Currently when linting translations, we check for any unused translation keys. This works fine for the default mods, which own the entire sets of translation files. For community mods, they often import the translation files from the common mod assets, but they may only use some of the translations provided. Currently, they would get warnings about not using translations from the common files they have imported.
Since the community mods don't own those translations, getting warnings about it is annoying. To solve this issue, introduce a AllowUnusedTranslationsInExternalPackages in the mod.yaml which defaults to true. This will prevent reporting of unused translation keys from external assets. Keys that are used for external assets will still be validated, and keys from the mod assets will be both validated and unused keys will be reported.
We default the new flag to true and don't provide an update rule. This means community mods will get the new behaviour. For the default mods, we do want to check the "external" assets, since we control those assets. So the default mods have their mod.yaml updated to disable the flag and retain the existing behaviour of checking everything.
Using the glory of regex, we can scrape any Lua script files that a map includes and locate calls to the UserInterface.Translate method. We can then treat them in the same way as C# fields marked with a TranslationReferenceAttribute. This allows the lint check to validate the translation invoked in the .lua script has a matching entry in the translation .ftl files, with all the required arguments (if any).
We can also locate any calls to AddPrimaryObjective or AddSecondaryObjective defined by the utils.lua script, which also accept translation keys.
The are a couple of restrictions:
- When linting the map, we don't check for keys in the ftl file that are unused. This is because the linter doesn't load all the keys when checking maps.
- In order to validate translation arguments with the regex, we require the Lua script to pass the table of arguments inline at the callsite. If it does not, we raise a warning so the user can adjust the code.
- Test all translation languages, not just English.
- Report any fields marked with TranslationReferenceAttribute that the lint pass lacked the knowledge to check.
- Improve context provided by lint messages.
Restructure code for readability.
- Rename the filename parameter to name and make it mandatory. Review all callers and ensure a useful string is provided as input, to ensure sufficient context is included for logging and debugging. This can be a filename, url, or any arbitrary text so include whatever context seems reasonable.
- When several MiniYamls are created that have similar content, provide a shared string pool. This allows strings that are common between all the yaml to be shared, reducing long term memory usage. We also change the pool from a dictionary to a set. Originally a Dictionary had to be used so we could call TryGetValue to get a reference to the pooled string. Now that more recent versions of dotnet provide a TryGetValue on HashSet, we can use a set directly without the memory wasted by having to store both keys and values in a dictionary.
This allows the LINQ spelling to be used, but benefits from the performance improvement of the specific methods for these classes that provide the same result.
When handling the Nodes collection in MiniYaml, individual nodes are located via one of two methods:
// Lookup a single key with linear search.
var node = yaml.Nodes.FirstOrDefault(n => n.Key == "SomeKey");
// Convert to dictionary, expecting many key lookups.
var dict = nodes.ToDictionary();
// Lookup a single key in the dictionary.
var node = dict["SomeKey"];
To simplify lookup of individual keys via linear search, provide helper methods NodeWithKeyOrDefault and NodeWithKey. These helpers do the equivalent of Single{OrDefault} searches. Whilst this requires checking the whole list, it provides a useful correctness check. Two duplicated keys in TS yaml are fixed as a result. We can also optimize the helpers to not use LINQ, avoiding allocation of the delegate to search for a key.
Adjust existing code to use either lnear searches or dictionary lookups based on whether it will be resolving many keys. Resolving few keys can be done with linear searches to avoid building a dictionary. Resolving many keys should be done with a dictionary to avoid quaradtic runtime from repeated linear searches.
This changeset is motivated by a simple concept - get rid of the MiniYaml.Clone and MiniYamlNode.Clone methods to avoid deep copying yaml trees during merging. MiniYaml becoming immutable allows the merge function to reuse existing yaml trees rather than cloning them, saving on memory and improving merge performance. On initial loading the YAML for all maps is processed, so this provides a small reduction in initial loading time.
The rest of the changeset is dealing with the change in the exposed API surface. Some With* helper methods are introduced to allow creating new YAML from existing YAML. Areas of code that generated small amounts of YAML are able to transition directly to the immutable model without too much ceremony. Some use cases are far less ergonomic even with these helper methods and so a MiniYamlBuilder is introduced to retain mutable creation functionality. This allows those areas to continue to use the old mutable structures. The main users are the update rules and linting capabilities.
