Provide an explicit size of 512, smaller than the default 2048. The cursors for all mods still pack onto this single sheet, so we avoid wasting memory on larger sheets.
Sorting the cursors also helps pack them onto the sheets more efficiently.
When a move is made where the source and target locations are the same and no actual moving is required, a path of length 0 is returned. When a move cannot be made as there is no valid route, a path of length 0 is also returned. This means Move is unable to tell the difference between no movement required, and no path is possible. Currently it will hit the `hadNoPath` case and report CompleteDestinationBlocked.
To fix the scenario where the source and target location match, track a alreadyAtDestination field. When this scenario triggers, report CompleteDestinationReached instead.
This fixes activities that were using this result to inform their next actions. e.g. MoveOntoAndTurn would previously cancel the Turn portion of the activity, believing that the destination could not be reached. Now, it knows the destination was reached (since we are already there!) and will perform the turn.
Sheets can be buffered - where a copy of their data is kept in RAM. This allows for modifications to the sheet to be batched in the RAM buffer, before later being committed to VRAM in a single operation. The SheetBuilder class allows for sprites to be allocated onto one or more sheets. Each time a sheet is filled, it will allocate a new sheet. Sheets allocated by the sheet builder will typically use the buffering mechanism.
Previously each time the builder allocated a new sheet, the buffer would get thrown away, and the next sheet would allocate a fresh buffer. These buffers can be large and may accumulate before the GC cleans them up. So although only one buffer will be live at a time, they can cause a spike in memory used by the process during loading.
We can avoid this issue by allowing the buffer from the previous sheet to be reused by the next sheet. This is possible because the sheet builder only has one live sheet for modifications at a time, and they are all the same type and size. By allocating only one buffer per builder instead of one per sheet, we reduce the peak memory required during loading.
- In CursorManager, we can release the buffer on the final sheet after loading cursors. We don't need to release the buffer on every sheet in the builder, as the builder will handle that.
- In SpriteCache, we don't need to call CreateBuffer explicitly, as the builder will do that for us.
- In RadarWidget, we don't need to call CreateBuffer explicitly, as GetData will do that for us.
When the process is running, we use a finally block to call Log.Dispose and flush any outstanding logs to disk before the process exits. This works when we handle any exception in a matching catch block.
When the exception is unhandled, then the finally block will not run and instead the process will just exit. To fix this, flush the logs inside a catch block instead before rethrowing the error. This ensures we get logs even when crashing.
Fixes a regression from 2c435c0506 - where the support actors must be able to path to the base actor in order to prevent them from spawning in isolated areas. If the base actor is immovable, they cannot path onto it because the base actor blocks them, so no support actors will spawn.
Fix this by allowing the support actors to path back to any cell adjacent to an immovable base actor, rather than requiring them to be able to path to its location directly.
If multiple tooltips are defined with conditionals attached, the Encyclopedia could not handle this. Now, it selects the tooltip EnabledByDefault to match usage across the rest of the code.
When HarvesterBotModule is ordering idle harvesters to nearby resources, it previously scanned from the harvester's current location. Instead, it now scans from the location of the nearest refinery. As the harvester will likely make many runs between the resource and the refinery, it is better to choose a location near the refinery. This will minimise overall distance travelled to harvest the resource patch.
The AI uses HarvesterBotModule to check for idle harvesters, and give them harvest orders. By default it scans every 50 ticks (2 seconds at normal speed), and for any idle harvesters locates an ore patch and issues a harvest order. This FindNextResource to scan for a suitable ore path is quite expensive. If the AI has to scan for ore patches for several harvesters, then this can produce a noticeable lag spike. Additionally, when there are no available ore patches, the scan will just keep repeating since the harvesters will always be idle - thus the lag spikes repeat every 50 ticks.
To reduce the impact, there already exists a randomization on the first scan interval so that multiple different AIs scan on different ticks. By ensuring the AI players scan at different times, we avoid a huge lag spike where they all operate on the same tick.
To reduce the impact even more, we make four additional changes:
- Scans continue to be done every 50 ticks to detect harvesters. But we spread out the searches for ore patches over multiple later ticks. We'll only perform one ore patch search per tick. This means instead of ordering e.g. 30 harvesters on a single tick and creating a spike, we order one on each tick over the next 30 ticks instead. This spreads out the performance impact.
- When a harvester fails to locate any suitable ore patch, we put it on a longer cooldown, by default 5x the regular cooldown. We don't need to scan as often for these harvesters, since it'll take time for new resources to appear.
- We change the path search in FindNextResource from FindPathToTargetCellByPredicate to FindPathToTargetCells. The format in an undirected path search that must flood fill from the start location. If ore is on the other side of the map, this entails searching the whole map which is very expensive. By maintaining a lookup of resource types per cell, we can instead give the target locations directly to the path search. This lookup requires a small overhead to maintain, but allows for a far more efficient path search to be carried out. The search can be directed towards the target locations, and the hierarchical path finder can be employed resulting in a path search that explores far fewer cells. A few tweaks are made to ResourceClaimLayer to avoid it creating empty list entries when this can be avoided.
- We adjust how the enemy avoidance cost is done. Previously, this search used world.FindActorsInCircle to check for nearby enemies, but this check was done for every cell that was searched, and is itself quite expensive. Now, we create a series of "bins" and cache the additional cost for that bin. This is a less fine grained approach but is sufficient for our intended goal of "avoid resource patches with too many enemies nearby". The customCost function is now less expensive so we can reuse the avoidance cost stored for each bin, rather than calculating fresh for every cell.
When this hits the case that "As both ends are accessible, we can freely swap them." - we must note that we are reversing the path search and pass the information into the HierarchicalPathFinder. When a normal path search occurs, the actor trying to pathfind will never check its own location - and thus never gets blocked by itself. When a search is reversed, the search will check the actors location. If we inform the search it is doing done in reverse, it will special case this scenario and avoid the actor blocking itself. But if it is not told about this scenario, then this special case is not applied and no path will be found when in fact a path is possible.
When spawning starting units, or spawning units when collecting a crate, nearby locations will be used. If a nearby location cannot be reached, e.g. it is on top of a cliff or blocked in by trees, then any unit spawned there will be isolated which is not ideal.
Use the PathMightExistForLocomotorBlockedByImmovable method to filter nearby locations to those where the spawned unit can path back to the original location. This ensures the spawned unit is not isolated.
The default Stream implementation of this method has to rent an array so it can call the overload that accepts an array, and then copy the output over. This is because the array overload is required and the span overload was only added more recently.
We can avoid the overhead of this by implementing the span overload and working with the destination span directly. Do so for all classes we have that derive from Stream, and redirect their array overload to the span overload for code reuse.
- AI places rally points at pathable locations. A pathable location isn't strictly required, but avoids the AI setting rally points at seemingly dumb locations. This is an addtional check on top of the existing buildability check.
- AI now evaluates rally points every AssignRallyPointsInterval (default 100 ticks). Invalid rally points aren't that harmful, so no need to check them every tick. Additionally we do a rolling update so rally points for multiple locations are spread across multiple ticks to reduce any potential lag spikes.
The ExtractEmmyLuaAPI utility command, invoked with `--emmy-lua-api`, produces a documentation file that is used by the [OpenRA Lua Language Extension](https://marketplace.visualstudio.com/items?itemName=openra.vscode-openra-lua) to provide documentation and type information is VSCode and VSCode compatible editors when editing the Lua scripts.
We improve the documentation and types produced by this utility in a few ways:
- Require descriptions to be provided for all items.
- Fix the type definitions of the base engine types (cpos, wpos, wangle, wdist, wvec, cvec) to match with the actual bindings on the C# side. Add some extra bindings for these types to increase their utility.
- Introduce ScriptEmmyTypeOverrideAttribute to allow the C# side of the bindings to provide a more specific type. The utility command now requires this to be used to avoid accidentally exporting poor type information.
- Fix a handful of scripts where the new type information revealed warnings.
The ability to ScriptEmmyTypeOverrideAttribute allows parameters and return types to provide a more specific type compared to the previous, weak, type definition. For example LuaValue mapped to `any`, LuaTable mapped to `table`, and LuaFunction mapped to `function`. These types are all non-specific. `any` can be anything, `table` is a table without known types for its keys or values, `function` is a function with an unknown signature.
Now, we can provide specific types. , e.g. instead of `table`, ReinforcementsGlobal.ReinforceWithTransport is able to specify `{ [1]: actor, [2]: actor[] }` - a table with keys 1 and 2, whose values are an actor, and a table of actors respectively. The callback functions in MapGlobal now have signatures, e.g. instead of `function` we have `fun(a: actor):boolean`. In UtilsGlobal, we also make use of generic types. These work in a similar fashion to generics in C#. These methods operate on collections, we can introduce a generic parameter named `T` for the type of the items in those collections. Now the return type and callback parameters can also use that generic type. This means the return type or callback functions operate on the same type as whatever type is in the collection you pass in. e.g. Utils.Do accepts a collection typed as `T[]` with a callback function invoked on each item typed as `fun(item: T)`. If you pass in actors, the callback operates on an actor. If you pass in strings, the callback operates on a string, etc.
Overall, these changes should result in an improved user experience for those editing OpenRA Lua scripts in a compatible IDE.
In 4312a4d3f4 MiniYaml merging was adjusted. One effect of this change was that duplicate keys in files that did not previously require merging was previously allowed, but was now an error. (Test case `TestMergeConflictsNoMerge`)
The installer files were relying on the previous behaviour to allow multiple `ContentPackage` keys. The above change caused a regression where attempting to manage mod content would crash due to now erroring on the duplicate keys.
We fix the issue by applying a unique ID suffix, as is a common pattern elsewhere in our yaml files, and teach InstallFromSourceLogic to recognise and strip it.
In 05ed9d9a73 we stopped caching the values with ToArray to resolve a desync. But even caching the enumerable can lead to a desync, so remove the caching entirely.
----
Let's explain how the code that cached values via ToArray could desync.
Usually, the cell given by `self.Location` matches with the cell given by `self.GetTargetablePositions()`. However if the unit is moving and close to the boundary between two cells, it is possible for the targetable position to be an adjacent cell instead.
Combined with the fact hovering over the unit will evaluate `CurrentAdjacentCells` only for the local player and not everybody, the following sequence becomes possible to induce a desync:
- As the APC is moving into the last cell before unloading, the local player hovers over it. `self.Location` is the last cell, but `self.GetTargetablePositions()` gives the *previous* cell (as the unit is close to the boundary between the cells)
- The local player then caches `CurrentAdjacentCells`. The cache key of `self.Location` is the final cell, but the values are calculated for `self.GetTargetablePositions()` of an *adjacent* cell.
- When the order to unload is resolved, the cache key of `CurrentAdjacentCells` is already `self.Location` and so `CurrentAdjacentCells` is *not* updated.
- The units unload into cells based on the *adjacent* cell.
Then, for other players in the game:
- The hover does nothing for these players.
- When the order is resolved, `CurrentAdjacentCells` is out of date and is re-evaluated.
- `self.Location` and `self.GetTargetablePositions()` are both the last cell, because the unit has finished moving.
- So the cache is updated with a key of `self.Location` and values from the *same* cell.
- The units unload into cells based on the *current* cell.
As the units unload into different cells, a desync occurs. Ultimately the cause here is that cache key is insufficient - `self.Location` can have the same value but the output can differ. The function isn't a pure function so memoizing the result via `ToArray()` isn't sound.
Reverting it to cache the enumerable, which is then lazily re-evaluated reduces the scope of possible desyncs but is NOT a full solve. The cached enumerable caches the result of `Actor.GetTargetablePositions()` which isn't a fully lazy sequence. A different result is returned depending on `EnabledTargetablePositions.Any()`. Therefore, if the traits were to enable/disable inbetween, then we can still end up with different results. Memoizing the enumerable isn't sound either!
Currently our only trait is `HitShape` which is enabled based on conditions. A condition that enables/disables it based on movement would be one way to trigger this scenario. Let's say you have a unit where you toggle between two hit shapes when it is moving and when it stops moving. That would allow you to replicate the above scenario once again.
Instead of trying to come up with a sound caching mechanism in the face of a series of complex inputs, we just give up on trying to cache this information at all.
