epic/OpenRA
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MiniYaml.From* methods support deferred execution.

Browse Source 
Previously, the MiniYaml.From* helpers such as FromStream would consume the entire input and then return a list of top-level nodes. Now, the input is processed using deferred execution and top-level nodes are yielded as they are resolved from the input.

The motivating use-case is MapCache, which currently manually buffers nodes before passing to MiniYaml.FromString in order to improve responsiveness when large payloads are processed. Now that MiniYaml.FromStream yields results back as they come in, we can switch to that without disadvantage.

The maintains the performance where map cache can update search results as each node comes in over the network rather than having to wait for the entire batch to be transferred. Now we can also remove the string buffer that captures each node, reducing memory pressure and simplifying the code.
This commit is contained in:
RoosterDragon
2025-04-11 20:22:35 +01:00
committed by Gustas Kažukauskas
parent 5676d7be40
commit 8f46247dc9
17 changed files with 171 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
OpenRA.Game/Map/MapCache.cs
View File

@@ -14,7 +14,6 @@ using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using OpenRA.FileSystem;
@@ -141,7 +140,7 @@ namespace OpenRA
}
void LoadMapInternal(string map, IReadOnlyPackage package, MapClassification classification, string oldMap,
MapGridType? gridType = null, IEnumerable<List<MiniYamlNode>> modDataRules = null)
MapGridType? gridType = null, MiniYamlNode[][] modDataRules = null)
{
IReadOnlyPackage mapPackage = null;
try
@@ -242,41 +241,19 @@ namespace OpenRA
{
var client = HttpClientFactory.Create();
var stringPool = new HashSet<string>(); // Reuse common strings in YAML
var buffer = new StringBuilder();
// Limit each query to 50 maps at a time to avoid request size limits
for (var i = 0; i < queryUids.Count; i += 50)
foreach (var batchUids in queryUids.Chunk(50))
{
var batchUids = queryUids.Skip(i).Take(50).ToList();
var url = repositoryUrl + "hash/" + string.Join(",", batchUids) + "/yaml";
using (new PerfTimer("RemoteMapDetails"))
{
try
{
await using (var resultStream = await client.GetStreamAsync(url))
{
using (var resultReader = new StreamReader(resultStream))
{
while (true)
{
var line = await resultReader.ReadLineAsync();
if (line == null || !line.StartsWith('\t'))
{
var yaml = MiniYaml.FromString(buffer.ToString(), url, stringPool: stringPool);
buffer.Clear();
foreach (var kv in yaml)
previews[kv.Key].UpdateRemoteSearch(MapStatus.DownloadAvailable, kv.Value, mapDetailsReceived);
if (line == null)
break;
}
buffer.Append(line);
buffer.Append('\n');
}
}
}
var result = await client.GetStreamAsync(url);
var yaml = MiniYaml.FromStream(result, url, stringPool: stringPool);
foreach (var kv in yaml)
previews[kv.Key].UpdateRemoteSearch(MapStatus.DownloadAvailable, kv.Value, mapDetailsReceived);
foreach (var uid in batchUids)
{