By adding a UpdateVisibilityNextTick flag against every FrozenActor to track when a visibility update is required, we can remove the dirtyFrozenActorIds set in FrozenActorLayer. In the Tick method we can now avoid a set lookup.
Also, don't create the frozenActorsToRemove list until we need one to avoid an allocation.
d010157611