201 lines
11 KiB
Plaintext
201 lines
11 KiB
Plaintext
Netcode for OpenRA
|
|
|
|
Current deficiencies:
|
|
bandwidth use to say *nothing*. we have to say nothing _somehow_.
|
|
of course, but it's currently the bulk of our cost. it will probably continue to be so.
|
|
it's mostly NODELAY'd ACKs, right?~half is reduction by half enough?
|
|
enough to host 8p on my link. good enough, then.
|
|
port forwarding
|
|
versioning
|
|
limited to 8 players -- why exactly? other than a dumb check in server? Just the dumb check i think, which i have removed. Untested - there are probably other stupid assumptions.
|
|
setting NODELAY doubles our bandwidth usage
|
|
various bits of the engine get confused when people drop -- dumb bug.
|
|
latency
|
|
A <-> server <-> B can longer than it needs to be if A is near B
|
|
Peer to peer in some aspect then?
|
|
udp makes this easier than current, but still interesting.
|
|
server probably still needs to get everything, even if there are other faster
|
|
ways some players can get it.
|
|
This is sounding like we'd need proper routing algorithms
|
|
maybe. i can see "attempt to connect to everyone" being the only useful strategy here, so we can just send our orders over every link we have. (and ask the server to pass on the rest, if neccessary)
|
|
need to be careful about how much upstream bandwidth we'd use
|
|
at 25fps, saying "no orders this frame" is ~25*28 bytes/sec/player.
|
|
even chris' connection can handle that for 8p.
|
|
700B/s/player
|
|
that's assuming we remove the '3 ticks per net tick' thing
|
|
if we keep that, divide by 3.
|
|
|
|
|
|
use JUST udp, or tcp too? the --> server link could easily be TCP (without NODELAY, if necessary) as long as our players graph is fully connected via udp.
|
|
can we make udp multicast work?
|
|
through the internet..... probably not.
|
|
LAN, we might as well broadcast. LAN bandwidth isnt a concern :S nor latency.
|
|
|
|
Solutions:
|
|
port forwarding
|
|
UPNP - fixes the problem on _some_ routers huge pain in the arse to implement
|
|
BS, there are OTS libraries we can use with .NET/mono.
|
|
UDP+punchthrough - better solution
|
|
requires a server with a udp port open and usable.
|
|
Can we abuse the master server in some way for this?
|
|
on our current host, probably not
|
|
- dreamhost: no
|
|
- master.open-ra.org: yes
|
|
- dchote mentioned that he knows people with a bunch of redundant servers we can use for persistant lobbies. maybe.
|
|
can we host ALL internet lobbies on external servers?
|
|
- This would be a nice feature.
|
|
- Provide a dedicated server for people to run their own.
|
|
|
|
versioning
|
|
we need at least mod versions + engine version.
|
|
|
|
player limit
|
|
easily removed
|
|
requires lobby upgrades > easy
|
|
|
|
NODELAY
|
|
tcp latency optimization, but bandwidth pessimization.
|
|
do we have the equiv of TCP_CORK sockopt on all platforms?
|
|
i don't think so, though we may be able to achieve the same result by toggling NODELAY.
|
|
|
|
Extra features?
|
|
- game chat <-> irc bridge
|
|
- MOTD etc broadcast from master server into game lobbies
|
|
- Desync communication from clients to the server (write desync diffs to server.log).
|
|
- actively send desync logs to us (for internet games at least)
|
|
- AOE(?) style desync detection stream
|
|
explain.
|
|
chrisf was talking about this a while back, they have a stream that they can insert arbitrary data into to check sync.
|
|
i like this plan.
|
|
|
|
|
|
|
|
Can we focus first on making our current setup with tcp sane? Or is this more work than throwing everything out and starting again
|
|
probably easier to start again; the current server is a quick hack with hacks added :D
|
|
dont forget the extra hacks added after that :D
|
|
Sounds like every other part of the engine :D
|
|
parts of it are good.
|
|
also, most of it is 'top-level' stuff that other code doesn't rely on.
|
|
|
|
Implementation:
|
|
TCP connection to servers run by us; udp between peers.
|
|
- what about latency? having N players in NZ, with a server in the US shouldn't make things suck.
|
|
- latency will either be
|
|
- 'longest path through the udp connection graph' (if we allow routing), or
|
|
- longest A <-> server <-> B without a connection A <-> B otherwise.
|
|
- ideally, we can get a totally connected graph and this isn't an issue.
|
|
server provides punchthrough introduction where necessary
|
|
udp provides:
|
|
- reliable, order transport.
|
|
- because we commonly have N frames of nothing before an order, we can optimise this.
|
|
- multiple streams (at least: chat, orders, sync. possibly lobby.)
|
|
sync: build arbitrary stream of sync data, send per-frame hashes to the server (only)
|
|
send stream for that frame on request
|
|
adding to the stream should be as easy as Game.AddToSyncLog( T )
|
|
|
|
who checks sync? how do we get individual players' syncreports together?
|
|
Can the server check either hashes or whatever new mechanism, and then send an
|
|
order to clients to dump their syncreport? Relying on clients to check is a bit dumb
|
|
-> server should be able to drop the players that desync
|
|
(who? if there are N distinct groups)
|
|
can we split the game into N distinct games, in that case?
|
|
drop any one-player games, since we know which client is at fault here.
|
|
We can do this in general; the faulty client can either automatically win,
|
|
or play a bot on their side.
|
|
Open Questions:
|
|
what level of connectivity do we _require_ wrt p2p?
|
|
can we achieve 100% reliably?
|
|
100% reachable nodes? 100% that we can _directly_ communicate with via udp.
|
|
For what reasons do we still need a server? Besides sync checking?
|
|
If lobbies are done by an external server controlled by us, user created servers are not really necessary are they?
|
|
in the case of a not-100% connected graph, a player quitting may disconnect it.
|
|
-> we should never kill a game when a player leaves.
|
|
perf degradation is bad but acceptable.
|
|
|
|
Protocol:
|
|
Handshake:
|
|
Client says: "Version, all mod versions available, player state(name,color,etc)"
|
|
Server says: "Game id, mods versions required, syncrandom, lobbystate" OR "GTFO"
|
|
-- client should be given the option to change mods if required and possible
|
|
-- this can be done by noting the required mods, dropping, reiniting appropriately, and connecting again.
|
|
The client still needs to be told by the server what it needs, for direct connect.
|
|
-- Would this include available slots?
|
|
It doesn't need to -> clients can be dumped into an "observers bin" on join, if there
|
|
isn't a free slot available
|
|
I'm not sure how nice that actually is -- having the server dump players into a
|
|
real slot if one is available is nicer, for noobs.
|
|
when you want to play with specific people, having some of them be unable to join until you kick someone is undesirable. -> observer bin. let host pro/demote people into/outof full slots.
|
|
"Heroes of Newerth" (a dota clone) is kinda like this.
|
|
their previous game was kinda a disaster wrt lobby stuff :D
|
|
(savage, s2)
|
|
HoN's netcode is REALLY nice. fps-style though, so not useful.
|
|
overflow into a bin, sure.
|
|
Client says : "ACK, NACK"?
|
|
--password? --banning?
|
|
Connection moves to "Lobby" protocol.
|
|
Lobby:
|
|
Server says: "assigned slot (as per above discussion), other player details(?)"
|
|
-- Player info exchange could be done via UDP at this stage
|
|
we probably want to use this stage to _connect_ udp, so probably not.
|
|
also, no reason; we care about neither latency nor bandwidth here.
|
|
Transfer of maps? mods?
|
|
One easy way to do this is for the host to upload to a repository
|
|
then the other clients can retrieve the missing bits using WebClient etc.
|
|
(Rather than reinventing a bulk transfer protocol)
|
|
Need a data structure for info that is needed for the lobby; slots, minimap(?)
|
|
Why should they not download everything immediately? maps arent big.
|
|
How do existing rts's solve this?
|
|
by not having the client determine lobby structure from the map.
|
|
(more generally, by not loading the map until game start)
|
|
Syncing client info in a less hacky way (currently its a hack that only works in frame 0). yes, we fix that.
|
|
|
|
|
|
|
|
protocol stages:
|
|
Handshake:
|
|
verify versions, password; choose player name/color; client is assigned a clientid
|
|
Lobby:
|
|
Sync lobby info; alter player name, color, team, etc.
|
|
transmit maps and any other transferable but not-currently-present data.
|
|
connect udp.
|
|
Game:
|
|
orders over udp; sync over tcp.
|
|
-> Return to Lobby mode at the end of a game (after postgame etc). Drop and reconnect is fine, if its automated.
|
|
|
|
Implementation stages:
|
|
Handshake:
|
|
- This can do nothing, as a first attempt.
|
|
Lobby:
|
|
- The udp connections are the most interesting bit here.
|
|
- As a first attempt, require 100% connectivity. i hate routing.
|
|
interesting or "interesting"?
|
|
udp punchthrough is the latter.
|
|
Do we need this in our first N attempts?
|
|
If we want to establish UDP connections with everyone, I imagine so.
|
|
We need punchthrough or upnp. (and, excepting broken routers, punchthrough isn't much more difficult. THAT is the "interesting" case.)
|
|
- Don't need transferrable data in first attempt.
|
|
Game:
|
|
- We need reliable, working, ordered, udp data transfer.
|
|
|
|
|
|
Other Issues: (to be solved later)
|
|
Where does this magical infinite webspace live?
|
|
maps are small.
|
|
If it's permanent you can't pretend a limit won't ever be hit
|
|
it isn't permanent; it only has to last the duration of the server.
|
|
Storing the maps indexed by sha is probably a very sane way to do it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|