Packet loss is a potential issue for multiplayer games and recognizing it earlier rather than later can be very helpful in making your game more enjoyable, especially for players with sub-optimal connections. In this article we’re going to explore the different types of packet loss, how to identify the issue and how to solve it when it appears.
» Under the hood
Before we discuss packet loss let’s review some of the basic concepts underlying this subject. SmartFoxServer uses several protocols such as TCP, UDP, HTTP, Websocket and SFS2X’s own custom protocol, which have different characteristics and also operate at different levels.
TCP and UDP are called “transport protocols” as they provide the essential foundations to transmit custom data. HTTP, Websocket and SFS2X’s own protocol instead work on top of these and define the rules (i.e. the protocol) of communication between applications.
The diagram below should clarify this hierarchy a bit further:
We can see that TCP works as a reliable and ordered transport, meaning that it already makes sure that packets get to their destination in the same order in which they were sent. To do so it exchanges messages between client and server to keep track of every packet.
Re-transmission is also supported in case one of the two sides didn’t acknowledge the reception of a previous packet. Finally “traffic control” allows TCP to avoid saturating the bandwidth when the other side of the communication is still dealing with older messages.
UDP on the other hand is a much simpler protocol, with no guarantee of packet delivery or ordering. Without retransmission and delivery checks, UDP can be much faster as it doesn’t need extra state messages, but its use is limited to scenarios where packet loss can be tolerated (multiplayer games can be one of them).
Finally when we talk about protocols such as HTTP or Websocket we are already implying their underlying transport (TCP in this case). SmartFoxServer’s own protocol can work with both TCP and UDP, making it more suitable for multiplayer games, among other things.
» Where did my packets go?
After introducing the basic features of TCP and UDP you might think that only UDP is a candidate for packet loss. After all, we’ve just shown how TCP guarantees data delivery and therefore we would expect that protocols built on top of it (HTTP, Websocket, and SFS2X protocol) would not suffer from such issues.
This is mostly correct but there are exceptions and, more importantly, the server has the last word on which packet might be discarded. To be more clear, SmartFoxServer does its best to avoid dropping packets but there are two situations in which this will happen:
- Incoming packet is too big: to avoid potential attacks SFS2X is configured to drop packets that are too big. By default any packet > 500KB will be rejected, as it’s quite unexpected for a multiplayer client to send requests that big. Of course the setting can be changed for different needs.
- The client’s outgoing queue is full: every connected client has a queue of outgoing messages. If the client is still busy dealing with previous packets the server will queue the new outgoing messages until the client is ready to receive more. When a client is very slow at processing packets, the server side queue might fill up and at this point the server is forced to drop messages.
The second scenario is the most common for TCP-based connections and SFS2X attempts to discard non-priority packets before anything else, to mitigate the situation. Should the queue become completely full there will be no other choice but to drop the next outgoing responses.
Finally as regards UDP the packet-loss aspect is included by design as a tradeoff for speed. Therefore, when you plan to use this protocol you should already work with its limitations in mind. Typical usages of this approach are real-time games where client-prediction and smoothing are employed to hide potential updates loss.
We have several examples of this approach, if you want to learn more:
» How do I know if my game is dropping packets?
We have mentioned that if you’re using UDP you will need to assume that packets will be dropped and include some form of compensation for it. When using TCP instead, you can check your AdminTool’s Dashboard to see if you have outgoing or incoming dropped packets.
In a live environment it is expected to see outgoing dropped packets, typically in the range of 5-10%. Within these values you should not worry about it as there always is a minority of slower clients that lag behind and cause the server to drop some messages. On the other hand if the outgoing rate is > 10-15% it’s probably best to investigate.
As regards incoming packets the value should always be 0%, as this only represents very large requests being discarded. (With 500KB as the default limit very few games -if any- should ever see an incoming dropped packet).
» What can be done to minimize packet drops?
Generally speaking in turn-based games we should always expect no packet loss or few rare instances for very slow clients. Instead real-time games are more likely to see some packet drops, typically for users with a slow device or laggy network.
The golden rule to minimize the issue is to avoid pushing more packets than your clients can realistically process and, while there isn’t a fixed value for this approach, a general guideline is this:
- TCP can safely handle up to 20-25pps (packets per second)
- UDP can use much higher frequency (e.g. 30-50pps, sometimes even higher) because packet drop is already accounted for.
So if you’re planning for a TCP-based action game with 30+ updates per second think twice as you’re asking too much and it will be very difficult to make it work smoothly, if at all possible.
The physical location of the servers plays an important role in this aspect, as a large distance from the client adds extra latency which in turn can reduce the packet-per-second rate.
Mobile connections have higher latency than regular home/office connections. If you’re planning a mobile game you should keep this aspect into account and run some tests in advance to see what is an acceptable pps for a mid tier phone.
Finally, even if it sounds obvious, dedicated hosting is the best hosting. Meaning that to run a good multiplayer game you need high quality bandwidth, which is only found from reputable hosting providers. It should go without saying, but we’ve had cases where this aspect wasn’t as clear as we thought it would be.
» Continue the discussion
We hope to have clarified some of the main aspects of this subject so you can improve your game performance, if necessary. If you have more questions or want to join the conversation please feel free to do so via our support forums.