Best of both worlds: SFS2X + server side Unity for realtime games (p2)

In part one of this article series we took a bird’s eye look at various client-server strategies for action multiplayer games. We then highlighted the advantages of running an hybrid solution with SmartFoxServer 2X and Unity on the server side to combine the best of both worlds.

In this second part we’ll be looking at the details of implementing such a solution, the potential difficulties and how to overcome them.

If you have skipped the first article we highly recommend to go back and read it, before you proceed.

» The basic architecture

To get started let’s review the basic architecture diagram from the previous article:

In this solution SmartFoxServer 2X acts as the main connection point for all clients, managing the session’s life cycle, providing the lobby and match making services etc., and orchestrating the creation and destruction of Unity servers.

Unity engines are attached to game Rooms to handle their game logic and removed when the match is over and the relative room is destroyed. In some cases, like for virtual worlds and MMOs, Unity engines can handle a larger portions of the world, based on the characteristics and size of the game map. In that case we will map a larger portion of the virtual world to as single Room.

In order to start a new game we create a Room and join all the players inside. Then  when clients are ready to start, we’ll spawn a new headless Unity server and tell each player to connect to it. This means that clients will use a maximum of 2 simultaneous connections, one toward SFS2X and the other toward the Unity server. With this approach everyone will be able to receive events from SFS2X (lobby updates, chat/buddy messages etc.) while the game is running.

» Headless Unity

Running Unity as a server is a bit uncommon compared to the normal use cases but it’s actually a very powerful solution for multiplayer games. In particular, the ability to create a server that runs on the same engine as the client is very convenient as we can share assets (e.g. game maps) and work with the same language and environment used for both sides.

In our use case we’ll be running “headless” Unity servers which means that Unity instances will not create a new desktop window and thus perform no rendering at all, as it is not needed on the server side.

To do this we just need to build our project as an executable for the OS used on the server side, and add a few parameters when launching it from command line.

How do I build my client/server Unity game exactly?

In this article we’re not going to develop a specific game. It would be beyond the scope of the tutorial and you can find innumerable videos and resources to learn how to create multiplayer games in Unity.

What we’re going to do, however, is we will describe how the Unity client and server are built and the server part is integrated with SmartFoxServer 2X. For the sake of example we can imagine to be building a multiplayer air-hockey game: we want it to be a 2D top down view that supports the browser via WebGL.

In order to write the client and server portions of the game we can use two different approaches:

1) Client and server logic in the same project

We can code both the client and server portions in the same Unity project and then activate one side via an external parameter. This is done by passing a command line argument to the Unity executable that is launched from the SFS2X side.

PROS:

  • we can manage all the code in one place
  • we can build both targets from the same project
  • code that is common to client and server can be reused without duplications
  • best for small or medium size projects

CONS:

  • there’s a potential for confusion and extra bugs when mixing server and client code
  • the app has to split itself into two so it can become cumbersome in large projects

2) Separate projects for client and server sides

This is probably the best approach for medium and large sized games as you can keep concerns and resources separated.

PROS:

  • More secure, as no server side code is present on the client side, so hackers can’t see what the server side does
  • Separation of concerns and resources
  • Easier to work with when there are teams of people dedicated to each side

CONS:

  • Some code duplication may occur, although shared libraries can usually mitigate the issue.

» Managing Unity engines

Now that we have a clearer picture of how to work with the client and server portions in Unity, it’s time to see how we can put it all together from the SmartFoxServer side.

In our example game we want to use a one-Unity-per-Room approach so that every game is managed by a separate Unity engine. To do this we can attach a custom Extension to each SFS2X game Room that will take care of running the Unity server.

To be able to run Unity from SmartFoxServer we will need to deploy the Unity executable to our Extension folder, for example like this:

Under our AirHockeyGame/ extension folder we deploy the Extension file (.jar file) and we create a Unity/ subdirectory were we can deploy the server side Unity engine. In our case we’re working with macOS where both the executable and relative assets are bundled in a single file.

Under Windows and Linux the build is comprised of two elements: the executable and a folder containing all the necessary assets. Make sure to copy both under the Unity/ directory.

Running multiple Unity on the same machine

For the sake of simplicity we will run the game servers on the same machine where SFS2X runs. This, in turn, requires to allocate a different TCP (or UDP) port number for each instance as it’s not possible for two apps to listen on the same port.

To do this we’ll use a simple PortManager class that allocates and deallocates a port value so we can pass it to the Unity engine itself.

package sfs2x.extension.unity;

public class PortManager
{
	private int portBaseValue = 7000;
	private int maxPorts = 100;
	
	private boolean [] ports = new boolean [maxPorts];
	
	public PortManager()
	{
		initPorts();
	}
	
	
	protected synchronized void initPorts()
	{
		for (int i = 0; i < maxPorts; i++)
		{
			ports[i] = false;
		}
	}
	
	public synchronized int getAvailablePort()
	{
		for (int i = 0; i < maxPorts; i++)
		{
			if (ports[i] == false)
			{
				ports[i] = true;
				return portBaseValue + i;
			}
		}
		
		throw new RuntimeException("All ports are currently taken. Cannot start a new server-side Unity instance");
	}

	public synchronized void releasePort(int value)
	{
		ports[value - portBaseValue] = false;
	}
}

The code should be straightforward: we keeps track of a series of numeric “slots” starting at 7000 (our initial port number) that can be taken and returned when the we’ve finished with the job. Notice the use of the synchronized mutator to make sure the class works with concurrent requests. We instantiate this class in the main Zone Extension of the game so that all Rooms can work with the same PortManager.

package sfs2x.extension.unity;

import com.smartfoxserver.v2.extensions.SFSExtension;

public class AirHockeyZoneExtension extends SFSExtension
{
	private PortManager pManager;
	
	@Override
	public void init()
	{
		pManager = new PortManager();
	}
	
	public PortManager getPortManager()
	{
		return pManager;
	}
}

Launching server side processes

Next we can move onto the core of the solution which is launching the Unity engines from our Room Extension:

package sfs2x.extension.unity;

import java.io.File;
import java.io.IOException;

import com.smartfoxserver.v2.extensions.SFSExtension;
import com.smartfoxserver.v2.entities.User;
import com.smartfoxserver.v2.entities.data.ISFSObject;

public class AirHockeyRoomExtension extends SFSExtension
{
	String unityExe = "OSXBuild";
	String workingDir = new File("extensions/AirHockey/Unity/OSXBuild.app/Contents/MacOS/").getAbsolutePath();
	Thread launcherThread;
	
	private class UnityRunner implements Runnable
	{
		private int tcpPort;
		
		public UnityRunner(int port)
		{
			this.tcpPort = port;
			
			try
			{
				getParentRoom().setVariable(new SFSRoomVariable("port", port));
			}
			catch(Exception ex)
			{
				trace(ex);
			}
		}
		
		@Override
		public void run()
		{
			try
			{
				String cmd = workingDir + "/" + unityExe;

				ProcessBuilder processBuilder = new ProcessBuilder(cmd, "-batchmode", "-logFile",  workingDir + "/unity.log", "--serverPort", String.valueOf(tcpPort)); 
				processBuilder.directory(new File(workingDir));
				
				Process proc = processBuilder.start();
				int exit = proc.waitFor();
				trace("Unity process terminated with code: " + exit);
			}
			catch(IOException ex)
			{
				trace("Error launching Unity executable: " + ex);
				ex.printStackTrace();
			}
			catch (Exception ex) 
			{
				trace("Unexpected exceptions: " + ex);
			}
			finally
			{
				((AirHockeyZoneExtension) getParentZone().getExtension()).getPortManager().releasePort(tcpPort);
			}
		}
	}

	@Override
	public void init()
	{
		AirHockeyZoneExtension ext = (AirHockeyZoneExtension) getParentZone().getExtension();
		int tcpPort = ext.getPortManager().getAvailablePort();
		
		launcherThread = new Thread(new UnityRunner(tcpPort), "unity-worker:" + tcpPort);
		launcherThread.start();

		trace("UnityLauncher started");
	}

	@Override
	public void destroy()
	{
		super.destroy();
		
		if (launcherThread.isAlive())
			launcherThread.interrupt();
		
		trace("UnityLauncher stopped");
	}
}

at the top of the class we define the path to our Unity executable relative to the SFS2X/ top folder. Under Windows and Linux you just have to point to the path of the executable but under macOS things are a little bit different, due to the bundling system we mentioned before.

To recap:

  • Windows / Linux: point to the folder where the executable was copied to, in other words under Unity/
  • macOS: you have to point to the bundle file as if were a directory (which it is) plus two extra steps. In other words Unity/<Name>.app/Content/MacOS/ where <Name> is the name of your build

Next we define an internal private class where the “magic” happens. The UnityRunner class implements Runnable and it is invoked by a separate thread that waits for the Process to complete its job and do some clean up before shutting down.

In the UnityRunner constructor we obtain a port value for the Unity instance and store it in the Room’s variables. Normally we would need to call getApi().setRoomVariables() method to update all clients, but we’re still in the init() method of our Extension where the server is not even accepting connections. In this situation it is fine to bypass the API and add variables directly (in any other situation, you would use the API instead).

The variable is going to be readable by clients when they join the Room, and it will be used to connect to the game server.

To run the Unity executable we use the convenient ProcessBuilder class from the JDK which allows to run system commands with any number of parameters. Let’s see them in detail:

  • -batchmode: this instructs Unity to run headless, with no rendering
  • -logFile: this allows us to specify where we want Unity to save a log file of all Unity’s runtime activity. This is particularly useful to debug unexpected issues, especially since we’ll have no visual output whatsoever
  • –serverPort: this is a custom command, one that it’s not recognized by Unity, and that we’ll be able to read it from inside the Unity server to obtain the port number assigned by our Extension

If we need to pass more parameters to the Unity server we can keep adding more custom commands preceded by a double dash (–) or any other similar convention. Just keep in mind that a single dash (-) is already used by the internal Unity commands and should be avoided.

Reading the command line parameters from Unity can be done via the System.Environment object:

	private int readServerPortFromCommandLine()
	{
		string[] args = System.Environment.GetCommandLineArgs();

		for (int i = 0; i < args.Length; i++) 
		{
			if (args[i] =="--serverPort"  && args.Length - 1) 
			{
				return int.Parse(args[i + 1]);
			}
		}

		return -1;
	}

Back to our Extension code we launch the process by calling the start() method on the ProcessBuilder instance and finally we call waitFor() to keep the new thread in pause until the process (i.e. Unity engine) is done.

The try/catch block takes care of potential runtime issues, such as a missing executable, in case we made some mistake with the deployment, and in the finally block takes care of releasing the port number before returning.

Deployment Notes:

To simplify the deployment of this example we would recommend copying the extension’s jar in the extensions/__lib__/ folder. This enables all classes to be loaded in the same classloader and make cross-extension communication easier.

Life cycle of a Unity engine

In this example we use the init() method of our Extension to bootstrap the Unity server instance so that it is ready as soon as the Room is created. An alternative to this approach would be to create the Room without starting the Unity server and wait for all the players to join before we launch it.

Both ways are perfectly valid and it all comes down to how our game works. For instance, a game that doesn’t need to wait for all players will use the first method and viceversa.

Similarly there can be different strategies to stop the Unity engine:

  • we wait for all players to leave the game server and then call Application.Quit() from Unity
  • we wait for all players to leave the SFS2X Room and the shut down the Unity process by calling Process.destroy() from our Extension

» Performance and scalability

Now that we have explored the basics of creating and managing Unity instances from a SmartFoxServer Extension, you may be wondering how resource intensive this could be, especially when we want to run hundreds of games simultaneously.

Unfortunately there is no clear-cut answer. The amount of resources (CPU and RAM especially) is heavily dependent on the complexity and type of game: in particular the number of players per room, the size and complexity of the game world, the physics, the amount of updates per second, they all contribute to the overall usage of resources.

Additionally there are the technical specifications of the machine running this system, which can also be highly variable.

As usual the best way to estimate the performance of a system is to benchmark it. In our case we can run one or more Unity servers and monitor the usage of CPU, RAM and network. Then we can extrapolate how many Unity engines can be run on our dedicated machine.

Supposing the tests give us a limit of approximately 50 engines per server (= 50 game rooms), how can we scale up our system to deal with a higher demands?

As you may expect the answer is to run multiple servers to host more Unity engines.
We can explore a couple of ways to do this:

  • Multi-game servers: we run a new dedicated server that will host a maximum of N Unity engines, based on our estimates. In our example we calculated 50 game Rooms,  which means each dedicated machine will host a max of 50 games, then a new machine needs to be setup or spawned
  • Cloud based micro servers: instead of running multiple games per machine we spawn a small dedicated server (e.g. like a t2.nano under AWS EC2) and run a single game per micro-server

Both approaches can be cloud-based and since most cloud providers offer Java API to automate operations in their environment, it can be convenient to integrate such API in the server side Extension to be able to manage on-demand servers.

This means that we can directly manage instances in our Extension code, creating new servers via the cloud API and removing them when they are no longer needed. This approach may sound complicated but in actuality most cloud vendors provide high level API that can add or remove a new server with a couple of lines of code.

» Next steps

We have explored the fundamental concepts to build an hybrid game server that takes advantage of both SmartFoxServer and Unity.

If you have any questions about what we have discussed so far, feel free to post your comments in our support board.

As a final note, we are planning to release a new API for SFS2X that will help you integrate Unity on the server side as we have illustrated, manage the Unity servers, load balancing etc.

Stay tuned for more.