Handling the server shutdown

In today’s article we’re going to discuss how to detect a server shut down, be it intentional or unexpected, and what tools can we use to exit cleanly without loosing precious data.

» Exit strategies

There are many ways in which SmartFoxServer can be shut down: via the AdminTool, a service manager, using a termination signal from the OS, issuing a System.exit() from code, a process crash etc.

In order to detect when the server process is shutting down, and react to it, we can use a native mechanism in Java called a shutdown hook, which allows to spawn a custom thread that will run right before the JVM shuts down.

This is a basic example of how to setup a shutdown hook:

Runtime.getRuntime().addShutdownHook
(
	new Thread() 
	{ 
  		public void run() 
  		{ 
			System.out.println("Shutdown Hook is running !"); 
  		} 
	}
);

The code is very straightforward and in theory the system allows us to register multiple hooks in the same application. However, because shutdown hooks run concurrently when the JVM is exiting, we would not recommend to use more than one unless you’ve good reasons to do so.

Also there are a few situations in which the mechanism will not trigger:

  • a crash of the JVM due to some native error
  • an OutOfMemoryError
  • a termination signal from the OS (such as SIGKILL in Unix, or equivalent in Windows)

Even though shutdown hooks are not guaranteed to run 100% of the times they are still a valid tool to deal with most common situations.

» Additional strategies

Now you might be asking, what about those cases that are not supported such as a system hard crash or a sudden process death? If your game maintains lots of critical in-memory data there are additional strategies you could employ to minimize, and possibly neutralize, the risk of data loss.

1) Frequent auto-saves

As simple as it may sound a basic auto-saving mechanism based on the server’s Task Scheduler would be sufficient to ensure that data is saved at regular intervals. In case of a server shutdown you can still rely on the shutdown hook mechanism to take care of the final save, and in case this doesn’t work you would have just lost a minimal amount of data.

2) Distributed in-memory database

A more radical approach to the problem is to use a distributed in-memory database. With this solution you would be physically running multiple copies of your in-memory data and if the main SmartFoxServer process (or the machine) dies, the data would be safe and sound, as it resides on more than one machine.

A good candidate for this scenario would be using something like Hazelcast.

This is a diagram from the white-paper linked above that shows how one or more SmartFoxServer instances interact with the distributed datastore. Essentially the datastore is made of two ore more nodes (servers) running Hazelcast and SmartFoxServer plugging into it via integrating Hazelcast in a server Extension. The beauty of this system is that you can work with regular POJOs and let the system deal with the complexity of distributing the copies in the shared memory.

Also notice the presence of an additional database as a secondary storage for further data security.

» Conclusions

Based on how critical is your in-memory game state you can choose and combine these different strategies to avoid loosing important data. If you want to learn more make sure to follow all the linked resources in this article and, if you want to discuss this topic further, join us on our support forum.