Cryptic Studios Interview

For their latest “Soapbox” feature, RPG Vault has published a two-page interview with Cryptic Studios’ Shannon Posniewski about the challenges associated with keeping MMORPG servers up and running.

Q: What does it take to get a crashed server back up and running?

A: Cities shards have only two points of failure that will bring one down, the database and the master control program (MCP). The database is very well tested and hardened. It is also backed up quite often. The Cities master control program is fairly simple and has also been battle-hardened. In addition, of course, the network ops team keeps an eye on the machines to make sure they are healthy and do preventative maintenance during scheduled times. These two critical systems haven’t failed in my memory. If the database failed, we’d need to get the last backup available on a functioning machine. This isn’t particularly difficult, and I wouldn’t expect it to take too long. If the MCP dies, then all we need to do is run it on another machine in the server set and the shard should come right up.

Besides those two, the loss of machines or server processes are automatically handled by the MCP as rapidly and transparently as possible.

Player data is flushed continuously to the database from all the servers.

Each player gets stored once every eight minutes on average, so if there is data loss, it should be minor.