Why CockroachDB not PostgreSQL

Game Design
1
  • Why Nakama default to use CockroachDB? Is there any customization or improvement on CockroachDB?
  • As you mentioned on use without cockroachdb, all tests are run on CockroachDB. Is it ok to use PostgreSQL instead?

I really appreciate your answer.

Thanks.

2

Hi @wxdyhj welcome :wave:

Why Nakama default to use CockroachDB? Is there any customization or improvement on CockroachDB?

We chose CockroachDB for a variety of reasons:

  • A fast, scalable, SQL database engine with a unique approach to scale out which does not give up true transactional semantics.
  • It’s open-source, modern, and cloud-native. Easier to deploy and operationalize than older databases.
  • Supports unique features like geotagging of data to specific regions as part of multi-regional replication.

I could go on but really you should read the documentation because it provides a lot of clarity for what makes it different to other SQL-orientated database engines: CockroachDB: Distributed SQL - Cockroach Labs

As you mentioned on use without cockroachdb, all tests are run on CockroachDB. Is it ok to use PostgreSQL instead?

You can use any Postgres wire-compatible database engine which supports the JSONB column type with Nakama. We only officially support CockroachDB at this stage because it’s a lot of work to run and maintain integration tests and performance tests with other database engines but we do work with game teams that use Postgres or AWS Aurora for Postgres.

1 Like
3

Hey,

This might be answered somewhere else already but I thought I’d jump on board here and ask it anyway.

So the key to Nakama’s ability to scale comes down to cockroach db? Because the database scales (globally), Nakama scales. So on top of the db, you have X frontend instances and these will of course be crucial with in-memory events like multiplayer games, but at the core it’s the database that scale.

Thanks!

1 Like
4

@nixarn That’s somewhat true but it doesn’t do justice to how the game server is designed. We should really create some detailed architecture docs but that will have to come in the new year after we’ve released the upcoming Nakama 3.0 version. In the meantime I can add more detail in this thread.

The game server is broken up into realtime and non-realtime features. This cleanly maps to what you see in most of the client SDKs as the client object and the socket object. The non-realtime features of the system like friends, groups, notifications, chat history, etc. all depend on carefully crafted queries and indexes and communicate directly with the database server(s).

A few of these game features like leaderboards and tournaments also use some in-memory caching capabilities inside the game server to handle certain features (like rank calculations). We use datastructures like skiplists to efficiently traverse and cache these large datasets in RAM. This means we can eliminate the need for secondary caches like Redis, or MemSQL which is a part of the goal of the technology. Minimize the amount of infrastructure needed to manage and scale highly successful games.

The realtime features of the game server utilize a combination of gossip protocols, inter-cluster communication, dotted version vectors, and distributed datastructures to create a cluster wide “view” of all socket connections and which users these sockets belong to. We typically call these presences and are internally represented as a tuple of { user_id, session_id, node_id }. This representation underpins all the realtime features like chat, relayed multiplayer, status events, etc, etc.

The authoritative multiplayer engine and matchmaker in the game server take advantage of the cluster system but actually also have some of their own special components to manage the way messages are broadcast, hook into the lifecycle of the server instance itself, and replicate information needed for players to find and form matches together.

The beauty of this design (at least as far as we believe) is that it treats the game server itself like an in-memory replicated database engine. So if you skew your vision :eyes: of the technology you could actually think of what we’ve built as an in-memory database which utilizes a persistent Postgres wire-compatible database for core data. It also minimizes the total infrastructure you need to manage down to:

  1. A load balancer to open sockets and handle requests (also do SSL termination).
  2. N-number of Nakama instances clustered together.
  3. N-number of database instances clustered together.

You end up with a scalable model with no single point of failure (except on DNS lookups but you can also set up secondary DNS if needed - depends on the budget for infrastructure in a game project) and can operate at very large scale as needed.

Sorry for the long post. Hope this helps.

PS: I’ve glossed over a few details but the majority of the important parts are covered above. :slight_smile:

4 Likes
5

Amazing answer! Thanks a lot @novabyte! Also excited to hear about Nakama 3.0!

1 Like