Consider a farmville-esque game where user could visit another person village. This could be easily done with getting their village data in the storage
The problem is how would I query a list of similar people?
Say I fire an rpc that would gather a list of people with certain category (village prestige 100-200), this data itself could be stored on the account metadata, or a separate storage object for each user.
How would I query according to that filter? Listing all storage collections with key “village_prestige” would work but I don’t think nakama storage could filter based on query that is passed (e.g not returning all collection keys and then filter it on rpc).
Thoughts on this?
I see no reason you couldn’t do this with custom SQL in your RPC.
@cloudAle This is a great question. There’s two ways that I would consider to implement this game design with Nakama but I’ll first summarize the asynchronous matchmaking that you’re interested in to check that we’re discussing the same design requirements.
In Boom Beach a player builds their island and explores the ocean around them to find opponents island’s to battle against. The game makes it feel as though the opponents are discovered as part of exploration but essentially the level of the opponent is within some ranges to make sure the game is competitive but also fun. This is an asynchronous matchmaking approach hidden within a fun discovery process for the player.
It’s not quite clear in Boom Beach on what the set of criteria is that opponents matchmake together but we can assume there’s a couple of fields like:
- Level
- Base Power
- There’s likely more to it but this is sufficient to demonstrate the approach.
In Nakama we already support both match listings and the matchmaker which are great when the search criteria for opponents needs to be applied for realtime matches. The reason it works so well is that the data which is indexed to search on is tied to the lifecycle of either a multiplayer match or the socket connection of the user (depends on whichever feature above is used).
In asynchronous multiplayer it’s tricker to determine how long the data to be used for matchmaking should live for and be considered valid. This is why up until now game teams that want to do async matchmaking with Nakama have used one of these two approaches with our help:
-
Use specialized indexes on a storage collection of objects and access it via optimized SQL queries.
This approach is great when you have no more than one field you want to query on over a range and perhaps one or two fields that you need to filter on. For example find opponents that have a base type “small-island” and a level above 20 but below 40. You can express this on top of the current storage engine with a few custom SQL indexes into the JSONB object structure and perform an efficient query over the data.
You must be careful not to attempt to express more than one range query or end up with fields you want to query on that are not covered by the right index structure; otherwise you’ll end up with full table scans which will perform very poorly as the dataset grows over time. This is a common problem that game teams that don’t use Nakama encounter and it doesn’t matter whether the database engine is SQL or NoSQL. Ultimately you cannot escape from the fact that the dataset must be easy to traverse on disk to reach the records that will be evaluated in RAM. This is true of any database engine.
It’s very important to run EXPLAIN statements over the queries you write to access the data and understand whether you’ll hit specialized indexes you’ve set up or if it will result in a table scan.
Despite all the caveats this approach is the simplest to achieve what you want and can scale very well if the criteria to search on involves no more than a few fields and you’ve indexed correctly on them.
-
Use the Bleve search engine with your own custom indexer on the dataset you want to matchmake over.
You’ve probably seen in our documentation that we use the Bleve search engine inside the game server. This is a powerful generalized search system that can be used with a query language to express searches over a dataset.
With some custom Go code and a careful approach to the design there’s no reason you cannot create your own Bleve index which stores data that is specific to the game that you’d like to matchmake players on.
You would initialize the index inside the InitModule “main” function and import the Go package for the Bleve engine. Then create a goroutine that would load data in the background into the index from storage objects in a collection and index just the parts of those objects which you want to be able to matchmake on. It’s important to index just the necessary fields because this keeps the memory utilization low and prevents out of memory errors where the indexes exceed RAM.
You could then use a goroutine to run in the background which would periodically sweep the dataset in your storage collection and refresh the index with the information. This keeps the dataset fresh as changes happen to player data, it enables you to cull the dataset to keep it as small as possible, and as a background operation lets you limit the execution overhead to a controlled speed.
Finally you’d implement an RPC function that would execute a Bleve search query over the index and return the information back to the player with the results of the matchmaker. This is essentially the same as what we do with match listings and the matchmaker feature in the game server.
You can see that the two approaches require deep and meaningful knowledge of the exact fields to matchmake on and how to maintain the dataset and queries in an optimal way. This makes it very hard for us to provide a first class API to do asynchronous matchmaking that is scalable and flexible enough for most games. Nevertheless its an area of real interest for me and I’ll continue to experiment with ways we could make this feel effortless for developers.
If you can share more specifics for the criteria you’d want to search on I can suggest more specific example code that would help with the game design requirements.
Hope it helps.
1 Like
Thanks @novabyte for the detailed explanation, while we’re going to go with the Bleve route, it seems that it’ll be smaller if we just index userId and power/village_prestige, while leaving the data in the storage itself not on the bleve index.
This means we do a bleve search, to first get user id and then do a storage get for village data for that user id
Would this perform better than the one you’ve described above? Not doing any goroutine check to bootstrap / update data for the matchmaking process. This also means user hitting bleve index directly with rpc not into storage objects
Would this approach be better? We haven’t done this preferably at scale so we probably be missing a lot of details / pitfalls
while we’re going to go with the Bleve route, it seems that it’ll be smaller if we just index userId and power/village_prestige, while leaving the data in the storage itself not on the bleve index
@cloudAle I would definitely recommend this approach. It keeps the in-memory index small and you can use the results of that index query to fetch the objects in a single batch read from the storage engine.
Not doing any goroutine check to bootstrap / update data for the matchmaking process. This also means user hitting bleve index directly with rpc not into storage objects […]
Would this approach be better? We haven’t done this preferably at scale so we probably be missing a lot of details / pitfalls
I don’t quite follow what you mean. Hit the Bleve index in the RPC is definitely what you’d want to do it’s what we do in both match listings and the matchmaker. The rest of what you mentioned is not clear to me.