Advice for data structure in Storage Engine with misc. filters

Game Design
, ,
1

Hi,

I’d like some advice for how to structure my Storage Engine setup; I’m working on a fairly simple project that uses async battles against other (mostly) random opponents.

It’s made up of just 2 RPC functions; one to submit an entry, and one to get an entry for a specific stage.

// Request
itemVersion: string
stage: number

// Response
success: boolean
entry: any
  • The entry is a Json object that has been submitted by a player.
  • Each player can submit multiple different entries, but should ideally not be able to be matched up against their own previous submissions.
  • The outcome of the battle is handled locally, nothing will happen to the user who submitted the entry.
  • I expect to cap the amount of entries to in the ballpark of 5-10.000 per stage, at which point I’ll start overwriting submissions.
  • There’s currently a limit of 20 stages.

I’ve structured the collections like this:

collection: builds{itemVersion}-{stage}
key: new UUID v4

Currently I went the quick’n’dirty route and use a custom Sql query in order to get a random element:

let collection = `builds${msg.itemVersion}-${msg.stage}`
let query = 'SELECT collection, value FROM storage WHERE collection = $1 ORDER BY RANDOM() LIMIT 1'
let queryArgs = [collection]
let list = nk.sqlQuery(query, queryArgs)

This has worked fine for now, but I’d like to clean it up going forward.

I’m planning to extend the logic with:

  • Bulk retrieval of entries for multiple stages
  • Server-side filtering based on user id (aka don’t fetch your own submissions)
  • Possibility for filtering on additional metadata (eg. score/ranking)

My thinking was to change the collection structure and then use storage indexing to query. However, I haven’t used indices before and am a bit unsure if this is the right way to go, so would appreciate some advice first :slight_smile:

The most important hurdle is the bulk retrieval; I have a feeling that ~20 storage read requests across collections containing ~5000 entries would be quite a heavy operation?

–

Currently using Typescript with CockroachDB. Existing data does not have to be migrated.

2

Hello @Theo-Rapidfire,

Could you elaborate a bit on what itemVersion and stage are and how they’re used? Can a player submit multiple entries per stage?

3

Sure!

itemVersion is a string that is used for compatibility purposes to ensure that people play against other submissions of the same app version. We are constantly adding a lot of new items, so this was a simple way to make sure the game has the required assets for the item.

The format of the string is a semVer string, with an optional -dev suffix for internal testing purposes. It is currently following the app version, so a new collection is used when we update the apps. Down the road we’re likely to allow assigning a custom value from a remote config, in case we eg. discover broken items in a live build.

stage is an integer value, which increases by 1 every time the player has finished a round. Starts at 0, and increases up to a potential max of ~20 if the player survives long enough.

The player must fight against opponents of the same stage, in order to progress their own stage; eg. if you start a battle at stage 5, we’ll find another submission also on stage 5, and send that back to you. This is where the bulk retrieval becomes a bit problematic, as it’s currently split up in different collections based on the stage value.

Yes, a player can submit multiple entries per stage. As we get more players, we’re likely to filter the submissions, so not all submissions are accepted.