Set of questions about multipe features

Runtime Framework
1

Hello,
I went over the documentation couple of time, and I have couple of questions about different features I could not saw. Checking if I missed them, or they exist in a different form.
I am putting them all in one thread if that is ok, but I can split per categories if you prefer ease future research.

Authentication:

  • Authentication system is mainly made for human. If I want an external system calling Nakama API, how can I add authentication for system and script?
  • I can link/authenticate any custom credential with CustomAuthenticate, but I can not see if I can add as many Custom Type as I want? How does this work if I want multiple custom credential?

DB:
-With Collection, can I have atomic operation over specific field? For example, can I do a “sum 5” operation to a integer value to add 5 to the field directly in db?

  • I am not a huge fan of them, but do we have Transaction feature for the database?
  • Is there collection system for binary data? If I want to save non JSON data, for example the replay of a game.

Hiro:

  • I am trying to find more info about the different Hiro API with the server framework, mainly on how error/conflict are managed. Do you have a bit more detail documentation about this? For example, are the claim reward API idempotent ? If I try to claim the same reward multiple time what will happen?

API Gateway:

  • If I want to add layer of caching for existant API or additional RPC, this has to be done through the before/after hook? Is there something easier/better?
  • Satori has a API key feature, does this exist for Nakama too? If I want to restrict access depending on the build, or limit API rate, is there something native for this?

Social:
-Can I have different type of friend connection? For example having Friend, having Neighbor, love relation etc
-In same style, you mention Twitter like link, but can I actually have non asymmetric link such as Twitter follower/subscriber? Where X following Y does not mean Y follows X.
-Do we have very large group (>100k Player) feature. I am thinking Faction here, like Hord vs Alliance group, where the goal is not to have the same level of details and feature as a normal group.
-Do we have shared objective feature, allowing all member of a group or a faction, to contribute to the same counter/reward system.

Event/Satori
-Not sure where this questions fits, but I wish to send internal event at the end of a call, to trigger custom RPC. What is the suggested way to achieve that?

2

Hello Jonathan,

I’ll be replying inline.

Authentication:

  • For Server-To-Server call you’d use the runtime HTTP key, see our docs here.
  • Custom IDs are intended for a 1:1 mapping from some other system, like a central accounts service which would already handle unifying user identities before presenting a single ID. To link multiple custom IDs directly in Nakama, what we usually recommend is using the AuthenticateDevice (there can be multiple device IDs per account), and use a before-hook to prefix the ID with some custom string. So an ID of “some-external-id” might become “external-system-name|some-external-id” in the before hook, and be processed accordingly by Nakama.

DB:

  • A Storage Object (SO) is identified by a composite key of Collection, Key and UserID, you can’t update a whole collection, but you can list all SOs within a given collection. You need to think how to best model your data to fit the Storage Engine layout.
    The standard way to update a value in the Storage Engine is a read/update/write model using conditional writes. So for your example, you’d read the value, add 5 to it in memory, and use a conditional write to make sure you’re updating the most up-to-date data. If the conditional write fails, you’ll need to re-read the value and repeat the process.
    If strictly required, it is possible to update values in a single query with custom SQL. Nakama’s runtime exposes direct SQL access handles and functions throughout, although usually we recommend against it unless there’s no way around it due to the risk of introducing query or connection handling bugs and their impact on the wider system. We’re happy to help write or review any custom queries or other runtime code, so we can proactively make changes as needed.
    We can also introduce additional core runtime functions as needed, if there are operations that make perfect sense but aren’t yet exposed as first class functions.
  • All operations done within the same API call (e.g.: StorageWrite, WalletsUpdate, MultiUpdate) are performed transactionally. If using the runtime’s SQL access it’s possible to handle transactions directly.
  • Large binary files should not be stored in the DB but rather on something like S3 or Cloud Storage if possible. Nakama can act as a gateway for this, it’s a pattern we’ve used frequently.
    On the write side Nakama accepts the write request via an RPC function, does validation or encoding changes as needed, and securely writes it to S3. It can also maintain a “directory” of S3 contents and metadata associated with them for filtered querying by users.
    Then on the read side it can create short-lived signed access URLs so clients can pull these files directly.
    This pattern typically scales very well and provides validation, access controls, and extended query functionality.

Hiro:

  • The error handling depends on each system, as each one has been purpose built for its own set of feature constraints. In the case of something like claiming rewards there is some commonality.
    Using Achievements as an example, let’s assume the user tries to claim an achievement completion reward twice. No matter how close together they may attempt to perform these claims (perhaps some kind of bad actor scenario) one will succeed and the other will fail. Normal clients would claim it once, the response would tell them the reward that was received, and the updated state will indicate there is no longer a reward pending - so they are not expected to try claiming again.
    Are there any more specific error cases you’re looking for?

API Gateway:

  • You can add some custom application-level caching with a before hook, but for API caching this is typically not done in the application server itself but at another layer of the deployment. We also do not frequently need to cache API responses as core data is likely to change continuously, but for example configuration is a good candidate. Worth noting Nakama’s operations are very efficient and don’t even use the database where it can be avoided - many realtime features such as chat messaging, following users, in-app notifications, matchmaking, multiplayer and so on are handled in-memory in Nakama rather than relying on the DB for message routing or other functionality.
  • It is unusual to feature gate in a single Nakama deployment based on client build. Typically we’d expect separate Nakama deployments for each environment (dev, QA, production etc) which could have their own rules they apply for all their own clients. This can cover feature restrictions, rate limits, maximum number of connections, and more.
    As a special case, API rate limiting is ideally done in a layer before Nakama. Perhaps a transparent gateway or proxy service, to ensure Nakama doesn’t even handle the connections far enough to then reject them - they should never reach the application layer at all in this case. We can help with this kind of setup if you’re looking for more in-depth recommendations.

Social:

  • By default a Friend is just a link in a graph of nodes, it cannot be labelled in the APIs directly. This could be easily complemented with some additional Storage Objects and searched with Storage Search, if required.
    We’ve also had a few feature requests for adding exactly this kind of Friend “state” so we’re potentially looking to introduce it as a first class feature within the next couple of months.
  • What you’re describing sounds like Nakama’s Status feature which subscribes for realtime updates on activity of friends or other users, if allowed. This might be online/offline notifications, rich presence, hot joining, and more.
  • Nakama has a Groups feature which is suited to support Guilds or Factions, although if you want to split all the player-base between two factions, there’s better approaches we can discuss. Usually a marker on the user’s core metadata indicating which Faction they belong to.
  • The Storage Engine does support shared storage objects that can be used for shared stashes, Guild chests or other uses, however this needs some careful thought as to not create points of contention. We’ve done this before even for whole playerbase shared objectives like you might see in Helldivers, we have very scalable patterns ready to use for this.

Event/Satori:

  • I think we’d need a better understanding of what you’re trying to achieve here, can you share some examples of what you need?
3 Likes
3

Thanks a lot for the complete answer !

Please let me add couple of follow up questions:
For the Hiro Error handling, what I am searching is documentation that state this for the different case. The example of reward is good, but is there a place explaining things like this, detailing the error code, or information about immediately/eventually consistency or idempotency of an API?

For API Gateway, configuration is indeed exactly what I had in mind when asking the question. Top Leaderboards page could be useful too, and a couple of minor API/RPC. Your second point about a layer before Nakama is interesting, can I add a API Gateway (such as AWS APIG or Kong) before Nakama?

For the shared resource, what I am looking for is, for example, to be able to have things like a shared counter that could get hundred of concurent update (increase by X).

For Event/Satori, let say after a Player called a Custom RPC which save his latest win, I want to to make an additional RPC call asynchrnously that will update a leaderboard with the win information.

Also, I just realized, the Device Authentication comes without any password arguement. If I want to prevent account stealing, I need to add password verification in a prehook?

4

No problem, responding to these as well:

For the Hiro Error handling, what I am searching is documentation that state this for the different case. The example of reward is good, but is there a place explaining things like this, detailing the error code, or information about immediately/eventually consistency or idempotency of an API?

We currently do not have documentation with these details, but it’s something that we should add in the future (cc @HeroicNathan). Hiro comes with an onboarding period where we can help clarifying some of these questions that may pop up during development.

For API Gateway, configuration is indeed exactly what I had in mind when asking the question. Top Leaderboards page could be useful too, and a couple of minor API/RPC. Your second point about a layer before Nakama is interesting, can I add a API Gateway (such as AWS APIG or Kong) before Nakama?

Yes, it is possible.

For the shared resource, what I am looking for is, for example, to be able to have things like a shared counter that could get hundred of concurent update (increase by X).

This is also easy to do, but it requires a small amount of custom code to ensure that the value is incremented in memory and flushed to the DB periodically (could be every second even), to ensure the db is not overwhelmed if these updates happen at scale. It’s a pattern we’ve used with some customers and that we can help with if needed.

For Event/Satori, let say after a Player called a Custom RPC which save his latest win, I want to to make an additional RPC call asynchrnously that will update a leaderboard with the win information.

Updating the leaderboard is something you’d do in the custom RPC itself or using Nakama’s event pipeline, Satori would just drive the live-events definitions and analytics capture.

1 Like
5

Thanks !
I totally missed the event pipeline, that seems to solve what I needed.

For the addition of an API Gateway, do you have any recommendation/guide about part to be careful with? I have not made my choice of APIG, but I am leaning strongly for AWS right now.

I have a couple of additional questions that came up:
-The Device Authentication comes without any password argument. If I want to prevent account stealing, I need to add password verification in a prehook or is there any existing out of the box solution?

-This is maybe a question for an other forum section, but is it possible to segregate node by type of request? I understand Nakama architecture has by default everything in the same node, but can I ask to create multiple type of nodes, each specialized in specific actions? The goal (among others) here is to be able to isolate some service that are critical to my game, from the services that are optional/risky.

6

For the addition of an API Gateway, do you have any recommendation/guide about part to be careful with? I have not made my choice of APIG, but I am leaning strongly for AWS right now.

This is something that would be better discussed with someone from our cloud team, we can set up a call with them included so we can walk through the options. We’ve used a number of different solutions so far with varying capabilities, I have no doubt we can fit your preferred one too.

-The Device Authentication comes without any password argument. If I want to prevent account stealing, I need to add password verification in a prehook or is there any existing out of the box solution?

For username and password authentication you should use AuthenticateEmail. Device authentication can be used for frictionless authentication using the device identifier as an unique ID for the user. Alternatively, Device or Custom auth can be used in conjunction with a BeforeHook to authenticate with an external Identity Provider using standard username/password or other SSO patterns - usually the authentication flow is handled in the client and then a small amount of code in a before hook is used to identify the Nakama user with the external identity provider ID.

Typically with a before hook on AuthenticateDevice the client is expected to submit an identity token that can either be verified cryptographically or via a server-to-server call to a trusted service. The token is then replaced with the underlying ID before further processing. Just knowing the underlying ID would never be enough for another user to assume the identity. Device identifiers are also not public, and not visible on the user’s profile or any similar endpoints that can be called by others.

-This is maybe a question for an other forum section, but is it possible to segregate node by type of request? I understand Nakama architecture has by default everything in the same node, but can I ask to create multiple type of nodes, each specialized in specific actions? The goal (among others) here is to be able to isolate some service that are critical to my game, from the services that are optional/risky.

This is possible if needed, but relatively unusual. Nakama in essence just handles traffic directed to it, there’s no need for it to be aware if that traffic is only a subset of the total volume and other Nakama nodes are handling different subsets.

1 Like
7

Understood for the first two points, thanks!

For the Nakama nodes segregation, the goal is really to isolate node based on the importance/risk of their tasks, so achieving blast radius reduction.

For example let say I add a feature that automatically moderate the chat message. This can be a resource heavy and slow down/crash the whole process if not done correctly, and it is also “acceptable” if this feature alone where is not working. But I do not want this feature to put at risk the request of authentication and inventory management.
By isolating the requests using the auto moderation feature in process that run on their own node, I am highly reducing the impact for the other node.

8

Hi @sesposito ,
I just wanted to add my interest to a feature that would allow custom Friend state.
Thanks
Andre