Large state data sync issue

Runtime Framework
1

In an authoritative match, initially match state has 5000 items which I need to sync with the client when someone joins.

What I am doing here in the M.match_join handler, I am sending those items in a small chunk, let’s say 100 items per call. so a total of 50 calls.

Here is the code sample. Basically what it’s doing here is sending a total of 5000 items, 100 at a time.

function M.match_join(context, dispatcher, tick, state, presences)
    for _, presence in ipairs(presences) do
        print("New player: ", presence.username, state.world_id, context.match_id)

        for _, items_table in pairs(state.world_items) do
            local world_items_array = convert_world_items_info_to_array(items_table)

            local paginated_items_array = {
                items = {}
            }

            for i = 1, #world_items_array.items do
                table.insert(paginated_items_array.items, world_items_array.items[i])
                if i % state.items_in_one_page == 0 or i == #world_items_array.items then
                    local encoded_msg = nk.json_encode(paginated_items_array)
                    dispatcher.broadcast_message(MatchMessageType.WorldItemsInfo, encoded_msg, {presence})
                    paginated_items_array.items = {}
                end
            end
        end

        state.presences[presence.username] = presence
        print("Player joined match successfully:", presence.username, state.world_id, context.match_id)
    end
    return state
end

Depending on the server and client location and network status, completing this M.match_join execution may take a few seconds, even a few minutes. But when the player leaves the match and disconnects the socket before completing these calls, Nakama runtime still tries to send the messages even though the socket is disconnected.

A side effect of this is I receive the M.match_leave fires when above 50 calls are completed.

If user tried to same session immediately after leaving the same session, we got some other wired behavior.

The documentation says Lua runtime does not give any error if the broadcast failed. What to do in this scenario?

  1. Nakama: Nakama: 3.1.2
  2. Server Framework Runtime language: Lua
  3. Unity SDK: 2.8.1
2

Hello @ahamed,

I’d suggest you change your design and consider alternative ways to send data to the player, so you don’t hold the join process hostage to your batch. One recommendation would be to keep the bulk of your data in the storage engine and have the client request it before or after the player has joined.

1 Like
3

@ahamed To go into more depth - the behaviour you’re seeing is correct. The match handler will not allow concurrent calls to its functions, if there is a match_leave then it will fire after match_join has returned. This holds even if the leave event happens “while” match_join is executing.

You also don’t specify how large your “items” are when encoded to broadcast, but I’ll assume the result is not small. If this is the case then it might be expected to have the process take a little bit of time.

For this use case it might be best to use a chunking pattern something like this:

  1. Allow the join to complete but mark the player as not yet ready in the state. To players already in the game this might look like seeing a player in a “connecting…” state, while to the player that’s just joining it would likely be hidden behind a loading/connecting screen.
  2. Over the next few match_loop executions send a few broadcasts at a time to the connecting player containing chunks of the 5000 items you must send. You should experiment to see how many items to send per loop, but it should ideally be one broadcast per loop per player - there’s rarely a benefit to doing 2 broadcasts to the same player in one loop.
  3. Keep track in the match state of how many items the player has been sent out of the 5000, and which chunk is next. You can use this to indicate progress.
  4. Once all chunks are sent over some number of match_loops (and perhaps acknowledged by the client via sending a message to the match), you can mark the player as “ready” in the match state, tell everyone else in the match that they’re now “connected” rather than “connecting…”, and allow them to play/communicate with the rest of the match participants.

Hope this helps, and gives you another option you can use! :sunglasses:

2 Likes
4

@ftkg and @zyro this is what we exectly did to solve the problem, and seems like it’s working. Thanks, both of you for your quick response :slight_smile: