Nakama js client has problems with decoding data

Client Libraries
1

Hi,

we have the following problem:

We are receiving the following data from the server:

{"match_data":{"match_id":"dab9224e-e04c-48e9-9a7d-322249e697b7.nakama-0", "data":"CgZqBAgBEAIKGiIYEAIYCCACKgYKBAgCEAEqCAoECAMQARA/CgQaAggCCkcSRQoVCgYKBAoCAAgKCCIGCgQIAxABEgE4ChUKBgoECgIAAQoIIgYKBAgDEAESATEKFQoGCgQKAgAJCggiBgoECAMQARIBOQo2WjQIAhIcaWkzblUzZU1WMWFWcVRMTmdyRDhuQUVGVmFBMhoMQm9yeXMtb25saW5lIAEqAlBMCjZaNAgDEhxBSmdXNDlIVWxNTUxpVEtySW1yNXJhckJKUkMzGgxCb3J5cy1vbmxpbmUgASoCUEwKCmIICAIQ4NQDGAEKCGIGCAMgASgCCgRSAhAB"}}

By reading the code, we can see that match_data.data is passed to b64DecodeUnicode:

function b64DecodeUnicode(str) {
  return decodeURIComponent(decode(str).split("").map(function(c) {
    return "%" + ("00" + c.charCodeAt(0).toString(16)).slice(-2);
  }).join(""));
}

and the function throws exception while trying to decode %fd:

Uncaught URIError: URI malformed at decodeURIComponent.

The following snippet shows the problem. The long string to be decoded is the one returned by decode(str) call in above snippet.

'%0a%06%6a%04%08%01%10%03%0a%1a%22%18%10%02%18%08%20%02%2a%06%0a%04%08%02%10%01%2a%08%0a%04%08%03%10%01%10%3f%0a%04%1a%02%08%02%0a%36%5a%34%08%02%12%1c%41%4a%67%57%34%39%48%55%6c%4d%4d%4c%69%54%4b%72%49%6d%72%35%72%61%72%42%4a%52%43%33%1a%0c%42%6f%72%79%73%2d%6f%6e%6c%69%6e%65%20%01%2a%02%50%4c%0a%36%5a%34%08%03%12%1c%69%69%33%6e%55%33%65%4d%56%31%61%56%71%54%4c%4e%67%72%44%38%6e%41%45%46%56%61%41%32%1a%0c%42%6f%72%79%73%2d%6f%6e%6c%69%6e%65%20%01%2a%02%50%4c%0a%0a%62%08%08%02%10%fd%fd%03%18%01%0a%08%62%06%08%03%20%01%28%02%0a%04%52%02%10%01'.split('%').forEach(v => { if (v != '') { console.log(v); decodeURIComponent('%' + v); }});

We have already tried using WebSocketAdapterPb adapter and the problem still exists. The error occures in the internals of the library.

So I attempted to fix this by replacing decodeURIComponent with unescape function and it started to decode properly:

function b64DecodeUnicode(str) {
  return unescape(decode(str).split("").map(function(c) {
    return "%" + ("00" + c.charCodeAt(0).toString(16)).slice(-2);
  }).join(""));
}

however unescape is deprecated function and decodeURIComponent should be used instead (but, as I already mentioned - it fails).

So, having this problem solved, another one occured. The result of this function is passed to JSON.parse:

message.match_data.data = message.match_data.data != null ? JSON.parse(b64DecodeUnicode(message.match_data.data)) : null;

which fails on parsing JSON, as the data passed to it is not JSON string, but binary.

We have verified and other client libraries (Unity and Android) works fine.

Please advice.

2

Please open this issue against nakama-js SDK repository.

In the meantime, what would happen if you don’t use nakama-js-proto and just use the regular nakama-js variant?

3

not using nakama-js-proto leads to the same wrong behavior as it just changes the transport protocol between library and server. The problem is within decoding match data, no matter what protocol have been used to get it.

4

Sure, feel free to open an issue on the Nakama-JS SDK - contribution and fixes are welcomed :slight_smile:

5

We have same problems and fixed it locally, but we have problems with building nakama-js from sources to make contribution :grinning: here this issue - How to build source · Issue #120 · heroiclabs/nakama-js · GitHub

6

By the way, our vision of the issue with decoding binary match (and party) data in nakama-js.

This issue has two parts.

First is binary data traffic between nakama-js-protobuf and nakama-js encoded with base64. This encoding is auto-generated code by ts-proto module for protobuf (nakama-js/realtime.ts at 6de7fd5127dd65da61748763fb6aecf1e0adb899 · heroiclabs/nakama-js · GitHub and nakama-js/realtime.ts at 6de7fd5127dd65da61748763fb6aecf1e0adb899 · heroiclabs/nakama-js · GitHub). There is no easy way to fix this (ts-proto has only fromJSON/toJSON functions and nothing like fromObject/toObject). We can ignore this i think. Yes, this is strange (we receive binary data inside nakama-js-protobuf, encode with base64, send it into nakama-js and decode base64 back to binary data), but this is on client side, and we can be ok with this.

Second is wrong base64 encoding implementation for binary data (nakama-js/socket.ts at 6de7fd5127dd65da61748763fb6aecf1e0adb899 · heroiclabs/nakama-js · GitHub) and hard-coded JSON serialization (nakama-js/socket.ts at 6de7fd5127dd65da61748763fb6aecf1e0adb899 · heroiclabs/nakama-js · GitHub). Wrong base64 encoding implementation is discussed in this topic. Binary data can’t be parsed as JSON and there is no sense to do this.

Our solution is adding new methods to socket implementation (like onpackmatchdata/onunpackmatchdata) that will be give opportunity to override default behavior with custom logic (right base64 encoding without json serialization in this case).

Or implementing these two behaviors (current and without json serialization) and select one as socket creation parameter (like createSocket(useSSL = false, verbose: boolean = false, adapter : WebSocketAdapter = new WebSocketAdapterText(), binary = false): Socket)

7

@Marcin here a PR for binary data, you can test it Binary match and party data by lugehorsam · Pull Request #124 · heroiclabs/nakama-js · GitHub (i doesn’t but i will this week)

8

@formatCvt Thank you, we’ll check it.

9

@Marcin @formatCvt please let me know how it goes and if there are any remaining issues, we can address them ASAP.

10

working fine for us, thanks!

1 Like
11

@lugehorsam sorry but we didn’t have time to check it, we will probably make it today.

12

Works well, thanks.

1 Like