Sending / Receiving byte data ( Lua )

[Eatos]

Hello everyone,
For i while now i have been trying to send and receive binary data in lua runtime instead to use json messages.
Why?
Even though current packets are small - avrage 200bytes, creating all that garbage within platforms is very necesserly. On other side i would like to access and store data more efficiently.
For instance: I have a packet that currently is compromise of 4 ints but there size or there could all be represented with single int:
Example:
CastSpell: <int actor (255), int lvl (10), int spId (255), int target (100) >
: Ideally i want this to pack to single int and then read bytes depending what i need to read and unpack.
Seams that both functions for string packing/unpacking are not supported in Lua.

Another thing what i would like to do and ease on loading/sending packets as json is their verification.
Current process is server receives packet, it looks for opcode is there additional handler/verifier, sends message data, message data gets decoded with nk.json_decoded and then based on keys i read what i need to read and additionally verify check before returning a go can it be done or not, or correcting packet it self.

This nk.json_decoded sounds to me a very not needed thing, and something that i can avoid if standard byte operations are supported within lua runtime.
I have tried client wise to do something like this:

"
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct MoveUnitPacket
{
[MarshalAs(UnmanagedType.I4)]
public int actorNumber;
};

private static byte[] StructToBytes(object packet)
{
    int size = Marshal.SizeOf(packet);
    byte[] bytes = new byte[size];
    IntPtr structPtr = Marshal.AllocHGlobal(size);
    Marshal.StructureToPtr(packet, structPtr, false);
    Marshal.Copy(structPtr, bytes, 0, size);
    Marshal.FreeHGlobal(structPtr);
    return bytes;
}

----------------------------------
MoveUnitPacket movePct = new MoveUnitPacket();
movePct.actorNumber = 13;
byte[] pctSerialized = StructToBytes(movePct);
await _socket.SendMatchStateAsync(matchId, 74, pctSerialized);

"
When this packet is received server side, it is received i hope as base64 string or something else, either way it is a string.
" {“match_id”:“da12bdd8-e7e5-4f2d-a62c-48ffd79f3ef3.nakama1”,“op_code”:74,“data”:“DQAAAA==”}} "
Notice: DQAAAA== string.

So next thing what i have tried is while reading this topic regarding FlatBuffers FlatBuffers with Lua It is said that by adapting that patch for lua 5.1 flatbuffers could be used in Lua runtime.
I have inspect a similar lib which i wanted to use and read it through to make sure it should be Lua 5.1 compilat and also Gopher-lua one.
Reading Gopher-Lua lib it is stated it supports byte operation with string.
Gopher-Lua

Yet when using Lua unpack it fails on string.bytes operation when trying to unpack, claiming it doesn’t exist.

So question is: Can this be integrated as working with byte data in runtime is pretty standalone procedure and it should be supported by default in runtime.

Alternatives are no go for me currently which are switch to Go language runtime or TypeScript one, as i have a lot of code in lua done already and plus i am not familiar with either of those two languages so switching to them would cause a big halt on production.

Thanks.

Current Nakam server runtime version is 3.12.0.

[ftkg]

Hi,
we might update the runtime in the future, but right now I’d recommend you move your performance critical endpoints to Go if you really need that performance boost.

[Eatos]

@ftkg Sorry to ressurect this but i have managed to do it

So my goal was to get msgpack to lua runtime it was tricky but in the end it works much better than raw encoding/decoding. I will leave a small guide for other folks if they want to try it.

This is for Unity but i belive with any engine it will work as long msgpack lib is valid one.

1. Download and import msgpack from https://github.com/neuecc/MessagePack-CSharp#quick-start
2. Make sure to correctly setup it for all platforms if you intended to use msgpack on your own types and what not.
3. For Lua, https://github.com/AleksandarEri/nakama-lua-msgpack/blob/main/message_pack.lua
   Import it, into your project, notice i have tried it with simple data types, mostly array based, i haven’t tried it with more complex, but i belive it should work.
4. In client, i had a bit trouble sending data until i noticed that i don’t need double to base64 for, as client already base64 any data that is being sent to match for instance.
5. Profit!

Lua example:

local msgpack = require(‘Libs.message_pack’)
local message = “lNIAAAAA0gAAAGXSAAAABqZLb2xiYXM=”
local t = msgpack.unpack(message)
print("Unpacked: ")
print(t[1],t[2],t[3],t[4])


When you run this base64 code in runtime you should get something like this as an output.

For client side:

And thats it, now you can use msgpack server/client side