We are experiencing a long delay in loading lua modules on server startup in our linux environments, is this a known problem? The same modules with the same version of nakama on mac loads almost instantly. If this is a known issue has it been resolved in the newest version of nakama?
Thanks.
@oscargoldman It depends a lot on what sort of attached disk you use with your Linux environments. There’s no reason that it’d be slow to load on Linux but near instant on macOS. The server pools VM instances used to handle runtime function execution in the server framework. This amortizes out the cost to load and evaluate the code at server startup. Right now its not clear to me there’s an issue at all.
How much code do you have that needs to get loaded?
This is nakama running in a VM on a windows machine. I’m also fairly sure that this is a recent behavior change as we were doing lua module dev on linux before and the modules were loading very quickly on server restart. The amount of lua code we load in is approx 70 kb, 26 modules in total.
Some additional info on this. When running nakama connecting to a local postgresql database, lua module loading is very fast, but when connecting to a remote database, lua modules take 1 minute or more. This is confirmed on both linux and mac vm’s.
@oscargoldman Do your modules run any kind of database work as part of their initialization - perhaps load data from the database to cache in-memory, or scan over users/storage etc?
Hey Andrei - well pretty well all of our modules use nk.storage, but only in the context of the rpc call itself. Would this require some kind of db handshake when the module is added to the server during the server start routine? If that was the case it makes sense the lua loading would take ages when using a remote db. We’re doing direct sql manipulation from our go module, both to our nakama db and another db we’re running, but the go module loads fine.
@oscargoldman
I am not seeing this issue my development env.
I have nakama server built from tip of the branch (as of today), and total of 12 modules (admittedly pretty simple modules, they are not dealing with storage or anything like that).
It is running on Fedora 31 linux on HyperV vm. My go version is 1.15
I have postgres server (v 12) running on ubuntu linux (native OS, no vm) – on another host (connected via pretty slow wireless network).
I am not seeing any delay.
I normally run local-2-local, but I saw this thread, and I was creating a new dev env – so figured let me try to see if I can reproduce this, but could not.
If you would like to me try something else to see I can repro this, I am happy to try.
@oscargoldman There is no “database handshake” required to set up the runtime - I was only asking specifically about database work you’ve built into your modules. That’s under your control and will definitely affect startup times. Storage operations defined inside RPC functions should not be a problem, those don’t actually execute unless called.
There are no issues I’m aware of with running the server against a remote database, in fact this is expected and encouraged for realistic deployments. Can you share your startup logs (at info level or more verbose, make sure you sanitise addresses/ports/passwords) including timestamps?
ok I will try another distro to run nakama on. We recently upgraded our development images to use Ubuntu 20.04 so it might be related to that. We used 18.04 LTS before and I’m pretty sure we didn’t have the problem. I will verify this before sending the startup log. Thanks.