Profiling/Debugging Custom RPCs

Runtime Framework
,
1

Hey, I have a dozen or so custom RPCs in my current Nakama game server, and one of this is just returning a config object immediately back to the requester:

func myRPC(ctx context.Context, logger runtime.Logger, db *sql.DB, nk runtime.NakamaModule, payload string) (string, error) {
		buf, err := json.Marshal(cfg.Some.Path)
		if err != nil {
			return "", fmt.Errorf("failed to marshal response: %w", err)
		}
		return string(buf), err
}

Looking at my nginx logs, I see this:

[redacted] - - [25/Jul/2023:05:12:49 +0200] "POST /v2/rpc/[redacted]?unwrap=true HTTP/1.1" 200 492 "[redacted]" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0" "[redacted]" 0.135

Explaning:

  • The request is taking 135(!)ms
  • The resulting json is only 495 bytes on the write (it’s gzip, but the actual raw json itself is ~3.5kb. Not large enough to take 135ms to marshal)
  • I can reproduce the average time for this from doing the request via the console explorer too

I would love to hear your suggestions to drill into specifically what’s taking so much time, if there’s any particular tools, etc. My initial thought would be to use something like pprof, but I’m not sure if there’s alternatives that work better with Nakama.

Thanks for taking the time!

2

Hello @jnowls,

Nakama exposes pprof endpoints in the console API (default port 7351) at /debug/pprof/ (see all the endpoints here.

You can use basic auth with the console admin credentials to access these endpoints.

Hope this helps.

1 Like
3

@sesposito is there a guide or some example how it could be accessed.
I have tried normally to access it at : http://127.0.0.1:7351/debug/pprof to see it and previously having logged to console as admin but no info is shown o.O

4

Hello @Eatos,

You would access the snapshot using curl and then pass it to go tool pprof.

curl -u '$USER:$PASSWORD' 'http://127.0.0.1:7351/debug/pprof/$PROFILE > $PROFILE.out' && go tool pprof $PROFILE.out

For more details about pprof, please check the package docs.