Push notifications are quite a complicated thing, that strictly depends on a target platform(device and operating system where you wish to see the notifications), and used provider, which are not so many, so let’s list them: apple apns, google FCM(GCM), HUAWEI notifications(HMS), Mozilla Services and Microsoft Notification Hubs. These services is monopolists of the notifications market, so no one can send a push notification bypassing them, including Nakama or other companies, which, unlike Nakama, promise to send, but in fact are only intermediaries between users and listed services.
First of all, you need to find out which target platform you would like to send push notifications to, and try to understand what you need for that. Each platform and provider has its own specifics. Nakama is not a Push provider so it can’t help you do that.
Possible solution: you can place near the Nakama kind of a push sender, like this, similar or your own solution, make your own RPC call that will decide: if a player in game online, then send him in-game message, otherway, send a message to this server that will pass it to all(configured) target device(s).