Plugin was built with a different version of package

Hey guys, I’ve been racking my brain trying to fix this error. As soon as I added https://github.com/awa/go-iap to verify receipts, I’m getting this “plugin was built with a different version of package golang.org/x/net/context/ctxhttp” error
I’ve tried to add
replace golang.org/x/net => golang.org/x/net v0.0.0-20200226121028-0de0cce0169b
to go.mod, matching the same version on Nakama, but it’s still failing

@hasbean I’ve solved this particular error many times before and had planned to describe how best to achieve what you want but I had no problem when I loaded the plugin with your dependency. I’ll describe what I’ve done.

$> go version
go version go1.14 linux/amd64
$> env GO111MODULE=on go get "github.com/awa/go-iap"
$> env GO111MODULE=on go mod vendor
$> env GO111MODULE=on go build --trimpath --mod=vendor --buildmode=plugin -o ./backend.so
...

This is the code I added to InitModule just to check the import (I’ve never used that library before):

client := appstore.New()
logger.Info("%+v", client)

I used Go 1.14 and Nakama 2.11.0 release. Hope this helps.

The error seems to come when I use the Google Play version.

try this one

	jsonKey, err := ioutil.ReadFile("jsonKey.json")
	if err != nil {
		log.Fatal(err)
	}

	client := playstore.New(jsonKey)

@hasbean I had to use code which looks like:

client, _ := playstore.New([]byte(""))
logger.Info("%+v", client)

I ignored the second error argument because I want to check the plugin compiles and loads rather than write well structured code and I received this error from the plugin loader:

plugin.Open("/nakama/data/modules/backend"): plugin was built with a different version of package golang.org/x/net/context/ctxhttp

I’ll write up my usual notes on how to solve it for you now :slight_smile:

@hasbean Ok I’ve solved it but it did prove to be quite tricky due to the cross-over of dependencies between what’s used in Nakama server and what’s pulled in transitively by the “go-iap” library. First I’ll explain why the plugin loader exposes this error when the shared object is loaded.

Due to this issue with Go plugins they bundle their own copy of the Go runtime inside the shared object. They also store a dependency list to ensure that undefined behaviour cannot occur if the host binary contains a different dependency tree to the plugin code.

If the dependencies between host binary and shared object (plugin) do not match exactly then the plugin loader will throw the error above. The problem next is how to make sure the dependencies can be matched exactly between the two pieces of binary code.

This is where Go makes things difficult because it uses minimal version selection. It’s quite hard to pin transitive dependencies because Go will select the minimal version that it thinks satisfies the dependency tree in just your plugin code project without any way to make it clear that there’s another dependency tree to consider. Fortunately there’s a handy trick although a little unorthodox to solve this problem.

You can use “blank imports” to trick the Go toolchain into considering the transitive dependencies from the server as a direct dependency in your plugin code just to allow you to pin them to the exact versions used in Nakama server.

In your case the “go-iap” has a number of dependencies which transitively overlap with Nakama so the solution needs this added to your imports at the top of your main.go file:

import (
	"context"
	"database/sql"
	"github.com/awa/go-iap/playstore"
	"github.com/heroiclabs/nakama-common/runtime"

	// NOTE: Required to pin transitive dependencies.
	_ "cloud.google.com/go/compute/metadata"
	_ "go.opencensus.io/trace"
	_ "golang.org/x/net/context"
	_ "golang.org/x/oauth2"
	_ "golang.org/x/sys/unix"
	_ "google.golang.org/api/googleapi"
	_ "google.golang.org/genproto/googleapis/rpc/status"
)

You can then update your mod file to track the correct set of transitives:

module github.com/username/somerepo

go 1.14

require (
	cloud.google.com/go v0.45.1
	github.com/awa/go-iap v1.3.0
	github.com/heroiclabs/nakama-common v1.4.0
	go.opencensus.io v0.22.1
	golang.org/x/net v0.0.0-20200226121028-0de0cce0169b
	golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
	golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae
	google.golang.org/api v0.10.0
	google.golang.org/genproto v0.0.0-20200226201735-46b91f19d98c
)

To check the exact set of dependencies in Nakama use vendor/modules.txt and the go.mod file. You must use these files on the correct tag that matches the release of Nakama you use and note that dependencies can and will be updated in future with new releases of the server.

Finally use env GO111MODULE=on go mod vendor as usual to vendor the dependencies and the rest of the steps as normal to compile and load your Go code.

Couldn’t get it to work up until I realized I forgot the “vendor” command.

Much appreciated! I have to figure out a way to smooth out upgrades between versions now, but it works!

1 Like

Hello,
Sadly i still get an error " plugin was built with a different version of package golang.org/x/net/context/ctxhttp", how can i solve this dependency?

@naskha01 What version of the game server do you use? What steps have you tried so far? What does your go.mod file look like? Which dependencies did you pin as shown above?

As you can guess just “i still get an error” really doesn’t help the community to help you.

@novabyte I am getting the same error with the github.com/dgrijalva/jwt-go package. Here is more information on how I have tried to solve this.

  • I am running Nakama version 2.9.1
  • In my main.go, I have added the following import:
    _ "github.com/dgrijalva/jwt-go"
  • In my go.mod, I have copied the dependency from nakama’s go.mod as follows:
    github.com/dgrijalva/jwt-go v3.2.1-0.20190620180102-5e25c22bd5d6+incompatible
  • Ran the following commands:
    go mod tidy
    env GO111MODULE=on go mod vendor
  • Tried to build the plugin as
    go build -buildmode=plugin -trimpath -mod=vendor -o <fullpath>/wb.so
  • Start the server

Still get the same error. Can you spot anything that I might have missed?

@gupta-vaibhav Looks like you’re on the right track :thinking:. What version of the Go toolchain do you see in the logs for the server startup? Do you build the game server from source? And what version of the Go toolchain do you currently build the shared object for?

@novabyte yes I do build from source. As a precaution, I rebuilt the nakama binary (just to ensure that they nakama and plugin are both built with the same version of go toolchain, i.e. 1.13.7). The command I use for the same is:
env CGO_ENABLED=1 go build -trimpath -mod=vendor

Here are the full logs from server startup as well:
{"level":"info","ts":"2020-07-18T03:40:55.841Z","msg":"Successfully loaded config file","path":"/home/ec2-user/goprojects/wb/go/config/conf.yml"} {"level":"warn","ts":"2020-07-18T03:40:55.841Z","msg":"WARNING: insecure default parameter value, change this for production!","param":"console.signing_key"} {"level":"info","ts":"2020-07-18T03:40:55.841Z","msg":"Nakama starting"} {"level":"info","ts":"2020-07-18T03:40:55.841Z","msg":"Node","name":"wb","version":"2.0.0+dev","runtime":"go1.13.7","cpu":1,"proc":1} {"level":"info","ts":"2020-07-18T03:40:55.841Z","msg":"Data directory","path":"./data/"} {"level":"info","ts":"2020-07-18T03:40:55.841Z","msg":"Database connections","dsns":["root@localhost:26257/wb?sslcert=./config/.cockroach-certs/client.root.crt&sslkey=./config/.cockroach-certs/client.root.key&sslmode=require"]} {"level":"debug","ts":"2020-07-18T03:40:55.841Z","msg":"Complete database connection URL","raw_url":"postgresql://root@localhost:26257/wb?sslcert=./config/.cockroach-certs/client.root.crt&sslkey=./config/.cockroach-certs/client.root.key&sslmode=require"} {"level":"info","ts":"2020-07-18T03:40:55.868Z","msg":"Database information","version":"CockroachDB CCL v19.2.6 (x86_64-unknown-linux-gnu, built 2020/04/06 18:05:31, go1.12.12)"} {"level":"info","ts":"2020-07-18T03:40:55.873Z","msg":"Initializing leaderboard rank cache"} {"level":"info","ts":"2020-07-18T03:40:55.873Z","msg":"Leaderboard rank cache initialization completed successfully","cached":[],"skipped":[]} {"level":"info","ts":"2020-07-18T03:40:55.873Z","msg":"Initialising runtime","path":"./modules/"} {"level":"info","ts":"2020-07-18T03:40:55.873Z","msg":"Initialising runtime event queue processor"} {"level":"info","ts":"2020-07-18T03:40:55.874Z","msg":"Runtime event queue processor started","size":65536,"workers":8} {"level":"info","ts":"2020-07-18T03:40:55.874Z","msg":"Initialising Go runtime provider","path":"./modules/"} {"level":"error","ts":"2020-07-18T03:40:55.886Z","msg":"Could not open Go module","path":"modules/wb.so","error":"plugin.Open(\"modules/wb\"): plugin was built with a different version of package github.com/dgrijalva/jwt-go","stacktrace":"github.com/heroiclabs/nakama/v2/server.openGoModule\n\tgithub.com/heroiclabs/nakama/v2@/server/runtime_go.go:1973\ngithub.com/heroiclabs/nakama/v2/server.NewRuntimeProviderGo\n\tgithub.com/heroiclabs/nakama/v2@/server/runtime_go.go:1887\ngithub.com/heroiclabs/nakama/v2/server.NewRuntime\n\tgithub.com/heroiclabs/nakama/v2@/server/runtime.go:453\nmain.main\n\tgithub.com/heroiclabs/nakama/v2@/main.go:130\nruntime.main\n\truntime/proc.go:203"} {"level":"error","ts":"2020-07-18T03:40:55.886Z","msg":"Error initialising Go runtime provider","error":"plugin.Open(\"modules/wb\"): plugin was built with a different version of package github.com/dgrijalva/jwt-go","stacktrace":"github.com/heroiclabs/nakama/v2/server.NewRuntime\n\tgithub.com/heroiclabs/nakama/v2@/server/runtime.go:455\nmain.main\n\tgithub.com/heroiclabs/nakama/v2@/main.go:130\nruntime.main\n\truntime/proc.go:203"} {"level":"fatal","ts":"2020-07-18T03:40:55.886Z","msg":"Failed initializing runtime modules","error":"plugin.Open(\"modules/wb\"): plugin was built with a different version of package github.com/dgrijalva/jwt-go","stacktrace":"main.main\n\tgithub.com/heroiclabs/nakama/v2@/main.go:132\nruntime.main\n\truntime/proc.go:203"}

@gupta-vaibhav I’m not sure why this doesn’t work at the moment. You’ve stepped through all the steps needed to synchronise the deps tree with Nakama when using libraries that Nakama already depends on. Please can you share the go.mod file and the vendor/modules.txt file with me?

My sincere apologies for the delay in responding. I was in a bit of a rush and had taken an alternate approach. However, I do realise that this issue needs to be resolved, as I am stuck with the same issue, but with another dependency now. This time I am trying to upgrade to nakama version 2.12.0.

This time the package causing the issue this time is:
golang.org/x/sys/unix

However this time I also noticed that I tried to follow the steps that you had suggested, running go mod vendor updates the version of this dependency from:

– as copied from nakama
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd
– to the latest
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d

Apologies if this is stupid question, but how do I prevent go from doing it. I am getting the same error:

{"level":"error","ts":"2020-08-01T14:42:35.914Z","caller":"server/runtime_go.go:2060","msg":"Could not open Go module","path":"modules/wb.so","error":"plugin.Open(\"modules/wb\"): plugin was built with a different version of package golang.org/x/sys/unix"}

Adding to it…

Since I am building nakama from source, I tried upgrading nakama dependencies where they were lower than the dependencies required by my project (please warn me if I should refrain from doing so at all costs)

I did manage to get past a few dependencies that were causing this issue, and got quite hopeful. However, now I am stuck at the following.

– From nakama’s vendor/module.txt
google.golang.org/grpc v1.28.1
– From my code’s vendor/module.txt
google.golang.org/grpc v1.28.1

Despite these two being exactly identical, I still get the same error:
{"level":"error","ts":"2020-08-01T15:15:44.957Z","caller":"server/runtime_go.go:2060","msg":"Could not open Go module","path":"modules/wb.so","error":"plugin.Open(\"modules/wb\"): plugin was built with a different version of package google.golang.org/grpc/backoff"}

Any other suggestions that you may have?

EDIT have added all the relevant go.mod and modules.txt files as once requested by @novabyte
plugin go.mod.txt (316 Bytes) plugin vendor_module.txt (2.5 KB) modified_nakama_go.mod.txt (3.4 KB) modified_nakama_vendor_module.txt (9.2 KB)

@gupta-vaibhav We spoke on Gitter. Were you able to resolve this yourself? I think the root cause was that you build the game server from source rather than use our official releases. This adds complexity to the plugin dependency compatibility.