StorageRead error with cockroachdb after upgrade to 3.17.0

Runtime Framework
1

Hi,

We are getting errors when doing reads using the storage read api.

This is happening after upgrading to 3.17.0 and only with cockroachdb. Postgres is working fine. We have run the migrate up on the db and tested with a clean db.

The error we are getting is: cannot convert [–the collection name–] to Text. The error is happening on line 479 of core_storage.go.

	err := ExecuteRetryablePgx(ctx, db, func(conn *pgx.Conn) error {
		rows, _ := conn.Query(ctx, query, params...)
		defer rows.Close()
		funcObjects := &api.StorageObjects{Objects: make([]*api.StorageObject, 0, len(objectIDs))}
**479	for rows.Next() {
			o := &api.StorageObject{CreateTime: &timestamppb.Timestamp{}, UpdateTime: &timestamppb.Timestamp{}}
			var createTime pgtype.Timestamptz
			var updateTime pgtype.Timestamptz

I see the query has change so I suspect it’s comming from there. Inspecting the data I see that the arrays of parameters are string, string and uuid.UUID all of length 1.

Anyone else facing this issue?

Thanks
Andre

2

Hi Andre; Can you please paste the entire error stack please? If sensitive info, feel free to email support@heroiclabs.com - thank you!

3

Hello @AndreParodi,

Thank you for reporting it. It seems to be an issue in how the driver serializes the arrays passed to it. I will look more into it and get back to you.

4

Hi, I’m getting the same problem after updating to Nakama 3.19.0. Also when I open the Nakama Console and try to look at a storage entry, the web browser says " An error has occurred!
An error occurred while reading storage object. Please refresh the page to try again."

5

Hello @Spuddy,

Which version of Cockroach are you running? Is it the same that is defined in the docker-compose.yaml file?

You could check by running psql postgres://root@0.0.0.0:26257/nakama -c ' SELECT version();' on Cockroach’s container.

6

I don’t have much knowledge of Linux so I couldn’t get the psql thing working to show Cockroach’s version but when I build its container, the following is output:

nakama-cockroachdb-1  | CockroachDB node starting at 2023-11-16 12:24:42.818315296 +0000 UTC (took 0.2s)
nakama-cockroachdb-1  | build:               CCL v20.2.19 @ 2022/02/09 18:48:48 (go1.13.14)

Does that tell you anything ? My docker-compose.yml file was copied from god knows where ages ago. The line that seems to select the cockroach version is:

image: cockroachdb/cockroach:latest-v20.2

Do I need to change that line to a newer version ?

7

It seems your are using a older version of the database. As you can see here in the docker-compose file of version 3.19.0, it uses CockroachDB v23.1.

8

Right, I didn’t realize I had to look at the docker-compose file in the server source code and merge it in to mine. I’ve changed my docker-compose file to this:

image: cockroachdb/cockroach:latest-v23.1

However, when I do docker compose up, I get the following error:

nakama-cockroachdb-1  | * ERROR: ERROR: server startup failed: failed to start server: failed to create engines: store has no min-version file; this can happen if the store was created by an old CockroachDB version that is no longer supported: dirname="/var/lib/cockroach": pebble: database already exists and is not pristine

Is there an easy way to fix this ?

9

I’ve managed to successfully update my existing CockroachDB database files from v20.2 that was used in Nakama 3.16 to CockroachDB v23.1 which is used from Nakama 3.17 to 3.19 by repeatedly changing the version number of cockroach in my docker-compose file from 20.2 to 21.1 to 21.2 to 22.1 to 22.2 and finally to 23.1, doing docker compose up each time I changed a version. Is there a less tedious way to do this in the future ? I found each production release version of CockroachDB here: Releases

10

If you care about that data stored in the volume, I don’t think there is another way. Otherwise, you could just destroy the volume and recreate it on the last version.