Importing `aws-sdk` services through Rollupjs and Babel throws fatal error

Runtime Framework
,
1

Looking to get some information on using the TS/JS Rollups/Babel pipeline with aws-sdk.

TL;DR:

  1. Followed Rollup/Babel setup here: TypeScript Setup - Heroic Labs Documentation
  2. Rollups/Babel work when bundling some Node Modules (tested on lodash)
  3. Does not work with aws-sdk

Setup Details:

  1. Versions: Nakama 3.8, Docker, Docker-Compose
  2. Server Framework Runtime language: TS/JS
  3. Using Rollup/Babel. Default configurations based on Nakama documentation.

Details

I want to use the AWS SDK to call into S3 buckets for content and asset management but I am running into issue importing the AWS sdk in the server’s main.ts entry point. I tried this with both the AWS version 2 and 3 SDK. Each time main.ts is transpiled, the resulting index.js file contains either import statements or require statements depending on babel’s configuration.

First 30 lines of index.js

import { parse } from 'url';
import { Buffer as Buffer$1 } from 'buffer';
import { request, Agent } from 'http';
import { readFile, lstatSync, readFileSync, createReadStream } from 'fs';
import { homedir, platform, release } from 'os';
import { join, sep } from 'path';
import { createHmac, createHash } from 'crypto';
import { Agent as Agent$1, request as request$1 } from 'https';
import { Readable, Writable } from 'stream';
import 'http2';
import { versions, env } from 'process';
import { exec } from 'child_process';

function _typeof(obj) {
  "@babel/helpers - typeof";

  if (typeof Symbol === "function" && typeof Symbol.iterator === "symbol") {
    _typeof = function (obj) {
      return typeof obj;
    };
  } else {
    _typeof = function (obj) {
      return obj && typeof Symbol === "function" && obj.constructor === Symbol && obj !== Symbol.prototype ? "symbol" : typeof obj;
    };
  }

  return _typeof(obj);
}

... etc

Based on the import statements in the above snippit, it looks like the Rollup is appending Node’s built-in modules (Node.js Built-in Modules) to the top of the index.js file. (Note, this is not happening with other node modules).

Output logs when running docker-compose up:

backend        | {"level":"info","ts":"2021-11-30T01:55:41.769Z","caller":"server/config.go:86","msg":"Successfully loaded config file","path":"/nakama/data/my-config.yml"}
backend        | {"level":"warn","ts":"2021-11-30T01:55:41.770Z","caller":"server/config.go:302","msg":"WARNING: insecure default parameter value, change this for production!","param":"console.signing_key"}
backend        | {"level":"warn","ts":"2021-11-30T01:55:41.770Z","caller":"server/config.go:314","msg":"WARNING: insecure default parameter value, change this for production!","param":"session.refresh_encryption_key"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.771Z","caller":"main.go:104","msg":"Nakama starting"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.771Z","caller":"main.go:105","msg":"Node","name":"nakama1","version":"3.4.0+9a295bde","runtime":"go1.16.5","cpu":8,"proc":8}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.771Z","caller":"main.go:106","msg":"Data directory","path":"./data/"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.771Z","caller":"main.go:117","msg":"Database connections","dsns":["root@cockroachdb:26257"]}
backend        | {"level":"debug","ts":"2021-11-30T01:55:41.771Z","caller":"main.go:259","msg":"Complete database connection URL","raw_url":"postgres://root@cockroachdb:26257/nakama?sslmode=prefer"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.777Z","caller":"main.go:120","msg":"Database information","version":"CockroachDB CCL v20.2.17 (x86_64-unknown-linux-gnu, built 2021/10/11 17:17:07, go1.13.14)"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.784Z","caller":"server/metrics.go:136","msg":"Starting Prometheus server for metrics requests","port":9100}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.803Z","caller":"server/leaderboard_rank_cache.go:116","msg":"Initializing leaderboard rank cache"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.803Z","caller":"server/leaderboard_rank_cache.go:214","msg":"Leaderboard rank cache initialization completed successfully","cached":[],"skipped":[]}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.804Z","caller":"server/runtime.go:569","msg":"Initialising runtime","path":"data/modules"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.804Z","caller":"server/runtime.go:576","msg":"Initialising runtime event queue processor"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.805Z","caller":"server/runtime.go:578","msg":"Runtime event queue processor started","size":65536,"workers":8}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.805Z","caller":"server/runtime_go.go:2396","msg":"Initialising Go runtime provider","path":"data/modules"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.806Z","caller":"server/runtime_go.go:2420","msg":"Go runtime modules loaded"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.806Z","caller":"server/runtime_lua.go:110","msg":"Initialising Lua runtime provider","path":"data/modules"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.807Z","caller":"server/runtime_lua.go:1080","msg":"Lua runtime modules loaded"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.807Z","caller":"server/runtime_lua.go:1083","msg":"Allocating minimum runtime pool","count":16}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.807Z","caller":"server/runtime_lua.go:1091","msg":"Allocated minimum runtime pool"}
backend        | {"level":"info","ts":"2021-11-30T01:55:41.807Z","caller":"server/runtime_javascript.go:550","msg":"Initialising JavaScript runtime provider","path":"data/modules","entrypoint":"build/index.js"}
backend        | {"level":"error","ts":"2021-11-30T01:55:41.906Z","caller":"server/runtime_javascript.go:1550","msg":"Could not compile JavaScript module","module":"index.js","error":"SyntaxError: index.js: Line 1:1 Unexpected reserved word (and 18 more errors)"}
backend        | {"level":"fatal","ts":"2021-11-30T01:55:41.906Z","caller":"server/runtime_javascript.go:554","msg":"Failed to load JavaScript files","error":"SyntaxError: index.js: Line 1:1 Unexpected reserved word (and 18 more errors)"}

Summary

I’m opening to any suggestions. Ultimately I need to be able to connect to specific amazon services, like S3, Lambda, etc. I’m also open to converting my backend to one of the other languages (Go or Lua) if there is no way to accomplish it with TS/JS. Hoping that there is a path here though.

2

I’ve also tried this and failed constantly, eventually realized that using Golang for this was the better option, I have a example of how i did it in golang here:

Oh… and dont be scared if you don’t know golang yet, I’ve got the hang of it quite quickly!

1 Like
3

Hi Linjoe, thanks for the insights. I’ll look at your post on getting the aws-sdk working with golang. I was hoping that the aws-sdk with TS/JS was just a simple configuration oversight, but maybe it is a deeper issue. Would appreciate if one of the Nakama devs could comment on this. @mofirouz any thoughts?

4

I’ve not actually used the aws-sdk in TS/JS - but from the log lines it looks like there are incompatibilities between our JavaScript virtual machine and what’s needed in the AWS SDK. I’d recommend using the SDK via Go as what @linjoe mentioned.

Only thing different is that I’d not use the env-var/ or .aws/credentials way to instantiate the AWS SDK - instead pass the access key/secret as a Nakama Runtime Environment and set up the AWS Client via code:

sess, err := session.NewSession(&aws.Config{
    Region:      aws.String("us-west-2"),
    Credentials: credentials.NewStaticCredentials("AKID", "SECRET_KEY", "TOKEN"),
})
1 Like
5

Right, that makes sense. Good to know that this a limitation of JS/TS and not a config issue. We’ll convert over our backend to Go and do the AWS integration that way