Request: Support for non-node runtimes

[saklani]

Currently, arweave.crypto.sign() returns different results in runtimes other than node like bun and deno.

Minimum Reproducible Example

import Arweave from "arweave";
import { readFileSync }  from "node:fs";
import { Buffer } from "node:buffer";

async function test() { 
  const arweave = Arweave.init({});
  const keyfile = JSON.parse(readFileSync("./keyfile.json").toString());
  console.log(keyfile);
  const sign = await arweave.crypto.sign(keyfile, Buffer('test'));
  const result = [];
  for (const i of sign.values()) {
    result.push(i);
  }
  console.log(result);
}

test();

Node v21.6.0 (changes every time)

[ 86, 248,  33, 216, 255, 242, 133, 113,  84, 150,  11,  97,
  193,  42,   8,  43, 229, 195, 174, 237, 200, 192,  36, 240,
   94, 164, 130, 234, 101, 249, 191, 106, 177, 206,  92, 148,
   27,  40, 162,  81, 113, 183,  65,  21,   0, 191,  91, 248,
   13, 101,  86, ...]

Bun v1.0.26 (consistent)

[ 7, 146, 211, 91, 31, 2, 185, 167, 35, 48, 68, 168, 183, 8, 156, 90, 205, 202, 40,
  58, 105, 252, 250, 146, 35, 42, 234, 244, 77, 72, 146, 226, 233, 178, 234, 108,
  229, 49, 93, 13, 73, 123, 131, 75, 155, 118, 207, 109, 250, 5, 195, 236, 87, 138,
  213, 43, 80, ...]

Deno v1.40.4 (consistent)

[ 38, 201,  37, 195, 237,  40, 113,  32, 210, 138,  14,  10,
  157, 103,  61, 247,  10, 252,  77,  47,  58, 178, 107, 167,
  209, 180, 194, 113,  94, 150,  36,  97,  19,  44, 112,   7,
  231, 140, 131,  57, 205, 108,  40, 214,  64,  84, 194,  96,
   23,  13, 106, ...]

A lot of downstream libraries seem to be using the crypto driver from this library
Including warp-contracts/warp-arbundles.
This is creating a bit problem since trying to sign a bundled data item seems to fail on non-node runtimes.

Any help with regards to this issue will be appreciated.

[rosmcmahon]

hi 👋 there's a couple of things i notice here, but the main thing is that this function doesn't accept a node:buffer as an argument. it's a Uint8Array.

perhaps you could test using something like
const sign = await Arweave.crypto.sign(key, new Uint8Array([1,2,3,4]) )

[rosmcmahon]

ah, if you wish the result to be the same each time, you need to take away the randomness added by default by salting. you can do this like so:
const sign = await Arweave.crypto.sign(key, new Uint8Array([1,2,3,4]), {saltLength: 0} )

i'm not a Deno or Bun guy, but if Deno & Bun are producing consistent results, perhaps they are using a default saltLength = 0.
this is what nodejs uses:

crypto.constants.RSA_PSS_SALTLEN_MAX_SIGN (default) sets it to the maximum permissible value.

[saklani]

Hey! Thanks for the help.

It seems that crypto.constants.RSA_PSS_SALTLEN_MAX_SIGN is undefined for bun.
For deno and node it is -2.
But the output don't change if I explicitly set the saltLength.

Updated Example

import Arweave from "arweave";
import { readFileSync }  from "node:fs";
import crypto from "node:crypto";

async function test() { 
  const arweave = Arweave.init({});
  const keyfile = JSON.parse(readFileSync("./keyfile.json").toString());
  
  const sign = await arweave.crypto.sign(
    keyfile, 
    new Uint8Array([1, 2, 3, 4]),
    { saltLength: -2 }
  ); // Explicit for bun
  const result = [];
  for (const i of sign.values()) {
    result.push(i);
  }
  console.log(result);
}

test();

I'll try to narrow it down more, seems a bun/deno issue.

[rosmcmahon]

I'm reopening this as I'd like to further investigate patching this. Supporting bun/deno would be nice!

[saklani]

Cool I'll link the associated issues for tracking:
oven-sh/bun#8831
oven-sh/bun#8832

[kay-is]

@rosmcmahon

I got it working by using the ./web/index.js in Bun.

Bun supports Web Crypto, which doesn't seem to have the compat issues from their node:crypto module.

It seems these are the exports that Bun can use:

"exports": {
    "bun": "./index.js",
    "worker": "./index.js",
    "node": "./index.js",
    "require": "./index.js", // if importer is CommonJS
    "import": "./index.mjs", // if importer is ES module
    "default": "./index.js",
}

So, in theory it should be possible to tell Bun to use the web/index.js, which, in turn uses Web Crypto.

---

Deno also supports Web Crypto, so I would assume this could also be a solution for Deno.

[rosmcmahon]

Fantastic @kay-is ! I have a plan to switch everything over to webcrypto anyhow

[Nehoko]

Hey @kay-is,
what is web/index.js exactly? Web Bundle?

[kay-is]

Hey @kay-is, what is web/index.js exactly? Web Bundle?

Yes.

The following defines a special browser export, but Bun ignores it because it isn't a browser.

arweave-js/package.json

Line 7 in 29901f4

"browser": "./web/index.js",

When I changed the following line to use ./web/index.js, it also worked in Bun.

arweave-js/package.json

Line 5 in 29901f4

"main": "./node/index.js",

[Nehoko]

Thanks for instant reply, @kay-is!

The following defines a special browser export, but Bun ignores it because it isn't a browser.

damn, I'm using Deno v2.1.5. Not sure how to configure it properly. Should I import https://unpkg.com/arweave/bundles/web.bundle.min.js instead of npm package? Any ideas?

Thank you for the assistance.

Upd.

1. Deno allows to import strictly from web without changes in library package.json.
2. When I try to import from web, it throws an error:

Uncaught (in promise) TypeError: Arweave.init is not a function
const arweave = Arweave.init({

3. When I try to edit package.json as @kay-is did, it throws the same error.
4. Same with constructor.

[kay-is]

Not sure, but have you tried:

import Arweave from "https://unpkg.com/[email protected]/web/index.js"

That file defines a global Arweave object, an esm default export, and a named Arweave export.

[Nehoko]

That file defines a global Arweave object, an esm default export, and a named Arweave export.

it says:

error: Uncaught SyntaxError: The requested module 'https://unpkg.com/[email protected]/web/index.js' does not provide an export named 'default'
import Arweave from "https://unpkg.com/[email protected]/web/index.js"

I guess it's not just that simple with Deno for some reason...

[Nehoko]

I'm thinking to extract our ArweaveService into separated micro service and use it with NodeJS while there is no Deno support.

I would also like to see this library @ jsr.io as Deno/Bun library in the future.
Thank you for your support anyway.

[rosmcmahon]

Just wanted to mention that this issue with bun should be fixed in a later release.

If you want to open a separate issue for demo we can take a look at that @Nehoko

[Nehoko]

Just wanted to mention that this issue with bun should be fixed in a later release.

If you want to open a separate issue for demo we can take a look at that @Nehoko

There might be an issue not with your library at the first place. I'm trying to communicate with arlocal by your library: maybe there is something wrong with it.

I'll investigate and let you know about the results (I think you might be interested).

Upd.
It is Deno issue: Node implementation works just fine in combination with arweave-js + arlocal. I'll create separate Request.