Error `Unexpected end of data` when parsing a JWT #544

Muhammad-Altabba · 2021-05-18T14:18:02Z

Muhammad-Altabba
May 18, 2021

Bug severity
5

Describe the bug
Not able to parse a simple JWT.

To Reproduce
Steps to reproduce the behaviour:

Try to Handle a Message that is a self-signed JWT:

    const message: IMessage = await agent.handleMessage({
      raw: verifiableCredential.proof.jwt,
      save: false,
    });

eyJhbGciOiJFUzI1NksiLCJ0eXAiOiJKV1QifQ.eyJ2YyI6eyJjcmVkZW50aWFsU3ViamVjdCI6eyJpZGVudGl0eUFsaWFzIjoibG9jYWxob3N0OmJjZjcxMjA0LWQzMDEtNGU3OC1iNWQ3LTViNmMyMWFjNmUwMiJ9LCJAY29udGV4dCI6WyJodHRwczovL3d3dy53My5vcmcvMjAxOC9jcmVkZW50aWFscy92MSJdLCJ0eXBlIjpbIlZlcmlmaWFibGVDcmVkZW50aWFsIl19LCJuYmYiOjE2MjEzNDE3OTgsImlzcyI6ImRpZDp3ZWI6bG9jYWxob3N0OmJjZjcxMjA0LWQzMDEtNGU3OC1iNWQ3LTViNmMyMWFjNmUwMiJ9.3045022100ba25a2e0661ea63a659e19e54b36cd32f307e65fe7a44beaddfc35ea07a602670220325075c71d34d52fbc1f4abc53c655cae3748e2300b3bb990db323cfb18d1aee

Observed behaviour
Getting an exception: 'Unsupported message type'

However, when debegging the following error is the actual one:
'Unexpected end of data'

stack:'SyntaxError: Unexpected end of data
    at decode (/home/muhammad/Repositories/interop-microservice/node_modules/multibase/src/rfc4648.js:52:11)
    at Object.decode (/home/muhammad/Repositories/interop-microservice/node_modules/multibase/src/rfc4648.js:118:14)
    at Base.decode (/home/muhammad/Repositories/interop-microservice/node_modules/multibase/src/base.js:46:23)
    at Object.u8a [as fromString] (/home/muhammad/Repositories/interop-microservice/node_modules/uint8arrays/from-string.js:46…ad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:11:30)
    at VerifierAlgorithm (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:54:34)
    at verifyJWSDecoded (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:239:38)
    at /home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:304:44
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
```
   
**Expected behaviour**
To be able to parse the JWT and save the message in the local DB.

**Additional context**
Add any other context about the problem here.

**Versions (please complete the following information):**
 - Veramo: 1.2.0
 - Node Version: 12.13.0

Muhammad-Altabba · 2021-05-19T05:56:47Z

Muhammad-Altabba
May 19, 2021
Author

Hello @mirceanis,
I have an update:

I think the issue is because the signature format I provided is wrong. So, I updated the code at our custom KeyManagementSystem class to have the following signJWT:

async signJWT({ key, data }: { key: IKey; data: string }): Promise<string> {
    let signatureString;
    try {
      l.debug('KeyManagementSystem -> signJWT(...)');

      const response = await axios.post(
        NETID_KMS_ENDPOINT + NETID_KMS_SIGN_PATH,
        {
          encPrivKey: key.privateKeyHex,
          value: data,
        }
      );

      signatureString = response.data.signature;
      l.debug(
        signatureString,
        'KeyManagementSystem -> signJWT: signatureString'
      );

      const signature = new Signature(signatureString, 'hex');
      const r = leftpad(signature.r.toString('hex'), 64);
      const s = leftpad(signature.s.toString('hex'), 64);

      return toJose(
        {
          r,
          s,
          recoveryParam: 1, // What is the correct value here? Not passing a value will cause the exception 'Signer did not return a recoveryParam'
        }
        // recoverable // Is it the best to not pass this parameter?
      );
    } catch (error) {
      l.error(signatureString);
      l.error(error);
      throw error;
    }
  }

The JWT generated is eyJhbGciOiJFUzI1NksiLCJ0eXAiOiJKV1QifQ.eyJ2YyI6eyJjcmVkZW50aWFsU3ViamVjdCI6eyJpZGVudGl0eUFsaWFzIjoibG9jYWxob3N0OjBlZTIzMWY0LTZhM2MtNDAxZC04NDg3LTZkNTFiMGU3ZTVmMSJ9LCJAY29udGV4dCI6WyJodHRwczovL3d3dy53My5vcmcvMjAxOC9jcmVkZW50aWFscy92MSJdLCJ0eXBlIjpbIlZlcmlmaWFibGVDcmVkZW50aWFsIl19LCJuYmYiOjE2MjE0MDMzMjcsImlzcyI6ImRpZDp3ZWI6bG9jYWxob3N0OjBlZTIzMWY0LTZhM2MtNDAxZC04NDg3LTZkNTFiMGU3ZTVmMSJ9.MkDSFVXXAata2FH4Kq6Mm3EZRZ_lrqAHRpZiKhsJ5KeZUVaze-ERfSiz0sQA832t36xnG5Q88HMJoOox-svNdQ

So, now, the error is:

Error: Unknown point format
  at ShortCurve.decodePoint (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/curve/base.js:291:9)
    at KeyPair._importPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:98:28)
    at new KeyPair (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:16:10)
    at Function.fromPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:24:10)
    at EC.keyFromPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/index.js:52:18)
    at find (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:65:24)
    at Array.find (<anonymous>)
    at VerifierAlgorithm (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:62:51)
    at verifyJWSDecoded (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:239:38)
    at /home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:304:44
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

What do you suggest?

Many thanks,

0 replies

Muhammad-Altabba · 2021-05-20T16:12:51Z

Muhammad-Altabba
May 20, 2021
Author

Hello @mirceanis
Do I need to provide more information? Or do I need to open another issue regarding the latest Error and close this one?
Many thanks,

0 replies

mirceanis · 2021-05-21T12:41:25Z

mirceanis
May 21, 2021
Maintainer

can you post an example of response.data.signature ?

From a distance, I would assume that the signature is a hex string representing 64 bytes of data.
If that is the case, you can try to directly return the base64url encoding of those bytes.
Something like:

return bytesToBase64url(hexToBytes(signatureString))

I don't know how your KeyManagementSystem backend encodes signatures so it is very hard to debug this remotely.

If you intend to use ES256K algorithm for JWT signing, then you can call toJose({r,s}, false)
You only need a recovery param if you are using ES256K-R
Which method call throws Signer did not return a recoveryParam for you?

L.E.: I marked this as wontfix since it is not a veramo issue, but we will try to do our best to help you debug this.

0 replies

Muhammad-Altabba · 2021-05-25T11:20:45Z

Muhammad-Altabba
May 25, 2021
Author

Thank you @mirceanis ,
Bellow is the details of the issue when using bytesToBase64url(hexToBytes(signatureString)):

signatureString:
3045022055cbbd468a99727ce019fce7e786d1dd32b2f5bb818fb3bad4186ff50c2346ba022100f4b2fc5f0a3cd5b3b7f2cbd4fc446006ff8457ed57c1665f0446c9fd90e8fa99

util.hexToBytes(signatureString):
Uint8Array(71)
[48, 69, 2, 32, 85, 203, 189, 70, 138, 153, 114, 124, 224, 25, 252, 231, 231, 134, 209, 221, 50, 178, 245, 187, 129, 143, 179, 186, 212, 24, 111, 245, 12, 35, 70, 186, 2, 33, 0, 244, 178, 252, 95, 10, 60, 213, 179, 183, 242, 203, 212, 252, 68, 96, 6, 255, 132, 87, 237, 87, 193, 102, 95, 4, 70, 201, 253, 144, 232, 250, 153]

util.bytesToBase64url(util.hexToBytes(signatureString)):
MEUCIFXLvUaKmXJ84Bn85-eG0d0ysvW7gY-zutQYb_UMI0a6AiEA9LL8Xwo81bO38svU_ERgBv-EV-1XwWZfBEbJ_ZDo-pk

Exception has occurred: Error: wrong signature length
  at toSignatureObject (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:13:5)
    at VerifierAlgorithm (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:54:34)
    at verifyJWSDecoded (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:239:38)
    at /home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:304:44
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

And web using util.toJose({r,s},false); the exception is:

Exception has occurred: Error: Unknown point format
  at ShortCurve.decodePoint (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/curve/base.js:291:9)
    at KeyPair._importPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:98:28)
    at new KeyPair (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:16:10)
    at Function.fromPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/key.js:24:10)
    at EC.keyFromPublic (/home/muhammad/Repositories/interop-microservice/node_modules/elliptic/lib/elliptic/ec/index.js:52:18)
    at find (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:65:24)
    at Array.find (<anonymous>)
    at VerifierAlgorithm (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/VerifierAlgorithm.ts:62:51)
    at verifyJWSDecoded (/home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:239:38)
    at /home/muhammad/Repositories/interop-microservice/node_modules/did-jwt/src/JWT.ts:304:44
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

What do you suggest?

Many thanks,

0 replies

mirceanis · 2021-05-25T15:57:28Z

mirceanis
May 25, 2021
Maintainer

It looks like the signature is 71 bytes, not 64, which suggests it's a DER encoding.
Your original approach with new Signature(signatureString, 'hex') should be right.
If it is then it means that perhaps it was only an issue with recoveryParam.
If you're using ES256K JWT algorithm, then something like this should work:

const signature = new Signature(signatureString, 'hex');
const r = leftpad(signature.r.toString('hex'), 64);
const s = leftpad(signature.s.toString('hex'), 64);
return toJose({r, s}, false);

0 replies

Muhammad-Altabba · 2021-05-26T09:21:00Z

Muhammad-Altabba
May 26, 2021
Author

Thanks @mirceanis,
Unfortunately, the proposed code is generating the same Unknown point format that is mentioned early.

Note: To ease debugging I am enabling just JwtMessageHandler and commenting out the other (is this ok just at Debugging time):

export const agent = createAgent<
  IDIDManager & IKeyManager & IDataStore & IDataStoreORM & IResolver
>({
  plugins: [
    new KeyManager({
      store: new KeyStore(dbConnection as any),
      kms: {
        local: new KeyManagementSystem(),
      },
    }),
    new DIDManager({
      store: new DIDStore(dbConnection as any),
      defaultProvider: 'did:web',
      providers: {
        'did:web': new WebDIDProvider({
          defaultKms: 'local',
        }),
      },
    }),
    new DIDResolverPlugin({
      resolver: new Resolver({
        web: webDidResolver().web as any,
        //resolve some other DID methods using the centralized `uniresolver.io` service
        key: uniResolver as any,
        elem: uniResolver as any,
      }),
    }),
    new CredentialIssuer(),
    new DataStore(dbConnection as any),
    new DataStoreORM(dbConnection as any),
    new MessageHandler({
      messageHandlers: [
        // new SdrMessageHandler(),
        // new DIDCommMessageHandler(),
        new JwtMessageHandler(),
        // new W3cMessageHandler(),
      ],
    }),
    new DIDComm(),
    new SelectiveDisclosure(),
  ],
});

Your suggestions are very valuable. Please, keep it till we succeed...
What do you suggest now?
Many thanks,

0 replies

mirceanis · 2021-05-26T11:49:21Z

mirceanis
May 26, 2021
Maintainer

unfortunately, I'm out of ideas.
I can't suggest anything more without knowing how your backend is producing the signature.

It doesn't look like it'a a problem with veramo, but a problem with the signature encoding.
Without knowing more about what you are working with, there's nothing that can be done from our end :(

0 replies

mirceanis · 2021-05-27T13:04:22Z

mirceanis
May 27, 2021
Maintainer

Note: To ease debugging I am enabling just JwtMessageHandler and commenting out the other

Yes, that should be ok.

Also, to help with debugging in VSCode make sure to enable sourcemaps and to include node_modules:
Here's an example launch configuration used for running a Jest test file called custom.kms.test.ts

  {
    type: 'node',
    request: 'launch',
    name: 'Jest CustomKMS test',
    program: '${workspaceFolder}/node_modules/.bin/jest',
    args: ['--runInBand', '--coverage=false', '--config=jest.json', 'custom.kms.test'],
    console: 'integratedTerminal',
    internalConsoleOptions: 'neverOpen',
    disableOptimisticBPs: true,
    windows: {
      program: '${workspaceFolder}/node_modules/jest/bin/jest',
    },
    sourceMaps: true,
    resolveSourceMapLocations: ['${workspaceFolder}/**', '**/node_modules/**'],
  }

I hope it helps

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error `Unexpected end of data` when parsing a JWT #544

{{title}}

Replies: 8 comments

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Error Unexpected end of data when parsing a JWT #544

Muhammad-Altabba May 18, 2021

Replies: 8 comments

Muhammad-Altabba May 19, 2021 Author

Muhammad-Altabba May 20, 2021 Author

mirceanis May 21, 2021 Maintainer

Muhammad-Altabba May 25, 2021 Author

mirceanis May 25, 2021 Maintainer

Muhammad-Altabba May 26, 2021 Author

mirceanis May 26, 2021 Maintainer

mirceanis May 27, 2021 Maintainer

Error `Unexpected end of data` when parsing a JWT #544

Muhammad-Altabba
May 18, 2021

Muhammad-Altabba
May 19, 2021
Author

Muhammad-Altabba
May 20, 2021
Author

mirceanis
May 21, 2021
Maintainer

Muhammad-Altabba
May 25, 2021
Author

mirceanis
May 25, 2021
Maintainer

Muhammad-Altabba
May 26, 2021
Author

mirceanis
May 26, 2021
Maintainer

mirceanis
May 27, 2021
Maintainer