UnhandledPromiseRejectionWarning due to HTTP request character limit violation

[mwagenma-maersk]

We're using the Detect action in a GitHub CI workflow to scan a multi-module Java Maven project which also includes an Angular webapp. After the scan, which terminated successfully, the workflow log contains an error stacktrace stating that an invalid HTTP request was sent. A notice annotation is added to the workflow run about policy violations, but I can not find the check results anywhere in order to take a look at the violations. Here is an excerpt from the workflow log:

`detect executed successfully.
detect executed in RAPID mode. Beginning reporting...
Attempting to upload Rapid Scan JSON...
Initiating authentication request to Black Duck...
Total size of all the files uploaded is 193266 bytes
Finished uploading artifact Rapid Scan JSON. Reported size is 193266 bytes. There were 0 items that failed to upload
Artifact Rapid Scan JSON has been successfully uploaded!
Successfully authenticated with Black Duck
Reporting complete.
Warning: Found dependencies violating policy!
(node:4123) UnhandledPromiseRejectionWarning: HttpError: Invalid request.

Only 65535 characters are allowed; 1067045 were supplied.
at /home/runner/work/_actions/synopsys-sig/detect-action/v0.3.2/webpack:/detect-action/node_modules/@octokit/request/dist-node/index.js:86:1
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:4123) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:4123) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.`

[mwagenma-maersk]

This seems to be caused by the GitHub checks API: https://github.community/t/undocumented-65535-character-limit-on-requests/117564. Would it be possible to split the result into smaller chunks and upload them separately?

[jcroall]

Hi @mwagenma-hsud, thanks for pointing this out. There should be a check in there to shorten the message if it exceeds the limit and truncate with message for more to check the Black Duck Hub UI, but we could certainly consider breaking it into multiple pieces.

The intent is to provide actionable feedback on pull requests - such a long list doesn't sound very actionable to me! Would you prefer to report only net-new (e.g. newly introduced by the code change) vulnerabilities instead of a long, spit up list? Only reporting issues in newly introduced or changed components?

[mwagenma-maersk]

Hi @jcroall, shortening the message with a reference to the UI sounded like a good option to me at first, but then it came to my mind that the Rapid scan does not send any scan results to the BlackDuck backend as far as I know, so this might not be an option.

The idea to only point out new issues seems to be a good option. This behavior could be switched on by an input parameter.

[dhananjayagupta]

@jcroall I am also getting the same error while scanning. Do we have any solution to continue the scan ?

[jcroall]

Hey @mwagenma-hsud and @dhananjayagupta, you can limit the rapid scan results to only newly introduced issues by adding:

        env:
          DETECT_BLACKDUCK_RAPID_COMPARE_MODE: BOM_COMPARE_STRICT

Really in a pull request, we want developers to focus on priorities - not get distracted by EVERYTHING in the backlog. If you really want to see everything in a PR comment, we may have to break it up into multiple comments - comments can only be, I think, 65535 bytes long!

[dhananjayagupta]

@jcroall This is also failing with same reason.

A nightly scheduled workflow or on push workflow should do a full Blackduck scan by uploading the results to dashboards. I think the RAPID scan is only helpful in PR workflows which will not upload anything to BD dashboard.

As per above comment what I understood is the problem is with size limit (65535 bytes) only for PR comment, but same issue is coming in non PR workflow. Is there any other issue ? Is there any option to disable commenting within pr comment body ?

Below is the workflow which I am using

name: NIGHTLY pipeline
on:
  workflow_dispatch:
  push:
    branches:
      - master
  #    - develop/**
  #    - release/**

  #  paths-ignore:
  #    - README.md

jobs:
  Build:
    name: Build
    runs-on: [self-hosted, research]
    container:
      image: internal-java8:latest
    timeout-minutes: 360
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
          
      - uses: actions/cache@v3
        with:
          path: |
            /root/.gradle/caches
            /root/.gradle/wrapper
          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
          restore-keys: |
            ${{ runner.os }}-gradle-    
                 
      - name: Build & Sonar
        run: | 
               BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/})
               chmod 775 *
               ./gradlew clean build --refresh-dependencies -x test -x asciidoctor
               
      - name: Scan blackduck
        uses: synopsys-sig/[email protected]
        with:
          scan-mode: INTELLIGENT
          github-token: ${{ secrets.ACCESS_TOKEN }}
          detect-version: 7.13.2
          blackduck-url: https://blackduck.org.XXXXXXXXXX.com
          blackduck-api-token: ${{ secrets.BLACKDUCK_TOKEN }}
        env:
          DETECT_PROJECT_NAME: core-github-test
          DETECT_PROJECT_VERSION_NAME: master
          DETECT_RISK_REPORT_PDF: true
          DETECT_BLACKDUCK_RAPID_COMPARE_MODE: BOM_COMPARE_STRICT

[jcroall]

@dhananjayagupta Ahh interesting - I think the same content is used to attach Markdown content to the policy check. I think we will have to simply truncate the message - with a note that for the full report, the results will have to be viewed in Black Duck Hub.

Looks like just working around for now is not enough! I'll see if I can get this scheduled into our next sprint.

[JeroenKnoops]

It turns out that you need to use the ${{secrets.GITHUB_TOKEN}} instead of a PAT

[dhananjayagupta]

Thanks Buddy. It worked.

[prashil-g]

I've created #60 shamelessly cherry picked fix from @JCastillo07 fork. Please help get this merged