[🐞] action$ multipart/form-data breaks production build

[steveblue]

Which component is affected?

Qwik City (routing)

Describe the bug

User cannot upload file with multipart/form-data in production environment using Form and action$.

Expected: User is able to upload files with multipart/form-data in production environment.

Here's the offending code snippet. The following in useSubmit should convert the Blob to an ArrayBuffer, then pass to the sharp library to optimize and output an image file in the file directory on the server. This works in development, but not production because form errors in production.

export const useSubmit = action$(async(form: any) => {
    const arrayBuffer = await form.upload.arrayBuffer();
    const buffer = await Buffer.from(arrayBuffer, 'base64');
    const filename = 'output.webp';
    await sharp(buffer)
        .toFile(`./assets/${filename}`, (err: any, info: any) => {
            console.log(err);
            console.log(info);
        })
    return {
      success: true
    }
});

export default component$(() => {
  useStylesScoped$(styles);
  const action = useSubmit();
  return (
    <>
      <h1>
        Upload Image
      </h1>
      <Form action={action}>
        <input type="file" name="upload" />
        <button type="submit">Upload</button>
      </Form>
    </>
  );
});

Few problems here:

• Expected form to be strictly typed. form.upload cannot be properly typed because form is typed as JSONObject and form.upload is a Blob. FileList probably also isn't a valid type for JSONObject.
• Expected form.upload to equal FileList but instead got Blob. FileList is an Array like interface of File. File inherits from Blob but has many useful properties which are missing.
• Expected filename to be accessible but filename isn't accessible because of Blob. FileList would include the filename in each File.
• Expected multiple attribute to work on <input type="file" name="upload" multiple /> which should produce a FileList but instead outputs Blob with last file in the list.
• Expected code to work in production build. Currently only single file upload works with npm run start but not npm run build and npm run deploy.

I realize FileList may not be readily serializable but it appears with the current API there is no way to serialize each File.

Reproduction

https://github.com/steveblue/whatipaint-v2/tree/45190bf3bb0e16e371b692928cc353fdf24d8be9

Steps to reproduce

Run npm install and npm run dev, visit /upload, upload an image file, observe upload created file in /assets directory.
Run npm run build and npm run deploy, visit /upload, upload another image file, observe Error.

TypeError: Error: Unexpected end of form
    at Multipart.<anonymous> (/node_modules/undici/lib/fetch/body.js:422:46)
    at Multipart.emit (node:events:390:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
TypeError: Error: Unexpected end of form
    at Multipart.<anonymous> (/node_modules/undici/lib/fetch/body.js:422:46)
    at Multipart.emit (node:events:390:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

System Info

System:
    OS: macOS 13.2.1
    CPU: (10) arm64 Apple M1 Max
    Memory: 6.98 GB / 64.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.13.0 - ~/.nvm/versions/node/v16.13.0/bin/node
    Yarn: 1.22.17 - ~/.nvm/versions/node/v16.13.0/bin/yarn
    npm: 8.1.0 - ~/.nvm/versions/node/v16.13.0/bin/npm
  Browsers:
    Chrome Canary: 98.0.4699.0
    Edge: 107.0.1418.62
    Firefox: 107.0.1

Additional Information

• Could not be reproduced with qwik.new because Stackblitz wisely prohibits this.
• Attempted with multiple approved versions of node 16 and 18

[steveblue]

Update. I tried a workaround proposed by @manucorporat.

It appears FileList and File are available on event.request.formData() but File still log typeof Blob which is confusing, but multiple uploads work by referencing this method on the request, although only for development. Production build still runs into the same error reported above.

export const useSubmit = action$(async(form: any, event: RequestEventAction<QwikCityPlatform>) => {
    const formData = await event.request.formData();
    const uploads: File[] = Array.from(formData.getAll('upload')) as File[];
    await Promise.resolve(uploads.map((file: File) => {
      return (async () => {
        const arrayBuffer: ArrayBuffer = await file.arrayBuffer();
        const buffer = Buffer.from(arrayBuffer as any, 'base64');
        return sharp(buffer)
            .toFile(`./assets/${file.name}`, (error: any, info: any) => {
                if (error) {
                  return {
                    error,
                    success: false
                  }
                } else {
                  return {
                    info,
                    success: true,
                  }
                }
            })
      })()
    }));
});

[jdgamble555]

Doesn't Qwik eagerly await the request object under-the-hood?

export const useAddUser = routeAction$(async (user) => {
  const userID = await db.users.add(user);
  return {
    success: true,
    userID,
  };
});

Under-the-hood might look like?

const data = await event.request.formData();
const user = data.get('user');
const userID = await db.users.add(user);

Doesn't this mean that any large files are also going to be eagerly awaited (which is undesirable for files)?

This seems to be why SolidJS and SvelteKit kept the request object with await formData() instead of allowing you to pass the data directly? That being said, I prefer less boilerplate, just wondering if there is a way to do this separately for files?

J

[furbox]

function getFileExtension(filename: string) {
  return filename.slice(((filename.lastIndexOf(".") - 1) >>> 0) + 2);
}

export const useSubmit = routeAction$(
  async (form: any, event: RequestEventAction<QwikCityPlatform>) => {
    const formData = await event.request.formData();
    const uploads: File[] = Array.from(formData.getAll("userfile")) as File[];
    //obtener y guardar el documento en la carpeta assets
    const file = uploads[0];
    const extension = getFileExtension(file.name);
    const name = file.name.slice(0, file.name.lastIndexOf("."));
    const uid = nanoid.nanoid(5);
    const path = `./assets/${name}_${uid}.${extension}`;
    const arrayBuffer = await form.userfile.arrayBuffer();
    const buffer = await Buffer.from(arrayBuffer, "base64");
    writeFileSync(path, buffer);
  }
);

[gioboa]

Hi @steveblue, thanks for opening this issue, did you solve this specific problem?
Thanks @jdgamble555 and @furbox for the discussion and for sharing details.

[josuealejo]

I'm having a not-so-good time with this, I have a basic Form with an input of type="file" and I got an ugly [vite] Internal server error: The body has already been consumed. None of the workarounds above are changing the result

[vite] Internal server error: The body has already been consumed.

export const useSubmit = routeAction$(
  12 |    async (form: any, event: RequestEventAction<QwikCityPlatform>) => {
  13 |      const formData = await event.request.formData();
     |                                            ^
  14 |      const uploads: File[] = Array.from(formData.getAll("csv")) as File[];

[lyquocnam]

i'm got the same too

[lyquocnam]

about 1 year for this issue ? I hope have someone check this. This really important for qwik.
Love qwik so much !

[gouroubleu]

routeAction$((form: any, event: RequestEventAction) => {
const formData = event.sharedMap.get('@actionFormData')
})

[rafalfigura]

I have the same issue. Does anybody have a working solution/workaround?

[wmertens]

@rafalfigura I think what @gouroubleu posted is the solution? If so, that needs documenting, PRs welcome.

[rafalfigura]

@wmertens I was able to fix the issue in fabian-hiller/modular-forms#194 . I've used the @gouroubleu solution :) Thanks

[gioboa]

@steveblue is it still a valid issue?

[github-actions]

Hello @steveblue. Please provide the missing information requested above.
Issues marked with STATUS-2: missing info will be automatically closed if they have no activity within 14 days.
Thanks 🙏