Price proof: convert images to webp before upload ?

[raphodn]

Story

In the Open Prices project we've been discussing the idea to convert as much of our images to webp. Why ? No quality impact, but a big size impact (factor of 5-7), all the seconds reduced to upload an image to the server.

This conversion is already done in the web app - prices.openfoodfacts.org.
What about doing it in the mobile app ?

Especially if we want to seperate the "proof upload" and "price upload" actions, and have the proof uploaded as soon as possible so that the server can return the "predictions" (proof type, price/product/location extraction...)

Linked to openfoodfacts/open-prices#577

[monsieurtanuki]

In the Open Prices project we've been discussing the idea to convert as much of our images to webp. Why ? No quality impact, but a big size impact (factor of 5-7), all the seconds reduced to upload an image to the server.

Could be done on the server side when receiving the image file, right?

This conversion is already done in the web app - prices.openfoodfacts.org. What about doing it in the mobile app ?

flutter is not very good converting images: something like slow actions that freeze the app.

Especially if we want to seperate the "proof upload" and "price upload" actions, and have the proof uploaded as soon as possible so that the server can return the "predictions" (proof type, price/product/location extraction...)

That would mean discarding the offline mode. Or maintaining both online and offline mode. What's supposed to happen with slow connectivity?

[raphodn]

Could be done on the server side when receiving the image file, right?

The idea is to have both. But if done on the mobile, users would send a ~1MB image, instead of a 10MB image. Good for slow connectivity, no ?

flutter is not very good converting images: something like slow actions that freeze the app.

I'm no expert, but we do background tasks already in the mobile app, we want to put some AI, so an image conversion shouldn't be that hard ?

[monsieurtanuki]

Good for slow connectivity, no ?

Actually we don't really care about speed as we do that in background.

an image conversion shouldn't be that hard

It's not hard, it takes resources, for instance we have to save and keep the image in BMP (instead of directly in JPEG) most of the time for performance reasons, and eventually convert it in JPEG but in background (cf. image_compute_container.dart)

And there are two different modes. Let's say the user selects a photo from the gallery. We support only JPEG here. Then:

• if the user leaves the photo untouched, we send it as is (no conversion)
• if the user crops the photo, we have to tap dance with the BMP then JPEG conversion (here it would be easy to convert to WEBP instead)

[monsieurtanuki]

@raphodn Which compression rate do you use for your webp conversions?

[raphodn]

@raphodn Which compression rate do you use for your webp conversions?

In the web app I use compressor.js
The only settings activated are:

• checkOrientation: true (default)
• retainExif: true
• mimeType: 'image/webp'
• quality: 0.8 (default)

I tested setting the quality to 1, and instead of dividing the size by 5-6, it actually increases it 👀

[monsieurtanuki]

@raphodn Things don't look that good in https://pub.dev/packages/flutter_image_compress#webp

Webp
Support android by the system api (speed very nice). The library also supports iOS. However, we're using third-party libraries, it is not recommended due to encoding speed. In the future, libwebp by google (C/C++) may be used to do coding work, bypassing other three-party libraries, but there are no plan for that currently.

Looks like a no-go for iOS: it would freeze the app (and for no apparent reason for the user as it's in background tasks).
Of course we could use webp for android and jpeg for ios, but that would add confusion to the code for a limited added value (on the Smoothie side at least).

That said, I've just noted that our current quality factor for JPEG is... 100 (percents I guess).
Lowering it down to 80% could be an easy win, wouldn't it?

[monsieurtanuki]

@raphodn I've run some tests around quality and compress format on an android device, regarding size in bytes:

x	jpeg	webp
quality 80	692,881	269,730
quality 100	2,633,929	4,820,580

Given that using webp in flutter would be problematic, as explained earlier, an easy win would be to lower the quality of compression from 100 to 80 for prices proofs.
The size of Smoothie proof images would be divided by 3.

What do you think about it?

[raphodn]

Thanks for the tests and the images !

I tried in the frontend using the current compressjs library, and doing the same thing (conversion to JPG 80%). Results here. The output size is 2x webp, but still a factor or 4-5 compared to the original image. And it retains EXIF.

So maybe an alternative path could indeed be :

• have the web & mobile UIs send compress JPEGs to save bandwith
• have the server manage these images (strip EXIF, compress again to WebP ?)

cc @raphael0202

[monsieurtanuki]

@raphodn @raphael0202

• when the image isn't cropped, we currently send it "as-is", no compression
  => we probably keep the EXIF but I'd have to double-check that
  => we may find it useful to compress it (more heavy computation on the device but less data to send), and there's a "keepExif" parameter (currently set to false)
• when the image is cropped, I don't know the standards but it would be strange if all EXIFs were supposed to be kept.
• had a look at EXIF packages in flutter, and apparently it's much safer (and more often possible) to read EXIF that to write them
  => wouldn't we be better off reading specific EXIF (e.g. GPS, date) and possibly passing them to the server (or keeping them locally for the "add price/proof" page)?
  => you would be able to keep your webp format without additional tap dancing with jpeg and EXIF