Abort on import

[bakape]

When thumbnailing certain files in the middle of a group of files, hydron receives a SIGABRT and crashes with one of a number of different error messages.
corrupted size vs. prev_size while consolidating
malloc(): corrupted top size
corrupted double-linked list
free(): invalid next size (normal)
Every file can be thumbnailed individually. If the import command is repeated, it will move past the point of crash but it might still crash on a later file. The order that the files are processed in seems to be important so to reproduce consistently hydron/import.go#41 should be n := 1
Both folders in the attached zip crash, I can get more examples if you like.
Debian 10, thumbnailer v2.6.4, hydron and deps up to date.

test data.zip

[Kamuso]

I "fixed" it by setting thumbnailer.c#93 to dst->data = malloc(dst->size + 25);
so av_image_get_buffer_size must be giving the wrong size for some files. I don't know where to go from here.

[bakape]

Why 25?

[Kamuso]

It's the smallest value that worked with the test data. I tried with 100 first.

[Kamuso]

I tried some imports running Hydron with Valgrind to see what's happening with memory. Running valgrind --read-var-info=yes --leak-check=no ./hydron import path shows invalid writes when importing the files that cause crashes. Pic related. Other files report no invalid writes.

Valgrind still reports an invalid write when using the malloc(dst->size + 25) I mentioned earlier, but malloc(dst->size + 64)
is fine.
All the bad images I've found have been jpgs with YUVJ420p colour encoding. Re-encoding them as png, or as jpg with YUVJ444p colour makes them thumbnail with no invalid writes, but not all YUVJ420p jpgs cause invalid writes.
I tried having a look at the ffmpeg source to see where the 8 bytes are coming from but I can't figure it out. Do you have any suggestions?

[ghost]

I believe this may be an issue with webp, as well.
example_image.tar.gz
wtf.log

[bakape]

I'll get to this eventually. Too much on my plate at the moment.

…

On Thu, 17 Dec 2020 at 05:22, チルノ ***@***.***> wrote: I believe this may be an issue with webp, as well. example_image.tar.gz <https://github.com/bakape/hydron/files/5706768/example_image.tar.gz> wtf.log <https://github.com/bakape/hydron/files/5706769/wtf.log> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB347MBSYXGVPLUPAKUTXCDSVF2QRANCNFSM4QK32OTA> .

[bakape]

I might have fixed it in the commit above. As you can reproduce, could you try the fix? Will try to reproduce before and after commit myself a later this week.

[Kamuso]

Still aborts. When trying to reproduce, running valgrind --read-var-info=yes --leak-check=no ./hydron import path and looking for invalid writes is more reliable than looking for crashes, by the way.