Change the file format of pxo files

[OverloadedOrama]

Now that we have layer blending (#911) and soon layer effects (#619, currently working on that), the pxo file format is going to just keep getting complicated. We have to store both the unedited image data and the image data with blending and effects applied, to allow for importing by other software and CLI exporting. Not to mention that some of the layer effect data may be textures (especially when we allow for custom shader importing), which have to be stored as binary to save space. I feel like it will all be a big mess.

Now that we're on the path from 0.x to 1.x, it is the perfect time to break compatibility! :)

Solution 1: ZIP

It might be worth it to just change the entire format and make it a .zip file in disguise. We have talked about this in the past on our Discord server, but ruled it out as an option because there was no easy way to do it, and doing it in GDScript could be slow. But now we use Godot 4, which has the new ZIPReader and ZIPPacker classes, which allow us to do just that. This approach is also done by other software, such as Krita, Pencil2D and PyxelEdit. I think this will make .pxo files a lot easier to understand, will simplify the loading and saving process, and could allow things such as previewing by file managers. Dolphin, KDE Plasma's default file manager for example, can preview Krita files.

Solution 2: SQLite

I am not familiar with SQLite myself, so I am not the most knowledgeable on the topic. However, it seems like one of Krita's developers wishes they had gone on this path instead of using a .zip + xml format. https://news.ycombinator.com/item?id=23509914
Godot does not work with SQLite natively from what I know, but it should be possible to do this using extensions such as this https://github.com/2shady4u/godot-sqlite

Feel free to discuss on which solution you think would be best, or if you have ideas for other solutions.

Of course, we would still have to keep the old importing code so 0.x pxo files can still be opened in 1.x.

[IcedQuinn]

I don't think SQLite is a great idea for an asset file format. It is actually quite complicated to reason about--something people don't really pay much attention to because they just go through the library and don't think about it. Designing your file format as an equivalent of record dumps isn't quite so bad and Microsoft did do that in .NET.

Zips are really old and we could probably do better in computer science (say, actually being able to support something like zstd.) But in principle its fine yeah.

[OverloadedOrama]

ZSTD would be a nice idea but I'm not sure how that would work. ZIPs are easy to work with due to Godot 4's ZIPPacker and ZIPReader classes, as seen in PR #952.

[IcedQuinn]

I get why people keep using ZIP I just wish we had something better.