Restoring [split] app data backups (i.e. any backup >1GB) fails with error "Input is not in the .gz format"

[UntrimmedParasail]

Please check before submitting an issue

• I know what my device, OS and App Manager versions are
• I know how to take logs
• I know how to reproduce the issue which may not be specific to my device

I have searched existing issues for keywords: restore & gz

Describe the bug

App Manager splits Backup archive files at 1GB boundaries and uses the filename convention data<#>.tar.gz.<part>

Any attempt to Restore a backup (Internal Data+Cache), where the backup archive had been split, fails with an error "Input is not in the .gz format"

Note: All backup splits were verified to match the SHA-256 hashes contained in checksums.txt

Note: App Manager v4.x.x has NOT been tested (to see if the problem has been resolved)

Combining the split archive parts and updating checksums.txt allowed restore operations to succeed, without any errors; ie

$ cat data0.tar.gz.* > joined.tar.gz.0
$ rm data0.tar.gz.*
$ mv joined.tar.gz.0 data0.tar.gz.0
$ tar -tf data0.tar.gz.0

To Reproduce

No response

Expected behavior

No response

Screenshots

No response

Logs

A sample restore failure:
(Int Data+Cache, approx 3.54GB, 4x data0 files)

====> op=BACKUP_RESTORE, mode=2 pkg=(it.vfsfitvnm.vimusic, 0)
io.github.muntashirakon.AppManager.backup.BackupException: Failed to restore data files for index 0.
	at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreData(RestoreOp.java:521)
	at io.github.muntashirakon.AppManager.backup.RestoreOp.runRestore(RestoreOp.java:200)
	at io.github.muntashirakon.AppManager.backup.BackupManager.restore(BackupManager.java:176)
	at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$io-github-muntashirakon-AppManager-batchops-BatchOpsManager(BatchOpsManager.java:338)
	at io.github.muntashirakon.AppManager.batchops.BatchOpsManager$$ExternalSyntheticLambda0.run(Unknown Source:12)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:920)
Caused by: java.io.IOException: Input is not in the .gz format
	at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.init(GzipCompressorInputStream.java:177)
	at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.<init>(GzipCompressorInputStream.java:147)
	at io.github.muntashirakon.AppManager.utils.TarUtils.extract(TarUtils.java:153)
	at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreData(RestoreOp.java:518)
	... 9 more

Device info

• Device: taimen (Pixel 2 XL)
• OS Version: Android 12 (LineageOS 19.1)
• App Manager Version: v3.1.6 [429] - F-Droid
• Mode: Root (Magisk)

Additional context

I am posting this on behalf of someone else who doesn't have a GitHub account.

[ctrlcver]

if report ERROR like gzip compressed data is corrupt,the data maybe not corrupt.
maybe your data is too large(eg:Total Quantity of Data > 12G, and data0.tar.gz.X COUNT > 10).

if data0.tar.gz.X COUNT > 10,should be used like this

# NOT cat data0.tar.gz.*,if use it, will echo invalid compressed data --format violated.and such as "...EOF..."
cat data0.tar.gz.0 data0.tar.gz.1 data0.tar.gz.2 data0.tar.gz.3 data0.tar.gz.4 data0.tar.gz.5 data0.tar.gz.6 data0.tar.gz.7 data0.tar.gz.8 data0.tar.gz.9 data0.tar.gz.10 data0.tar.gz.11 data0.tar.gz.12 data0.tar.gz.13 > join0.tar.gz.0
mv data0.tar.gz.0 data0.tar.gz.0.bak
mv join0.tar.gz.0 data0.tar.gz.0
tar -tf data0.tar.gz.0 # the data0 is intact.
sha256sum data0.tar.gz.0 # the new data0 is different form original.you should edit checksums.txt.

cp the new data0.tar.gz.0 in your phone,and restore it.

from my experience.

[MuntashirAkon]

if report ERROR like gzip compressed data is corrupt,the data maybe not corrupt.
maybe your data is too large(eg:Total Quantity of Data > 12G, and data0.tar.gz.X COUNT > 10).

Nice catch! This issue is actually present in App Manager.

[MuntashirAkon]

if report ERROR like gzip compressed data is corrupt,the data maybe not corrupt.
maybe your data is too large(eg:Total Quantity of Data > 12G, and data0.tar.gz.X COUNT > 10).

Fixed in b8bcd37