FAILED: You probably need to delete the target dataset to fix this.

[ozgurerdogan]

When moving source dataset (of proxmox) to different node / server, I am getting "FAILED: You probably need to delete the target dataset to fix this." when running zfs_autobackup.

So everytime I have to delete target datacenter and let zfs_autobackup to copy whole dataset again which causes bandwidth.

So my question is, is there a workaround for such situations so I do not have to delete target dataset.

[psy0rz]

depends on your setup. are you using the proxmox example script?

which version of zfs-autobackup?

can you should a zfs list -r -t snapshot for both the original dataset and the dataset after moving. and for the backup target.

[ozgurerdogan]

I am not using that script. I am using 3.1.3

My crons is like:
zfs-autobackup -v --ssh-target [email protected] s1d8-backup D3 --keep-source=40 --keep-target=40 --strip-path=1 --clear-mountpoint --clear-refreservation --ignore-transfer-errors --no-holds --snapshot-format %Y-%m-%d-%H:%M:%S --destroy-missing=10d --force --destroy-incompatible"

But I am using proxmox builtin pve-zsync tool to move dataset to an other server.?! It probably does not move its snapshots..

Should I use some other method to move dataset?

[psy0rz]

maybe, but the common snapshot can be the one that pve-zsync created.

try this:

• do the zsync like before
• run zfs-autobackup with the old server as source, but use --other-snapshots. This will download the snapshot created by pve-zsync.
• now run zfs-autobackup like you normally would on the new server. It should recognise that the snapshot there is the same as the one on the target, and continue from there.

[ozgurerdogan]

Ok I will try.. Thank you.

[ozgurerdogan]

Sorry it does not work I guess. I added --other-snapshots but still getting:FAILED: You probably need to delete the target data set to fix this.

[psy0rz]

ok, can you run both comands with --other-snapshots and --debug, and post the parameters and output here?

[ozgurerdogan]

Ok this is one of them:

! [Target] D4/vm-999-disk-0: Cant find common snapshot with source.
! [Source] D5/vm-999-disk-0: FAILED: You probably need to delete the target dataset to fix this.
! Exception: You probably need to delete the target dataset to fix this.
Traceback (most recent call last):
File "/usr/local/bin/zfs-autobackup", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/init.py", line 9, in cli
failed_datasets=zfs_autobackup.run()
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/ZfsAutobackup.py", line 594, in run
fail_count = self.sync_datasets(
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/ZfsAutobackup.py", line 433, in sync_datasets
source_dataset.sync_snapshots(target_dataset, show_progress=self.args.progress,
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/ZfsDataset.py", line 1025, in sync_snapshots
self._plan_sync(target_dataset=target_dataset, also_other_snapshots=also_other_snapshots)
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/ZfsDataset.py", line 958, in _plan_sync
common_snapshot = self.find_common_snapshot(target_dataset)
File "/usr/local/lib/python3.9/dist-packages/zfs_autobackup/ZfsDataset.py", line 801, in find_common_snapshot
raise (Exception("You probably need to delete the target dataset to fix this."))
Exception: You probably need to delete the target dataset to fix this.

And this is command:

ssh s6.xxx.com "zfs-autobackup -v --ssh-target [email protected] s6d5-backup D4 --keep-source=40 --keep-target=40 --strip-path=1 --clear-mountpoint --clear-refreservation --other-snapshots --ignore-transfer-errors --no-holds --snapshot-format %Y-%m-%d-%H:%M:%S --destroy-missing=10d --force --destroy-incompatible --debug";

Actually, if you can add an option to overwrite/delete target dataset in this case, it would help a lot. Because everytime, I have to manually delete target dataset.

[psy0rz]

did you run that on the original source? where you migrated away from? because you should do that first.

[psy0rz]

automaticly deleting/overwriting is a bit too dangerous. it would mean you would delete and recreate the backup every time, and everytime you are without backup for the duration of that first full operation.

its better to solve the problem in a clean way.

[ozgurerdogan]

I just noted that this issue happening even on nonmigrated datasets. I am running this script for around 150 datasets with interval of 1 hour. Could this be main issue?
On next issue happens, I will post it with debug here.

[ozgurerdogan]

This error is also appears on existing running tasks (with no move). FAILED: You probably need to delete the target dataset to fix this And I have to delete target dataset. Is there easier solution for this that I can apply ?

[ozgurerdogan]

I think it could be backup tasks are so often and maybe prev backup is not completed and next set is created.. maybe..

[psy0rz]

normally this shouldnt happen. Try running without --no-holds to prevent proxmox deleting the common snapshot.

[ozgurerdogan]

Thank you for your responds. My swiches are:
--keep-source=40 --keep-target=40 --clear-mountpoint --clear-refreservation --other-snapshots --ignore-transfer-errors --no-holds --snapshot-format %Y-%m-%d-%H:%M:%S --force --destroy-missing=10d";