bug: ingestions fail to take place after sync failure

[de-sh]

/about output:
v1.6, distributed mode

Description:

Ingestion API returns 200, but events aren't in s3. Logs show retries reaching limit(5):

2024-11-18T18:13:43Z WARN  parseable::sync] failed to sync local data with object store. ConnectionError(UnhandledError(Reqwest { retries: 5, max_retries: 5, elapsed: 6.842013835s, retry_timeout: 30s, source: reqwest::Error { kind: Status(500), url: ".../date%3D2024-11-15/ingestor.8677e4a099f5a08.manifest.json" } }))

[nitisht]

As we discussed, we'll add detailed warning about how to notify users if sync is getting delayed and there is a arrow to parquet conversion lag due to high CPU usage.