Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add strong consistency check for data on disk #1604

Merged
merged 7 commits into from
Jan 15, 2025

Conversation

vibhansa-msft
Copy link
Member

@vibhansa-msft vibhansa-msft commented Jan 6, 2025

✅ What

Data being downloaded by block-cache is saved to local disk, if user choses to enable persistence. This feature is useful if application is reading the same files multiple times and we do not have enough in memory cache to hold all of them. When a block is downloaded it's saved on local disk. There are chances of accidental overwrite on the disk file which might corrupt the data. Later when same block is read by application Blobfuse will load the data from this disk file and end up serving corrupt data to user application.

Having a CRC check on the data when it was downloaded, protects us from those accidental corruptions.

🤔 Why

To provide strong data consistency checks and protect data persisted on local disk.

👩‍🔬 How to validate if applicable

If user choses to persist the downloaded content then this additional flag can provide and extra layer of security.

🔖 Related links

NA

README.md Show resolved Hide resolved
@jainakanksha-msft
Copy link
Collaborator

jainakanksha-msft commented Jan 10, 2025

@vibhansa-msft Please add the whole use case to validate the scenario also add the correct links in the PR.

@jainakanksha-msft
Copy link
Collaborator

@vibhansa-msft , Do we need to update any public documentation for this new argument?
Additionally, do we need to verify our default config generation script
or any update in E2E test cases?

@vibhansa-msft vibhansa-msft merged commit 37149a5 into blobfuse/2.4.1 Jan 15, 2025
8 checks passed
@vibhansa-msft vibhansa-msft deleted the vibhansa/diskcrc branch January 15, 2025 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants