Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider using the --memory-mapping flag on kraken2 to avoid high RAM usage #23

Open
dfornika opened this issue Oct 12, 2022 · 0 comments

Comments

@dfornika
Copy link
Member

We've been experiencing some memory-related issues (filled swap space) on one of our systems recently. It's likely (though we haven't proven) that our kraken2 analyses are filling up main memory and spilling over into the swap space.

The way this pipeline is currently configured is quite inefficient in terms of RAM usage. Each job loads the kraken2 database into RAM. When using the standard database, that involves loading ~60 GB of data into RAM for each job.

kraken2 offers a --memory-mapping flag that avoids loading the database into RAM. It will almost certainly be slower, but may still be acceptable.

Allow users to use the --memory-mapping flag on the kraken2 process. Consider which mode should be the default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant