Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Maintenance] update threads and benchmark files for annotation rules #1529

Open
3 tasks
mathiasbio opened this issue Feb 7, 2025 · 0 comments · May be fixed by #1528
Open
3 tasks

[Maintenance] update threads and benchmark files for annotation rules #1529

mathiasbio opened this issue Feb 7, 2025 · 0 comments · May be fixed by #1528
Assignees
Labels
Maintenance Something which needs to be updated
Milestone

Comments

@mathiasbio
Copy link
Collaborator

Task

The cluster has been quite clogged in this period in early 2025 and @Karl-Svard in prodbioinfo notified me about a job in balsamic cadd_annotate_somaticINDEL_research which was hardly using any threads or memory on the node on ganglia but it required 36 cores and booked up the whole node. It seems that this rule doesn't at all require this amount of resources and the cluster could be free:d up a bit if we lowered it.

I then looked at other similar rules and saw a few bcftools commands that was also run on the whole node. I don't think this should be necessary at all since the sizes of the VCFs are rarely even in the scale of 1gb.

On top of this the benchmark files specified in the rules also had the same name and should overwrite each other, meaning that we cannot track the benchmark of these rules.

Suggested approach

Lower the threads to 4 for the majority of the rules and give unique names to the rules. Then test if the cases can pass, and look into the benchmark files to see if they were at any point at risk of crashing, to help determine if 4 is a reasonable thread to posit.

Deviation

No response

Risk assessment

  • Needed
  • Not needed

Risk assessment link

No response

System requirements assessed

  • Yes, I have reviewed the system requirements

Requirements affected

No response

Can be closed when

No response

Blockers

No response

@mathiasbio mathiasbio added the Maintenance Something which needs to be updated label Feb 7, 2025
@github-project-automation github-project-automation bot moved this to Todo in BALSAMIC Feb 7, 2025
@mathiasbio mathiasbio self-assigned this Feb 7, 2025
@mathiasbio mathiasbio moved this from Todo to In Progress in BALSAMIC Feb 7, 2025
@mathiasbio mathiasbio linked a pull request Feb 7, 2025 that will close this issue
58 tasks
@mathiasbio mathiasbio added this to the Release 17 milestone Feb 7, 2025
@mathiasbio mathiasbio linked a pull request Feb 7, 2025 that will close this issue
58 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Something which needs to be updated
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

1 participant