Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic slurm allocations #794

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

MilesCranmer
Copy link
Owner

This adds the cluster_manager="slurm_native" option which uses https://github.com/kleinhenz/SlurmClusterManager.jl. The idea with this is that you could potentially just have

sbatch -N 2 --ntasks-per-node=64 search.py

and it will "just work" and set up the search over the cluster. This is the much more standard way to use slurm. This will get the right slurm configurations from the environment variables.

TODO:

  • Do a multi-node test. Maybe @wkharold would be interested in testing too?
  • Set up heap size hint We don't actually need to do this! Since slurm will partition the memory for us and each Julia process will be aware.
  • Add a detailed example to docs for how to set this up.

@coveralls
Copy link

coveralls commented Jan 2, 2025

Pull Request Test Coverage Report for Build 12581536478

Details

  • 11 of 25 (44.0%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.7%) to 92.706%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pysr/sr.py 0 1 0.0%
pysr/julia_extensions.py 9 14 64.29%
pysr/julia_helpers.py 2 10 20.0%
Totals Coverage Status
Change from base Build 12512611011: -0.7%
Covered Lines: 1398
Relevant Lines: 1508

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants