WIP: Parallel trial scheduler stub using multiprocessing.Pool #939
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@jsfreischuetz here's the stub idea I had for making a parallel trial scheduler using
multiprocessing.Pool
.This would be implemented in something like a class
ParallelTrialScheduler
in themlos_bench.schedulers
module.Perhaps some aspects of the APIs would have to change, and certainly lots and lots of tests and some error handling and interaction with the Storage backend instead of just local
dicts
for state management.I think we should probably start with something simple like this in order to complete #380. Maybe for starters it can just handle fanning out the execution of trial repeats across backend workers.
Then later your work for #926 can expand it do budgeted config scheduling and then later model adjusted scoring and unstable config detection.
For testing, I'm thinking we setup something like
LocalEnv
and just have them run a simple script that returns known data (e.g., maybe even just an echo back the value it's given in the config or a computation of it or some such).Let me know if you have questions.