Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND thread-safety for MetricsLogger. #48868

Merged
merged 93 commits into from
Dec 3, 2024
Merged

[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND thread-safety for MetricsLogger. #48868

merged 93 commits into from
Dec 3, 2024

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Nov 22, 2024 •
edited
Loading

APPO (new API stack) enhancements vol 05: Auto-sleep time AND thread-safety for MetricsLogger.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

sven1977 added 30 commits October 10, 2024 19:21
@sven1977
wip
2b09969 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
5089e12 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
490e254 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
0c8fb9e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into fix_…
8d46658 
…accumulation_of_results_in_algorithm
@sven1977
wip
12185ca 
Signed-off-by: sven1977 <[email protected]>
@sven1977
ppo reporting everything ok now.
6e4652b 
Signed-off-by: sven1977 <[email protected]>
@sven1977
fix episodes/episodes-lifetime in env runners.
ef549b8 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
1171ccf 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
85c48e8 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
bd5a884 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
937ff49 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into fix_…
4673c96 
…accumulation_of_results_in_algorithm

Signed-off-by: sven1977 <[email protected]>

# Conflicts:
#	rllib/utils/metrics/metrics_logger.py
@sven1977
wip
a686fc7 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
a6fcc37 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
b6ef29e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
666ba01 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
70939e7 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
18fbb91 
Signed-off-by: sven1977 <[email protected]>
@sven1977
fix
dbf2d07 
Signed-off-by: sven1977 <[email protected]>
@sven1977
fix
75f761f 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into fix_…
6c1aa7a 
…accumulation_of_results_in_algorithm
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into impa…
b164f31 
…la_appo_aggregation_workers_new_api_stack
@sven1977
fix
b65d49c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
2342bc9 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into add_…
3707a13 
…off_policyness_metric_to_new_api_stack
@sven1977
Merge branch 'add_off_policyness_metric_to_new_api_stack' into impala…
4d1a4ec 
…_appo_aggregation_workers_new_api_stack
@sven1977
wip
ec7159c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
7327a4e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
64c09e4 
Signed-off-by: sven1977 <[email protected]>
@sven1977 sven1977 enabled auto-merge (squash) December 2, 2024 20:48
@github-actions github-actions bot disabled auto-merge December 2, 2024 20:48
@sven1977
wip
eef1302
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into appo…
180305f 
…_enhancements_05_auto_sleep_and_thread_safety_for_metrics
@sven1977
wip
00a8fef
@sven1977 sven1977 requested a review from a team as a code owner December 3, 2024 13:25
@sven1977 sven1977 enabled auto-merge (squash) December 3, 2024 13:42
@sven1977 sven1977 added rllib RLlib related issues rllib-algorithms An RLlib algorithm/Trainer is not learning. rllib-newstack labels Dec 3, 2024
@github-actions github-actions bot disabled auto-merge December 3, 2024 14:01
@sven1977 sven1977 merged commit f06833e into ray-project:master Dec 3, 2024
4 of 5 checks passed
simonsays1980 pushed a commit to simonsays1980/ray that referenced this pull request Dec 3, 2024
@sven1977 @simonsays1980
[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND…
3b92d16 
… thread-safety for MetricsLogger. (ray-project#48868)
jecsand838 pushed a commit to jecsand838/ray that referenced this pull request Dec 4, 2024
@sven1977 @jecsand838
[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND…
2a166ab 
… thread-safety for MetricsLogger. (ray-project#48868)

Signed-off-by: Connor Sanders <[email protected]>
@sven1977 sven1977 deleted the appo_enhancements_05_auto_sleep_and_thread_safety_for_metrics branch December 4, 2024 09:15
dentiny pushed a commit to dentiny/ray that referenced this pull request Dec 7, 2024
@sven1977 @dentiny
[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND…
2d31ec3 
… thread-safety for MetricsLogger. (ray-project#48868)

Signed-off-by: hjiang <[email protected]>
ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Dec 17, 2024
@sven1977 @ujjawal-khare
[RLlib] APPO (new API stack) enhancements vol 05: Auto-sleep time AND…
97a64d8 
… thread-safety for MetricsLogger. (ray-project#48868)

Signed-off-by: ujjawal-khare <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simonsays1980 simonsays1980 simonsays1980 approved these changes

@scottjlee scottjlee Awaiting requested review from scottjlee

@bveeramani bveeramani Awaiting requested review from bveeramani

@raulchen raulchen Awaiting requested review from raulchen

@stephanie-wang stephanie-wang Awaiting requested review from stephanie-wang

@omatthew98 omatthew98 Awaiting requested review from omatthew98

@alexeykudinkin alexeykudinkin Awaiting requested review from alexeykudinkin

@srinathk10 srinathk10 Awaiting requested review from srinathk10

Assignees

@simonsays1980 simonsays1980

Labels
go add ONLY when ready to merge, run all tests rllib RLlib related issues rllib-algorithms An RLlib algorithm/Trainer is not learning. rllib-newstack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sven1977 @simonsays1980