Update flaky shadow mode #677
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #677 +/- ##
=======================================
Coverage 98.07% 98.07%
=======================================
Files 434 434
Lines 36638 36721 +83
=======================================
+ Hits 35932 36015 +83
Misses 706 706
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #677 +/- ##
=======================================
Coverage 98.07% 98.07%
=======================================
Files 434 434
Lines 36638 36721 +83
=======================================
+ Hits 35932 36015 +83
Misses 706 706
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #677 +/- ##
=======================================
Coverage 98.07% 98.07%
=======================================
Files 434 434
Lines 36638 36721 +83
=======================================
+ Hits 35932 36015 +83
Misses 706 706
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Changes have been made to critical files, which contain lines commonly executed in production. Learn more ✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## main #677 +/- ##
==========================================
+ Coverage 98.11% 98.13% +0.01%
==========================================
Files 474 474
Lines 37964 38428 +464
==========================================
+ Hits 37250 37711 +461
- Misses 714 717 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 7 files with indirect coverage changes This change has been scanned for critical changes. Learn more |
services/test_results.py
Outdated
| return ( | ||
| FLAKY_TEST_DETECTION.check_value(identifier=repoid, default=False) | ||
| and read_yaml_field(commit_yaml, ("test_analytics", "flake_detection"), False) | ||
| or FLAKY_SHADOW_MODE.check_value(identifier=repoid, default=False) |
There was a problem hiding this comment.
If there's some problem where we want to turn off displaying the results to users, then which flag would we want to flip?
There was a problem hiding this comment.
Turning off FLAKY_SHADOW_MODE would also disable writing. So, I think we might want another flag to just disable the reads.
There was a problem hiding this comment.
i updated the changes such that FLAKY_SHADOW_MODE retains it's current functionality on the write path and we're going to use FLAKY_DETECTION to toggle the beta, so we can separately disable the read path without affecting the write path
we should write if: FLAKY_SHADOW_MODE or FLAKE_DETECTION feature flags are enabled AND the user has not disabled flake detection via the config we should read if: FLAKE_DETECTION feature flag is enabled AND the user has not disabled flake detection via the config
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
This PR does 2 things: