Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding metric wrapper classes #203

Merged
merged 11 commits into from
Nov 21, 2023
Merged

Adding metric wrapper classes #203

merged 11 commits into from
Nov 21, 2023

Conversation

drewoldag
Copy link
Collaborator

@drewoldag drewoldag commented Nov 8, 2023

Problem & Solution Description (including issue #)

The variety of different metric APIs that are present in qp currently make it challenging to write consistent evaluator stages in RAIL. By introducing wrapper classes we can abstract and unify the API (as much as possible) while making it easier to interface with RAIL evaluator stages.

TODO

Need to add simple unit test coverage for the classes
Include PIT Metric wrapper class.

Code Quality

  • My code follows the code style of this project
  • I have written unit tests or justified all instances of #pragma: no cover; in the case of a bugfix, a new test that breaks as a result of the bug has been added
  • My code contains relevant comments and necessary documentation for future maintainers; the change is reflected in applicable demos/tutorials (with output cleared!) and added/updated docstrings use the NumPy docstring format
  • Any breaking changes, described above, are accompanied by backwards compatibility and deprecation warnings

@drewoldag drewoldag self-assigned this Nov 8, 2023
Copy link

codecov bot commented Nov 8, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a578c66) 99.95% compared to head (838ba13) 99.95%.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #203    +/-   ##
========================================
  Coverage   99.95%   99.95%            
========================================
  Files          33       35     +2     
  Lines        2294     2463   +169     
========================================
+ Hits         2293     2462   +169     
  Misses          1        1            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@drewoldag drewoldag force-pushed the awo-metric-wrapper-classes branch from 638a0ee to 68963b3 Compare November 8, 2023 22:49
@drewoldag drewoldag requested a review from eacharles November 9, 2023 05:12
@drewoldag drewoldag marked this pull request as ready for review November 9, 2023 05:12
@drewoldag drewoldag marked this pull request as draft November 9, 2023 05:14
@drewoldag drewoldag removed the request for review from eacharles November 9, 2023 05:14
@drewoldag drewoldag marked this pull request as ready for review November 14, 2023 00:31
Copy link
Collaborator

@eacharles eacharles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, I had a couple minor comments about adding docstrings to the base classes and if the placeholder functions should be no-ops (i.e., just pass) instead of NotImplemented. Up to you how if you want to implement those changes.

@drewoldag drewoldag merged commit b7b6aa4 into main Nov 21, 2023
6 checks passed
@drewoldag drewoldag deleted the awo-metric-wrapper-classes branch November 21, 2023 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants