Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cambium API and GREET Excel Integration #380

Merged
merged 20 commits into from
Oct 25, 2024
Merged

Conversation

dakotaramos
Copy link
Collaborator

@dakotaramos dakotaramos commented Oct 23, 2024

Cambium and GREET integration
Added:
cambium_data.py: to manage the Cambium API call and parsing of data
greet_data.py: to handle manipulation and extraction of data from GREET excel docs
resource_files/cambium/: directory that stores cambium resource files, files currently in this folder are defaults for testing
resource_files/greet/: directory that stores 2023 versions of GREET, a preprocessed yaml file with relevant data points to be used in GreenHEART LCA analysis, and associated README.md
tests/hopp/test_cambium_resource.py: pytests for cambium integration
tests/hopp/test_greet_resource.py: pytests for greet integration

Edited:
tests/hopp/utils.py: created default file paths for cambium and greet testing
hopp/simulation/technologies/resource/init.py: made cambium and greet data classes importable
pyproject.toml: added "pytest-dependency" under develop dependencies, used in cambium pytests

Related issue

#375

Impacted areas of the software

pyproject.toml, pytest suite, and resources

Additional supporting information

This work is being completed for the LCA integration, the data pulling and parsing will live in HOPP, the LCA calculations with that data will live in GreenHEART

NOTE:

  • Additional variables can possibly be added to the Cambium API call to pull curtailment, end use costs, busbar costs, clean energy fractions, distribution losses, transmissions totals, and various load values if needed.

  • Spoke with Cambium API folks (James Morris). 1) they do not have any plans in FY25 to add a bulk requests functionality to the API but will let us know when it becomes available. 2) Communicated possible increased API requests from this integration so they can monitor system health / computational resources needed.

  • Many more data points can be extracted from GREET excel, but requires time and effort to interpret GREET calculations and determine proper values.

Test results, if applicable

tests/hopp/test_cambium_resource.py .....
tests/hopp/test_greet_resource.py ...

…class, added another test for processing and saving GREET data to non-default filename
…_init__, added cambium_data.py, updated pyproject.toml to include pytest-dependency, added test_cambium_resource.py, updated tests/hopp/utils.py with default cambium file
@bayc bayc added this to the NAWEA workshop-ready release milestone Oct 23, 2024
@bayc bayc added enhancement New feature or request high priority Need to tackle soon labels Oct 23, 2024
@bayc bayc linked an issue Oct 23, 2024 that may be closed by this pull request
@dakotaramos
Copy link
Collaborator Author

dakotaramos commented Oct 23, 2024

After thought for the cambium api integration, it will likely be useful to allow users to input the specific data columns / variables they want to pull from the API as an argument instead of pulling a preset list of "most" variables available. However, this can be added as a follow up PR, I'd prefer to get this in as is for now given the NAWEA release. Open to other's opinions.

Copy link
Collaborator

@johnjasa johnjasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review, I'll dig in more later on! Great work, love to see this coming in.

@dakotaramos
Copy link
Collaborator Author

Flagging dates of possible Cambium API slow downs due to database updates (James Morris):

"Late-Nov or early-Dec we'll be uploading the new Standard Scenarios which might make the db a little slow as we're bulk writing records. January will be Cambium which is larger and will almost definitely slow the db down a bit during the couple of days/week we're ingesting that project."

Copy link
Collaborator

@johnjasa johnjasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled down this branch and played around a bit, I dig it! I have some minor comments, nothing major. Really big effort here to get both tools integrated into HOPP, thank you for that.

hopp/simulation/resource_files/greet/README.md Outdated Show resolved Hide resolved
hopp/simulation/technologies/resource/cambium_data.py Outdated Show resolved Hide resolved
hopp/simulation/technologies/resource/greet_data.py Outdated Show resolved Hide resolved
hopp/simulation/technologies/resource/greet_data.py Outdated Show resolved Hide resolved
hopp/simulation/technologies/resource/greet_data.py Outdated Show resolved Hide resolved
hopp/simulation/technologies/resource/greet_data.py Outdated Show resolved Hide resolved
@johnjasa johnjasa merged commit 8998ec4 into NREL:develop Oct 25, 2024
4 checks passed
This was referenced Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request high priority Need to tackle soon
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cambium Call
4 participants