Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REFACTOR][AGENCE-BIO] Use DataProcessor #449

Merged
merged 3 commits into from
Jan 7, 2025
Merged

Conversation

hacherix
Copy link
Contributor

@hacherix hacherix commented Jan 6, 2025

Related to #405

DAG has been refactored to use the DataProcessor class.
A side improvement is that the API calls to agence-bio now also include the records with no département.

image

Leading underscore removed since the method is not private.
Copy link
Contributor

@HAEKADI HAEKADI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool 🦾
Just the same little remark on script names. adding the data source is not coherent with what we did so far ^^

workflows/data_pipelines/agence_bio/agence_bio_dag.py Outdated Show resolved Hide resolved
@hacherix hacherix force-pushed the refactor-agence-bio-dag branch from 001a9b5 to b5643b1 Compare January 6, 2025 15:03
Remove the filter during the API calls on the départements. Agence BIO
API now accepts calls without it. Several hundreds of records has no
départements.
@hacherix hacherix force-pushed the refactor-agence-bio-dag branch from b5643b1 to 1533cff Compare January 7, 2025 09:31
@hacherix hacherix merged commit ec73542 into main Jan 7, 2025
4 checks passed
@hacherix hacherix deleted the refactor-agence-bio-dag branch January 7, 2025 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants