Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REFACTOR]Introduce a data processor for Egapro #409

Merged
merged 24 commits into from
Nov 26, 2024
Merged

Conversation

HAEKADI
Copy link
Contributor

@HAEKADI HAEKADI commented Oct 25, 2024

related to #405

Implement DataProcessor and Egapro functionality:

This PR introduces significant enhancements to the data pipeline workflow, including the creation of the DataProcessor class and the EgaproProcessor class. The following key changes have been made:

  1. Creation of DataProcessor Class:

    • Implemented the DataProcessor class, which serves as a base class for processing various data sources. This class includes common functionality that can be reused across different processors.
  2. Implementation of Egapro Functionality:

    • Created the EgaproProcessor class, which extends the DataProcessor class to handle specific data processing tasks related to Egapro.
  3. New Configuration Class:

    • Introduced the DataSourceConfig class to encapsulate configuration settings related to data sources. This class centralizes parameters such as URLs, file paths, and other relevant settings, making it easier to manage and modify configurations as needed.

@HAEKADI HAEKADI changed the title Refactor egapro [REFACTOR]Introduce a data processor for Egapro Nov 12, 2024
@HAEKADI HAEKADI self-assigned this Nov 12, 2024
@HAEKADI HAEKADI requested a review from hacherix November 12, 2024 15:19
@HAEKADI HAEKADI marked this pull request as ready for review November 12, 2024 15:19
@HAEKADI HAEKADI requested a review from MKCG as a code owner November 12, 2024 15:19
@HAEKADI HAEKADI removed the request for review from MKCG November 12, 2024 16:08
workflows/data_pipelines/egapro/config.py Show resolved Hide resolved
helpers/data_processor.py Outdated Show resolved Hide resolved
helpers/data_processor.py Show resolved Hide resolved
@HAEKADI HAEKADI requested a review from hacherix November 26, 2024 14:08
@HAEKADI HAEKADI merged commit 7ed92f7 into main Nov 26, 2024
4 checks passed
@HAEKADI HAEKADI deleted the refactor-egapro branch November 26, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants