Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multi-step processing #50

Open
khaeru opened this issue Jan 7, 2021 · 1 comment
Open

Allow multi-step processing #50

khaeru opened this issue Jan 7, 2021 · 1 comment
Labels
enh Enhancements & new features historical Historical transport statistics

Comments

@khaeru
Copy link
Contributor

khaeru commented Jan 7, 2021

The initial processing scripts for input data set have a “1-step” design pattern: a single function process() does all the work.

This must be extended to allow multiple layers of processing steps, forming a flowchart, graph, or network of processing steps that can be arbitrarily long. This is necessary because, for instance, the step of harmonizing or joining 2 or more data sets must be kept distinct from the step of cleaning/adapting the input data for each one.

This graph pattern is already implemented by other software, e.g. ixmp.reporting—code should be reused instead of reimplemented.

@khaeru khaeru added enh Enhancements & new features historical Historical transport statistics labels Jan 7, 2021
@khaeru
Copy link
Contributor Author

khaeru commented May 4, 2021

This graph pattern is already implemented by other software, e.g. ixmp.reporting

This is now genno.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enh Enhancements & new features historical Historical transport statistics
Projects
None yet
Development

No branches or pull requests

1 participant