Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config file not checking data defenitions from MathProg format #151

Closed
trevorb1 opened this issue Apr 4, 2023 · 2 comments
Closed

Config file not checking data defenitions from MathProg format #151

trevorb1 opened this issue Apr 4, 2023 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@trevorb1
Copy link
Member

trevorb1 commented Apr 4, 2023

If I have a MathProg data file that has a parameter which is not defined in the config.yaml file, otoole breaks. For example, running the command:

otoole convert datafile excel data.txt data.xlsx config.yaml

Will traceback to an amply error along the lines of:

packages\amply\amply.py", line 308, in eval
    assert len(self.records[0]) == 1
TypeError: object of type 'TabularRecord' has no len()

We should add a check that all definitions in the config file match the definitions once the data has been read in. The error is thrown in the following function. Specifically while the context manager is open and the file is being loaded in.

def read_in_datafile(self, path_to_datafile: str, config: Dict) -> Amply:
"""Read in a datafile using the Amply parsing class
Arguments
---------
path_to_datafile: str
config: Dict
"""
parameter_definitions = self._load_parameter_definitions(config)
datafile_parser = Amply(parameter_definitions)
with open(path_to_datafile, "r") as datafile:
datafile_parser.load_file(datafile)
return datafile_parser

@trevorb1 trevorb1 added the bug Something isn't working label Apr 4, 2023
@trevorb1 trevorb1 self-assigned this Apr 4, 2023
@trevorb1
Copy link
Member Author

trevorb1 commented Apr 4, 2023

I confirmed where this becomes an issue. With a parameter removed from the config file, the following table can be generated. Note that the datafile format is otoole, not momani.

from format to format Works Error
csv datafile Runs, but does not copy missing data none
csv excel Runs, but does not copy missing data none
excel csv no OtooleExcelNameMismatchError: AccumulatedAnnualDemand -> Excel tab name not found in config file
excel datafile no OtooleExcelNameMismatchError: AccumulatedAnnualDemand -> Excel tab name not found in config file
datafile csv no AmplyError: Param AccumulatedAnnualDemand not previously defined
datafile excel no AmplyError: Param AccumulatedAnnualDemand not previously defined

The from format excel is the functionality I think we want. While the from format datafile works, I think this issue is only with long formatted data. When the data is wide foratted (momani), then the exception in the issue description in shown. When the from format is csv, no warning is given to the user which isnt great either.

@trevorb1
Copy link
Member Author

Replaced by issue #160 and PR #157

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant