Update loading functions, fix various issues and complete pipeline by athrado · Pull Request #44 · nestauk/asf_core_data

Commits on Jul 19, 2022

S3 loading for raw EPC data and suppl data (#22 )

Julia Suter
authored and
Julia Suter
committed

Commits on Jul 22, 2022

Fix requirements

Julia Suter
authored and
Julia Suter
committed

Commits on Aug 30, 2022

Updates for loading from S3 (#19 )

Julia Suter
authored and
Julia Suter
committed
Loading and downlaoding from S3 and fixes (#19 , #24 , #25 )

Julia Suter
authored and
Julia Suter
committed
Load batches (download control) (#24 )

Julia Suter
authored and
Julia Suter
committed
Show download options (#24 )

Julia Suter
authored and
Julia Suter
committed

Commits on Aug 31, 2022

Fix column loading (#24 )

Julia Suter
authored and
Julia Suter
committed
Minor loading fixes (#24 )

Julia Suter
authored and
Julia Suter
committed
Fix batch laoding (#24 )

Julia Suter
authored and
Julia Suter
committed
Remove MACOSX dirs (#24 )

Julia Suter
authored and
Julia Suter
committed

Commits on Sep 1, 2022

Minor loading fixes (#24 )

Julia Suter
authored and
Julia Suter
committed
Minor loading fixes (#24 )

Julia Suter
authored and
Julia Suter
committed
Fix paths (#24 )

Julia Suter
authored and
Julia Suter
committed

Commits on Sep 2, 2022

Minor loading fixes (#24 )

Julia Suter
authored and
Julia Suter
committed
Remove debug prints (#24 )

Julia Suter
authored and
Julia Suter
committed
Upload notebook (#24 )

Julia Suter
authored and
Julia Suter
committed

Commits on Sep 14, 2022

Flexible bucket name (#19 )

Julia Suter
authored and
Julia Suter
committed

Commits on Sep 15, 2022

Allow loading geojson files (#19 )

Julia Suter
authored and
Julia Suter
committed

Commits on Sep 16, 2022

Fix geopandas loading function (#19 )

Julia Suter
authored and
Julia Suter
committed

Commits on Oct 21, 2022

Fix issues with data loading (#19 )

Julia Suter
authored and
Julia Suter
committed

Commits on Oct 25, 2022

Minor fixes to EPC with MCS installation dates (#25 )

Julia Suter
authored and
Julia Suter
committed

Commits on Dec 21, 2022

Pull in MCS code and adjust processing functions (#28 )

Julia Suter
authored and
Julia Suter
committed
Add guide for processing new data (#28 )

Julia Suter
authored and
Julia Suter
committed
First attempt at describing feature processing - put on hold (#29 )

Julia Suter
authored and
Julia Suter
committed
Update gitignore and remove old files

Julia Suter
authored and
Julia Suter
committed

Commits on Dec 22, 2022

Update doc strings (#28 )

Julia Suter
authored and
Julia Suter
committed

Commits on Jan 6, 2023

Postcode format for backwards compatibility

Julia Suter
authored and
Julia Suter
committed
Fix data loading (paths)

Julia Suter
authored and
Julia Suter
committed
Fix data loading (paths)

Julia Suter
authored and
Julia Suter
committed
Fix data loading (encoding)

Julia Suter
authored and
Julia Suter
committed

Commits on Jan 18, 2023

Fixes for data loading (#19 )

Julia Suter
authored and
Julia Suter
committed
Update notebook (#19 )

Julia Suter
authored and
Julia Suter
committed
Update bucket name

Julia Suter
authored and
Julia Suter
committed

Commits on Jan 27, 2023

Batch handling

Julia Suter
authored and
Julia Suter
committed

Commits on Jan 30, 2023

Update notebook

Julia Suter
authored and
Julia Suter
committed

Commits on Feb 14, 2023

Make capping optional (#33 )

Julia Suter
authored and
Julia Suter
committed
Add geopandas (#30 )

Julia Suter
authored and
Julia Suter
committed

Commits on Feb 15, 2023

Simplify IMD data loader (#32 )

Julia Suter
authored and
Julia Suter
committed
Make features optional and fix duplication (#34 )

Julia Suter
authored and
Julia Suter
committed
Fix indixing issue (#35 )

Julia Suter
authored and
Julia Suter
committed

Commits on Feb 16, 2023

Improve loading by country (#36 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 1, 2023

Notebook with completed outputs

Julia Suter
authored and
Julia Suter
committed
Take over quarter 4 fix from #43

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 2, 2023

Update outputs

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 3, 2023

Remove main function (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove no longer used imports (#44 )

Julia Suter
authored and
Julia Suter
committed
Improve doc string (#44 )

Julia Suter
authored and
Julia Suter
committed
Add geojson to supported file type (#44 )

Julia Suter
authored and
Julia Suter
committed
Convert string to path (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove unnecessary S3 call (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix docstring (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix usecols for old batches (#44 )

Julia Suter
authored and
Julia Suter
committed
Update column renaming (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix loading function for certificates (#44 )

Julia Suter
authored and
Julia Suter
committed
Rename EPC batch function (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 7, 2023

Fix doc string (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove print statements (#44 )

Julia Suter
authored and
Julia Suter
committed
Rename column (#44 )

Julia Suter
authored and
Julia Suter
committed
Refer to bucket via config (#44 )

Julia Suter
authored and
Julia Suter
committed
Main call to run from terminal (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix docstrings and kwargs (#44 )

Julia Suter
authored and
Julia Suter
committed
Adjust output for final merged version (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 8, 2023

Improve postcode loading (#44 )

Julia Suter
authored and
Julia Suter
committed
Data formatting in function (#44 )

Julia Suter
authored and
Julia Suter
committed
Postcode merging (#44 )

Julia Suter
authored and
Julia Suter
committed
Update doc string (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove potential duplicates in usecols (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix input options (#44 )

Julia Suter
authored and
Julia Suter
committed
Shorter code (#44 )

Julia Suter
authored and
Julia Suter
committed
Change output to outputs/input to inputs (#44 )

Julia Suter
authored and
Julia Suter
committed
Updated doc string (#44 )

Julia Suter
authored and
Julia Suter
committed
Updated doc string (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove old code fragments (#44 )

Julia Suter
authored and
Julia Suter
committed
Updates (#44 )

Julia Suter
authored and
Julia Suter
committed
Comment and fix reload raw data (#44 )

Julia Suter
authored and
Julia Suter
committed
Update comments (#44 )

Julia Suter
authored and
Julia Suter
committed
Update docstring (#44 )

Julia Suter
authored and
Julia Suter
committed
Update docstring (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 9, 2023

Update pipeline with usecol options and docstrings (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix line break (#44 )

Julia Suter
authored and
Julia Suter
committed
Rename and update install date computation (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove unnecessary gitkeeps (#44 )

Julia Suter
authored and
Julia Suter
committed
Update configs for MCS (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 10, 2023

Remove notebook as python file (#44 )

Julia Suter
authored and
Julia Suter
committed
Delete zip file after unzipping and unneccesary code removed (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 12, 2023

Update comments and docstrings (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove duplicate loading function (#44 )

Julia Suter
authored and
Julia Suter
committed
Update MCS feature selection (#44 )

Julia Suter
authored and
Julia Suter
committed
Rename script and add necessary MCS fields (#44 )

Julia Suter
authored and
Julia Suter
committed
Update gitignore (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 15, 2023

Update notebooks (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove duplicates (#44 )

Julia Suter
authored and
Julia Suter
committed
Merge issues (#44 )

Julia Suter
authored and
Julia Suter
committed
Merge issues (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 17, 2023

Remove notebook tags (#44 )

Julia Suter
authored and
Julia Suter
committed
Add kwargs (#44 )

Julia Suter
authored and
Julia Suter
committed
Outputs after final runs (#44 )

Julia Suter
authored and
Julia Suter
committed
Rename file and remove notebook tags (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove duplicated file (#44 )

Julia Suter
authored and
Julia Suter
committed
Resolve merge conflicts (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits on Mar 19, 2023

Adjust kwargs for S3 loading and saving functions (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove old function for latest batch (#44 )

Julia Suter
authored and
Julia Suter
committed
Adjust output path for merged file (#44 )

Julia Suter
authored and
Julia Suter
committed
Fix kwarg dtype (#44 )

Julia Suter
authored and
Julia Suter
committed
Update notebooks (#44 )

Julia Suter
authored and
Julia Suter
committed
Remove dupl file (#44 )

Julia Suter
authored and
Julia Suter
committed

Commits

Commits on Jul 19, 2022

Commits on Jul 22, 2022

Commits on Aug 30, 2022

Commits on Aug 31, 2022

Commits on Sep 1, 2022

Commits on Sep 2, 2022

Commits on Sep 14, 2022

Commits on Sep 15, 2022

Commits on Sep 16, 2022

Commits on Oct 21, 2022

Commits on Oct 25, 2022

Commits on Dec 21, 2022

Commits on Dec 22, 2022

Commits on Jan 6, 2023

Commits on Jan 18, 2023

Commits on Jan 27, 2023

Commits on Jan 30, 2023

Commits on Feb 14, 2023

Commits on Feb 15, 2023

Commits on Feb 16, 2023

Commits on Feb 20, 2023

Commits on Feb 24, 2023

Commits on Feb 25, 2023

Commits on Feb 27, 2023

Commits on Feb 28, 2023

Commits on Mar 1, 2023

Commits on Mar 2, 2023

Commits on Mar 3, 2023

Commits on Mar 7, 2023

Commits on Mar 8, 2023

Commits on Mar 9, 2023

Commits on Mar 10, 2023

Commits on Mar 12, 2023