CSV Concatenator

[max-zilla]

This adds an extractor that will try to concatenate CSV/TSV/XLSX files when uploaded to Clowder.

To test:

export CLOWDER_VERSION=2
python concatenate.py

(there's also a Dockerfile)

Then upload two CSVs to a dataset and see that concatenated.csv is created. If you already have 2+ CSVs in the dataset and you upload a new one, all will be merged into the output. Uploading additional CSVs will update the concatenated file as it goes.

File types are currently separated, meaning CSVs will get their own merge separate from Excel etc. This would be easy to change but not sure if there's a good use case for that?

Uses pyclowder files.delete so requires this version of pyclowder: clowder-framework/pyclowder#92

[longshuicy]

Test works. But I haven't tested the built image in docker

[ddey2]

I tested in pycharm. I had to install pandas and set Clowder_version. It works well.

Approving this. I guess we can merge once you address the other comments.