Support .json.gz files #114
Conversation
File format detection based on file extension facilitates support for .json.gz files in a future commit.
nextstrain-bot
temporarily deployed
to
auspice-us-victorlin-su-8diu77
victorlin
force-pushed
the
victorlin/support-json-gz
branch
from
0c20fec to
15ca8fd
Compare
There was a problem hiding this comment.
I didn't know browsers now exposed decompression APIs - that's great!
We should also support gzipped sidecar files as part of this PR. For instance, loading a pair of gzipped main + frequencies JSONs doesn't work and the console shows:
Read measurements-panel_flu_seasonal_h3n2_ha_tip-frequencies.json.gz as a main dataset JSON file
Read measurements-panel_flu_seasonal_h3n2_ha.json.gz as a main dataset JSON fileI'd suggest an implementation whereby the linking of main & sidecars is compression agnostic, e.g. we allow a gzipped main and non-gzipped sidecar, or vice versa, however I'm not wedded to this so let me know if you think there's good reason not to.
Prepare for supporting .json.gz files. The file extension should not be used in the key since it can be either .json or .json.gz. Simply use the filename without extension. This is simplified with a new function to get the dataset name from any Auspice JSON file.
Auspice JSONs can be large, and in such cases they are often compressed using gzip. Adding the ability to load these files directly removes the need for users to manually decompress the file. Done using the Compression Streams API¹. ¹ <https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API>
Narratives and Newick file descriptions specify file extensions. Do the same for Auspice JSONs.
victorlin
force-pushed
the
victorlin/support-json-gz
branch
from
15ca8fd to
f89f81e
Compare
Thanks for flagging sidecar files. I just force-pushed some changes to allow any combination of .json and .json.gz. |
Description of proposed changes
Auspice JSONs can be large, and in such cases they are often compressed using gzip. Adding the ability to load these files directly removes the need for users to manually decompress the file.
Done using the Compression Streams API¹.
¹ https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API
Related issue(s)
Closes #112
Checklist
nextstrain remote download https://next.nextstrain.org/ncov/gisaid/global/6mand compressed withgzip