Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor update hdf5 page #184

Merged
merged 3 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/prs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ jobs:
with:
python-version: 3.9
- name: Set up JDK 11 for jNeuroML etc
uses: actions/setup-java@v1
uses: actions/setup-java@v3
with:
java-version: 11
distribution: 'temurin'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ jobs:
with:
python-version: 3.9
- name: Set up JDK 11 for jNeuroML etc
uses: actions/setup-java@v1
uses: actions/setup-java@v3
with:
java-version: 11
distribution: 'temurin'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand Down
9 changes: 5 additions & 4 deletions source/Userdocs/HDF5.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@
# HDF5 support

The XML serializations of large NeuroML models can be prohibitive to store.
For such cases, NeuroML also includes support for saving models in the binary [HDF5](https://www.hdfgroup.org/solutions/hdf5) format via the [NeuroMLhdf5Writer in libNeuroML](https://libneuroml.readthedocs.io/en/stable/userdocs/writers.html#neuroml.writers.NeuroMLHdf5Writer).
For such cases, NeuroML also includes support for saving models in the binary [HDF5](https://www.hdfgroup.org/solutions/hdf5) format via the [NeuroMLHdf5Writer in libNeuroML](https://libneuroml.readthedocs.io/en/stable/userdocs/writers.html#neuroml.writers.NeuroMLHdf5Writer). The same format can be exported also from the Java API ([example](https://github.com/NeuroML/org.neuroml.model/blob/master/src/test/java/org/neuroml/model/test/HDF5Test.java)).

The format of the export is documented below:


- {ref}`Network <schema:network>` is exported as a `network` group with `id`, `notes`, and the `temperature` (optional) stored as attributes.
- {ref}`Network <schema:network>` is exported as a `network` HDF5 group with `id`, `notes`, and the `temperature` (optional) stored as attributes.
- {ref}`Population <schema:population>` is exported as a group with id `population_<id of the population>` with `id`, `component`, `size`, `type`, and `property` tags stored as attributes.
- If the population is a {ref}`population list <schema:populationlist>` that includes {ref}`instances <schema:instance>` of cells, the locations of cells (x, y, z), these are stored in a 3 column table ("chunked array") with a row per instance.

Expand All @@ -18,6 +19,6 @@ The format of the export is documented below:
- {ref}`InputList <schema:inputlist>` is exported similar to Projection with the {ref}`Input <schema:input>`, and {ref}`InputW <schema:inputw>` entries stored in tables.


The source code of these export functions can be seen [here in the libNeuroML repository](https://github.com/NeuralEnsemble/libNeuroML/blob/2d8112178d8d82b07a20f8395ec22a23a6323a6c/neuroml/nml/helper_methods.py#L2548).
For more details, the source code of these export functions can be seen [here in the libNeuroML repository](https://github.com/NeuralEnsemble/libNeuroML/blob/2d8112178d8d82b07a20f8395ec22a23a6323a6c/neuroml/nml/helper_methods.py#L2548) and [here in org.neuroml.model](https://github.com/NeuroML/org.neuroml.model/blob/master/src/main/java/org/neuroml/model/util/hdf5/NeuroMLHDF5Writer.java).

HDF5 NeuroML files can also be passed to `jnml` like `XML` files.
HDF5 NeuroML files can be read and processed by `jnml` and `pynml` in the same way as XML files (see [here](https://github.com/OpenSourceBrain/OpenCortex/tree/master/examples/HDF5) for LEMS Simulation file examples which reference HDF5 NeuroML models).