-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ENH] microelectrode electrophysiology specification (BEP032) #1705
base: master
Are you sure you want to change the base?
Conversation
src/schema/objects/columns.yaml
Outdated
@@ -42,6 +42,38 @@ age: | |||
for privacy purposes. | |||
type: number | |||
unit: year | |||
alpha_rotation: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree -- very adhoc. Let's discuss in google doc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you chime in? I think the other guy commenting might be amenable to accepting this as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).
For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):
Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).
Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).
Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.
- yaw: clockwise when looking down
- pitch: In the direction of the electrode face
- roll: clockwise when looking down at the probe
The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.
We need to add a “probe_model”, which references probeinterface_library
NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.
@@ -391,6 +462,12 @@ reference__ieeg: | |||
- type: string | |||
enum: | |||
- n/a | |||
reference_atlas: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The “reference atlas” if you visually verify the area is basically the atlas you looked at before you nod and said “eh, good enough”. This seems at once an overly detailed (does this really matter? the coordinates are commonly given with sub-mm precision) and underdetermined (what coordinates did you look at in the atlas?). I think it's best just to drop this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to double check the setting again, but I think the point is that in some animal studies there are no real coordinates but only semantic statements about the location of a probe/electrode. In these cases it is important to know which atlas they are semantically referring to. Also: in rodents (as far as I know) the atlases are used to plan the surgery. So here it identifies the target area (in coordinates and semantics of the atlas), while the actual location might of cause differ (hopefully only a bit).
Also relevant if you'd like to comment. → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIzHGpUU |
Also I forgot to link to this when I wrote it → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIGPAMOw |
height__probes: | ||
name: height | ||
display_name: Height | ||
description: | | ||
Physical height of the probe, for example, '5'. | ||
This dimension corresponds to the y’axis of the Euler transformation defined by | ||
alpha, beta and gamma rotations values below. | ||
type: number |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
height__probes: | |
name: height | |
display_name: Height | |
description: | | |
Physical height of the probe, for example, '5'. | |
This dimension corresponds to the y’axis of the Euler transformation defined by | |
alpha, beta and gamma rotations values below. | |
type: number |
I don't understand what this is supposed to be
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So these are about geometry of the probes. Thus relating to
width__probes: | ||
name: width | ||
display_name: Width | ||
description: | | ||
Physical width of the probe, for example, '5'. | ||
This dimension corresponds to the x’axis of the Euler transformation defined by | ||
alpha, beta and gamma rotations values. | ||
type: number |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
width__probes: | |
name: width | |
display_name: Width | |
description: | | |
Physical width of the probe, for example, '5'. | |
This dimension corresponds to the x’axis of the Euler transformation defined by | |
alpha, beta and gamma rotations values. | |
type: number |
I don't understand what this is supposed to be.
Co-authored-by: Ben Dichter <[email protected]>
* origin/master: Update blood.tsv example Convert participants.tsv and samples.tsv fix: Use default YAML loader Update all TSV blocks MNT: Only check yaml syntax ENH: Add code to generate tables from TSV fence blocks Do explicitly "allow" for having dotfiles and explicitly ignore them Move description of n/a above "String values" to avoid any association
It follows the [general BIDS specifications to describe participants](../modality-agnostic-files.md#participants-file). | ||
|
||
On top of the existing columns that can be present in this file and that are described in the BIDS specifications (`participant_id`, `species`, `strain`, `strain_rrid`, `sex`, `handedness`, and `age`), we propose to allow adding the following ones: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this section describing the additional fields of the participants table is missing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bendichter The additional fields were introduced in #1839. If I had implemented the macro for them, it would have crashed since it is not included in this pull request. That said, I think we should delete this section because, according to the current implementation, all these fields would be part of the participants section in the Modality-Agnostic Files (aka the main one), making it redundant.
However, I don't think it’s a good practice to add them to the general participants section. This could lead some people to include birthdate
for human data, which is not acceptable. On the other hand, birthdate
is necessary for animal data. Unfortunately, I don’t have a solution for enforcing this selectively.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Peyman-N there is no issue with the participant sections regarding sensitivity in the model agnostic files section. This can be handled through REQUIRED, RECOMMENDED, OPTIONAL plus an awareness comment in the Description of the general column names. So I would agree in removing this section from the modality-specific section and updating instead the model agnostic files section
|
||
## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`) | ||
|
||
This file provides metadata on the global coordinate system in which the electrodes are placed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file provides metadata on the global coordinate system in which the electrodes are placed. | |
This file provides metadata on the coordinate system in which the electrodes are placed. |
I think the word "global" is confusing here. Is it serving a purpose I am missing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume this was added to make sure that only this is only used for the subjects native coordinate system or the common coordinate system? (compared to a coordinate system of a probe for defining the relative coordinates of electrodes outside of a subject)
src/modality-specific-files/microelectrode-electrophysiology.md
Outdated
Show resolved
Hide resolved
|
||
|
||
## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We recently settled on the idea that the coordinate system of the standard electrodes.tsv file (without a "space" designation") should be in probe space. This means that x, and y, (and maybe z), should be relative positions of contacts on the probe. It would be idea if the (0,0) point is the tip of the probe, but honestly this is not really important for spike sorting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's a bit strange to discuss coordinate systems before we discuss probes and electrodes. Is there a motivation for having these sections in the current order or can I move things around a bit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bendichter I agree with you that probes and electrodes should be discussed first. And in iEEG and EEG channels and electrodes descriptions also come prior to the coordinate system description. Feel free to move it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SUGGESTED ORDER:
- channels first
- electrodes second
- probes third
- coordinate systems last
|
||
This file contains the following information: | ||
- The electrode name | ||
- The electrode coordinates in 3 columns (`xyz`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- The electrode coordinates in 3 columns (`xyz`) | |
- The electrode coordinates in 2 or 3 columns (`xy[z]`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depends if we assume always 3 columns with z put to n/a for 2D
- The electrode coordinates in 3 columns (`xyz`) | ||
- The ID of the probe the electrode is located on | ||
|
||
The coordinates are relative to the position on the probe. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The coordinates are relative to the position on the probe. | |
When the 'space' label is not defined, the coordinates are relative position on the probe. To specify electrode positions in subject-space, an atlas space, or some other space, use another electrodes.tsv file with the [space-<label>](https://bids-specification.readthedocs.io/en/stable/appendices/entities.html#space) entity: (`*[_space-<label>]_electrodes.tsv`). Each of these additional electrodes.tsv files should be accompanied by a matching *_space-<label>_coordsystem.json file. | |
For example: | |
└─ sub-01/ | |
├─ sub-01_space-CCFv3_electrodes.tsv | |
├─ sub-01_space-CCFv3_coordsystem.json | |
└─ ... | |
|
||
We propose to store all metadata that is not directly related to one of the other metadata files (probe/electrode/channel information) into a single JSON file: `_ephys.json`. | ||
|
||
There should be one such JSON file for each data file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not really sure what this means
### Probes | ||
|
||
It is RECOMMENDED to use the following structure to provide more metadata for each probe: | ||
|
||
```JSON | ||
"ProbeContours": { | ||
"probe_infoid": { | ||
"<my_probe_id>": { | ||
"Contour": <list of contour points>, | ||
"Unit": "<my spatial unit>" | ||
} | ||
} | ||
} | ||
``` | ||
|
||
|
||
Whereas `<my_probe_id>` has to exist also in the `probes.tsv` file and `<list of contour points>` is | ||
a list of x, y(, z) tuples providing the contour of the probe in the same reference frame as the | ||
electrode coordinates (see `electrodes.tsv`). | ||
In case of different units used than in the `electrodes.tsv`, the key `Unit` can be used here to | ||
provide the spatial unit of the coordinate points. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an accurate depiction of what was discussed in the November 2024, meeting, but in the December 2024 meeting we decided to punt on the countour specification, since it's a bit tricky to include, we might want to refer to ProbeInterface, and it's not critical for spike sorting.
Task Events documentation. | ||
Note that this file can also be used to describe stimulation performed during the recording. For this, | ||
please follow the iEEG stimulation documentation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To do: add links here.
|
||
## Multi-part recordings | ||
|
||
Two different procedures are supported to handle multi-part recordings. In short, the two options are: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two different procedures are supported to handle multi-part recordings. In short, the two options are: | |
Two different procedures are supported to handle multi-part recordings. The two options are: |
in the `*_events.tsv` file. | ||
These two options are made available to support different usages and habits of the experimenters, as | ||
well as to benefit from the capability of the supported data formats (NWB and NIX). | ||
They are described in the following subsections, and made explicit through some of the example data sets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My read of the BIDS standard is that (i) _scans.tsv is the more canonical approach. What is the justification for supporting both methods?
session (for example, recording start times in case the recording was paused and restarted) | ||
when the data from each of these different recordings is stored in separate files. | ||
Each data file should have a name that contains a `_task-XX` and/or `_run-XX` suffix, and | ||
should be described by at most one row in the `*_scans.tsv` file. See also the BIDS Scans |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be described by at most one row in the `*_scans.tsv` file. See also the BIDS Scans | |
should be described by one row in the `*_scans.tsv` file. See also the BIDS Scans |
why "at most"? Shouldn't it be exactly 1, so this scan file can specify the start time?
be expressed in the following format 2009-06-15T13:45:30 (year, month, day, hour (24h), | ||
minute, second; this is equivalent to the RFC3339 “date-time” format, time zone is always | ||
assumed as local time). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
be expressed in the following format 2009-06-15T13:45:30 (year, month, day, hour (24h), | |
minute, second; this is equivalent to the RFC3339 “date-time” format, time zone is always | |
assumed as local time). | |
be expressed in the RFC3339 “date-time” format, for example 2009-06-15T13:45:30 (year, month, day, hour (24h), minute, second. Time zone is always assumed as local time. |
@@ -391,6 +462,12 @@ reference__ieeg: | |||
- type: string | |||
enum: | |||
- n/a | |||
reference_atlas: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to double check the setting again, but I think the point is that in some animal studies there are no real coordinates but only semantic statements about the location of a probe/electrode. In these cases it is important to know which atlas they are semantically referring to. Also: in rodents (as far as I know) the atlases are used to plan the surgery. So here it identifies the target area (in coordinates and semantics of the atlas), while the actual location might of cause differ (hopefully only a bit).
@@ -342,12 +342,12 @@ split: | |||
display_name: Split | |||
description: | | |||
In the case of long data recordings that exceed a file size of 2Gb, | |||
`.fif` files are conventionally split into multiple parts. | |||
Each of these files has an internal pointer to the next file. | |||
`.fif`, `.nwb`, `.nix` files are conventionally split into multiple parts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bendichter is NWB also providing internal pointers to the next split file? The next comments only explain the .fif
split files further.
@ree-gupta is this split also true for .nix
files?
It follows the [general BIDS specifications to describe participants](../modality-agnostic-files.md#participants-file). | ||
|
||
On top of the existing columns that can be present in this file and that are described in the BIDS specifications (`participant_id`, `species`, `strain`, `strain_rrid`, `sex`, `handedness`, and `age`), we propose to allow adding the following ones: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Peyman-N there is no issue with the participant sections regarding sensitivity in the model agnostic files section. This can be handled through REQUIRED, RECOMMENDED, OPTIONAL plus an awareness comment in the Description of the general column names. So I would agree in removing this section from the modality-specific section and updating instead the model agnostic files section
|
||
|
||
## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bendichter I agree with you that probes and electrodes should be discussed first. And in iEEG and EEG channels and electrodes descriptions also come prior to the coordinate system description. Feel free to move it.
|
||
## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`) | ||
|
||
This file provides metadata on the global coordinate system in which the electrodes are placed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume this was added to make sure that only this is only used for the subjects native coordinate system or the common coordinate system? (compared to a coordinate system of a probe for defining the relative coordinates of electrodes outside of a subject)
Example of * _electrodes.tsv: | ||
|
||
```tsv | ||
probe_id impedance x y z material location |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
electrode_id is missing form the table
The electrode positions and properties are stored in a `.tsv` file (amplifier information is in `channels.tsv`). | ||
|
||
This file contains the following information: | ||
- The electrode name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
electrode name or ID ?
|
||
|
||
```tsv | ||
channel_id electrode_id gain type units sampling_frequency status |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sampling_frequency has a set unit?
|
||
**Recommended Channel Type Values** | ||
|
||
For the `type` column we recommend to use the following terms (adopted from [iEEG](intracranial-electroencephalography.md#channelselectrode-description-_channelselectrodestsv)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
types of example are not occurring in that table. should be added??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this is an enum list we may not need this table hardcoded but have it in a schema and pulled in?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replace with macro (if macro is there)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replace EXT in example above with more specific channel type (e.g. LFP, MUA, SPIKES,etc )
|
||
There should be one such JSON file for each data file. | ||
|
||
The `*_ephys.json` file can be used to store any ephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The `*_ephys.json` file can be used to store any ephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`. | |
The `*_microephys.json` file can be used to store any microephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`. |
[FIX] Bep032 - fixing Markdown
Replaces #1352 submitted from a fork outside of bids-specification.
Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal
Note
We meet regularly and everyone is welcome
Next meeting: insert date on URL to join
Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure
Tip
HTML preview of this BEP
bids-validator
with a custom schema. (attn @TheChymera)TODOs
To add your name, please edit our Contributors wiki and add your name with the type of contribution.
For assistance, please tag @bids-standard/maintainers.
To see the checks and preview, scroll down and click on the
show all checks
link.From the list, select the
Details
link of theci/circleci: build_docs artifact
check to see the preview of the BIDS specification.bids-validator
using schema in this PRc557d1f
to1c30c6e
legacy-validator#1798 is the first one trying it on whitelisted set of packages, and I think we should create a helper action for that : https://github.com/bids-standard/bids-validator/issues/1931bids-examples
adding sample dandisets : Draft examples for BEP032 on animal electrophysiology bids-examples#430 (@robertoostenveld )bids-validator
on sample datasets and this modified schema (@yarikoptic)type
column for electrodes[ie]cephys/
sectionIssues this PR would likely to address
Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):
Other issues which relate but not in scope here and provided for reference/backreference