Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] microelectrode electrophysiology specification (BEP032) #1705

Open
wants to merge 40 commits into
base: master
Choose a base branch
from

Conversation

yarikoptic
Copy link
Collaborator

@yarikoptic yarikoptic commented Feb 14, 2024

Replaces #1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal

Note

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure


TODOs

Issues this PR would likely to address

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

Other issues which relate but not in scope here and provided for reference/backreference

@@ -42,6 +42,38 @@ age:
for privacy purposes.
type: number
unit: year
alpha_rotation:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- very adhoc. Let's discuss in google doc

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Could you chime in? I think the other guy commenting might be amenable to accepting this as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Copy link
Collaborator

@TheChymera TheChymera Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).

For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):

Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).

Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).

Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.

  • yaw: clockwise when looking down
  • pitch: In the direction of the electrode face
  • roll: clockwise when looking down at the probe

The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.

We need to add a “probe_model”, which references probeinterface_library

  • versioning of probe library files (raised here)
  • annotation of “tip” position (raised here)

NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.

@@ -391,6 +462,12 @@ reference__ieeg:
- type: string
enum:
- n/a
reference_atlas:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The “reference atlas” if you visually verify the area is basically the atlas you looked at before you nod and said “eh, good enough”. This seems at once an overly detailed (does this really matter? the coordinates are commonly given with sub-mm precision) and underdetermined (what coordinates did you look at in the atlas?). I think it's best just to drop this.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to double check the setting again, but I think the point is that in some animal studies there are no real coordinates but only semantic statements about the location of a probe/electrode. In these cases it is important to know which atlas they are semantically referring to. Also: in rodents (as far as I know) the atlases are used to plan the surgery. So here it identifies the target area (in coordinates and semantics of the atlas), while the actual location might of cause differ (hopefully only a bit).

@TheChymera TheChymera closed this Mar 7, 2024
@TheChymera TheChymera deleted the bep032 branch March 7, 2024 22:13
@TheChymera TheChymera restored the bep032 branch March 7, 2024 22:15
@TheChymera TheChymera reopened this Mar 11, 2024
@TheChymera
Copy link
Collaborator

TheChymera commented Mar 12, 2024

@TheChymera
Copy link
Collaborator

@Remi-Gau Remi-Gau changed the title [ENH] Add BEP032 (microelectrode electrophysiology) specification [ENH] microelectrode electrophysiology specification (BEP032) Apr 18, 2024
Comment on lines +371 to +378
height__probes:
name: height
display_name: Height
description: |
Physical height of the probe, for example, '5'.
This dimension corresponds to the y’axis of the Euler transformation defined by
alpha, beta and gamma rotations values below.
type: number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
height__probes:
name: height
display_name: Height
description: |
Physical height of the probe, for example, '5'.
This dimension corresponds to the y’axis of the Euler transformation defined by
alpha, beta and gamma rotations values below.
type: number

I don't understand what this is supposed to be

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these are about geometry of the probes. Thus relating to

Comment on lines +1116 to +1123
width__probes:
name: width
display_name: Width
description: |
Physical width of the probe, for example, '5'.
This dimension corresponds to the x’axis of the Euler transformation defined by
alpha, beta and gamma rotations values.
type: number
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
width__probes:
name: width
display_name: Width
description: |
Physical width of the probe, for example, '5'.
This dimension corresponds to the x’axis of the Euler transformation defined by
alpha, beta and gamma rotations values.
type: number

I don't understand what this is supposed to be.

yarikoptic and others added 7 commits December 18, 2024 14:47
* origin/master:
  Update blood.tsv example
  Convert participants.tsv and samples.tsv
  fix: Use default YAML loader
  Update all TSV blocks
  MNT: Only check yaml syntax
  ENH: Add code to generate tables from TSV fence blocks
  Do explicitly "allow" for having dotfiles and explicitly ignore them
  Move description of n/a above "String values" to avoid any association
It follows the [general BIDS specifications to describe participants](../modality-agnostic-files.md#participants-file).

On top of the existing columns that can be present in this file and that are described in the BIDS specifications (`participant_id`, `species`, `strain`, `strain_rrid`, `sex`, `handedness`, and `age`), we propose to allow adding the following ones:

Copy link
Contributor

@bendichter bendichter Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this section describing the additional fields of the participants table is missing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bendichter The additional fields were introduced in #1839. If I had implemented the macro for them, it would have crashed since it is not included in this pull request. That said, I think we should delete this section because, according to the current implementation, all these fields would be part of the participants section in the Modality-Agnostic Files (aka the main one), making it redundant.
However, I don't think it’s a good practice to add them to the general participants section. This could lead some people to include birthdate for human data, which is not acceptable. On the other hand, birthdate is necessary for animal data. Unfortunately, I don’t have a solution for enforcing this selectively.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Peyman-N there is no issue with the participant sections regarding sensitivity in the model agnostic files section. This can be handled through REQUIRED, RECOMMENDED, OPTIONAL plus an awareness comment in the Description of the general column names. So I would agree in removing this section from the modality-specific section and updating instead the model agnostic files section


## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`)

This file provides metadata on the global coordinate system in which the electrodes are placed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This file provides metadata on the global coordinate system in which the electrodes are placed.
This file provides metadata on the coordinate system in which the electrodes are placed.

I think the word "global" is confusing here. Is it serving a purpose I am missing?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this was added to make sure that only this is only used for the subjects native coordinate system or the common coordinate system? (compared to a coordinate system of a probe for defining the relative coordinates of electrodes outside of a subject)



## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We recently settled on the idea that the coordinate system of the standard electrodes.tsv file (without a "space" designation") should be in probe space. This means that x, and y, (and maybe z), should be relative positions of contacts on the probe. It would be idea if the (0,0) point is the tip of the probe, but honestly this is not really important for spike sorting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a bit strange to discuss coordinate systems before we discuss probes and electrodes. Is there a motivation for having these sections in the current order or can I move things around a bit?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bendichter I agree with you that probes and electrodes should be discussed first. And in iEEG and EEG channels and electrodes descriptions also come prior to the coordinate system description. Feel free to move it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTED ORDER:

  • channels first
  • electrodes second
  • probes third
  • coordinate systems last


This file contains the following information:
- The electrode name
- The electrode coordinates in 3 columns (`xyz`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- The electrode coordinates in 3 columns (`xyz`)
- The electrode coordinates in 2 or 3 columns (`xy[z]`)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends if we assume always 3 columns with z put to n/a for 2D

- The electrode coordinates in 3 columns (`xyz`)
- The ID of the probe the electrode is located on

The coordinates are relative to the position on the probe.
Copy link
Contributor

@bendichter bendichter Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The coordinates are relative to the position on the probe.
When the 'space' label is not defined, the coordinates are relative position on the probe. To specify electrode positions in subject-space, an atlas space, or some other space, use another electrodes.tsv file with the [space-<label>](https://bids-specification.readthedocs.io/en/stable/appendices/entities.html#space) entity: (`*[_space-<label>]_electrodes.tsv`). Each of these additional electrodes.tsv files should be accompanied by a matching *_space-<label>_coordsystem.json file.
For example:
└─ sub-01/
├─ sub-01_space-CCFv3_electrodes.tsv
├─ sub-01_space-CCFv3_coordsystem.json
└─ ...


We propose to store all metadata that is not directly related to one of the other metadata files (probe/electrode/channel information) into a single JSON file: `_ephys.json`.

There should be one such JSON file for each data file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure what this means

Comment on lines 387 to 407
### Probes

It is RECOMMENDED to use the following structure to provide more metadata for each probe:

```JSON
"ProbeContours": {
"probe_infoid": {
"<my_probe_id>": {
"Contour": <list of contour points>,
"Unit": "<my spatial unit>"
}
}
}
```


Whereas `<my_probe_id>` has to exist also in the `probes.tsv` file and `<list of contour points>` is
a list of x, y(, z) tuples providing the contour of the probe in the same reference frame as the
electrode coordinates (see `electrodes.tsv`).
In case of different units used than in the `electrodes.tsv`, the key `Unit` can be used here to
provide the spatial unit of the coordinate points.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an accurate depiction of what was discussed in the November 2024, meeting, but in the December 2024 meeting we decided to punt on the countour specification, since it's a bit tricky to include, we might want to refer to ProbeInterface, and it's not critical for spike sorting.

Comment on lines +484 to +486
Task Events documentation.
Note that this file can also be used to describe stimulation performed during the recording. For this,
please follow the iEEG stimulation documentation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do: add links here.


## Multi-part recordings

Two different procedures are supported to handle multi-part recordings. In short, the two options are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Two different procedures are supported to handle multi-part recordings. In short, the two options are:
Two different procedures are supported to handle multi-part recordings. The two options are:

in the `*_events.tsv` file.
These two options are made available to support different usages and habits of the experimenters, as
well as to benefit from the capability of the supported data formats (NWB and NIX).
They are described in the following subsections, and made explicit through some of the example data sets.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My read of the BIDS standard is that (i) _scans.tsv is the more canonical approach. What is the justification for supporting both methods?

session (for example, recording start times in case the recording was paused and restarted)
when the data from each of these different recordings is stored in separate files.
Each data file should have a name that contains a `_task-XX` and/or `_run-XX` suffix, and
should be described by at most one row in the `*_scans.tsv` file. See also the BIDS Scans
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
should be described by at most one row in the `*_scans.tsv` file. See also the BIDS Scans
should be described by one row in the `*_scans.tsv` file. See also the BIDS Scans

why "at most"? Shouldn't it be exactly 1, so this scan file can specify the start time?

Comment on lines +511 to +513
be expressed in the following format 2009-06-15T13:45:30 (year, month, day, hour (24h),
minute, second; this is equivalent to the RFC3339 “date-time” format, time zone is always
assumed as local time).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
be expressed in the following format 2009-06-15T13:45:30 (year, month, day, hour (24h),
minute, second; this is equivalent to the RFC3339 “date-time” format, time zone is always
assumed as local time).
be expressed in the RFC3339 “date-time” format, for example 2009-06-15T13:45:30 (year, month, day, hour (24h), minute, second. Time zone is always assumed as local time.

@@ -391,6 +462,12 @@ reference__ieeg:
- type: string
enum:
- n/a
reference_atlas:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to double check the setting again, but I think the point is that in some animal studies there are no real coordinates but only semantic statements about the location of a probe/electrode. In these cases it is important to know which atlas they are semantically referring to. Also: in rodents (as far as I know) the atlases are used to plan the surgery. So here it identifies the target area (in coordinates and semantics of the atlas), while the actual location might of cause differ (hopefully only a bit).

@@ -342,12 +342,12 @@ split:
display_name: Split
description: |
In the case of long data recordings that exceed a file size of 2Gb,
`.fif` files are conventionally split into multiple parts.
Each of these files has an internal pointer to the next file.
`.fif`, `.nwb`, `.nix` files are conventionally split into multiple parts.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bendichter is NWB also providing internal pointers to the next split file? The next comments only explain the .fif split files further.

@ree-gupta is this split also true for .nix files?

It follows the [general BIDS specifications to describe participants](../modality-agnostic-files.md#participants-file).

On top of the existing columns that can be present in this file and that are described in the BIDS specifications (`participant_id`, `species`, `strain`, `strain_rrid`, `sex`, `handedness`, and `age`), we propose to allow adding the following ones:

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Peyman-N there is no issue with the participant sections regarding sensitivity in the model agnostic files section. This can be handled through REQUIRED, RECOMMENDED, OPTIONAL plus an awareness comment in the Description of the general column names. So I would agree in removing this section from the modality-specific section and updating instead the model agnostic files section



## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bendichter I agree with you that probes and electrodes should be discussed first. And in iEEG and EEG channels and electrodes descriptions also come prior to the coordinate system description. Feel free to move it.


## Coordinate System JSON (`*_coordsystem.json`) & Photos of electrode positions (`_photo.jpg`)

This file provides metadata on the global coordinate system in which the electrodes are placed.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this was added to make sure that only this is only used for the subjects native coordinate system or the common coordinate system? (compared to a coordinate system of a probe for defining the relative coordinates of electrodes outside of a subject)

Example of * _electrodes.tsv:

```tsv
probe_id impedance x y z material location
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

electrode_id is missing form the table

The electrode positions and properties are stored in a `.tsv` file (amplifier information is in `channels.tsv`).

This file contains the following information:
- The electrode name
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

electrode name or ID ?



```tsv
channel_id electrode_id gain type units sampling_frequency status
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sampling_frequency has a set unit?


**Recommended Channel Type Values**

For the `type` column we recommend to use the following terms (adopted from [iEEG](intracranial-electroencephalography.md#channelselectrode-description-_channelselectrodestsv))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

types of example are not occurring in that table. should be added??

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is an enum list we may not need this table hardcoded but have it in a schema and pulled in?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace with macro (if macro is there)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace EXT in example above with more specific channel type (e.g. LFP, MUA, SPIKES,etc )


There should be one such JSON file for each data file.

The `*_ephys.json` file can be used to store any ephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `*_ephys.json` file can be used to store any ephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`.
The `*_microephys.json` file can be used to store any microephys-specific metadata for the dataset. We recommend storing all setup-related metadata in a dedicated node of the JSON file called `Setup`.

[FIX] Bep032 - fixing Markdown
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.