Skip to content

Commit

Permalink
Merge pull request #101 from iscoe/dev
Browse files Browse the repository at this point in the history 
Dev -> Main Part 2: Electric Boogaloo
  • Loading branch information
@griffinmilsap
griffinmilsap authored Feb 29, 2024
2 parents 4abfa3f + 50d52b1 commit 7732124
Show file tree
Hide file tree
Showing 126 changed files with 7,632 additions and 1,394 deletions.
37 changes: 18 additions & 19 deletions .github/workflows/python-publish-ezmsg-sigproc.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,24 @@ permissions:

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.8'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-sigproc
run: python -m build extensions/ezmsg-sigproc
- name: Publish ezmsg-sigproc
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_SIGPROC }}
packages_dir: extensions/ezmsg-sigproc/dist
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.8"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-sigproc
run: python -m build extensions/ezmsg-sigproc
- name: Publish ezmsg-sigproc
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_SIGPROC }}
packages_dir: extensions/ezmsg-sigproc/dist
37 changes: 18 additions & 19 deletions .github/workflows/python-publish-ezmsg-websocket.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,24 @@ permissions:

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.8'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-websocket
run: python -m build extensions/ezmsg-websocket
- name: Publish ezmsg-websocket
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_WEBSOCKET }}
packages_dir: extensions/ezmsg-websocket/dist
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.8"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-websocket
run: python -m build extensions/ezmsg-websocket
- name: Publish ezmsg-websocket
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_WEBSOCKET }}
packages_dir: extensions/ezmsg-websocket/dist
37 changes: 18 additions & 19 deletions .github/workflows/python-publish-ezmsg-zmq.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,24 @@ permissions:

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.8'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-zmq
run: python -m build extensions/ezmsg-zmq
- name: Publish ezmsg-zmq
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_ZMQ }}
packages_dir: extensions/ezmsg-zmq/dist
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.8"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg-zmq
run: python -m build extensions/ezmsg-zmq
- name: Publish ezmsg-zmq
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN_ZMQ }}
packages_dir: extensions/ezmsg-zmq/dist
35 changes: 17 additions & 18 deletions .github/workflows/python-publish-ezmsg.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,23 @@ permissions:

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.8'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg
run: python -m build
- name: Publish ezmsg
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN }}
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.8"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build ezmsg
run: python -m build
- name: Publish ezmsg
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
password: ${{ secrets.PYPI_API_TOKEN }}
77 changes: 40 additions & 37 deletions .github/workflows/python-tests.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,51 +2,54 @@ name: Test package

on:
push:
branches: [ main ]
branches: [main]
pull_request:
branches: [ main ]
branches: [main]
workflow_dispatch:

jobs:
build:

strategy:
matrix:
python-version: [3.8, 3.9, '3.10']
python-version: [3.8, 3.9, "3.10"]
os:
- 'ubuntu-latest'
- 'windows-latest'
- 'macos-latest'
- "ubuntu-latest"
- "windows-latest"
- "macos-latest"
runs-on: ${{matrix.os}}

steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8
pip install -e ".[test]"
pip install -e extensions/ezmsg-sigproc
pip install -e extensions/ezmsg-websocket
pip install -e extensions/ezmsg-zmq
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
flake8 . --count --exit-zero --statistics
- name: Test ezmsg
run: |
python -m pytest -v tests
- name: Test ezmsg-sigproc
run: |
python -m pytest -v extensions/ezmsg-sigproc/tests
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Upgrade pip and install pipx
run: |
python -m pip install --upgrade pip pipx flake8
- name: Install Poetry via pipx
run: |
pipx install poetry
echo "$HOME/.local/bin/" >> $GITHUB_PATH
- name: Install dependencies with Poetry
run: |
poetry install -E zmq -E websocket -E sigproc --with test
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test ezmsg
run: |
poetry run python -m pytest -v tests
- name: Test ezmsg-sigproc
run: |
poetry run python -m pytest -v extensions/ezmsg-sigproc/tests
74 changes: 59 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,87 @@

Messaging and Multiprocessing.

`ezmsg` is a pure-Python implementation of a directed acyclic graph (DAG) pub/sub messaging pattern based on [`labgraph`](https://github.com/facebookresearch/labgraph) which is optimized and intended for use in constructing real-time software. `ezmsg` implements much of the `labgraph` API (with a few notable differences), and owes a lot of its design to the `labgraph` developers/project.
`ezmsg` is a pure-Python implementation of a directed acyclic graph (DAG) pub/sub messaging pattern based on [`labgraph`](https://github.com/facebookresearch/labgraph) which is optimized and intended for use in constructing real-time software. `ezmsg` implements much of the `labgraph` API (with a few notable differences), and owes a lot of its design to the `labgraph` developers/project.

`ezmsg` is very fast and uses Python's `multiprocessing.shared_memory` module to facilitate efficient message passing without C++ or any compilation/build tooling.

## Installation

`pip install ezmsg`

## Dependencies

* Due to reliance on `multiprocessing.shared_memory`, `ezmsg` requires __minimum Python 3.8__.
* `typing_extensions`
- Due to reliance on `multiprocessing.shared_memory`, `ezmsg` requires **minimum Python 3.8**.
- `typing_extensions`

Testing `ezmsg` requires:

Testing `ezmsg` requires:
* `pytest`
* `pytest-cov`
* `pytest-asyncio`
* `numpy`
- `pytest`
- `pytest-cov`
- `pytest-asyncio`
- `numpy`

## Setup (Development)
``` bash

```bash
$ python3 -m venv env
$ source env/bin/activate
(env) $ pip install --upgrade pip
(env) $ pip install -e ".[test]"
(env) $ pip install --upgrade pip poetry
(env) $ poetry install --with test

(env) $ python -m pytest tests # Optionally, Perform tests
```

Note that it is generally recommended to install poetry into it's own standalone venv via the `pipx` cli tool.

## Documentation

https://ezmsg.readthedocs.io/en/latest/

`ezmsg` is very similar to [`labgraph`](https://www.github.com/facebookresearch/labgraph), so you might get a primer with their documentation and examples. Additionally, there are many examples provided in the examples/tests directories strewn throughout this repository.

## Extensions

`ezmsg` extensions can be installed individually or all at once. To install all the extension packages in one go, you can use the following command:

```bash
pip install "ezmsg[all_ext]"
```

This will install all the available public extension packages for `ezmsg` that are listed in `pyproject.toml`.
If you prefer to install the extension packages individually, you can use the following command:

```bash
pip install "ezmsg[zmq,sigproc,...]"
```

Please note that the `ezmsg` package itself can still be installed without any additional extensions using `pip install ezmsg`.

See the extension directory for more details
* `ezmsg-sigproc` -- Timeseries signal processing modules
* `ezmsg-websocket` -- Websocket server and client nodes for `ezmsg` graphs
* `ezmsg-zmq` -- ZeroMQ pub and sub nodes for `ezmsg` graphs
* ... More to come!

- `ezmsg-sigproc` -- Timeseries signal processing modules
- `ezmsg-websocket` -- Websocket server and client nodes for `ezmsg` graphs
- `ezmsg-zmq` -- ZeroMQ pub and sub nodes for `ezmsg` graphs
- ... More to come!

Additionally, the following extensions are contained in external repositories:

- [ezmsg-panel](https://github.com/griffinmilsap/ezmsg-panel) -- Plotting tools for `ezmsg` that use [panel](https://github.com/holoviz/panel)
- [ezmsg-blackrock](https://github.com/griffinmilsap/ezmsg-blackrock) -- Interface for Blackrock Cerebus ecosystem (incl. Neuroport) using `pycbsdk`
- [ezmsg-unicorn](https://github.com/griffinmilsap/ezmsg-unicorn) -- g.tec Unicorn Hybrid Black integration for `ezmsg`
- [ezmsg-gadget](https://github.com/griffinmilsap/ezmsg-gadget) -- USB-gadget with HID control integration for Raspberry Pi (Zero/W/2W, 4, CM4)
- [ezmsg-openbci](https://github.com/griffinmilsap/ezmsg-openbci) -- OpenBCI Cyton serial interface for `ezmsg`
- [ezmsg-ssvep](https://github.com/griffinmilsap/ezmsg-ssvep) -- Tools for running SSVEP experiments with `ezmsg`
- [ezmsg-vispy](https://github.com/pperanich/ezmsg-vispy) -- `ezmsg` visualization toolkit using PyQt6 and vispy.

## Publications

A collection of academic papers, journals, and other publications that have cited or utilized `ezmsg` in research and development.
These publications provide insights into the practical applications and impact of `ezmsg` in various fields.

- [A click-based electrocorticographic brain-computer interface enables long-term high-performance switch-scan spelling](https://doi.org/10.21203/rs.3.rs-3158792/v1)
- [Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months](https://doi.org/10.1002/advs.202304853)

## Financial Support

`ezmsg` is supported by Johns Hopkins University (JHU), the JHU Applied Physics Laboratory (APL), and by the Wyss Center for Bio and Neuro Engineering.
Loading

0 comments on commit 7732124

Please sign in to comment.