Load .mrc using Distributed and with Direct Electron Metadata

[CSSFrancis]

Description of the change

This adds support for loading .mrc files using the dask-distributed backend and memory mapping. This is helpful for loading large files using multiple threads split across many machines.

The methods in the _distributed file should be fairly extendable and can be applied to other binary files which use the np.memmap function.

Changes:

• Added memmap_distributed function
• Add load metadata.txt file from DirectElectron
• Added distributed and chunks kwargs to the .mrc file loader

Progress of the PR

• Change implemented (can be split into several points),
  • Add memmap_distributed function
  • Add load metadata.txt file from DirectElectron
• add tests
  • Test loading using distributed option
  • Test using metadata.txt file (Need to make small test files with different sizes)
• update docstring (if appropriate),
• update user guide (if appropriate),
• add a changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
• Check formatting of the changelog entry (and eventual user guide changes) in the docs/readthedocs.org:rosettasciio build of this PR (link in github checks)
• ready for review.

Minimal example of the bug fix or the new feature

from rsciio.mrc import file_reader
file_reader("your_mrc_file.msa", 
                    distributed=True,  # can use `dask-distributed` if wanted 
                    lazy=True, # If lazy= False will still use distributed loading but then compute
                    metadata_file = "de_metadata.txt",  # Direct Electron saved  metadata file 
                    )

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (13507d4) 86.02% compared to head (5a7eb28) 86.12%.

Files	Patch %	Lines
rsciio/mrc/_api.py	98.75%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #162      +/-   ##
==========================================
+ Coverage   86.02%   86.12%   +0.10%     
==========================================
  Files          81       82       +1     
  Lines       10387    10471      +84     
  Branches     2253     2274      +21     
==========================================
+ Hits         8935     9018      +83     
  Misses        931      931              
- Partials      521      522       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[CSSFrancis]

pre-commit.ci autofix

[CSSFrancis]

@ericpre so it seems like the Packaging CI seems to fail when adding a new file. Is that correct? It seems like it is trying to pull the file from the registered list to package (and then subsequently prune). Is this something worth trying to fix? Or just wait until the file is merged into main.

[ericpre]

Yes, the packaging failure is normal, it will pass once the PR is merged.