added link_data --> clear_cache relationship to support repacking zarr nwbfiles

[pauladkisson]

Motivation

See catalystneuro/neuroconv#1003

Checklist

• Did you update CHANGELOG.md with your changes?
• Have you checked our Contributing document?
• Have you ensured the PR clearly describes the problem and the solution?
• Is your contribution compliant with our coding style? This can be checked running ruff from the source directory.
• Have you checked to ensure that there aren't other open Pull Requests for the same change?
• Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

[pauladkisson]

@oruebel, can you enable tests for this PR as well?

[pauladkisson]

@rly @oruebel CI tests pls?

[pauladkisson]

@rly @oruebel CI tests pls?

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.53%. Comparing base (46bdb77) to head (cd235b0).

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #215      +/-   ##
==========================================
- Coverage   87.13%   85.53%   -1.61%     
==========================================
  Files           5        5              
  Lines        1174     1182       +8     
  Branches      287      291       +4     
==========================================
- Hits         1023     1011      -12     
- Misses        100      115      +15     
- Partials       51       56       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pauladkisson]

Looks like this is passing all the tests, can we merge?

@rly @oruebel

[oruebel]

@rly @mavaylon1 can you please review. Export is more your area of expertise

[mavaylon1]

@pauladkisson @oruebel @rly I can take a gander this week.

[pauladkisson]

@mavaylon1, Still waiting for that gander...has it flown off-course? 🪿

[mavaylon1]

@pauladkisson Said gander has indeed flown off into the wilderness due to SFN. Let me review this tomorow.

[pauladkisson]

@mavaylon1, giving this a bump.

[mavaylon1]

So I am taking a look but I need more context. can you walk me through it since there are multiple links pointing elsewhere. I'll continue looking at this right now, but let me know as well @pauladkisson

You are repacking zarr files but then you added "data is an h5py.Dataset" so I assume you are exporting at some point.

Also in general it is better to be more explicit with variable names. Can you be more explicit instead using "o" and "c".

[pauladkisson]

So I am taking a look but I need more context. can you walk me through it since there are multiple links pointing elsewhere.

Sure! I am working on a function to repack NWB files with more appropriate chunking/compression. But in order to do so, I need an option to redo chunking/compression when exporting -- hence the link_data -> clear_cache relationship.

Another related bug that pops when I enable this option: some datasets with strings are normally already decoded when writing from scratch, but when exporting, they need to be decoded again.

[pauladkisson]

Also in general it is better to be more explicit with variable names. Can you be more explicit instead using "o" and "c".

This code is pretty much exact copy-paste from L1290-1300, so should be just as readable.

[mavaylon1]

Also in general it is better to be more explicit with variable names. Can you be more explicit instead using "o" and "c".

This code is pretty much exact copy-paste from L1290-1300, so should be just as readable.

Yeah both places aren't ideal, but I understand why you kept the style. I thought we cleaned that up already. All good though.

[mavaylon1]

@pauladkisson i am working on a change for exporting, but I think this PR should be good to go regardless.

[pauladkisson]

@pauladkisson i am working on a change for exporting, but I think this PR should be good to go regardless.

Fantastic! Thanks for pushing this through @mavaylon1.

[pauladkisson]

FYI I'm not authorized to merge.