Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[on hold] Support for sub arrays of dynamic size (jagged arrays) #305

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

bernhardmgruber
Copy link
Member

@bernhardmgruber bernhardmgruber commented Jun 23, 2021

This PR adds support for fields of dynamic array type. Specifically:

  • allow dynamic field types in the record dimension
  • add specializations to most of the core functions
  • add llama::dynamic to signal a dynamic array member in a RecordCoord
  • extend VirtualRecord to allow holding dynamic indices
  • extend blobNrAndOffset to allow for additional dynamic indices
  • add OffsetTable mapping
  • add customization allowing to dump OffsetTable mappings
  • add a few unit tests

@bernhardmgruber bernhardmgruber force-pushed the jagged_array branch 12 times, most recently from cc4b647 to dd45ecc Compare June 29, 2021 13:35
@bernhardmgruber bernhardmgruber force-pushed the jagged_array branch 6 times, most recently from bf0f7fc to 641b014 Compare July 7, 2021 15:39
@bernhardmgruber
Copy link
Member Author

Given a small HEP style data structure:

    struct run {};
    struct luminosityBlock {};
    struct Electrons {};
    struct Muons {};
    struct Eta{};
    struct Mass{};
    struct Phi{};

    using Electron = llama::Record<
        llama::Field<Eta, float>,
        llama::Field<Mass, float>,
        llama::Field<Phi, float>
    >;
    using Muon = llama::Record<
        llama::Field<Eta, float>,
        llama::Field<Mass, float>,
        llama::Field<Phi, float>
    >;
    using Event = llama::Record<
        llama::Field<run, std::int32_t>,
        llama::Field<luminosityBlock, std::int32_t>,
        llama::Field<Electrons, Electron[]>,
        llama::Field<Muons, Muon[]>
    >;
....
    auto mapping = llama::mapping::OffsetTable<llama::ArrayDims<1>, Event>{
        llama::ArrayDims{30},
        llama::ArrayDims{50},
        llama::ArrayDims{40}};

Here is a dump of the mapping using AoS for the sub mappings:
image
Using SoA for the sub mappings:
image
Using Aos for the events and SoA for the electrons and muons:
image

@bernhardmgruber bernhardmgruber force-pushed the jagged_array branch 2 times, most recently from 60c4dca to 7ab012e Compare July 7, 2021 16:47

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* allow dynamic field types in the record dimension
* add specializations to most of the core functions
* add llama::dynamic to signal a dynamic array member in a RecordCoord
* extend VirtualRecord to allow holding dynamic indices
* extend blobNrAndOffset to allow for additional dynamic indices
* add OffsetTable mapping
* add customization allowing to dump OffsetTable mappings
* add a few unit tests
@bernhardmgruber bernhardmgruber changed the title Support for sub arrays of dynamic size Support for sub arrays of dynamic size (jagged arrays) Mar 16, 2023
@bernhardmgruber bernhardmgruber changed the title Support for sub arrays of dynamic size (jagged arrays) [on hold] Support for sub arrays of dynamic size (jagged arrays) Sep 27, 2023
@costeastefan
Copy link

I am interested in this. Any ETA available?

@bernhardmgruber
Copy link
Member Author

I am interested in this. Any ETA available?

Hi! There is no ETA on this feature, since nobody is working on it. However, you can model jagged arrays yourself by creating replacing a dynamic sub array by an integral offset and creating a separate view for the dynamic sub array's structural type. This also corresponds to the implementation for jagged arrays in ROOT or Awkward Array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants