Do not manage to read a TTree with a structure of arrays of basic types #298
Comments
|
@tamasgal this thing hits fID equals -2, I think we're missing something fundamental here |
|
Actually the only missing thing in this case is the leaf type support for With #299 the following works (without, you will fail reading the julia> using UnROOT
julia> f = ROOTFile("/Users/tamasgal/Downloads/Output_REC.root")
ROOTFile with 3 entries and 51 streamers.
/Users/tamasgal/Downloads/Output_REC.root
├─ runs (TTree)
│ └─ "PARAMETERS"
├─ events (TTree)
│ ├─ "AllCaloHitContributionsCombined"
│ ├─ "_AllCaloHitContributionsCombined_particle"
│ ├─ "BeamCal_Hits"
│ ├─ "⋮"
│ ├─ "YokeEndcapCollection"
│ ├─ "_YokeEndcapCollection_contributions"
│ └─ "PARAMETERS"
└─ podio_metadata (TTree)
├─ "events___idTable"
├─ "events___CollectionTypeInfo"
├─ "runs___idTable"
├─ "runs___CollectionTypeInfo"
├─ "PodioBuildVersion"
└─ "EDMDefinitions"
julia> LazyBranch(f, "podio_metadata/events___idTable/m_names")
1-element LazyBranch{SubArray{String, 1, Vector{String}, Tuple{UnitRange{Int64}}, true}, UnROOT.Offsetjagg, ArraysOfArrays.VectorOfVectors{String, Vector{String}, Vector{Int32}, Vector{Tuple{}}}}:
["AllCaloHitContributionsCombined", "EventHeader", "BeamCalClusters", "BeamCalClusters_particleIDs", "BeamCalCollection", "BeamCalRecoParticles", "BeamCalRecoParticles_particleIDs", "BeamCal_Hits", "BuildUpVertices", "BuildUpVertices_RP" … "TightSelectedPandoraPFOs", "InnerTrackerBarrelHitsRelations", "InnerTrackerEndcapHitsRelations", "OuterTrackerBarrelHitsRelations", "OuterTrackerEndcapHitsRelations", "RefinedVertexJets_rel", "RelationCaloHit", "RelationMuonHit", "VXDEndcapTrackerHitRelations", "VXDTrackerHitRelations"]
julia> LazyBranch(f, "podio_metadata/events___idTable/m_collectionIDs")
1-element LazyBranch{SubArray{UInt32, 1, Vector{UInt32}, Tuple{UnitRange{Int64}}, true}, UnROOT.Offsetjagg, ArraysOfArrays.VectorOfVectors{UInt32, Vector{UInt32}, Vector{Int32}, Vector{Tuple{}}}}:
UInt32[0x3a25675d, 0xd793ab91, 0xf0d073dd, 0x1d19206c, 0xc298a348, 0xc29370d2, 0x3954b563, 0xd2b19e7b, 0xfd03f5d0, 0x310a0f04 … 0x5fa7cf93, 0x029be193, 0x743732ae, 0xc42bbbee, 0xd1211017, 0x8dac6bb6, 0x603a5016, 0xdf24625a, 0xbb4cff22, 0x178c9330]
julia> LazyTree(f, "podio_metadata", [Regex("events___idTable/(.*)") => s"\1"])
Row │ m_names m_collectionIDs ⋯ │ SubArray{String SubArray{UInt32 ⋯─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ ["AllCaloHitContributionsCombined", "EventHeader", "BeamC [975529821, 3616779153, 4040192989, 488185964, ⋯ 1 column omitted
|
|
Fixed in @peremato let me know if it works for you. Btw. just a little bit of clarification: the custom parsing always applies to a branch and not a tree (or set of branches). It's usually needed when the split-level is low (so that one needs to deserialise compound structures) or if the type for a specific branch is simply not supported. |
|
huh, I don't know why this doesn't error due to |
|
How did you get the |
|
Ah I see: julia> UnROOT.LazyTree(f, "podio_metadata", ["events___idTable"])
fID = -2 # <- added a @show here...
ERROR: BoundsError: attempt to access 2-element Vector{Any} at index [-1]
Stacktrace:
[1] getindex(A::Vector{Any}, i1::Int64)
@ Base ./essentials.jl:13
[2] streamerfor(f::ROOTFile, branch::UnROOT.TBranchElement_10)
@ UnROOT ~/Dev/UnROOT.jl/src/root.jl:161Yes, that negative EDIT: and yes, if you go to the deepest split level and there is an interpretation (like the one for |
|
In this case the It all boils down to take the automatic parser generation into this level so that it works without using the split-branches. I need to study what |
|
yeah, from my very quick look, |
|
Yes... I mean, obviously the information is sitting right in front of us ;) So in that case |
|
It's just a bit weird that this works fine in so many cases 😆 : Line 160 in 77b75d8 |
First, thanks very much @tamasgal. It works great once you know how to do it. It is very confusing still for me the way to select the branches and leaves (perhaps is a lack of proper documentation or pre-knowledge of the ROOT file organisation). This works nicely: but what I would do naively does not the flowing works but the names of the columns are wrong I did also try the naming convention that was used for the other tree "events" with |
|
Yes, the problem is indeed that you need to know a little bit about the ROOT structure's subtleties. As you can see, I really hope that I will find a longer time slot (2-4 weeks) next year to spend a significant amount of time on refactoring UnROOT. |
|
Regarding the julia> LazyTree(f, "events", [r"BeamCal_Hits/BeamCal_Hits.*\.(\w+)$" => s"\1"])
Row │ time x energyError energy y ⋯
│ SubArray{Float3 SubArray{Float3 SubArray{Float3 SubArray{Float3 Sub ⋯
─────┼──────────────────────────────────────────────────────────────────────────
1 │ [] [] [] [] [] ⋯
2 │ [] [] [] [] [] ⋯
3 │ [] [] [] [] [] ⋯
4 │ [] [] [] [] [] ⋯
5 │ [] [] [] [] [] ⋯
6 │ [] [] [] [] [] ⋯
7 │ [] [] [] [] [] ⋯
8 │ [] [] [] [] [] ⋯
9 │ [] [] [] [] [] ⋯
10 │ [] [] [] [] [] ⋯
11 │ [] [] [] [] [] ⋯
12 │ [] [] [] [] [] ⋯
13 │ [0.0, 0.0, [-8.2, -8. [0.0, 0.0, [0.0267, 0 [63 ⋯
14 │ [] [] [] [] [] ⋯
15 │ [] [] [] [] [] ⋯
16 │ [] [] [] [] [] ⋯
17 │ [] [] [] [] [] ⋯
18 │ [] [] [] [] [] ⋯
19 │ [0.0, 0.0] [3.17, 3.2 [0.0, 0.0] [0.0305, 0 [-1 ⋯
20 │ [] [] [] [] [] ⋯
21 │ [] [] [] [] [] ⋯
22 │ [0.0, 0.0] [151.0, 15 [0.0, 0.0] [0.0128, 0 [-8 ⋯
⋮ │ ⋮ ⋮ ⋮ ⋮ ⋱
4 columns and 3 rows omitted
|
|
I was not doing this. If I do and the leaves get the name |
|
I mean, technically we can do this LazyTree creation on the fly automatically but I could not come up with a way which works reliably, especially with all those funny (read weird) namings and dot-madness. So eventually we need to ask the user to provide the regex to help Actually that is basically the place where we would need to incorporate the original streamer which tells you how to name them and how the hierarchy is structures, but it's quite complex and ...painful fact: if you let That's why I kind of like the that we simply use On the other hand, you can of course provide your custom structs and make I use both techniques with great performance. |
Yes that works too, if you are fine with the UnROOT naming ;) |
|
Hi Tom. I agree we can do several things and hide the UnROOT level. I you want have a look at what I have been doing with EDM4hep.jl. I am mapping a simple Julia type (isbits) to a set of columns in the LazyTree within a StructArray in a recursive manner. This is very convenient and good performance for some use cases. There are some examples like |
EDM4hep root files store in a tree called
podio_metadataan object of the typeThe following is a reproducer:
The test file can be downloaded from https://github.com/peremato/EDM4hep.jl/blob/main/examples/Output_REC.root
The text was updated successfully, but these errors were encountered: