Dictionary loading, RootMap forward declarations failed #30
Comments
|
Could it be this line ? This line depends on |
|
Nevermind, the issue was the |
|
I now seem to be failing in This occurs after the shared library has been loaded into cppyy, and I attempt to access a class/namespace using
The python stack trace |
|
Firstly, dictionaries may not give you the hoped for speed-up. Upstream has been touting modules for the longest time, but after never quite pulling it off (or getting the promised speed-up), they now tell me to change the scripts to allow adding headers to the PCH (you already can, it's just manual). A big problem with modules is that they have a custom version (for I/O and to be redistributable) rather than using the native Clang ones. They have another trick up their sleeves: a parser that only picks up the declarations, not the definitions. That would be game changing. They say it's available, but I haven't seen it work. Second, the dictonaries will still either contain references to headers to be parsed or the headers as strings (is an option, but doesn't work on Windows b/c it is has a size limit on literal strings). The only speedup that might be seen then, is that not everything is loaded at startup, with the class loader ensuring only loading what is actually used. Third, what I really would like to see, is multiple interpreters and priority-based linkage. There's nothing stopping me from doing that already, just that the interpreter right now is coded upstream as a global variable. As for the crash, there's no useful information in the stack trace. Probably deep inside Clang (which is linked statically into |
|
Thanks for the info, I'll give Cppyy 3.5 a shot. |
|
Using Cppyy 3.5 didn't resolve the issue, the issue seems to be within Clang like you said, do I still need to pass the include paths to Cppyy ?
|
|
Ouch. But I don't understand ... The include paths will be backed into the dictionary; just have to make sure they're relative (and you can add other search paths it as well). |
|
The paths are not relative. is that the issue ? EDIT: The root map file has relative headers, however the |
|
Sounds good. The relative headers is purely to allow redistribution of binaries. Yes, they then got resolved, so makes sense that |
|
Is there a full working example of creating a shared library, generating a dictionary and loading it into Cppyy ?, I feel like I'm missing something fundamental here. |
|
The docs has an example: https://cppyy.readthedocs.io/en/latest/utilities.html#dictionaries Most of it is pretty simple/straightforward, though. There really shouldn't be all that much to it: it's a vehicle for putting everything (the headers, include paths, classes, as well as linking with any dependencies) in a single package. But in the end, it's still Cling parsing the same headers and loading the same libraries if done directly in Python code. |
|
It seems it should "just work" which is a shame. |
|
I managed to compile Clang and Cling as debug and managed to get the following stack trace
With the following assertion being raised |
|
Turns that an error occurs during
|
|
The shadows container within |
|
Yes, it's not until an object is used when the headers are parsed. (I'll add that |
|
Currently all the headers used to generate the dictionary are all working via cppyy.include(...), albeit very slowly.... |
|
I'll ask whether there's anything that they know of that could cause this. |
|
Could it be the type alias referenced in #29, it's nested within a class and was previously causing issues |
|
That failure itself is in meta, which would be called until parsing completed successfully. But otherwise yes, it's a |
|
I checked, and for so it definitely take a different path through the code. |
|
Can you try setting: before loading anything? The above is a RAII, so auto-parsing is (should be) disabled until that variable is deleted. |
That worked, however I now have a trivial issue, |
|
Upstream asks whether using a |
I'll try that and provide a reproducible example if I can later today. |
I'm most likely going to drop attempting to generate a dictionary, given the problems I've encountered trying to generate a dictionary for the existing code base. My usual approach to loading the headers using 'cppyy.include' works perfectly, however this is incredibly slow. Is there a different approach that will speed up the parsing of the headers?, is it possible to inject 3rd party header only (Which I suspect are the culprits leading to the slow-down) libraries into the PCH ? |
|
There's a file called However, if you give it a try, you at least will have an indication of what type of speedup is theoretically possible. |
Thank you, I'll give this a try when I have time. |
|
Something like this: compiler-research/CppInterOp#302 (comment) may eventually lead to using multiple PCHs and faster parsing/instantiation times. Note that there is a solution to multiple PCHs, namely modules. However, Cling doesn't use Clang native modules and does not track versioning or compiler options, so as a practical matter I've never moved to them. Furthermore, modules do not solve the problem that multiple packages all place their code in the same global namespace, so over time, parsing gets slower and slower (in particular template instantiations). |
Hey,
I'm attempting to load my shared library and rootmap, however I get
cling::Interpreter::kFailurewhen[libCling.so] CppyyLegacy::TCling::LoadLibraryMapattempts to declare the forward declarations from the*.rootmapfile.I'll be honest, I don't really understand whats happening here, my goal is move away for loading header files at startup as this takes around 30-60 seconds, I was hoping that using the dictionary will speed things up. Do I even need a LinkDef.h file ?, it just contains
#pragma link C++ defined_instatements for every header inTARGET_INTERFACE_HEADERSanyway.My call to root is as follows
[libCling.so] CppyyLegacy::TCling::LoadLibraryMap TCling.cxx:5330uniqueString: #line 1 "Forward declarations from ./libalgorithms.rootmap" namespace deep_learning { }namespace signals { namespace clustering { } }namespace statistics { namespace distributions { namespace distance { } } }namespace tracking { namespace data_streams { } }namespace common { }namespace common { template <typename T> class TypedColumn; }namespace common { template <typename T> class TypedColumnView; }namespace tracking { namespace modelling { namespace spatial { } } }namespace tracking { namespace modelling { } }namespace gating { }namespace signals { }namespace signals { namespace kalman { } }namespace signals { namespace classification { namespace generic { } } }namespace signals { namespace classification { namespace beam { } } }namespace profiling { namespace metadata { } }namespace tracking { namespace metadata { } }namespace statistics { }namespace assignment { }template <const char*, typename RecordType> class Streamable;template <> class Streamable<tracking::data_streams::best_estimates,std::vector<utils::StrongId<unsigned long,data_aggregator::EntityIdTag>,std::pmr::polymorphic_allocator<utils::StrongId<unsigned long,data_aggregator::EntityIdTag>>>>;namespace tracking { namespace data_records { } }namespace tracking { namespace data_records { class Hypothesis; } }template <> class Streamable<tracking::data_streams::hypotheses,tracking::data_records::Hypothesis>;namespace tracking { namespace data_records { class MatchingScores; } }template <> class Streamable<tracking::data_streams::matching_scores,tracking::data_records::MatchingScores>;namespace tracking { namespace data_records { class Measurement; } }template <> class Streamable<tracking::data_streams::measurements,tracking::data_records::Measurement>;namespace tracking { namespace filter { } }namespace tracking { namespace filter { namespace local_hypothesis { } } } #line 1 "Forward declarations from /opt/project/core/algorithms/target/libalgorithms.rootmap"The text was updated successfully, but these errors were encountered: