Pyinstaller with onnxruntime

[pjvazquez]

My installation is an Ubuntu 22.04 LTS + Python 3.12 + CUDA 12.6 + CUDnn 9 + ONNRUNTIME-GPU 1.19

To generate the bundle I run

pyinstaller command_line.py --name aps_cv --noconfirm --add-data aps_cv/config:aps_cv/config --add-data aps_cv/models:aps_cv/models --add-data aps_cv/static:aps_cv/static --add-data aps_cv/docs:aps_cv/docs --add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/onnxruntime/capi/*.so:lib --add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/nvidia/cudnn/lib/*.so.8:lib --add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/:lib  --upx-dir=/usr/local/cuda-12.6/targets/x86_64-linux/lib

and the app runs but without using the GOU due to the error

[E:onnxruntime:Default, provider_bridge_ort.cc:1992 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1586 void onnxruntime::ProviderSharedLibrary::Ensure() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_shared.so with error: libonnxruntime_providers_shared.so: cannot open shared object file: No such file or directory

Any idea on how to solve it

[rokm]

--add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/onnxruntime/capi/*.so:lib

For shared libraries that are bundled with packages (as seems to be the case with onnxruntime/capi/libonnxruntime_providers_shared.so), it is usually important that they are collected into correct location. In this case, it should likely be onnxruntime/capi instead of lib.

--add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/nvidia/cudnn/lib/*.so.8:lib

Same goes for CUDA libs installed via PyPI wheel.

--add-binary /home/pjvazquez/miniconda3/envs/aps2/lib/python3.12/site-packages/:lib

Why would you do this?

--upx-dir=/usr/local/cuda-12.6/targets/x86_64-linux/lib

And what is this supposed to be?