Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No module named 'tensorrt_llm.bindings'` error message #2656

Open
maulikmadhavi opened this issue Jan 3, 2025 · 1 comment
Open

No module named 'tensorrt_llm.bindings'` error message #2656

maulikmadhavi opened this issue Jan 3, 2025 · 1 comment
Labels
triaged Issue has been triaged by maintainers

Comments

@maulikmadhavi
Copy link

Installing using [pixi](https://pixi.sh/latest/) library management tool. The pixi.toml manifest file looks like this.

[project]
description = "Add a short description here"
name = "TensorRT-LLM"
platforms = ["linux-64"]
version = "0.1.0"
channels = ["nvidia", "conda-forge", {channel = "pytorch", priority = -1}]

[pypi-options]
index-url = "https://pypi.nvidia.com"
extra-index-urls = ["https://pypi.org/simple"]

[dependencies]
python = "==3.12.3"
mpi4py = ">=4.0.1,<5"

[pypi-dependencies]
torch = { version = "==2.5.1+cu118", index = "https://download.pytorch.org/whl/cu118/" }
tqdm = ">=4.66.5, <5"
torchvision = { version = "==0.20.1+cu118", index = "https://download.pytorch.org/whl/cu118/" }
nvidia-cutlass = ">=3.6.0.0, <4"
tensorrt_cu12_libs = { version = ">=10.7.0", index = "https://pypi.nvidia.com" }
tensorrt_cu12_bindings = { version = ">=10.7.0", index = "https://pypi.nvidia.com" }
tensorrt = { version = ">=10.7.0", index = "https://pypi.nvidia.com" }
tensorrt-llm = {version = "==0.16.0",  index = "https://pypi.nvidia.com" }
onnx = ">=1.17.0, <2"
huggingface-hub = ">=0.27.0, <0.28"

Upon running pixi shell can build the environment setup is successful. However, on checking import tensorrt_llm, it failed with ModuleNotFoundError: No module named 'tensorrt_llm.bindings' error message.

Can anyone suggest me with any missing inputs or anything?

Thanks
Maulik

@nv-guomingz
Copy link
Collaborator

#1194
Hi @maulikmadhavi Please refer to above issue link. We don't verify the pixi yet so we still recommend u follow the doc to install/run trt-llm

@nv-guomingz nv-guomingz added the triaged Issue has been triaged by maintainers label Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triaged Issue has been triaged by maintainers
Projects
None yet
Development

No branches or pull requests

2 participants