Installing torchvision for ppc64le IBM architecture

[brando90]

I am in a weird scenario were I am forced to use torch 1.3.1 (due to hardware see: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/). I read from the pytorch docs that it's corresponding version of torchvision is 0.4.1 (https://pypi.org/project/torchvision/):

Installation

We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

torch	torchvision	python
master / nightly	master / nightly	>=3.6
1.5.0	0.6.0	>=3.5
1.4.0	0.5.0	==2.7, >=3.5, <=3.8
1.3.1	0.4.2	==2.7, >=3.5, <=3.7
1.3.0	0.4.1	==2.7, >=3.5, <=3.7
1.2.0	0.4.0	==2.7, >=3.5, <=3.7
1.1.0	0.3.0	==2.7, >=3.5, <=3.7
<=1.0.1	0.2.2	==2.7, >=3.5, <=3.7

but for some reason I have the wrong version of it:

torchvision 0.2.2 pypi_0 pypi

is there a way to install the right version of torchvision?

---

What I've tried:

First I tried force installing the right version with conda. Conda couldn't find the version of torchvision that I need:

$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Then I proceeded to try to install it regardless with pip

$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2

got an error too.

Is there anything else to try?

---

I tried but it failed:

conda install torchvision==0.4.2 -c pytorch

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://conda.anaconda.org/pytorch/linux-ppc64le
  - https://conda.anaconda.org/pytorch/noarch
  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

---

related:

• crossposted SO: https://discuss.pytorch.org/t/force-installing-torchvision/97279
• crossposted reddit pytorch: https://www.reddit.com/r/pytorch/comments/iyf2qn/force_installing_torchvision/
• crossposted reddit ibm: https://www.reddit.com/r/IBM/comments/iyhzex/force_installing_torchvision_042_when_i_am_forced/
• real problem is installing torchmeta: help installing torchmeta (for ppc64le IBM architecture) tristandeleu/pytorch-meta#95
• https://www.ibm.com/mysupport/s/forumsquestion?id=0D50z00006gaxV9CAI
• quora: https://www.quora.com/unanswered/How-does-one-install-specific-Python-packages-in-Conda-from-IBM-architectures
• reddit ibm2: https://www.reddit.com/r/newIBM/comments/iyij10/force_installing_torchvision_042_when_i_am_forced/
• gitissue for ibm: Installing torchvision for ppc64le architecture IBM/powerai#268

cc @fmassa @vfdev-5

[brando90]

IBM/powerai#268

[zou3519]

The torchvision issue tracker might be a better place for this: https://github.com/pytorch/vision/issues

[vfdev-5]

@brando90 interesting issue. I just tried in a clean conda env (x86 linux) to install both pytorch and torchvision :

conda install pytorch=1.3.1 torchvision -c pytorch

and it proposes to install the correct version:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cffi-1.14.3                |   py36he30daa8_0         222 KB
    intel-openmp-2020.2        |              254         786 KB
    mkl-2020.2                 |              256       138.3 MB
    mkl_fft-1.2.0              |   py36h23d657b_0         149 KB
    ninja-1.10.1               |   py36hfd86e86_0         1.4 MB
    numpy-1.19.1               |   py36hbc911f0_0          21 KB
    numpy-base-1.19.1          |   py36hfa32c7d_0         4.1 MB
    olefile-0.46               |           py36_0          48 KB
    pillow-7.2.0               |   py36hb39fc2d_0         619 KB
    pycparser-2.20             |             py_2          94 KB
    pytorch-1.3.1              |py3.6_cuda10.1.243_cudnn7.6.3_0       428.0 MB  pytorch
    torchvision-0.4.2          |       py36_cu101        14.8 MB  pytorch

Can you try to run the command with pytorch=1.3.1 which normally shouldn't reinstall existing pytorch ?

EDIT: Probably, the issue is with linux-ppc64le architecture and the only noarch binary is 0.2.2 according to anaconda: https://anaconda.org/pytorch/torchvision

Can you try to build it from sources, from github branch v0.4.2 : https://github.com/pytorch/vision/tree/v0.4.2 ?

[pmeier]

@vfdev-5 Your hunch is correct. As of torchvision==0.3 the provided binaries are precompiled for a few selected platforms. Thus, if you try to install with an unsupported platform it falls back to the last version without platform requirements, i. e. torchvision==0.2.2.

[owlesia]

In order to install the correct version of torchvision on PPC64le I found it easy to just install the IBM MLDL framework which includes correct versions of pytorch and torchvision-base:

conda install powerai

[escanore981]

PackagesNotFoundError: The following packages are not available from current channels: powerai