Broken environment due to numpy version

[Kirill888]

Describe the bug

Running the following image in the sandbox:

543785577597.dkr.ecr.af-south-1.amazonaws.com/geoscienceaustralia/sandbox:1.0.7

import numba

raises the following:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/tmp/ipykernel_103/2257432204.py in <module>
----> 1 import numba

/env/lib/python3.8/site-packages/numba/__init__.py in <module>
    198 
    199 _ensure_llvm()
--> 200 _ensure_critical_deps()
    201 
    202 # we know llvmlite is working as the above tests passed, import it now as SVML

/env/lib/python3.8/site-packages/numba/__init__.py in _ensure_critical_deps()
    138         raise ImportError("Numba needs NumPy 1.18 or greater")
    139     elif numpy_version > (1, 21):
--> 140         raise ImportError("Numba needs NumPy 1.21 or less")
    141 
    142     try:

ImportError: Numba needs NumPy 1.21 or less

Running pip check reports issues:

numba 0.55.1 has requirement numpy<1.22,>=1.18, but you have numpy 1.22.2.
datacube-ows 1.8.26 has requirement pyparsing<3,>=2.2.1, but you have pyparsing 3.0.7.
awscli 1.19.106 has requirement PyYAML<5.5,>=3.10, but you have pyyaml 6.0.

I see that you have this pin:

https://github.com/digitalearthafrica/dea-sandbox/blob/604aa17470dc7decb942d855c3abe2f70b82dc2a/docker/requirements.txt#L905

but that's not the version that is installed. A newer version of numpy is installed and, it's installed using root user, so normal user can not override it even temporarily. scipy, astropy, hdstats are also installed as root, so I'm guessing there is somewhere pip install that runs afterwards, runs as root and updates numpy while breaking numba dependency.

find /env -type d -uid 0 -name "*-info"
/env/lib/python3.8/site-packages/PyYAML-6.0.dist-info
/env/lib/python3.8/site-packages/astropy-5.0.1.dist-info
/env/lib/python3.8/site-packages/hdstats-0.1.8.post1.dist-info
/env/lib/python3.8/site-packages/numpy-1.22.2.dist-info
/env/lib/python3.8/site-packages/packaging-21.3.dist-info
/env/lib/python3.8/site-packages/pyerfa-2.0.0.1.dist-info
/env/lib/python3.8/site-packages/pyparsing-3.0.7.dist-info
/env/lib/python3.8/site-packages/scipy-1.8.0.dist-info

other libraries depend on numba and won't work because of that issue:

pip show numba
Name: numba
Version: 0.55.1
Summary: compiling Python code using LLVM
Home-page: https://numba.pydata.org
Author: 
Author-email: 
License: BSD
Location: /env/lib/python3.8/site-packages
Requires: llvmlite, numpy, setuptools
Required-by: dask-ml, datashader, quantecon, rvlib, segregation

To Reproduce
Steps to reproduce the behavior.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

[dominicschaff]

Hi @Kirill888, this repository is for a new version of the sandbox image in which we are replacing and upgrading the libraries.

For reference the work in progress image can be found here: https://hub.docker.com/r/digitalearthafrica/deafrica-sandbox

I have now checked the new image to make sure it doesn't have the above issue as well, and I did need to correct some of the requirements.

You can see in the following screenshot that both libraries are now able to be imported, and their respective version.

I will keep this issue open until we deploy this new image