Skip to content

Commit

Permalink
docker: add migration page
Browse files Browse the repository at this point in the history
  • Loading branch information
ntarocco authored and zzacharo committed Mar 17, 2023
1 parent 082669a commit 8e94c28
Show file tree
Hide file tree
Showing 5 changed files with 102 additions and 4 deletions.
6 changes: 5 additions & 1 deletion docs/install/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ InvenioRDM depends on the following requirements to be installed on your local s
- In case that `invenio-cli` (and other commands installed via `pip`) cannot be found after installing, you may have to update your `$PATH` to include the install directory (e.g. `PATH="$PATH:$HOME/.local/bin"` on Linux).
- [Docker](https://docs.docker.com/) 1.13.0+
- [Docker](https://docs.docker.com/) 20.10.10+
- [Docker-Compose](https://docs.docker.com/compose/) 1.17.0+
For running and building the application locally you will also need:
Expand All @@ -32,6 +32,10 @@ For running and building the application locally you will also need:
!!! warning "Supported Python distributions"
InvenioRDM targets CPython 3.9. Anaconda Python in particular is not currently supported and other Python distributions are not tested.
!!! note "ARM-based CPUs"
If you are developing locally with an ARM-based CPU, notably a recent Apple M1/M2 Mac, the minimum support version of InvenioRDM is v10.
Previous versions cannot be installed because of a `node-sass` incompatibility.
### Services
InvenioRDM depends on the following services. During the installation we start these services in containers, but you could as well use externally hosted services for them:
Expand Down
7 changes: 4 additions & 3 deletions docs/maintenance/docker-images.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# RDM Docker Images
# RDM Docker images

The official RDM Docker image(s) are available in the [docker-invenio](https://github.com/inveniosoftware/docker-invenio) repo.

Expand All @@ -14,8 +14,9 @@ the current RDM Docker image.

Docker images are pushed to the CERN registry [inveniosoftware]](https://registry.cern.ch/harbor/projects/1825/repositories) project. This is done to:

1. Avoid [Docker Hub rate limits](https://www.docker.com/increase-rate-limits/).
2. Take advantage of the automatic security scan provided by Harbor.
1. March 2023: Docker announced that Free Team organizations, such as `inveniosoftware`, will be removed.
2. Avoid [Docker Hub rate limits](https://www.docker.com/increase-rate-limits/).
3. Take advantage of the automatic security scan provided by Harbor.

The project is configured with tag retention policies, so that the disk space is not filled up too quickly (see below).

Expand Down
87 changes: 87 additions & 0 deletions docs/releases/upgrading/migrate-docker-images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Migrate Docker images

Docker terminated the Free Team organizations subscription on April 14, 2023. This means that the `inveniosoftware` on Docker Hub, a Free Team organization, will no longer exist.

## Impact on InvenioRDM

The base Docker images for InvenioRDM v9.x and v10.x were hosted on Docker Hub, under the `inveniosoftware` organization. When Docker will remove the organization, such images might not be available anymore. In a [second announcement](https://www.docker.com/blog/we-apologize-we-did-a-terrible-job-announcing-the-end-of-docker-free-teams/), Docker clarified that:

> ...public images will only be removed from Docker Hub if their maintainer decides to delete them.
Depending on what will happen, building the Docker images on InvenioRDM v9/v10 with the commands `invenio-cli containers ` might fail.

With InvenioRDM v11, released on January 26, 2023, we have migrated the Docker images to the CERN registry, see [here the release notes](../versions/version-v11.0.0.md#deployment). This was done for a couple of reasons:

1. Avoid [Docker Hub rate limits](https://www.docker.com/increase-rate-limits/).
2. Take advantage of the automatic security scan provided by the CERN registry.

With this new Docker announcement, it is hard to trust them in keeping such images available and free to use. Hosting images on CERN registry is now necessary.

## Migration

### Am I impacted?

If you are running InvenioRDM v9 or v10, you might be impacted at some point.

Independently of the InvenioRDM version that you are running, you can check if you are impacted by opening the `Dockerfile` file in your instance and by checking the first command `FROM`.

⚠️ These lines might fail:

```
FROM inveniosoftware/centos7-python:3.7
```
```
FROM inveniosoftware/centos8-python:3.8
```
```
FROM inveniosoftware/centos7-python:3.9
```

✅ These lines will work:

```
FROM registry.cern.ch/inveniosoftware/centos7-python:3.7
```
```
FROM registry.cern.ch/inveniosoftware/centos8-python:3.8
```
```
FROM registry.cern.ch/inveniosoftware/centos7-python:3.9
```
```
FROM registry.cern.ch/inveniosoftware/almalinux:1
```

### Should I act now?

It is not necessary to act immediately, but you might want to consider this migration in a near future. While Docker clarified with a second announcement that public images will still be available, it is not clear for how long and if they will be limited at some point.

### Migration

There are two possible ways to migrate:

1. Upgrade your InvenioRDM to v11. If you are running InvenioRDM v9, make sure that you upgrade first to v10, and then to v11. Be aware that InvenioRDM v11 is **not** a LTS version.
2. Change the Dockerfile to use `registry.cern.ch/inveniosoftware/*` instead of `inveniosoftware/*`.

```diff
- FROM inveniosoftware/centos7-python:3.9
+ FROM registry.cern.ch/inveniosoftware/centos7-python:3.9
```

or, using the latest AlmaLinux image:

```diff
- FROM inveniosoftware/centos7-python:3.9
+ FROM registry.cern.ch/inveniosoftware/almalinux:1
```

In this last case of AlmaLinux, make sure that you are running Python 3.9, which is the Python version that comes with the `almalinux` image. You should follow the [v11 Python version change](./upgrade-v11.0.md#python-version-change) instructions to make sure that you are taking the necessary steps.

!!! warning "Test!"
The change of the base Docker image can lead to unexpected issues. We recommend testing your instance extensively in a test environment, before deployment to production.

## About images registry

The CERN registry for the InvenioRDM images is available to everyone. However, it is not a service with a guaranteed uptime, as the Docker Hub registry.

You might want to explore the usage of proxy caches within your organization (for example, using the open source [Harbor](https://goharbor.io/) or [Nexus](https://www.sonatype.com/products/nexus-repository)): it will help you having the InvenioRDM images always available, to dramatically increase network speed (local network download vs internet download) and to eventually by-pass any organization's restriction in bandwidth or speed.
5 changes: 5 additions & 0 deletions docs/releases/upgrading/upgrade-v11.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ Update the base Docker image in the `Dockerfile`, and rebuild the image if neces
+++FROM registry.cern.ch/inveniosoftware/almalinux:1
```

!!! Note

Testing the Docker images locally requires Docker version `20.10.10+`.


**Local development**

Changing the Python version in your development environment highly
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ nav:
- Upgrade policy: releases/upgrading/index.md
- Upgrade from v10 to v11: releases/upgrading/upgrade-v11.0.md
- Upgrade from v9 to v10: releases/upgrading/upgrade-v10.0.md
- Migrate Docker images: releases/upgrading/migrate-docker-images.md
- Migrate v9 to OpenSearch: releases/upgrading/migrate-opensearch-v9.md
- Upgrade from v8 to v9: releases/upgrading/upgrade-v9.0.md
- Upgrade from v7 to v8: releases/upgrading/upgrade-v8.0.md
Expand Down

0 comments on commit 8e94c28

Please sign in to comment.