Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCI container creation at the LPC #18

Merged
merged 6 commits into from
Jun 28, 2021

Conversation

aperloff
Copy link

Given the presence of buildah, the containerize.sh script will create an OCI style image containing the current CMSSW release. The only prerequisites are (which are checked before running):

  1. Buildah is installed on the system
  2. The users id is given a set of subuids and subgids in /etc/subuid and /etc/subgid
  3. /cvmfs/cms.cern.ch is mounted on the host machine
  4. There is sufficient space to store the images

The basic workflow would be:

ssh -Y <username>@cmslpcheavy-sl7.fnal.gov
cd <path to cmssw>
cmsenv
<path to lpc_scripts>/containerize.sh -t <name:tag> -b <base_image>
podman run --rm -it <name>:<tag>

Alexx Perloff added 2 commits June 25, 2021 18:12
@aperloff aperloff requested review from kpedro88 and mtonjes June 25, 2021 23:27
containerize/Dockerfile Outdated Show resolved Hide resolved
containerize/Dockerfile Outdated Show resolved Hide resolved
containerize/Dockerfile Outdated Show resolved Hide resolved
containerize/Dockerfile Outdated Show resolved Hide resolved
containerize/containerize.sh Outdated Show resolved Hide resolved
containerize/containerize.sh Outdated Show resolved Hide resolved
containerize/containerize.sh Outdated Show resolved Hide resolved
containerize/containerize.sh Show resolved Hide resolved
containerize/containerize.sh Show resolved Hide resolved
containerize/containerize.sh Outdated Show resolved Hide resolved
@aperloff
Copy link
Author

@kpedro88 I believe I have addressed all of your review comments. Please let me know if you're happy with the results.

containerize/Dockerfile Outdated Show resolved Hide resolved
ARG CMSSW_VERSION
ARG TAR

COPY ${TAR} /home/cmsusr/${CMSSW_VERSION}.tar.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to previous comment, could this be:

COPY ${TAR} ${WORKDIR}/${CMSSW_VERSION}.tar.gz

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to /home/${USER} as the ${WORKDIR} and ${HOME} variables are not available and misleading, respectively.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is ${WORKDIR} not available? In fact, it's defined in this file but then never used anywhere...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not an argument or environment variable. The line:

WORKDIR /home/${USER}

Is the location the user will be dropped into when they open the container, as opposed to '/'.

Copy link
Contributor

@kpedro88 kpedro88 Jun 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, it's a special Docker directive. In that case, I propose:

ARG USERDIR /home/${USER}
WORKDIR ${USERDIR}

then ${USERDIR} can be referenced where useful.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, take a look. I think I've implemented what you asked for.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is better.

containerize/Dockerfile Outdated Show resolved Hide resolved
Alexx Perloff added 2 commits June 28, 2021 14:47
…option to override default username in container and some changes to the Dockerfile for sustainability.
ARG NONPRIVILEGED_USER=cmsusr
ARG USERDIR=/home/${NONPRIVILEGED_USER}
WORKDIR ${USERDIR}
COPY --from=builder --chown=cmsusr:cmsusr /home/cmsusr/${CMSSW_VERSION} ${USERDIR}/${CMSSW_VERSION}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If ${USERDIR} works here, can't you define ARG USER_BUILD cmsusr and then USER ${USER_BUILD}, etc. to work around the #19 issue?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can certainly do what you're suggesting, and it does partially get around the problem. However, it won't get around #19 because I still won't be able to evaluate arguments in the parameters (chown) of the COPY command. Let me know what you think now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an improvement. Can you update #19 to clarify the point about using arguments specifically in the parameters of the COPY command?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, #19 is updated.

@aperloff aperloff force-pushed the feature-containerize branch from 2be2c3e to 9431c84 Compare June 28, 2021 21:27
@kpedro88
Copy link
Contributor

+1

@aperloff aperloff merged commit a225f1f into FNALLPC:master Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants