buildah mount fails in buildah container (fuse-overlayfs?)

[cgwalters]

$ podman run -q -v /dev/fuse --cap-add=all --rm -ti quay.io/buildah/stable:v1.38.1 bash -c 'id=$(buildah from busybox); buildah mount $id'
Resolved "busybox" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/busybox:latest...
Getting image source signatures
Copying blob 9c0abc9c5bd3 done   | 
Copying config af47096251 done   | 
Writing manifest to image destination
Error: mount /var/lib/containers/storage/overlay:/var/lib/containers/storage/overlay, flags: 0x1000: operation not permitted
WARN[0000] failed to shutdown storage: "mount /var/lib/containers/storage/overlay:/var/lib/containers/storage/overlay, flags: 0x1000: operation not permitted" 

I'm trying to build on coreos/rpm-ostree#5268 to have it support being run in the same setup as the buildah container, which is widely used for nested builds (container-in-container).

In the end I don't actually need mount here necessarily; we could also copy the image to an OCI and operate, but I'd like to avoid another copy.

I haven't looked at the code here but I think this is also related to containers/podman#25241 in that the "mount" logic is different from the "run" flow.

[cgwalters]

OK, I think I narrowed in on this. It works if I create a new user+mounts via e.g. unshare -U -m --keep-caps -r. And I think what's going on here is that when we go to do a build, buildah always unshares and makes a new mount namespace, so we can remount /var/lib/containers/storage in that mountns.

But in this scenario by default /var/lib/containers is actually set up by the outer runtime and we don't have permissions to modify it. The unshare works around that, and we can just run everything under that.

But that said, ISTM we could just skip doing this mount (there's even an option for it via skip_mount_home). Ah but yeah if I set that then we just get further errors because we can't mutate the /var/lib/containers mounts.

OK, dunno hmm, I guess we can just add something to the man page about this? There's already things related to rootless that are similar.

Tangentially related, wow buildah unshare totally blows up in a buildah container, lots of failures to construct the

WARN[0000] Reading allowed ID mappings: reading subuid mappings for user "root" and subgid mappings for group "root": no subuid ranges found for user "root" in /etc/subuid 
...
memfd_create(): Invalid argument

Wait we failed to create a memfd?? 🤔 Must be related to seccomp? A quick strace shows:

214  mount("/memfd:buildah-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace-in-a-user-namespace (deleted)", "/tmp/containers.EyK1zL", NULL, MS_B
IND, NULL <unfinished ...>
1203  <... nanosleep resumed>NULL)      = 0
1214  <... mount resumed>)              = -1 ENAMETOOLONG (File name too long)
1203  nanosleep({tv_sec=0, tv_nsec=20000},  <unfinished ...>
1214  unlink("/tmp/containers.EyK1zL")  = 0
1214  openat(AT_FDCWD, "/proc/self/exe", O_RDONLY|O_CLOEXEC) = 3
1214  fstat(3,  <unfinished ...>
1203  <... nanosleep resumed>NULL)      = 0
1214  <... fstat resumed>{st_mode=S_IFREG|0777, st_size=32937928, ...}) = 0
1203  nanosleep({tv_sec=0, tv_nsec=20000},  <unfinished ...>
1214  memfd_create("buildah-in-a-user-namespace-in-a"..., MFD_CLOEXEC|MFD_ALLOW_SEALING) = -1 EINVAL (Invalid argument)

Heh no the real problem is it looks like we enter some sort of endless loop if we fail to create a userns, and then try to create a memfd that exceeds NAME_MAX of 255 bytes.