Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebuild Rocky Linux 9 image with missing packages and newer cloud-init #721

Merged
merged 2 commits into from
Nov 3, 2023

Conversation

mnasiadka
Copy link
Member

New rocky-container-stackhpc element contains containerfile that installs 'Minimal Install' group which fixes missing packages like microcode_ctl and brings in new cloud-init from c9s that supports NetworkManager and fixes some interface renaming issues.

New rocky-container-stackhpc element contains containerfile that
installs 'Minimal Install' group which fixes missing packages
like microcode_ctl and brings in new cloud-init from c9s that
supports NetworkManager and fixes some interface renaming issues.
@mnasiadka mnasiadka force-pushed the image_cloud_init_c9s branch from 0de8324 to 4a9a62b Compare October 26, 2023 09:08
@mnasiadka mnasiadka marked this pull request as ready for review November 3, 2023 10:30
@mnasiadka mnasiadka requested a review from a team as a code owner November 3, 2023 10:30
@jovial
Copy link
Contributor

jovial commented Nov 3, 2023

We did still see the race occurring on at least one node with cloud-init-23.1.1-12.el9.noarch.rpm and the NetworkManager cloud init renderer. I don't think we had the Minimal install group:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    25 |                                                                                                                        | 2023-10-31 14:43 | Install        |    2   
    24 |                                                                                                                        | 2023-10-31 14:43 | Removed        |    2 EE
    23 |                                                                                                                        | 2023-10-31 14:43 | Install        |   19  <
    22 | install tree                                                                                                           | 2023-10-31 14:17 | Install        |    1 > 
    21 |                                                                                                                        | 2023-10-31 14:02 | Install        |    2   
    20 |                                                                                                                        | 2023-10-31 14:02 | Install        |    1 EE
    19 |                                                                                                                        | 2023-10-31 14:02 | Install        |   13 EE
    18 |                                                                                                                        | 2023-10-31 14:01 | Install        |    5   
    17 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    16 | -v -y install lvm2                                                                                                     | 2023-10-16 11:30 | Install        |    6 EE
    15 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    14 | -v -y install cryptsetup                                                                                               | 2023-10-16 11:30 | Install        |    1   
    13 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    6   
    12 | -v -y install grub2-tools grub2 efibootmgr grub2-efi-x64 grub2-efi-x64-modules shim-x64                                | 2023-10-16 11:29 | Install        |   47 EE
    11 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |    9   
    10 | -v -y install logrotate net-tools vim git less python3 NetworkManager-config-server linux-firmware cloud-utils-growpar | 2023-10-16 11:29 | Install        |   19 EE
     9 | -v -y remove linux-firmware-whence linux-firmware                                                                      | 2023-10-16 11:29 | Removed        |    2   
     8 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |   26   
     7 | -v -y install dnf-plugins-core python3-libselinux traceroute python3-policycoreutils selinux-policy-targeted tcpdump a | 2023-10-16 11:28 | Install        |  132 EE
     6 | -y update                                                                                                              | 2023-10-16 11:28 | Upgrade        |   16   
     5 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    1   
     4 | -v -y install iscsi-initiator-utils                                                                                    | 2023-10-16 11:28 | Install        |    3 EE
     3 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    2   
     2 | -v -y install gettext python3-PyYAML                                                                                   | 2023-10-16 11:28 | Install        |    2   
     1 | install -y findutils util-linux sudo python3 NetworkManager https://mirror.stream.centos.org/9-stream/AppStream/x86_64 | 2023-10-10 15:59 | I, U           |   77 EE
Failed to store expired repos cache: [Errno 13] Permission denied: '/var/cache/dnf/expired_repos.json'
[stack@compute-12 ~]$ 

@mnasiadka mnasiadka merged commit 51c6878 into stackhpc/yoga Nov 3, 2023
@mnasiadka mnasiadka deleted the image_cloud_init_c9s branch November 3, 2023 11:03
@mnasiadka
Copy link
Member Author

We did still see the race occurring on at least one node with cloud-init-23.1.1-12.el9.noarch.rpm and the NetworkManager cloud init renderer. I don't think we had the Minimal install group:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    25 |                                                                                                                        | 2023-10-31 14:43 | Install        |    2   
    24 |                                                                                                                        | 2023-10-31 14:43 | Removed        |    2 EE
    23 |                                                                                                                        | 2023-10-31 14:43 | Install        |   19  <
    22 | install tree                                                                                                           | 2023-10-31 14:17 | Install        |    1 > 
    21 |                                                                                                                        | 2023-10-31 14:02 | Install        |    2   
    20 |                                                                                                                        | 2023-10-31 14:02 | Install        |    1 EE
    19 |                                                                                                                        | 2023-10-31 14:02 | Install        |   13 EE
    18 |                                                                                                                        | 2023-10-31 14:01 | Install        |    5   
    17 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    16 | -v -y install lvm2                                                                                                     | 2023-10-16 11:30 | Install        |    6 EE
    15 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    14 | -v -y install cryptsetup                                                                                               | 2023-10-16 11:30 | Install        |    1   
    13 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    6   
    12 | -v -y install grub2-tools grub2 efibootmgr grub2-efi-x64 grub2-efi-x64-modules shim-x64                                | 2023-10-16 11:29 | Install        |   47 EE
    11 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |    9   
    10 | -v -y install logrotate net-tools vim git less python3 NetworkManager-config-server linux-firmware cloud-utils-growpar | 2023-10-16 11:29 | Install        |   19 EE
     9 | -v -y remove linux-firmware-whence linux-firmware                                                                      | 2023-10-16 11:29 | Removed        |    2   
     8 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |   26   
     7 | -v -y install dnf-plugins-core python3-libselinux traceroute python3-policycoreutils selinux-policy-targeted tcpdump a | 2023-10-16 11:28 | Install        |  132 EE
     6 | -y update                                                                                                              | 2023-10-16 11:28 | Upgrade        |   16   
     5 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    1   
     4 | -v -y install iscsi-initiator-utils                                                                                    | 2023-10-16 11:28 | Install        |    3 EE
     3 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    2   
     2 | -v -y install gettext python3-PyYAML                                                                                   | 2023-10-16 11:28 | Install        |    2   
     1 | install -y findutils util-linux sudo python3 NetworkManager https://mirror.stream.centos.org/9-stream/AppStream/x86_64 | 2023-10-10 15:59 | I, U           |   77 EE
Failed to store expired repos cache: [Errno 13] Permission denied: '/var/cache/dnf/expired_repos.json'
[stack@compute-12 ~]$ 

Minimal Install basically adds mainly microcode_ctl which adds CPU CVE mitigations (which was missing) - I'm not aware it fixes anything else.

@jovial
Copy link
Contributor

jovial commented Nov 3, 2023

We did still see the race occurring on at least one node with cloud-init-23.1.1-12.el9.noarch.rpm and the NetworkManager cloud init renderer. I don't think we had the Minimal install group:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    25 |                                                                                                                        | 2023-10-31 14:43 | Install        |    2   
    24 |                                                                                                                        | 2023-10-31 14:43 | Removed        |    2 EE
    23 |                                                                                                                        | 2023-10-31 14:43 | Install        |   19  <
    22 | install tree                                                                                                           | 2023-10-31 14:17 | Install        |    1 > 
    21 |                                                                                                                        | 2023-10-31 14:02 | Install        |    2   
    20 |                                                                                                                        | 2023-10-31 14:02 | Install        |    1 EE
    19 |                                                                                                                        | 2023-10-31 14:02 | Install        |   13 EE
    18 |                                                                                                                        | 2023-10-31 14:01 | Install        |    5   
    17 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    16 | -v -y install lvm2                                                                                                     | 2023-10-16 11:30 | Install        |    6 EE
    15 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    1   
    14 | -v -y install cryptsetup                                                                                               | 2023-10-16 11:30 | Install        |    1   
    13 |                                                                                                                        | 2023-10-16 11:30 | Reason Change  |    6   
    12 | -v -y install grub2-tools grub2 efibootmgr grub2-efi-x64 grub2-efi-x64-modules shim-x64                                | 2023-10-16 11:29 | Install        |   47 EE
    11 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |    9   
    10 | -v -y install logrotate net-tools vim git less python3 NetworkManager-config-server linux-firmware cloud-utils-growpar | 2023-10-16 11:29 | Install        |   19 EE
     9 | -v -y remove linux-firmware-whence linux-firmware                                                                      | 2023-10-16 11:29 | Removed        |    2   
     8 |                                                                                                                        | 2023-10-16 11:29 | Reason Change  |   26   
     7 | -v -y install dnf-plugins-core python3-libselinux traceroute python3-policycoreutils selinux-policy-targeted tcpdump a | 2023-10-16 11:28 | Install        |  132 EE
     6 | -y update                                                                                                              | 2023-10-16 11:28 | Upgrade        |   16   
     5 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    1   
     4 | -v -y install iscsi-initiator-utils                                                                                    | 2023-10-16 11:28 | Install        |    3 EE
     3 |                                                                                                                        | 2023-10-16 11:28 | Reason Change  |    2   
     2 | -v -y install gettext python3-PyYAML                                                                                   | 2023-10-16 11:28 | Install        |    2   
     1 | install -y findutils util-linux sudo python3 NetworkManager https://mirror.stream.centos.org/9-stream/AppStream/x86_64 | 2023-10-10 15:59 | I, U           |   77 EE
Failed to store expired repos cache: [Errno 13] Permission denied: '/var/cache/dnf/expired_repos.json'
[stack@compute-12 ~]$ 

Minimal Install basically adds mainly microcode_ctl which adds CPU CVE mitigations (which was missing) - I'm not aware it fixes anything else.

We only seemed to hit on one node, so seems much more reliable, but possibly doesn't 100% fix the issue. @dawud can confirm.

@dawud
Copy link

dawud commented Nov 3, 2023

Hey @jovial , I think you meant to mention somebody else, not me. Cheers!

@Alex-Welsh
Copy link
Member

Sorry about that @dawud, I think the intended target was @technowhizz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants