Automated packer building for Talos images (Multicloud)

[Timotej979]

Hello 👋

I have stumbled upon a problem with a solution I'm trying to implement for building multiple images of TalosOS using packer so that I can then push them to the packer registry and pull them from it and use them in my IaC in Terraform.

The general idea is to have a dedicated repository for building this images for each cloud provider, so for example the following structure:

Repository structure:

• Makefile
• .gitignore
• .gitlab-ci.yml
• aws/
  • templates/
    • aws.pkr.hcl
  • variables/
    • aws.pkvars.hcl
  • scripts
    • common.sh
    • install-talos.sh
    • aws-setup.sh
  • cloud-config/
    • aws.json
• azure/
  • ...
• gcp/
  • ...
• ...

So the idea is to have everything separated by cloud provider so that management of new releases can be more controlled, also I'm not sure whether to use the latest release directly in the install-talos.sh script and just use actual short-term infrastructure setup (Small test cluster to test image compatibility in the pipeline) to test the new releases of Talos?

I know that Talos supports the imager tool and image factory, however these do not exactly fit into my use-case of automating the whole deployment process of Talos for multiple clouds. They can however be used and I'm not against using them, however some clouds are not yet supported by Talos which I want to implement a custom image for.

Finally my current development repository is available here, however there is not much progress in it as I still do not have a clear idea on how to implement the building of Talos using packer.

https://github.com/Timotej979/Homelab-infrastructure-talos

Kind regards and any help is much appreciated as this is not a simple topic, also any kind of already created examples are much appreciated. 😄 🙏

[rothgar]

Can you give an example of what you're customizing for these clouds? I looked at your repo and looked at some of the scripts but they were all empty. I'm not sure if you're actually trying to customer Talos or just distribute talos images.

[rothgar]

One of the philosophies we have with Talos is to eliminate unnecessary processes and software. This includes eliminating software you don't need inside the Linux machine (eg systemd) and processes to manage Linux images.

I often call the image factory the "packer killer" because packer was designed for taking general purpose Linux distributions and creating and distributing artifacts for consumption. In the world of Talos creating artifacts is mostly unnecessary, but distributing artifacts is something we currently don't solve.

It looks like your current process for distributing images is to run a bunch of cloud VMs, snapshot them, convert them to images, tag them.

If there were a CLI tool to take a schematic from the image factory and push it directly to a cloud provider would that solve your distribution problem?

[Timotej979]

Yes, exactly this solution/potential-tool that you proposed would solve such a problem 😄, upon new version releases of Talos I can then just run the dev/stage environment to test if everything is compatible with production and then deploy to production and transfer the deployments and anything that might be important to transfer.

In my case however I would like to support 13 different clouds (This will be my Master's thesis so I do like to make my life hard, however it is a great learning opportunity 😅). As such I'm now wondering what to do for the clouds that do not have currently built images of Talos OS in the image factory, like Alicloud/Tencent/Huawei/OVH. For this I would probably require some custom build process or use a universal cloud image that might work? Here was the direction of using packer as a build tool considered.

[smira]

Packer won't get you to a cloud that Talos doesn't support, as Talos support for clouds is built-in.

Some clouds might successfully boot generic metal image, or support nocloud metadata. It all depends on the cloud, the critical part is getting network up, and Talos would do DHCPv4 by default.

[Timotej979]

Okay, thanks for the answer 😄, I will check what works on the clouds that I will be using and report what works and what not, so that if someone else needs it later for development/testing or compatibility checking. I can either report the results here or somewhere else if you think it might be needed.

[smira]

But to make my point really clear: you can use Packer e.g. to download an image from the Image Factory, and upload that image to the cloud of your choice, but as Packer is designed around "old" Linux distros, it won't be able to modify Talos image in any way, so it might only take care of uploading.