New kubernetes backend #21796

grihabor · 2024-12-22T23:24:32Z

While helm backend already exists and works fine for kubernetes, creating a helm chart might be an overkill for a small thing like a single configmap or a secret. New kubernetes backend can deploy single object easily given a yaml file.

See docs for more details on usage.

I'm also planning to contribute our integration with python_format_string target (I will open a pr) that can handle simple python templating in text files.

> fd _category_ docs/docs/ --max-depth 2 --exec rg position '{}' --with-filename --line-number | sort -k 3,3 -n docs/docs/introduction/_category_.json:3: "position": 1 docs/docs/getting-started/_category_.json:3: "position": 2 docs/docs/using-pants/_category_.json:3: "position": 3 docs/docs/python/_category_.json:3: "position": 4 docs/docs/go/_category_.json:3: "position": 5 docs/docs/jvm/_category_.json:3: "position": 6 docs/docs/shell/_category_.json:3: "position": 7 docs/docs/docker/_category_.json:3: "position": 8 docs/docs/kubernetes/_category_.json:3: "position": 9 docs/docs/helm/_category_.json:3: "position": 10 docs/docs/terraform/_category_.json:3: "position": 11 docs/docs/sql/_category_.json:3: "position": 12 docs/docs/ad-hoc-tools/_category_.json:3: "position": 13 docs/docs/javascript/_category_.json:3: "position": 13 docs/docs/writing-plugins/_category_.json:3: "position": 14 docs/docs/releases/_category_.json:3: "position": 15 docs/docs/contributions/_category_.json:3: "position": 16 docs/docs/tutorials/_category_.json:3: "position": 17

cburroughs · 2025-01-07T20:08:56Z

cc @tgolsson who I know has done some k8s related work at https://github.com/tgolsson/pants-backends/tree/main/pants-plugins/k8s

Broad topic not intended to detail this whole PR: Something we have struggled with internally is that we would like to be able to generate the final reified k8s yaml for inspection or alternative deployments. (I have a PoC plugin that does this for helm) This is along the lines of 'package' but not not quite what helm means by 'package', and I'm unsure what to call it. It would be nice if all the Pant k8s generators could eventually both "spit out the yaml" and "deploy".

grihabor · 2025-01-08T11:16:04Z

Something we have struggled with internally is that we would like to be able to generate the final reified k8s yaml for inspection or alternative deployments.

Haha, we have exactly this problem, I've called the goal render and implemented it for helm_deployment and k8s_bundle

lilatomic

Looks good!
My only real sticking point is about the semantics of the requirement for context

lilatomic · 2025-01-08T14:03:22Z

src/python/pants/backend/k8s/goals/tailor.py

+    putative_targets = []
+
+    if k8s.tailor_source_targets:
+        all_k8s_files_globs = req.path_globs("*.yaml")


include "*.yml" too? or maybe make this a customisable option? Not necessary for a first pass

lilatomic · 2025-01-08T14:17:02Z

src/python/pants/backend/k8s/goals/tailor.py

+    if k8s.tailor_source_targets:
+        all_k8s_files_globs = req.path_globs("*.yaml")
+        all_k8s_files = await Get(Paths, PathGlobs, all_k8s_files_globs)
+        unowned_k8s_files = set(all_k8s_files.files) - set(all_owned_sources)


I think that globbing every yaml file will be too broad, but I don't know that there's a standard way of detecting if a manifest is a k8s manifest (kubectl apply --validate=true --dry-run=client seems to need a server). Worst case people can turn off tailoring.

lilatomic · 2025-01-08T14:36:50Z

build-support/bin/external_tool_versions.py

+
+
+@dataclass(frozen=True)
+class VersionHash:


You could use pants.core.util_rules.external_tool.ExternalToolVersion

lilatomic · 2025-01-08T14:49:36Z

build-support/bin/external_tool_versions.py

+            )
+
+    backward_platform_mapping = {v: k for k, v in platform_mapping.items()}
+    for result in results:


you could have these output in semver order (instead of lexical order) by using from packaging.version import Version, by collecting the versions and then sorting with something like sorted(versions, key=lambda e: Version(e.version))

lilatomic · 2025-01-08T15:12:03Z

build-support/bin/external_tool_versions.py

+
+    backward_platform_mapping = {v: k for k, v in platform_mapping.items()}
+    for result in results:
+        v = result.get(60)


clarify that this is for the timeout

Suggested change

v = result.get(60)

v = result.get(timeout=60)

lilatomic · 2025-01-08T15:19:47Z

src/python/pants/backend/k8s/goals/deploy.py

+    kubectl_tool = await Get(
+        DownloadedExternalTool, ExternalToolRequest, kubectl.get_request(platform)
+    )
+    digest = await Get(Digest, MergeDigests([kubectl_tool.digest, request.input_digest]))


you could use Process.immutable_input_digest for the tool. Here's how it's done in the Helm backend

pants/src/python/pants/backend/helm/util_rules/tool.py

Lines 454 to 457 in 1d1e93e

immutable_input_digests = {

**request.extra_immutable_input_digests,

**helm_binary.immutable_input_digests,

}

lilatomic · 2025-01-08T15:22:09Z

src/python/pants/backend/k8s/goals/deploy.py

+) -> DeployProcess:
+    context = field_set.context.value
+    if context is None:
+        raise ValueError(f"Missing `{K8sBundleContextField.alias}` field")


we could make this more helpful by including the FS's address with field_set.address.spec

lilatomic · 2025-01-08T15:27:06Z

src/python/pants/backend/k8s/goals/deploy.py

+    platform: Platform,
+) -> DeployProcess:
+    context = field_set.context.value
+    if context is None:


I think I've missed something in the logic here. Why do we force the field_set.context to have a value even when kubectl.pass_context is False?

grihabor added 15 commits December 21, 2024 13:48

Copy internal plugin as is

a594341

Fix linters

c13360d

Fix env var

e9cbcbf

Add a test

420d6b9

Fixing the test...

9409599

Download kubectl via TemplatedExternalTool

7a0d00d

Fix the tests

8ff08ab

Implement tailor goal

596e947

Add a script to fetch kubectl versions

fb2c8e5

Add more known versions

db2d216

Add more docs

f5e11d6

Add notes

4e5ba6b

pants fix

f5df270

Fix mypy

4b8a446

lilatomic reviewed Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New kubernetes backend #21796

New kubernetes backend #21796

grihabor commented Dec 22, 2024 •

edited

Loading

cburroughs commented Jan 7, 2025

grihabor commented Jan 8, 2025

lilatomic left a comment

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

lilatomic Jan 8, 2025

	immutable_input_digests = {
	**request.extra_immutable_input_digests,
	**helm_binary.immutable_input_digests,
	}



		@dataclass(frozen=True)
		class VersionHash:

New kubernetes backend #21796

Are you sure you want to change the base?

New kubernetes backend #21796

Conversation

grihabor commented Dec 22, 2024 • edited Loading

cburroughs commented Jan 7, 2025

grihabor commented Jan 8, 2025

lilatomic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grihabor commented Dec 22, 2024 •

edited

Loading