diff --git a/helm/kube-image-keeper/README.md.gotmpl b/helm/kube-image-keeper/README.md.gotmpl index 62cf795a..4853ddfc 100644 --- a/helm/kube-image-keeper/README.md.gotmpl +++ b/helm/kube-image-keeper/README.md.gotmpl @@ -8,23 +8,6 @@ kube-image-keeper (a.k.a. *kuik*, which is pronounced /kwɪk/, like "quick") is a container image caching system for Kubernetes. It saves the container images used by your pods in its own local registry so that these images remain available if the original becomes unavailable. -## Upgrading - -### From 1.2.0 to 1.3.0 - -***ACTION REQUIRED*** - -In v1.3.0, we removed the finalizer `pod.kuik.enix.io/finalizer` from pods that were rewritten by kuik. - -To avoid having these pods stuck in `Terminating` state after a delete action or a rolling update, you will need to manually remove the finalizer from these pods once you upgrade to 1.3.0. -This can be achieved using the following command: - -```bash -kubectl get pods --all-namespaces -l kuik.enix.io/images-rewritten=true -o json | jq '.items[].metadata.finalizers=null' | kubectl replace -f - -``` - -***Ensure you run this command AFTER having fully upgraded to version 1.3.0. Otherwise, the kuik controllers will continuously re-add the finalizer to the rewritten pods.*** - ## Why and when is it useful? At [Enix](https://enix.io/), we manage production Kubernetes clusters both for our internal use and for various customers; sometimes on premises, sometimes in various clouds, public or private. We regularly run into image availability issues, for instance: @@ -71,13 +54,13 @@ kuik has been developed for, and tested with, Kubernetes 1.24 to 1.28; but the c ## How it works -When a pod is created, kuik's **mutating webhook** rewrites its images on the fly, adding a `localhost:{port}/` prefix (the `port` is 7439 by default, and is configurable). +When a pod is created, kuik's **mutating webhook** rewrites its images on the fly to point to the local caching registry, adding a `localhost:{port}/` prefix (the `port` is 7439 by default, and is configurable). This means that you don't need to modify/rewrite the source registry url of your manifest/helm chart used to deploy your solution, kuik will take care of it. On `localhost:{port}`, there is an **image proxy** that serves images from kuik's **caching registry** (when the images have been cached) or directly from the original registry (when the images haven't been cached yet). One **controller** watches pods, and when it notices new images, it creates `CachedImage` custom resources for these images. -Another **controller** watches these `CachedImage` custom resources, and copies images from source registries to kuik's caching registry accordingly. +Another **controller** watches these `CachedImage` custom resources, and copies images from source registries to kuik's caching registry accordingly. When images come from a private registry, the controller will use the `imagePullSecrets` from the `CachedImage` spec, those are set from the pod that produced the `CachedImage`. Here is what our images look like when using kuik: @@ -148,7 +131,7 @@ That's it! Our container images are available across multiple registries for reliability. You can find them on [Github Container Registry](https://github.com/enix/kube-image-keeper/pkgs/container/kube-image-keeper), [Quay](https://quay.io/repository/enix/kube-image-keeper) and [DockerHub](https://hub.docker.com/r/enix/kube-image-keeper). -CAUTION: If you use a storage backend that runs in the same cluster as kuik but in a different namespace, ensure you filter-out the storage backend's pods. Failure to do so may lead to interdependency issues, making it impossible to start both kuik and its storage backend if either encounters an issue. +CAUTION: If you use a storage backend that runs in the same cluster as kuik but in a different namespace, be sure to filter the storage backend's pods. Failure to do so may lead to interdependency issues, making it impossible to start both kuik and its storage backend if either encounters an issue. {{ template "chart.valuesSection" . }} @@ -195,6 +178,12 @@ This logic isn't implemented by the kuik controllers or webhook directly, but th Keep in mind that kuik will ignore pods scheduled into its own namespace. +#### Image pull policy + +In the case of a container configured with `imagePullPolicy: Never`, the container will always be filtered out as it makes no sense to cache an image that would never be cached and always read from the disk. + +In the case of a container configured with `imagePullPolicy: Always`, or with the tag `latest`, or with no tag (defaulting to `latest`), by default, the container will be filtered out in order to keep the default behavior of kubernetes, which is to always pull the new version of the image (thus not using the cache of kuik). This can be disabled by setting the value `controllers.webhook.ignorePullPolicyAlways` to `false`. + ### Cache persistence Persistence is disabled by default. You can enable it by setting the Helm value `registry.persistence.enabled=true`. This will create a PersistentVolumeClaim with a default size of 20 GiB. You can change that size by setting the value `registry.persistence.size`. Keep in mind that enabling persistence isn't enough to provide high availability of the registry! If you want kuik to be highly available, please refer to the [high availability guide](https://github.com/enix/kube-image-keeper/blob/main/docs/high-availability.md). @@ -267,6 +256,10 @@ rootCertificateAuthorities: You can of course use as many insecure registries or root certificate authorities as you want. In the case of a self-signed certificate, you can either use the `insecureRegistries` or the `rootCertificateAuthorities` value, but trusting the root certificate will always be more secure than allowing insecure registries. +### Registry UI + +For debugging reasons, it may be useful to be able to access the registry through an UI. This can be achieved by enabling the registry UI with the value `registryUI.enabled=true`. The UI will not be publicly available through an ingress, you will need to open a port-forward from port `80`. You can set a custom username and password with values `registryUI.auth.username` (default is `admin`) and `registryUI.auth.password` (empty by default). + ## Garbage collection and limitations When a CachedImage expires because it is not used anymore by the cluster, the image is deleted from the registry. However, since kuik uses [Docker's registry](https://docs.docker.com/registry/), this only deletes **reference files** like tags. It doesn't delete blobs, which account for most of the used disk space. [Garbage collection](https://docs.docker.com/registry/garbage-collection/) allows removing those blobs and free up space. The garbage collecting job can be configured to run thanks to the `registry.garbageCollectionSchedule` configuration in a cron-like format. It is disabled by default, because running garbage collection without persistence would just wipe out the cache registry.