diff --git a/charts/twins-api/README.md b/charts/twins-api/README.md
index c14406b6b..a3031db05 100644
--- a/charts/twins-api/README.md
+++ b/charts/twins-api/README.md
@@ -23,48 +23,76 @@ Use this Helm chart to deploy API Twins service, which is a part of 2GIS's [On-P
| `image.tag` | Twins API service image tag. | `1.6.0` |
| `image.pullPolicy` | Image [pull policy](https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy) | `IfNotPresent` |
+### Deployment Artifacts Storage settings
+
+| Name | Description | Value |
+| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
+| `dgctlStorage.host` | S3 host. Format: `host:port`. **Required** | `""` |
+| `dgctlStorage.bucket` | S3 bucket name. **Required** | `""` |
+| `dgctlStorage.accessKey` | S3 access key for accessing the bucket. **Required** | `""` |
+| `dgctlStorage.secretKey` | S3 secret key for accessing the bucket. **Required** | `""` |
+| `dgctlStorage.manifest` | The path to the [manifest file](https://docs.2gis.com/en/on-premise/overview#nav-lvl2@paramCommon_deployment_steps). Format: `manifests/0000000000.json`
This file contains the description of pieces of data that the service requires to operate. **Required** | `""` |
+
### API service settings
-| Name | Description | Value |
-| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
-| `api.strategy.type` | Type of Kubernetes deployment. Can be `Recreate` or `RollingUpdate`. | `RollingUpdate` |
-| `api.strategy.rollingUpdate.maxUnavailable` | Maximum number of pods that can be created over the desired number of pods when doing [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment). | `0` |
-| `api.strategy.rollingUpdate.maxSurge` | Maximum number of pods that can be unavailable during the [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment) process. | `1` |
-| `api.keys.url` | URL of the Keys service, ex: http://{keys-api}.svc. This URL should be accessible from all the pods within your Kubernetes cluster. **Required** | `""` |
-| `api.keys.token` | Keys service API key **Required** | `""` |
-| `api.keys.requestTimeout` | Timeout for requests to the Keys API. | `5s` |
-| `api.replicas` | A replica count for the pod. | `1` |
-| `api.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
-| `api.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
-| `api.podAnnotations` | Kubernetes [pod annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
-| `api.podLabels` | Kubernetes [pod labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
-| `api.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
-| `api.affinity` | Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity). | `{}` |
-| `api.tolerations` | Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
-| `api.service.annotations` | Kubernetes [service annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
-| `api.service.labels` | Kubernetes [service labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
-| `api.service.type` | Kubernetes [service type](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types). | `ClusterIP` |
-| `api.service.port` | Service port. | `80` |
-| `api.ingress.enabled` | If Ingress is enabled for the service. | `false` |
-| `api.ingress.className` | Name of the Ingress controller class. | `nginx` |
-| `api.ingress.hosts[0].host` | Hostname for the Ingress service. | `twins-api.example.com` |
-| `api.ingress.hosts[0].paths[0].path` | Path of the host for the Ingress service. | `/` |
-| `api.ingress.hosts[0].paths[0].pathType` | Type of the path for the Ingress service. | `Prefix` |
-| `api.ingress.tls` | TLS configuration | `[]` |
-| `api.hpa.enabled` | If HPA is enabled for the service. | `false` |
-| `api.hpa.minReplicas` | Lower limit for the number of replicas to which the autoscaler can scale down. | `1` |
-| `api.hpa.maxReplicas` | Upper limit for the number of replicas to which the autoscaler can scale up. | `2` |
-| `api.hpa.scaleDownStabilizationWindowSeconds` | Scale-down window. | `""` |
-| `api.hpa.scaleUpStabilizationWindowSeconds` | Scale-up window. | `""` |
-| `api.hpa.targetCPUUtilizationPercentage` | Target average CPU utilization (represented as a percentage of requested CPU) over all the pods; if not specified the default autoscaling policy will be used. | `80` |
-| `api.hpa.targetMemoryUtilizationPercentage` | Target average memory utilization (represented as a percentage of requested memory) over all the pods; if not specified the default autoscaling policy will be used. | `""` |
+| Name | Description | Value |
+| ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
+| `api.strategy.type` | Type of Kubernetes deployment. Can be `Recreate` or `RollingUpdate`. | `RollingUpdate` |
+| `api.strategy.rollingUpdate.maxUnavailable` | Maximum number of pods that can be created over the desired number of pods when doing [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment). | `0` |
+| `api.strategy.rollingUpdate.maxSurge` | Maximum number of pods that can be unavailable during the [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment) process. | `1` |
+| `api.keys.url` | URL of the Keys service, ex: http://{keys-api}.svc. This URL should be accessible from all the pods within your Kubernetes cluster. **Required** | `""` |
+| `api.keys.token` | Keys service API key **Required** | `""` |
+| `api.keys.requestTimeout` | Timeout for requests to the Keys API. | `5s` |
+| `api.replicas` | A replica count for the pod. | `1` |
+
+### api.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+
+| Name | Description | Value |
+| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
+| `api.resources.requests.cpu` | A CPU request. | `50m` |
+| `api.resources.requests.memory` | A memory request. | `128Mi` |
+| `api.resources.limits.cpu` | A CPU limit. | `1` |
+| `api.resources.limits.memory` | A memory limit. | `256Mi` |
+| `api.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
+| `api.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
+| `api.podAnnotations` | Kubernetes [pod annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
+| `api.podLabels` | Kubernetes [pod labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
+| `api.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
+| `api.affinity` | Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity). | `{}` |
+| `api.tolerations` | Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |
+| `api.service.annotations` | Kubernetes [service annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
+| `api.service.labels` | Kubernetes [service labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
+| `api.service.type` | Kubernetes [service type](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types). | `ClusterIP` |
+| `api.service.port` | Service port. | `80` |
+| `api.ingress.enabled` | If Ingress is enabled for the service. | `false` |
+| `api.ingress.className` | Name of the Ingress controller class. | `nginx` |
+| `api.ingress.hosts[0].host` | Hostname for the Ingress service. | `twins-api.example.com` |
+| `api.ingress.hosts[0].paths[0].path` | Path of the host for the Ingress service. | `/` |
+| `api.ingress.hosts[0].paths[0].pathType` | Type of the path for the Ingress service. | `Prefix` |
+| `api.ingress.tls` | TLS configuration | `[]` |
+| `api.hpa.enabled` | If HPA is enabled for the service. | `false` |
+| `api.hpa.minReplicas` | Lower limit for the number of replicas to which the autoscaler can scale down. | `1` |
+| `api.hpa.maxReplicas` | Upper limit for the number of replicas to which the autoscaler can scale up. | `2` |
+| `api.hpa.scaleDownStabilizationWindowSeconds` | Scale-down window. | `""` |
+| `api.hpa.scaleUpStabilizationWindowSeconds` | Scale-up window. | `""` |
+| `api.hpa.targetCPUUtilizationPercentage` | Target average CPU utilization (represented as a percentage of requested CPU) over all the pods; if not specified the default autoscaling policy will be used. | `80` |
+| `api.hpa.targetMemoryUtilizationPercentage` | Target average memory utilization (represented as a percentage of requested memory) over all the pods; if not specified the default autoscaling policy will be used. | `""` |
### Migrate service settings
-| Name | Description | Value |
-| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | ----- |
-| `migrate.initialDelaySeconds` | Delay in seconds at the service startup. | `0` |
-| `migrate.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
+| Name | Description | Value |
+| ----------------------------- | ---------------------------------------- | ----- |
+| `migrate.initialDelaySeconds` | Delay in seconds at the service startup. | `0` |
+
+### migrate.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+
+| Name | Description | Value |
+| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ------ |
+| `migrate.resources.requests.cpu` | A CPU request. | `10m` |
+| `migrate.resources.requests.memory` | A memory request. | `32Mi` |
+| `migrate.resources.limits.cpu` | A CPU limit. | `100m` |
+| `migrate.resources.limits.memory` | A memory limit. | `64Mi` |
+| `migrate.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
### Database access settings
@@ -87,20 +115,58 @@ Use this Helm chart to deploy API Twins service, which is a part of 2GIS's [On-P
| `postgres.rw.username` | PostgreSQL username. **Required** | `""` |
| `postgres.rw.password` | PostgreSQL password. **Required** | `""` |
-### Limits
-
-| Name | Description | Value |
-| ----------------------------------- | ---------------------------------- | ------- |
-| `api.resources` | **Limits for the API service** | |
-| `api.resources.requests.cpu` | A CPU request. | `50m` |
-| `api.resources.requests.memory` | A memory request. | `128Mi` |
-| `api.resources.limits.cpu` | A CPU limit. | `1` |
-| `api.resources.limits.memory` | A memory limit. | `256Mi` |
-| `migrate.resources` | **Limits for the Migrate service** | |
-| `migrate.resources.requests.cpu` | A CPU request. | `10m` |
-| `migrate.resources.requests.memory` | A memory request. | `32Mi` |
-| `migrate.resources.limits.cpu` | A CPU limit. | `100m` |
-| `migrate.resources.limits.memory` | A memory limit. | `64Mi` |
+### Kubernetes Importer job settings
+
+| Name | Description | Value |
+| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------- |
+| `importer` | **Common settings** | |
+| `importer.enabled` | If importer is enabled for the service | `false` |
+| `importer.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) | `{}` |
+| `importer.initialDelaySeconds` | Number of seconds after the container has started before liveness or readiness probes are initiated | `1` |
+| `importer.retry.download.maxAttempts` | The maximum number of retries download before stopping | `3` |
+| `importer.retry.download.delay` | Delay until the retry attempts download | `1s` |
+| `importer.retry.execute.maxAttempts` | The maximum number of retries execute psql command before stopping | `3` |
+| `importer.retry.execute.delay` | Delay until the retry attempts execute | `1s` |
+
+### importer.postgres **Database settings**
+
+| Name | Description | Value |
+| --------------------------------------- | ---------------------------------------------- | ------ |
+| `importer.postgres.schemaSwitchEnabled` | Automatic switch PostgreSQL schema on releases | `true` |
+
+### importer.persistentVolume **Persistent Volume settings**
+
+| Name | Description | Value |
+| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ------------------- |
+| `importer.persistentVolume.enabled` | If [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) is enabled for the service | `false` |
+| `importer.persistentVolume.accessModes` | Persistent Volume [Access Mode](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes) | `["ReadWriteOnce"]` |
+| `importer.persistentVolume.storageClass` | Kubernetes [Storage Classes](https://kubernetes.io/docs/concepts/storage/storage-classes/) | `topolvm-ext4` |
+| `importer.persistentVolume.size` | Volume size | `50Gi` |
+
+### importer.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+
+| Name | Description | Value |
+| ------------------------------------ | ---------------- | -------- |
+| `importer.resources.requests.cpu` | A CPU request | `256m` |
+| `importer.resources.requests.memory` | A memory request | `512Mi` |
+| `importer.resources.limits.cpu` | A CPU limit | `2` |
+| `importer.resources.limits.memory` | A memory limit | `2048Mi` |
+
+### importer.cleaner **Cleaner scheme settings**
+
+| Name | Description | Value |
+| ------------------------------- | ------------------------------------------- | ------ |
+| `importer.cleaner.enabled` | If clean schemes is enabled for the service | `true` |
+| `importer.cleaner.versionLimit` | Number of backup schemes | `2` |
+
+### importer.cleaner.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+
+| Name | Description | Value |
+| -------------------------------------------- | ---------------- | ------- |
+| `importer.cleaner.resources.requests.cpu` | A CPU request | `50m` |
+| `importer.cleaner.resources.requests.memory` | A memory request | `128Mi` |
+| `importer.cleaner.resources.limits.cpu` | A CPU limit | `1000m` |
+| `importer.cleaner.resources.limits.memory` | A memory limit | `512Mi` |
## Maintainers
diff --git a/charts/twins-api/templates/_helpers.tpl b/charts/twins-api/templates/_helpers.tpl
index 7d63fc111..6031be71e 100644
--- a/charts/twins-api/templates/_helpers.tpl
+++ b/charts/twins-api/templates/_helpers.tpl
@@ -10,6 +10,10 @@
{{ include "twins.name" . }}-migrate
{{- end }}
+{{- define "twins.importer.name" -}}
+{{ include "twins.name" . }}-importer
+{{- end }}
+
{{- define "twins.secret.deploys.name" -}}
{{ include "twins.name" . }}-secret-deploys
{{- end }}
@@ -44,6 +48,21 @@ app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
+{{- define "twins.importer.labels" -}}
+app.kubernetes.io/name: {{ .Chart.Name }}-importer
+app.kubernetes.io/instance: {{ .Release.Name }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+
+{{- define "twins.manifestCode" -}}
+{{- base $.Values.dgctlStorage.manifest | trimSuffix ".json" }}
+{{- end }}
+
+{{- define "twins.env.loglevel" -}}
+- name: TWINS_LOG_LEVEL
+ value: "{{ .Values.api.logLevel }}"
+{{- end }}
+
{{- define "twins.env.db" -}}
- name: TWINS_DB_RO_HOST
value: "{{ required "A valid .Values.postgres.ro.host required" .Values.postgres.ro.host }}"
@@ -51,8 +70,13 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
value: "{{ .Values.postgres.ro.port }}"
- name: TWINS_DB_RO_NAME
value: "{{ required "A valid .Values.postgres.ro.name required" .Values.postgres.ro.name }}"
+{{- if .Values.importer.postgres.schemaSwitchEnabled }}
+- name: TWINS_DB_RO_SCHEMA
+ value: "{{ include "twins.manifestCode" . }}"
+{{- else }}
- name: TWINS_DB_RO_SCHEMA
value: "{{ .Values.postgres.ro.schema }}"
+{{- end }}
- name: TWINS_DB_RO_CONNECTION_TIMEOUT
value: "{{ .Values.postgres.ro.timeout }}"
- name: TWINS_DB_RO_USERNAME
@@ -61,8 +85,13 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
value: "{{ required "A valid .Values.postgres.rw.host required" .Values.postgres.rw.host }}"
- name: TWINS_DB_RW_PORT
value: "{{ .Values.postgres.rw.port }}"
+{{- if .Values.importer.postgres.schemaSwitchEnabled }}
+- name: TWINS_DB_RW_SCHEMA
+ value: "{{ include "twins.manifestCode" . }}"
+{{- else }}
- name: TWINS_DB_RW_SCHEMA
value: "{{ .Values.postgres.rw.schema }}"
+{{- end }}
- name: TWINS_DB_RW_CONNECTION_TIMEOUT
value: "{{ .Values.postgres.rw.timeout }}"
- name: TWINS_DB_RW_NAME
@@ -87,6 +116,7 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- define "twins.env.db.jobs" -}}
{{ include "twins.env.db" . }}
+{{ include "twins.env.loglevel" . }}
- name: TWINS_DB_RO_PASSWORD
valueFrom:
secretKeyRef:
@@ -100,6 +130,7 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
{{- define "twins.env.api"}}
+{{ include "twins.env.loglevel" . }}
{{ include "twins.env.db.deploys" . }}
- name: TWINS_AUTH_ENDPOINT
value: "{{ required "A valid .Values.api.keys.url required" .Values.api.keys.url }}"
@@ -112,6 +143,38 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
value: "{{ .Values.api.keys.requestTimeout }}"
{{- end }}
+{{- define "twins.env.importer" -}}
+{{ include "twins.env.db.jobs" . }}
+- name: TWINS_IMPORTER_DB_SCHEMA_SWITCH_ENABLED
+ value: "{{ .Values.importer.postgres.schemaSwitchEnabled }}"
+- name: TWINS_S3_ENDPOINT
+ value: "{{ .Values.dgctlStorage.host }}"
+- name: TWINS_S3_BUCKET
+ value: "{{ .Values.dgctlStorage.bucket }}"
+- name: TWINS_S3_ACCESS_KEY
+ valueFrom:
+ secretKeyRef:
+ name: {{ include "twins.secret.jobs.name" . }}
+ key: dgctlStorageAccessKey
+- name: TWINS_S3_SECRET_KEY
+ valueFrom:
+ secretKeyRef:
+ name: {{ include "twins.secret.jobs.name" . }}
+ key: dgctlStorageSecretKey
+- name: TWINS_IMPORTER_MANIFEST_PATH
+ value: "{{ required "A valid .Values.dgctlStorage.manifest entry required" .Values.dgctlStorage.manifest }}"
+- name: TWINS_IMPORTER_NUMBER_SCHEMA_BACKUPS
+ value: "{{ .Values.importer.cleaner.versionLimit }}"
+- name: TWINS_S3_RETRY_MAX_ATTEMPTS
+ value: "{{ .Values.importer.retry.download.maxAttempts }}"
+- name: TWINS_S3_RETRY_DELAY
+ value: "{{ .Values.importer.retry.download.delay }}"
+- name: TWINS_IMPORTER_PSQL_RETRY_MAX_ATTEMPTS
+ value: "{{ .Values.importer.retry.execute.maxAttempts }}"
+- name: TWINS_IMPORTER_PSQL_RETRY_DELAY
+ value: "{{ .Values.importer.retry.execute.delay }}"
+{{- end }}
+
{{/*
Return the target Kubernetes version
*/}}
diff --git a/charts/twins-api/templates/importer/cleaner/job.yaml b/charts/twins-api/templates/importer/cleaner/job.yaml
new file mode 100644
index 000000000..b59fb8573
--- /dev/null
+++ b/charts/twins-api/templates/importer/cleaner/job.yaml
@@ -0,0 +1,33 @@
+{{- if and .Values.importer.cleaner.enabled .Values.importer.postgres.schemaSwitchEnabled }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ include "twins.importer.name" . }}-cleaner
+ labels:
+ {{- include "twins.importer.labels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": post-install,post-upgrade
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+spec:
+ backoffLimit: 0
+ template:
+ metadata:
+ name: {{ include "twins.importer.name" . }}-cleaner
+ labels:
+ {{- include "twins.importer.labels" . | nindent 8 }}
+ spec:
+ restartPolicy: Never
+ containers:
+ - name: cleaner-job
+ image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}
+ imagePullPolicy: {{ .Values.imagePullPolicy }}
+ command: [ "twins-importer", "clear" ]
+ resources:
+ {{- toYaml .Values.importer.cleaner.resources | nindent 12 }}
+ env:
+ {{- include "twins.env.importer" . | nindent 12 }}
+ {{- with .Values.importer.nodeSelector }}
+ nodeSelector:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+{{- end }}
diff --git a/charts/twins-api/templates/importer/job.yaml b/charts/twins-api/templates/importer/job.yaml
new file mode 100644
index 000000000..74f16b12a
--- /dev/null
+++ b/charts/twins-api/templates/importer/job.yaml
@@ -0,0 +1,53 @@
+{{- if .Values.importer.enabled }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: {{ include "twins.importer.name" . }}
+ labels:
+ {{- include "twins.importer.labels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": pre-install,pre-upgrade
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+ "helm.sh/hook-weight": "10"
+spec:
+ backoffLimit: 0
+ template:
+ metadata:
+ name: {{ include "twins.importer.name" . }}
+ labels:
+ {{- include "twins.importer.labels" . | nindent 8 }}
+ spec:
+ restartPolicy: Never
+ {{- if .Values.importer.initialDelaySeconds }}
+ initContainers:
+ - name: delay
+ image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}
+ command: [ "sh", "-c", "sleep {{ .Values.importer.initialDelaySeconds }}" ]
+ resources:
+ {{- toYaml .Values.importer.resources | nindent 12 }}
+ {{- end }}
+ containers:
+ - name: importer
+ image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}
+ imagePullPolicy: {{ .Values.imagePullPolicy }}
+ command: [ "twins-importer", "import" ]
+ resources:
+ {{- toYaml .Values.importer.resources | nindent 12 }}
+ env:
+ {{- include "twins.env.importer" . | nindent 12 }}
+ {{- if .Values.importer.persistentVolume.enabled }}
+ volumeMounts:
+ - name: {{ include "twins.importer.name" . }}-pv
+ mountPath: "/tmp"
+ {{- end }}
+ {{- if .Values.importer.persistentVolume.enabled }}
+ volumes:
+ - name: {{ include "twins.importer.name" . }}-pv
+ persistentVolumeClaim:
+ claimName: {{ include "twins.importer.name" . }}
+ {{- end }}
+ {{- with .Values.importer.nodeSelector }}
+ nodeSelector:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+{{- end }}
\ No newline at end of file
diff --git a/charts/twins-api/templates/importer/pvc.yaml b/charts/twins-api/templates/importer/pvc.yaml
new file mode 100644
index 000000000..c64ae1a75
--- /dev/null
+++ b/charts/twins-api/templates/importer/pvc.yaml
@@ -0,0 +1,21 @@
+{{- if .Values.importer.persistentVolume.enabled }}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: {{ include "twins.importer.name" . }}
+ labels:
+ {{- include "twins.importer.labels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": pre-install,pre-upgrade
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+ "helm.sh/hook-weight": "-20"
+spec:
+ storageClassName: {{ .Values.importer.persistentVolume.storageClass }}
+ {{- with .Values.importer.persistentVolume.accessModes }}
+ accessModes:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+ resources:
+ requests:
+ storage: {{ .Values.importer.persistentVolume.size }}
+{{- end }}
diff --git a/charts/twins-api/templates/migrate/job.yaml b/charts/twins-api/templates/migrate/job.yaml
index 42db6506f..747a199e5 100644
--- a/charts/twins-api/templates/migrate/job.yaml
+++ b/charts/twins-api/templates/migrate/job.yaml
@@ -1,7 +1,8 @@
+{{- if not .Values.importer.enabled }}
apiVersion: batch/v1
kind: Job
metadata:
- name: {{ include "twins.migrate.name" . }}-migrate
+ name: {{ include "twins.migrate.name" . }}
labels:
{{- include "twins.migrate.labels" . | nindent 4 }}
annotations:
@@ -38,3 +39,4 @@ spec:
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
+{{- end }}
diff --git a/charts/twins-api/templates/secret-post.yaml b/charts/twins-api/templates/secret-post.yaml
new file mode 100644
index 000000000..847e4f903
--- /dev/null
+++ b/charts/twins-api/templates/secret-post.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Secret
+metadata:
+ name: {{ include "twins.secret.jobs.name" . }}
+ labels:
+ {{- include "twins.labels" . | nindent 4 }}
+ annotations:
+ "helm.sh/hook": post-install,post-upgrade
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+ "helm.sh/hook-weight": "-20"
+type: Opaque
+data:
+ dbROPassword: {{ required "A valid .Values.postgres.ro.password required" .Values.postgres.ro.password | b64enc }}
+ dbRWPassword: {{ required "A valid .Values.postgres.rw.password required" .Values.postgres.rw.password | b64enc }}
+ dgctlStorageAccessKey: {{ required "A valid .Values.dgctlStorage.accessKey required" .Values.dgctlStorage.accessKey | b64enc }}
+ dgctlStorageSecretKey: {{ required "A valid .Values.dgctlStorage.secretKey required" .Values.dgctlStorage.secretKey | b64enc }}
diff --git a/charts/twins-api/templates/secret-jobs.yaml b/charts/twins-api/templates/secret-pre.yaml
similarity index 68%
rename from charts/twins-api/templates/secret-jobs.yaml
rename to charts/twins-api/templates/secret-pre.yaml
index ce1f88c71..4e113f2dd 100644
--- a/charts/twins-api/templates/secret-jobs.yaml
+++ b/charts/twins-api/templates/secret-pre.yaml
@@ -12,3 +12,5 @@ type: Opaque
data:
dbROPassword: {{ required "A valid .Values.postgres.ro.password required" .Values.postgres.ro.password | b64enc }}
dbRWPassword: {{ required "A valid .Values.postgres.rw.password required" .Values.postgres.rw.password | b64enc }}
+ dgctlStorageAccessKey: {{ required "A valid .Values.dgctlStorage.accessKey required" .Values.dgctlStorage.accessKey | b64enc }}
+ dgctlStorageSecretKey: {{ required "A valid .Values.dgctlStorage.secretKey required" .Values.dgctlStorage.secretKey | b64enc }}
diff --git a/charts/twins-api/values.yaml b/charts/twins-api/values.yaml
index 2eadb2a72..9fa720c16 100644
--- a/charts/twins-api/values.yaml
+++ b/charts/twins-api/values.yaml
@@ -14,6 +14,21 @@ dgctlDockerRegistry: ''
imagePullSecrets: []
+# @section Deployment Artifacts Storage settings
+
+# @param dgctlStorage.host S3 host. Format: `host:port`. **Required**
+# @param dgctlStorage.bucket S3 bucket name. **Required**
+# @param dgctlStorage.accessKey S3 access key for accessing the bucket. **Required**
+# @param dgctlStorage.secretKey S3 secret key for accessing the bucket. **Required**
+# @param dgctlStorage.manifest The path to the [manifest file](https://docs.2gis.com/en/on-premise/overview#nav-lvl2@paramCommon_deployment_steps). Format: `manifests/0000000000.json`
This file contains the description of pieces of data that the service requires to operate. **Required**
+
+dgctlStorage:
+ host: ''
+ bucket: ''
+ accessKey: ''
+ secretKey: ''
+ manifest: ''
+
image:
repository: 2gis-on-premise/twins-api
tag: 1.6.0
@@ -22,7 +37,8 @@ image:
# @section API service settings
api:
-
+ # @param api.logLevel Log level (debug|info|warning|error)
+ logLevel: info
# @param api.strategy.type Type of Kubernetes deployment. Can be `Recreate` or `RollingUpdate`.
# @param api.strategy.rollingUpdate.maxUnavailable Maximum number of pods that can be created over the desired number of pods when doing [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment).
# @param api.strategy.rollingUpdate.maxSurge Maximum number of pods that can be unavailable during the [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment) process.
@@ -45,6 +61,12 @@ api:
replicas: 1
+ # @section api.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+ # @param api.resources.requests.cpu A CPU request.
+ # @param api.resources.requests.memory A memory request.
+ # @param api.resources.limits.cpu A CPU limit.
+ # @param api.resources.limits.memory A memory limit.
+
resources:
requests:
cpu: 50m
@@ -131,6 +153,12 @@ migrate:
initialDelaySeconds: 0
+ # @section migrate.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+ # @param migrate.resources.requests.cpu A CPU request.
+ # @param migrate.resources.requests.memory A memory request.
+ # @param migrate.resources.limits.cpu A CPU limit.
+ # @param migrate.resources.limits.memory A memory limit.
+
resources:
requests:
cpu: 10m
@@ -187,16 +215,82 @@ postgres:
password: ''
-# @section Limits
+# @section Kubernetes Importer job settings
+
+# @extra importer **Common settings**
+# @param importer.enabled If importer is enabled for the service
+# @param importer.nodeSelector Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector)
+# @param importer.initialDelaySeconds Number of seconds after the container has started before liveness or readiness probes are initiated
+# @param importer.retry.download.maxAttempts The maximum number of retries download before stopping
+# @param importer.retry.download.delay Delay until the retry attempts download
+# @param importer.retry.execute.maxAttempts The maximum number of retries execute psql command before stopping
+# @param importer.retry.execute.delay Delay until the retry attempts execute
+
+importer:
+ enabled: true
+ nodeSelector: {}
+ initialDelaySeconds: 1
+ retry:
+ download:
+ maxAttempts: 3
+ delay: 1s
+ execute:
+ maxAttempts: 3
+ delay: 1s
+
+ # @section importer.postgres **Database settings**
+ # @param importer.postgres.schemaSwitchEnabled Automatic switch PostgreSQL schema on releases
+
+ postgres:
+ schemaSwitchEnabled: true
+
+ # @section importer.persistentVolume **Persistent Volume settings**
-# @extra api.resources **Limits for the API service**
-# @param api.resources.requests.cpu A CPU request.
-# @param api.resources.requests.memory A memory request.
-# @param api.resources.limits.cpu A CPU limit.
-# @param api.resources.limits.memory A memory limit.
+ # @param importer.persistentVolume.enabled If [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) is enabled for the service
+ # @param importer.persistentVolume.accessModes Persistent Volume [Access Mode](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes)
+ # @param importer.persistentVolume.storageClass Kubernetes [Storage Classes](https://kubernetes.io/docs/concepts/storage/storage-classes/)
+ # @param importer.persistentVolume.size Volume size
+
+ persistentVolume:
+ enabled: false
+ accessModes:
+ - ReadWriteOnce
+ storageClass: topolvm-ext4
+ size: 50Gi
-# @extra migrate.resources **Limits for the Migrate service**
-# @param migrate.resources.requests.cpu A CPU request.
-# @param migrate.resources.requests.memory A memory request.
-# @param migrate.resources.limits.cpu A CPU limit.
-# @param migrate.resources.limits.memory A memory limit.
+ # @section importer.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+ # @param importer.resources.requests.cpu A CPU request
+ # @param importer.resources.requests.memory A memory request
+ # @param importer.resources.limits.cpu A CPU limit
+ # @param importer.resources.limits.memory A memory limit
+
+ resources:
+ requests:
+ cpu: 256m
+ memory: 512Mi
+ limits:
+ cpu: 2
+ memory: 2048Mi
+
+ # @section importer.cleaner **Cleaner scheme settings**
+ # @param importer.cleaner.enabled If clean schemes is enabled for the service
+ # @param importer.cleaner.versionLimit Number of backup schemes
+
+
+ cleaner:
+ enabled: true
+ versionLimit: 2
+
+ # @section importer.cleaner.resources **Kubernetes [resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) settings**
+ # @param importer.cleaner.resources.requests.cpu A CPU request
+ # @param importer.cleaner.resources.requests.memory A memory request
+ # @param importer.cleaner.resources.limits.cpu A CPU limit
+ # @param importer.cleaner.resources.limits.memory A memory limit
+
+ resources:
+ requests:
+ cpu: 50m
+ memory: 128Mi
+ limits:
+ cpu: 1000m
+ memory: 512Mi