Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to configure times to suspend chaos. #50

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ RUN go build -o /bin/chaoskube -v \
FROM alpine:3.6
MAINTAINER Linki <[email protected]>

RUN apk --no-cache add tzdata

RUN addgroup -S chaoskube && adduser -S -g chaoskube chaoskube
COPY --from=builder /bin/chaoskube /bin/chaoskube

Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,26 @@ spec:
...
```

## Limiting the Chaos

You can limit when chaos introduced. To turn on the feature, add the `--limit-chaos` option and the `--location` option, which requires a timezone name from the [(IANA) tz databse](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Alternatively, you can use `UTC` or `Local` as the location.
By default, this will only allow chaos to be introduced between 9:30 and 14:30, and not on Saturday or Sunday. You can also explicitly add a list of `YYYY-MM-DD`-formatted dates as "holidays". See the options chart below for more details.

## Options

| Option | Description | Default |
|------------------|---------------------------------------------------------------------|------------------------|
| `--interval` | interval between pod terminations | 10m |
| `--labels` | label selector to filter pods by | (matches everything) |
| `--annotations` | annotation selector to filter pods by | (matches everything) |
| `--namespaces` | namespace selector to filter pods by | (all namespaces) |
| `--dry-run` | don't kill pods, only log what would have been done | true |
| `--limit-chaos` | limit chaos according to specified times/days | false |
| `--location` | timezone from tz database, e.g "America/New_York", "UTC" or "Local" | (none) |
| `--off-days` | days when chaos is to be suspended. (Or "none") | "Saturday,Sunday" |
| `--chaos-hrs` | start and end time for introducing chaos (24hr time) | "start:9:30,end:14:30" |
| `--holidays` | comma-separated, "YYYY-MM-DD" days to skip chaos | (empty list) |

## Contributing

Feel free to create issues or submit pull requests.
11 changes: 11 additions & 0 deletions examples/helm/chaoskube/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
appVersion: 0.6.2
description: Chaoskube periodically kills random pods in your Kubernetes cluster.
engine: gotpl
home: https://github.com/linki/chaoskube
maintainers:
- email: [email protected]
name: Martin Linkhorst
name: chaoskube
sources:
- https://github.com/linki/chaoskube
version: 0.6.2
71 changes: 71 additions & 0 deletions examples/helm/chaoskube/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Chaoskube Helm Chart

[chaoskube](https://github.com/linki/chaoskube) periodically kills random pods in your Kubernetes cluster.

## TL;DR;

```console
$ helm install stable/chaoskube
```

## Installing the Chart

To install the chart with the release name `my-release`:

```console
$ helm install --name my-release stable/chaoskube
```

The command deploys chaoskube on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.

## Uninstalling the Chart

To uninstall/delete the my-release deployment:

```console
$ helm delete my-release
```

The command removes all the Kubernetes components associated with the chart and deletes the release.

## Configuration

By default `chaoskube` runs in dry-run mode so it doesn't actually kill anything.
If you're sure you want to use it run `helm` with:

```console
$ helm install stable/chaoskube --set dryRun=false
```

| Parameter | Description | Default |
|---------------------------|-----------------------------------------------------|-----------------------------------|
| `name` | container name | chaoskube |
| `image` | docker image | quay.io/linki/chaoskube |
| `imageTag` | docker image tag | v0.4.0 |
| `replicas` | number of replicas to run | 1 |
| `interval` | interval between pod terminations | 10m |
| `labels` | label selector to filter pods by | "" (matches everything) |
| `annotations` | annotation selector to filter pods by | "" (matches everything) |
| `namespaces` | namespace selector to filter pods by | "" (all namespaces) |
| `dryRun` | don't kill pods, only log what would have been done | true |
| `limitChaos` | limit chaos according to specified times/days | false |
| `location` | timezone from tz database, e.g "America/New_York" | (none) |
| `offDays` | days when chaos is to be suspended. (Or "none") | "Saturday,Sunday" |
| `chaosHrs.start` | start time for introducing chaos (24hr time) | 9:30 |
| `chaosHrs.end` | end time for introducing chaos (24hr time) | 14:30 |
| `holidays` | comma-separated, "YYYY-MM-DD" days to skip chaos | "" (none) |
| `resources.cpu` | cpu resource requests and limits | 10m |
| `resources.memory` | memory resource requests and limits | 16Mi |

Setting label and namespaces selectors from the shell can be tricky but is possible (example with zsh):

```console
$ helm install \
--set labels='app=mate\,stage!=prod',namespaces='!kube-system\,!production' \
stable/chaoskube --debug --dry-run | grep -A4 args
args:
- --in-cluster
- --interval=10m
- --labels=app=foo,stage!=prod
- --namespaces=!kube-system,!production
```
9 changes: 9 additions & 0 deletions examples/helm/chaoskube/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
chaoskube is running and will kill arbitrary pods every {{ .Values.interval }}.

You can follow the logs to see what chaoskube does:

POD=$(kubectl get pods -l app={{ printf "%s-%s" .Release.Name .Values.name }} --namespace {{ .Release.Namespace }} --output name)
kubectl logs -f $POD --namespace={{ .Release.Namespace }}
{{ if .Values.dryRun }}
You are running in dry-run mode. No pod is actually terminated.
{{ end -}}
44 changes: 44 additions & 0 deletions examples/helm/chaoskube/templates/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: {{ printf "%s-%s" .Release.Name .Values.name }}
labels:
app: {{ printf "%s-%s" .Release.Name .Values.name }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
replicas: {{ .Values.replicas }}
template:
metadata:
labels:
app: {{ printf "%s-%s" .Release.Name .Values.name }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
containers:
- name: {{ .Values.name }}
image: {{ .Values.image }}:{{ .Values.imageTag }}
args:
- --interval={{ .Values.interval }}
- --labels={{ .Values.labels }}
- --annotations={{ .Values.annotations }}
- --namespaces={{ .Values.namespaces }}
- --location={{ .Values.location }}
- --off-days={{ .Values.offDays }}
- --holidays={{ .Values.holidays }}
- --chaos-hours=start:{{ .Values.chaosHrs.start }},end:{{ .Values.chaosHrs.end }}
{{- if not .Values.dryRun }}
- --no-dry-run
{{- end }}
{{- if .Values.limitChaos }}
- --limit-chaos
{{- end }}
resources:
requests:
cpu: {{ .Values.resources.cpu }}
memory: {{ .Values.resources.memory }}
limits:
cpu: {{ .Values.resources.cpu }}
memory: {{ .Values.resources.memory }}
50 changes: 50 additions & 0 deletions examples/helm/chaoskube/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# container name
name: chaoskube

# docker image
image: quay.io/linki/chaoskube

# docker image tag
imageTag: v0.6.2

# number of replicas to run
replicas: 1

# interval between pod terminations
interval: 10m

# label selector to filter pods by, e.g. app=foo,stage!=prod
labels:

# annotation selector to filter pods by, e.g. chaos.alpha.kubernetes.io/enabled=true
annotations:

# namespace selector to filter pods by, e.g. '!kube-system,!production' (use quotes)
namespaces:

# don't kill pods, only log what would have been done
dryRun: true

# Limit chaos times (default: false)
limitChaos: true

# Timezone location from the "tz database" (e.g. "America/Los_Angeles") NO DEFAULT
# Also allowed: "UTC", "Local"
location: Local

# Comma-separated list of weekday names when chaos is suspended. Default: "Saturday, Sunday".
# (Use "none" for no off days.)
offDays:

# Daily start and end times for introducing chaos. Default: start: 9:30, end: 16:30
chaosHrs:
start: 9:30
end: 13:30

# A list of ISO 8601 dates (YYYY-MM-DD) when chaos is suspended. Default: empty list.
holidays:

# resource requests and limits
resources:
cpu: 10m
memory: 16Mi
15 changes: 14 additions & 1 deletion examples/chaoskube.yaml → examples/kubernetes/chaoskube.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec:
serviceAccountName: chaoskube
containers:
- name: chaoskube
image: quay.io/linki/chaoskube:v0.6.1
image: quay.io/linki/chaoskube:v0.6.2
args:
# kill a pod every 10 minutes
- --interval=10m
Expand All @@ -26,6 +26,19 @@ spec:
- --namespaces=!kube-system
# terminate pods for real: this disables dry-run mode which is on by default
- --no-dry-run
# limit chaos times as specified in following options (default is off)
# --no-limit-chaos or missing options turns this off
- --limit-chaos
# timezone location from the "tz database" (e.g. "America/Los_Angeles") NO DEFAULT
# also allowed: "UTC" or "Local"
- --location=Local
# comma-separated list of weekday names when chaos is suspended. Default: "Saturday,Sunday".
# (Use "none" for no off days.)
- --off-days=
# list of ISO 8601 dates (YYYY-MM-DD) when chaos is suspended. Default: empty list.
- --holidays=
# daily start and end times for introducing chaos. Default: start:9:30,end:16:30
- --chaos-hours=

---

Expand Down
File renamed without changes.
Loading