Skip to content

Commit

Permalink
Add support to configure times to suspend chaos.
Browse files Browse the repository at this point in the history
  • Loading branch information
twildeboer committed Dec 8, 2017
1 parent fc73a95 commit 39c68fe
Show file tree
Hide file tree
Showing 12 changed files with 990 additions and 14 deletions.
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ RUN go build -o /bin/chaoskube -v \
FROM alpine:3.6
MAINTAINER Linki <[email protected]>

RUN apk --no-cache add tzdata

RUN addgroup -S chaoskube && adduser -S -g chaoskube chaoskube
COPY --from=builder /bin/chaoskube /bin/chaoskube

Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,26 @@ spec:
...
```

## Limiting the Chaos

You can limit when chaos introduced. To turn on the feature, add the `--limit-chaos` option and the `--location` option, which requires a timezone name from the [(IANA) tz databse](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Alternatively, you can use `UTC` or `Local` as the location.
By default, this will only allow chaos to be introduced between 9:30 and 14:30, and not on Saturday or Sunday. You can also explicitly add a list of `YYYY-MM-DD`-formatted dates as "holidays". See the options chart below for more details.

## Options

| Option | Description | Default |
|------------------|---------------------------------------------------------------------|------------------------|
| `--interval` | interval between pod terminations | 10m |
| `--labels` | label selector to filter pods by | (matches everything) |
| `--annotations` | annotation selector to filter pods by | (matches everything) |
| `--namespaces` | namespace selector to filter pods by | (all namespaces) |
| `--dry-run` | don't kill pods, only log what would have been done | true |
| `--limit-chaos` | limit chaos according to specified times/days | false |
| `--location` | timezone from tz database, e.g "America/New_York", "UTC" or "Local" | (none) |
| `--off-days` | days when chaos is to be suspended. (Or "none") | "Saturday,Sunday" |
| `--chaos-hrs` | start and end time for introducing chaos (24hr time) | "start:9:30,end:14:30" |
| `--holidays` | comma-separated, "YYYY-MM-DD" days to skip chaos | (empty list) |

## Contributing

Feel free to create issues or submit pull requests.
2 changes: 1 addition & 1 deletion chaoskube/chaoskube_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"k8s.io/client-go/kubernetes/fake"
"k8s.io/client-go/pkg/api/v1"

"github.com/linki/chaoskube/util"
"github.com/twildeboer/chaoskube/util"
)

var logOutput = bytes.NewBuffer([]byte{})
Expand Down
11 changes: 11 additions & 0 deletions examples/helm/chaoskube/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
appVersion: 0.6.2
description: Chaoskube periodically kills random pods in your Kubernetes cluster.
engine: gotpl
home: https://github.com/linki/chaoskube
maintainers:
- email: [email protected]
name: Martin Linkhorst
name: chaoskube
sources:
- https://github.com/linki/chaoskube
version: 0.6.2
71 changes: 71 additions & 0 deletions examples/helm/chaoskube/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Chaoskube Helm Chart

[chaoskube](https://github.com/linki/chaoskube) periodically kills random pods in your Kubernetes cluster.

## TL;DR;

```console
$ helm install stable/chaoskube
```

## Installing the Chart

To install the chart with the release name `my-release`:

```console
$ helm install --name my-release stable/chaoskube
```

The command deploys chaoskube on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.

## Uninstalling the Chart

To uninstall/delete the my-release deployment:

```console
$ helm delete my-release
```

The command removes all the Kubernetes components associated with the chart and deletes the release.

## Configuration

By default `chaoskube` runs in dry-run mode so it doesn't actually kill anything.
If you're sure you want to use it run `helm` with:

```console
$ helm install stable/chaoskube --set dryRun=false
```

| Parameter | Description | Default |
|---------------------------|-----------------------------------------------------|-----------------------------------|
| `name` | container name | chaoskube |
| `image` | docker image | quay.io/linki/chaoskube |
| `imageTag` | docker image tag | v0.4.0 |
| `replicas` | number of replicas to run | 1 |
| `interval` | interval between pod terminations | 10m |
| `labels` | label selector to filter pods by | "" (matches everything) |
| `annotations` | annotation selector to filter pods by | "" (matches everything) |
| `namespaces` | namespace selector to filter pods by | "" (all namespaces) |
| `dryRun` | don't kill pods, only log what would have been done | true |
| `limitChaos` | limit chaos according to specified times/days | false |
| `location` | timezone from tz database, e.g "America/New_York" | (none) |
| `offDays` | days when chaos is to be suspended. (Or "none") | "Saturday,Sunday" |
| `chaosHrs.start` | start time for introducing chaos (24hr time) | 9:30 |
| `chaosHrs.end` | end time for introducing chaos (24hr time) | 14:30 |
| `holidays` | comma-separated, "YYYY-MM-DD" days to skip chaos | "" (none) |
| `resources.cpu` | cpu resource requests and limits | 10m |
| `resources.memory` | memory resource requests and limits | 16Mi |

Setting label and namespaces selectors from the shell can be tricky but is possible (example with zsh):

```console
$ helm install \
--set labels='app=mate\,stage!=prod',namespaces='!kube-system\,!production' \
stable/chaoskube --debug --dry-run | grep -A4 args
args:
- --in-cluster
- --interval=10m
- --labels=app=foo,stage!=prod
- --namespaces=!kube-system,!production
```
9 changes: 9 additions & 0 deletions examples/helm/chaoskube/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
chaoskube is running and will kill arbitrary pods every {{ .Values.interval }}.

You can follow the logs to see what chaoskube does:

POD=$(kubectl get pods -l app={{ printf "%s-%s" .Release.Name .Values.name }} --namespace {{ .Release.Namespace }} --output name)
kubectl logs -f $POD --namespace={{ .Release.Namespace }}
{{ if .Values.dryRun }}
You are running in dry-run mode. No pod is actually terminated.
{{ end -}}
46 changes: 46 additions & 0 deletions examples/helm/chaoskube/templates/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: {{ printf "%s-%s" .Release.Name .Values.name }}
labels:
app: {{ printf "%s-%s" .Release.Name .Values.name }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
replicas: {{ .Values.replicas }}
template:
metadata:
labels:
app: {{ printf "%s-%s" .Release.Name .Values.name }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
imagePullSecrets:
- name: {{ .Values.docker.secretName }}
containers:
- name: {{ .Values.name }}
image: {{ .Values.image }}:{{ .Values.imageTag }}
args:
- --interval={{ .Values.interval }}
- --labels={{ .Values.labels }}
- --annotations={{ .Values.annotations }}
- --namespaces={{ .Values.namespaces }}
- --location={{ .Values.location }}
- --off-days={{ .Values.offDays }}
- --holidays={{ .Values.holidays }}
- --chaos-hours=start:{{ .Values.chaosHrs.start }},end:{{ .Values.chaosHrs.end }}
{{- if not .Values.dryRun }}
- --no-dry-run
{{- end }}
{{- if .Values.limitChaos }}
- --limit-chaos
{{- end }}
resources:
requests:
cpu: {{ .Values.resources.cpu }}
memory: {{ .Values.resources.memory }}
limits:
cpu: {{ .Values.resources.cpu }}
memory: {{ .Values.resources.memory }}
50 changes: 50 additions & 0 deletions examples/helm/chaoskube/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# container name
name: chaoskube

# docker image
image: quay.io/linki/chaoskube

# docker image tag
imageTag: v0.6.1

# number of replicas to run
replicas: 1

# interval between pod terminations
interval: 10m

# label selector to filter pods by, e.g. app=foo,stage!=prod
labels:

# annotation selector to filter pods by, e.g. chaos.alpha.kubernetes.io/enabled=true
annotations:

# namespace selector to filter pods by, e.g. '!kube-system,!production' (use quotes)
namespaces:

# don't kill pods, only log what would have been done
dryRun: true

# Limit chaos times (default: false)
limitChaos: true

# Timezone location from the "tz database" (e.g. "America/Los_Angeles") NO DEFAULT
# Also allowed: "UTC", "Local"
location: Local

# Comma-separated list of weekday names when chaos is suspended. Default: "Saturday, Sunday".
# (Use "none" for no off days.)
offDays:

# Daily start and end times for introducing chaos. Default: start: 9:30, end: 16:30
chaosHrs:
start: 9:30
end: 13:30

# A list of ISO 8601 dates (YYYY-MM-DD) when chaos is suspended. Default: empty list.
holidays:

# resource requests and limits
resources:
cpu: 10m
memory: 16Mi
15 changes: 14 additions & 1 deletion examples/chaoskube.yaml → examples/kubernetes/chaoskube.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec:
serviceAccountName: chaoskube
containers:
- name: chaoskube
image: quay.io/linki/chaoskube:v0.6.1
image: quay.io/linki/chaoskube:v0.6.2
args:
# kill a pod every 10 minutes
- --interval=10m
Expand All @@ -26,6 +26,19 @@ spec:
- --namespaces=!kube-system
# terminate pods for real: this disables dry-run mode which is on by default
- --no-dry-run
# limit chaos times as specified in following options (default is off)
# --no-limit-chaos or missing options turns this off
- --limit-chaos
# timezone location from the "tz database" (e.g. "America/Los_Angeles") NO DEFAULT
# also allowed: "UTC" or "Local"
- --location=Local
# comma-separated list of weekday names when chaos is suspended. Default: "Saturday,Sunday".
# (Use "none" for no off days.)
- --off-days=
# list of ISO 8601 dates (YYYY-MM-DD) when chaos is suspended. Default: empty list.
- --holidays=
# daily start and end times for introducing chaos. Default: start:9:30,end:16:30
- --chaos-hours=

---

Expand Down
File renamed without changes.
Loading

0 comments on commit 39c68fe

Please sign in to comment.