Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syslog-ng custom log metrics #1554

Merged
merged 10 commits into from
Nov 1, 2023
Merged

syslog-ng custom log metrics #1554

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
1ab0dca
feat: syslogng source metrics
pepov Oct 20, 2023
2e25a4c
feat: syslogng output metrics
pepov Oct 20, 2023
358a911
feat: syslog-ng add more granular info to output metrics
pepov Oct 27, 2023
5d74c1d
feat: syslog-ng custom metrics config
pepov Oct 27, 2023
de1482f
feat: syslog-ng handle empty flow match same way as if it was undefin…
pepov Oct 27, 2023
eceaaf2
feat: syslog-ng custom metrics, fix tests
pepov Oct 27, 2023
3f31b3d
feat: syslog-ng custom metrics, rename variable for clarity
pepov Oct 31, 2023
69cf2af
feat: syslog-ng custom metrics, add logging name to source metrics
pepov Oct 31, 2023
e00a6de
feat: syslog-ng custom metrics, add logging to output metrics as well
pepov Oct 31, 2023
edbb436
feat: syslog-ng custom metrics, simplify example
pepov Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ codegen: ${CONTROLLER_GEN} tidy ## Generate code

.PHONY: install
install: manifests ## Install CRDs into the cluster in ~/.kube/config
kubectl create -f config/crd/bases || kubectl replace -f config/crd/bases
kubectl apply -f config/crd/bases --server-side --force-conflicts

.PHONY: license-check
license-check: ${LICENSEI} .licensei.cache ## Run license check
Expand Down
13 changes: 13 additions & 0 deletions charts/logging-operator/crds/logging.banzaicloud.io_loggings.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14652,6 +14652,19 @@ spec:
type: object
skipRBACCreate:
type: boolean
sourceMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
statefulSet:
properties:
metadata:
Expand Down
13 changes: 13 additions & 0 deletions charts/logging-operator/crds/logging.banzaicloud.io_syslogngclusterflows.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,19 @@ spec:
- pattern
type: object
type: object
outputMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
type: object
status:
properties:
Expand Down
13 changes: 13 additions & 0 deletions charts/logging-operator/crds/logging.banzaicloud.io_syslogngflows.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,19 @@ spec:
- pattern
type: object
type: object
outputMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
type: object
status:
properties:
Expand Down
13 changes: 13 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_loggings.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14652,6 +14652,19 @@ spec:
type: object
skipRBACCreate:
type: boolean
sourceMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
statefulSet:
properties:
metadata:
Expand Down
13 changes: 13 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_syslogngclusterflows.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,19 @@ spec:
- pattern
type: object
type: object
outputMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
type: object
status:
properties:
Expand Down
13 changes: 13 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_syslogngflows.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,19 @@ spec:
- pattern
type: object
type: object
outputMetrics:
items:
properties:
key:
type: string
labels:
additionalProperties:
type: string
type: object
level:
type: integer
type: object
type: array
type: object
status:
properties:
Expand Down
55 changes: 55 additions & 0 deletions config/samples/syslog-ng-custom-metrics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
kind: Logging
apiVersion: logging.banzaicloud.io/v1beta1
metadata:
name: logging
spec:
controlNamespace: default
fluentbit: {}
syslogNG:
metrics: {}
sourceMetrics:
- key: custom_input
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGFlow
metadata:
name: all1
spec:
match: {}
outputMetrics:
- key: custom_output
labels:
flow: all1
localOutputRefs:
- http
globalOutputRefs:
- http2
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGClusterFlow
metadata:
name: all2
spec:
match: {}
outputMetrics:
- key: custom_output
labels:
flow: all2
globalOutputRefs:
- http2
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGOutput
metadata:
name: http
spec:
file:
path: "/tmp/log"
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGClusterOutput
metadata:
name: http2
spec:
file:
path: "/tmp/log"
37 changes: 37 additions & 0 deletions config/samples/syslog-ng.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ apiVersion: logging.banzaicloud.io/v1beta1
metadata:
name: logging
spec:
controlNamespace: default
syslogNG:
bufferVolumeMetrics:
mount_name: buffers
Expand All @@ -26,3 +27,39 @@ spec:
resources:
requests:
storage: 3Gi
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGFlow
metadata:
name: all1
spec:
match: {}
localOutputRefs:
- http
- http2
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGFlow
metadata:
name: all2
spec:
match: {}
localOutputRefs:
- http
- http2
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGOutput
metadata:
name: http
spec:
file:
path: "/tmp/log"
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGOutput
metadata:
name: http2
spec:
file:
path: "/tmp/log"
4 changes: 4 additions & 0 deletions docs/configuration/crds/v1beta1/syslogng_clusterflow_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ Default: -

Default: -

### outputMetrics ([]filter.MetricsProbe, optional) {#syslogngclusterflowspec-outputmetrics}

Default: -


## SyslogNGClusterFlowList

Expand Down
4 changes: 4 additions & 0 deletions docs/configuration/crds/v1beta1/syslogng_flow_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ Default: -

Default: -

### outputMetrics ([]filter.MetricsProbe, optional) {#syslogngflowspec-outputmetrics}

Default: -


## SyslogNGFilter

Expand Down
4 changes: 4 additions & 0 deletions docs/configuration/crds/v1beta1/syslogng_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ Default: -

Default: -

### sourceMetrics ([]filter.MetricsProbe, optional) {#syslogngspec-sourcemetrics}

Default: -


## SyslogNGTLS

Expand Down
4 changes: 4 additions & 0 deletions docs/configuration/plugins/syslogng-filters/parser.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ Default: -

## MetricsProbe

### - (struct{}, required) {#metricsprobe--}

Default: -

### key (string, optional) {#metricsprobe-key}

The name of the counter to create. Note that the value of this option is always prefixed with syslogng_, so for example key("my-custom-key") becomes syslogng_my-custom-key.
Expand Down
Loading