Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance the Grafana JSON file #134

Merged
merged 4 commits into from
Sep 24, 2024
Merged

Enhance the Grafana JSON file #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4a13791
enhance the grafana json file
LifengWang Sep 19, 2024
0daa1d8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 19, 2024
a5836c0
update the readme for importing Grafana json file
LifengWang Sep 24, 2024
5ae6744
Merge branch 'main' of https://github.com/LifengWang/GenAIEval
LifengWang Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions evals/benchmark/grafana/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,12 @@ If you have any Grafana installation issue please check this [link](https://gra

The next step is to configure the data source for Grafana to scrape metrics from. Click on the "Data Source" button, select Prometheus, and specify the Prometheus url `localhost:9090`.

Then you need to upload a dashboard JSON file in the Grafana UI under `Home > Dashboards > Import dashboard`. You can use a file like [tgi_grafana.json](https://github.com/huggingface/text-generation-inference/blob/main/assets/tgi_grafana.json).

## 3. Import Grafana Dashboard
After setup the Grafana server, then you can import a Grafana Dashboard through uploading a dashboard JSON file in the Grafana UI under `Home > Dashboards > Import dashboard`. You can use a file like [tgi_grafana.json](https://github.com/huggingface/text-generation-inference/blob/main/assets/tgi_grafana.json).
Open the dashboard, and you will see different panels displaying the metrics data.

In this folder, we also provides some Grafana dashboard JSON files for your reference.
- `chatqna_megaservice_grafana.json`: A sample Grafana dashboard JSON file for visualizing the metrics of ChatQnA microservices. Selecting different job_name options in the top-left of the dashboard displays the metrics for the corresponding microservices.
- `tei_grafana.json`: A sample Grafana dashboard JSON file for visualizing TEI metrics.
- `tgi_grafana.json`: A sample Grafana dashboard JSON file for visualizing TGI metrics.
- `redis_grafana.json`: A sample Grafana dashboard JSON file for visualizing the Redis metrics. For importing the redis metrics, you need to add the new connection and Redis data source in Grafana. Please refer this [link](https://grafana.com/grafana/plugins/redis-datasource/?tab=installation) for more details.
101 changes: 72 additions & 29 deletions evals/benchmark/grafana/chatqna_megaservice_grafana.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,47 @@
{
"__inputs": [
{
"name": "DS_PROMETHEUS",
"label": "Prometheus Data Source",
"description": "",
"type": "datasource",
"pluginId": "prometheus",
"pluginName": "Prometheus"
}
],
"__elements": {},
"__requires": [
{
"type": "panel",
"id": "gauge",
"name": "Gauge",
"version": ""
},
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "10.0.2"
},
{
"type": "panel",
"id": "heatmap",
"name": "Heatmap",
"version": ""
},
{
"type": "datasource",
"id": "prometheus",
"name": "Prometheus",
"version": "1.0.0"
},
{
"type": "panel",
"id": "timeseries",
"name": "Time series",
"version": ""
}
],
"annotations": {
"list": [
{
Expand All @@ -18,14 +61,14 @@
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 30,
"id": 1,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -111,17 +154,17 @@
"$$hashKey": "object:638",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "rate(process_cpu_seconds_total{container=\"$container_name\"}[1m])",
"expr": "rate(process_cpu_seconds_total{job=\"$job_name\"}[1m])",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
"interval": "",
"intervalFactor": 1,
"legendFormat": "cpu",
"legendFormat": "{{ job }}",
"range": true,
"refId": "A",
"useBackend": false
Expand All @@ -133,7 +176,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -218,17 +261,17 @@
"$$hashKey": "object:638",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "process_resident_memory_bytes{container=\"$container_name\"}",
"expr": "process_resident_memory_bytes{job=\"$job_name\"}",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
"interval": "",
"intervalFactor": 1,
"legendFormat": "mem",
"legendFormat": "{{ job }}",
"range": true,
"refId": "A",
"useBackend": false
Expand All @@ -240,7 +283,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -327,11 +370,11 @@
"$$hashKey": "object:214",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "code",
"expr": "sum by(handler) (rate(http_requests_total{container=\"$container_name\", handler!~\"/metrics|/v1/health_check|none\"}[1m]))",
"expr": "sum by(handler) (rate(http_requests_total{job=\"$job_name\", handler!~\"/metrics|/v1/health_check|none\"}[1m]))",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
Expand All @@ -349,7 +392,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -467,11 +510,11 @@
"$$hashKey": "object:140",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "sum by(status) (rate(http_requests_total{container=\"$container_name\"}[1m]))",
"expr": "sum by(status) (rate(http_requests_total{job=\"$job_name\"}[1m]))",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
Expand All @@ -489,7 +532,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -592,11 +635,11 @@
"$$hashKey": "object:766",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "sum(rate(http_requests_total{status=\"5xx\", container=\"$container_name\"}[30s]))",
"expr": "sum(rate(http_requests_total{status=\"5xx\", job=\"$job_name\"}[30s]))",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
Expand All @@ -614,7 +657,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"description": "",
"fieldConfig": {
Expand Down Expand Up @@ -704,11 +747,11 @@
"$$hashKey": "object:426",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"disableTextWrap": false,
"editorMode": "code",
"expr": "histogram_quantile(0.$Percentage, rate(http_request_duration_seconds_bucket{container=\"$container_name\", handler!~\"/metrics|/v1/health_check|none\"}[1m])) ",
"expr": "histogram_quantile(0.$Percentage, rate(http_request_duration_seconds_bucket{job=\"$job_name\", handler!~\"/metrics|/v1/health_check|none\"}[1m])) ",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
Expand All @@ -726,7 +769,7 @@
{
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"fieldConfig": {
"defaults": {
Expand Down Expand Up @@ -814,10 +857,10 @@
"$$hashKey": "object:146",
"datasource": {
"type": "prometheus",
"uid": "e4584a9f-5364-4b3d-a851-7abbc5250820"
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "http_request_duration_seconds_sum{handler!=\"none\",container=\"$container_name\", handler!~\"/metrics|/v1/health_check|none\"} / http_request_duration_seconds_count",
"expr": "http_request_duration_seconds_sum{handler!=\"none\",job=\"$job_name\", handler!~\"/metrics|/v1/health_check|none\"} / http_request_duration_seconds_count",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
Expand All @@ -843,17 +886,17 @@
},
"datasource": {
"type": "prometheus",
"uid": "P1809F7CD0C75ACF3"
"uid": "${DS_PROMETHEUS}"
},
"definition": "label_values(http_requests_total{namespace=\"opea-rag\"},container)",
"definition": "label_values(http_requests_total{},job)",
"hide": 0,
"includeAll": false,
"multi": false,
"name": "container_name",
"name": "job_name",
"options": [],
"query": {
"qryType": 1,
"query": "label_values(http_requests_total{namespace=\"opea-rag\"},container)",
"query": "label_values(http_requests_total{},job)",
"refId": "PrometheusVariableQueryEditor-VariableQuery"
},
"refresh": 1,
Expand Down Expand Up @@ -925,6 +968,6 @@
"timezone": "",
"title": "ChatQnA MegaService Dashboard",
"uid": "_eX4mpl0",
"version": 12,
"version": 1,
"weekStart": ""
}
Loading