Skip to content

Commit

Permalink
Prometheus: prevented caching of metric values.
Browse files Browse the repository at this point in the history
Notably, the "angie_http/stream_upstreams_peers_state" metrics in the
"prometheus_all.conf" template could output the same value cached from
the first evaluation of $p8st_all_ups_state for two reasons:

 1. By default, the "map" evaluation was always cached.

 2. Even non-cacheable variable values weren't flushed properly.

Meanwhile, other metrics that use $p8s_value directly worked fine only
because the "valid" flag was not set.

--HG--
branch : 1.3
  • Loading branch information
VBart committed Nov 22, 2023
1 parent fcfb771 commit ee6fcee
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 3 deletions.
1 change: 1 addition & 0 deletions conf/prometheus_all.conf
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,7 @@ angie_connections_idle $p8s_value
}

map $p8s_value $p8st_all_ups_state {
volatile;
"up" 1;
"down" 2;
"unavailable" 3;
Expand Down
16 changes: 16 additions & 0 deletions docs/xml/angie/changes.xml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,22 @@

<changes ver="1.3.2" date="">

<change type="bugfix">
<para lang="ru">
были возможны некорректные значения метрик в формате Prometheus, в значениях
которых использовались отличные от $p8s_value переменные; на практике
проблема могла наблюдаться с "angie_http_upstreams_peers_state"
и "angie_stream_upstreams_peers_state" из стандартного шаблона
"prometheus_all.conf".
</para>
<para lang="en">
possible incorrect values of metrics in Prometheus output that used variables
other than $p8s_value for their values; in practice the issue could occur with
"angie_http_upstreams_peers_state" and "angie_stream_upstreams_peers_state"
from the standard "prometheus_all.conf" template.
</para>
</change>

<change type="bugfix">
<para lang="ru">
некоторые попытки соединения с проксируемыми серверами могли не учитываться
Expand Down
7 changes: 4 additions & 3 deletions src/http/modules/ngx_http_prometheus_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ ngx_http_prometheus_render_metric(ngx_uint_t m,

metric = ctx->metrics->elts;

vv = ngx_http_get_indexed_variable(ctx->request, metric[m].value);
vv = ngx_http_get_flushed_variable(ctx->request, metric[m].value);

if (vv == NULL || vv->not_found || vv->len == 0) {
return NGX_OK;
Expand Down Expand Up @@ -558,8 +558,6 @@ ngx_http_prometheus_variable(ngx_http_request_t *r,
return NGX_OK;
}

v->not_found = 0;

switch (item->type) {

case NGX_DATA_INTEGER_TYPE:
Expand Down Expand Up @@ -597,6 +595,9 @@ ngx_http_prometheus_variable(ngx_http_request_t *r,
return NGX_ERROR;
}

v->valid = 1;
v->not_found = 0;

return NGX_OK;
}

Expand Down

0 comments on commit ee6fcee

Please sign in to comment.