Mediapipe graph process time metric #2942
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bstrzele
requested review from
dtrawins and
dkalinowski
and removed request for
dtrawins
bstrzele
force-pushed
the
mp_process_time_metric
branch
from
08f22b8 to
8493488
Compare
dtrawins
reviewed
Dec 23, 2024
bstrzele
force-pushed
the
mp_process_time_metric
branch
from
9fd076f to
8174bf7
Compare
bstrzele
force-pushed
the
mp_process_time_metric
branch
2 times, most recently
from
2df7c56 to
ff2fa6b
Compare
dkalinowski
reviewed
Jan 7, 2025
| @@ -196,6 +203,9 @@ class MediapipeGraphExecutor { | |||
| INCREMENT_IF_ENABLED(this->mediapipeServableMetricReporter->getGraphErrorMetric(executionContext)); | |||
| } | |||
| MP_RETURN_ON_FAIL(status, "graph wait until done", mediapipeAbslToOvmsStatus(status.code())); | |||
| timer.stop(PROCESS); | |||
| double processTime = timer.template elapsed<std::chrono::microseconds>(PROCESS); | |||
| OBSERVE_IF_ENABLED(this->mediapipeServableMetricReporter->getProcessingTimeMetric(executionContext), processTime); | |||
There was a problem hiding this comment.
It is not clear to me if we want or don't want to register processing times of failed execution @dtrawins @bstrzele
In either way, I don't think this is correct because:
a) we do not register time in all error return codepaths
b) we do register time in one specific error return codepath (when if (outputPollers.size() != outputPollersWithReceivedPacket.size()) {)
dkalinowski
reviewed
Jan 7, 2025
docs/metrics.md
Outdated
| @@ -241,6 +241,7 @@ For [MediaPipe Graphs](./mediapipe.md) execution there are 4 generic metrics whi | |||
| | counter | ovms_responses | Useful to track number of packets generated by MediaPipe graph. Keep in mind that single request may trigger production of multiple (or zero) packets, therefore tracking number of responses is complementary to tracking accepted requests. For example tracking streaming partial responses of LLM text generation graphs. | | |||
| | gauge | ovms_current_graphs | Number of graphs currently in-process. For unary communication it is equal to number of currently processing requests (each request initializes separate MediaPipe graph). For streaming communication it is equal to number of active client connections. Each connection is able to reuse the graph and decide when to delete it when the connection is closed. | | |||
| | counter | ovms_graph_error | Counts errors in MediaPipe graph execution phase. For example V3 LLM text generation fails in LLMCalculator due to missing prompt - calculator returns an error and graph cancels. | | |||
| | histogram | ovms_graph_processing_time_us | Time for which mediapipe graph was opened. | | |||
There was a problem hiding this comment.
Please explain more - for successful only? for failed as well? which fails count?
bstrzele
force-pushed
the
mp_process_time_metric
branch
2 times, most recently
from
ae8dd5a to
2455f22
Compare
dkalinowski
approved these changes
Jan 10, 2025
bstrzele
force-pushed
the
mp_process_time_metric
branch
from
f6c43b3 to
a51b6e7
Compare
dtrawins
reviewed
Jan 28, 2025
docs/metrics.md
Outdated
| @@ -241,6 +241,7 @@ For [MediaPipe Graphs](./mediapipe.md) execution there are 4 generic metrics whi | |||
| | counter | ovms_responses | Useful to track number of packets generated by MediaPipe graph. Keep in mind that single request may trigger production of multiple (or zero) packets, therefore tracking number of responses is complementary to tracking accepted requests. For example tracking streaming partial responses of LLM text generation graphs. | | |||
| | gauge | ovms_current_graphs | Number of graphs currently in-process. For unary communication it is equal to number of currently processing requests (each request initializes separate MediaPipe graph). For streaming communication it is equal to number of active client connections. Each connection is able to reuse the graph and decide when to delete it when the connection is closed. | | |||
| | counter | ovms_graph_error | Counts errors in MediaPipe graph execution phase. For example V3 LLM text generation fails in LLMCalculator due to missing prompt - calculator returns an error and graph cancels. | | |||
| | histogram | ovms_graph_processing_time_us | Time for which mediapipe graph was opened and has been successfully closed. | | |||
There was a problem hiding this comment.
I think in this table we should have also a column indicating for which type of request is applicable each metric. Right know everything is mixed. Some are for MP graphs only and some for individual model only. It could be adjusted in a separate PR.
There was a problem hiding this comment.
Every metric in this table should only be relevant for mediapipe
There was a problem hiding this comment.
But this one probably could use come kind of indicator
bstrzele
force-pushed
the
mp_process_time_metric
branch
2 times, most recently
from
b38186a to
02d5b8e
Compare
bstrzele
force-pushed
the
mp_process_time_metric
branch
from
02d5b8e to
71d8530
Compare
dtrawins
approved these changes
Jan 29, 2025
Co-authored-by: Trawinski, Dariusz <[email protected]>
bstrzele
force-pushed
the
mp_process_time_metric
branch
from
71d8530 to
f838549
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-160188
A Prometheus metric ovms_graph_processing_time_us for tracking graph processing of histogram type.
Uses the same buckets like for single model execution metric.
The metric tracks time allocated by the graph to serve a single user request for unary calls or the session time for the stream requests.
🧪 Checklist
``