Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: prometheus metrics #62

Merged
merged 11 commits into from
Mar 19, 2024
Merged

feat: prometheus metrics #62

merged 11 commits into from
Mar 19, 2024

Conversation

jachro
Copy link
Contributor

@jachro jachro commented Mar 18, 2024

This PR:

  • adds a new GET /metrics endpoint exposing Prometheus metrics to both services
  • make the metrics list the standard JVM metrics
  • make the metrics list the standard HTTP service metrics (the search-api service)
  • make the metrics list two custom Redis metrics: the total size of the queues and a number of unprocessed events (the search-provision service)

Copy link
Member

@eikek eikek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼

).orNotFound
)

lazy val router: HttpApp[F] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now this is named router but it's an HttpApp and above is named searchAndOp.Routes and is created using the Router class. It looks a bit strange, but maybe it's just me :). Perhaps this could be app or application or just routes? The last orNotFound could also be added when the HttpApp is really needed (when added to EmberBuilder)…?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a good point. I'll try to streamline things a bit.


object Microservice extends IOApp:

private val port = port"8080"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can make this configurable as well (port and bind address)? (just a thought, because it's being addressed here anyways :))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

.map(new HttpApplication[F](_).router)

final class HttpApplication[F[_]: Async](metricsRoutes: HttpRoutes[F])
extends Http4sDsl[F]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need the dsl here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we don't

metrics <- Prometheus.metricsOps[F](registry, prefix = "server")
yield PrometheusExportService(registry).routes <+> Metrics[F](metrics)(measuredRoutes)

def makeRoutes: Resource[F, HttpRoutes[F]] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I understand these overloaded methods. What would be the default to measure when no measuredRoutes are supplied? Or are they for different things?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the reason why we need MetricsRoutes in two flavours is:

  • in the case of search-api service we collect metrics of the HTTP API (latencies, number of calls, etc) so we pass the measuredRoutes (business APIs) for which we'd like to gather the metrics.
  • in the case of search-provision we don't want any HTTP metrics to be collected but still want to create routes that are exposed by the microservice.
    Would that make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, thank you! Sorry, I somehow couldn't make the connection it refers to the two services

import io.prometheus.client.{CollectorRegistry, Collector as JCollector}
import org.http4s.metrics.prometheus.PrometheusExportService

final class CollectorRegistryBuilder[F[_]: Sync]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is mutable and we usually don't have this, should we maybe make it very clear by putting Mutable in the name? Like it is sometimes done in scala standard api (e.g. ListBuffer or something similar so you see immediately it's mutable), for me not looking at the source and having these with… methods, I would assume an immutable thing :). Or could be made immutable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I tried to make it immutable at first but ended up with code I wasn't happy with. Now, I tried again now and I think it's a bit better.

val script =
"""local xrange = redis.call('XRANGE', KEYS[1], ARGV[1], ARGV[2])
|return #xrange - 1""".stripMargin
createStringCommands.use(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!

@jachro jachro merged commit c0a94b2 into development Mar 19, 2024
2 checks passed
@jachro jachro deleted the metrics branch March 19, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants