serving-dinov2

Contents

Serving dinov2 onnx model with triton. (Artifact throughput: 48.927 infer/sec)
Gradio Demo.
Docker Compose.
K8s Setting(Triton, Traefik, Promtail, Loki, Prometheus, Grafana).
Serving dinov2 TensorRT Model. (Artifact throughput: 222.66 infer/sec)
~~Serving dinov2 onnx model with Fastertransformer~~ (fastertransformer_backend don't support vit yet.)

Docker Compose

Check docker-compose

Kubernetes

Check Kubernetes