Skip to content

Commit

Permalink
fix: don't read more than max bytes from a request (#1276)
Browse files Browse the repository at this point in the history
## Which problem is this PR solving?

- If the compressed input to an HTTP request is too big, it can cause
Refinery to have difficulties.

## Short description of the changes

- Set a request max for the HTTP inputs /1/batch and /1/events
- Remove bogus syntax from test config

I don't have a good way to test this in CI, but it was extensively
tested locally.
  • Loading branch information
kentquirk authored Aug 15, 2024
1 parent e24f371 commit f5fc674
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 5 deletions.
7 changes: 4 additions & 3 deletions route/route.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ const (
numZstdDecoders = 4
traceIDShortLength = 8
traceIDLongLength = 16
GRPCMessageSizeMax int = 5000000 // 5MB
GRPCMessageSizeMax int = 5_000_000 // 5MB
HTTPMessageSizeMax = 5_000_000 // 5MB
defaultSampleRate = 1
)

Expand Down Expand Up @@ -687,7 +688,7 @@ func (r *Router) getMaybeCompressedBody(req *http.Request) (io.Reader, error) {
defer gzipReader.Close()

buf := &bytes.Buffer{}
if _, err := io.Copy(buf, gzipReader); err != nil {
if _, err := io.Copy(buf, io.LimitReader(gzipReader, HTTPMessageSizeMax)); err != nil {
return nil, err
}
reader = buf
Expand All @@ -703,7 +704,7 @@ func (r *Router) getMaybeCompressedBody(req *http.Request) (io.Reader, error) {
return nil, err
}
buf := &bytes.Buffer{}
if _, err := io.Copy(buf, zReader); err != nil {
if _, err := io.Copy(buf, io.LimitReader(zReader, HTTPMessageSizeMax)); err != nil {
return nil, err
}

Expand Down
2 changes: 0 additions & 2 deletions test/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,11 @@ Logger:
LegacyMetrics:
Enabled: true
Dataset: refinery_metrics
APIKey: $REFINERY_HONEYCOMB_API_KEY
APIHost: https://api-dogfood.honeycomb.io

OTelMetrics:
Enabled: true
Dataset: refinery_metrics_otel
APIKey: $REFINERY_HONEYCOMB_API_KEY
APIHost: https://api-dogfood.honeycomb.io

RefineryTelemetry:
Expand Down

0 comments on commit f5fc674

Please sign in to comment.