🚀 feat: Add Load-Shedding Middleware #3264
Conversation
ErfanMomeniii
requested review from
gaby,
sixcolors,
ReneWerner87 and
efectn
and removed request for
a team
|
Thanks for opening this pull request! 🎉 Please check out our contributing guidelines. If you need help or want to chat with us, join us on Discord https://gofiber.io/discord |
WalkthroughThe pull request introduces a new load-shedding middleware for the Fiber web framework in Changes
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
middleware/loadshedding/loadshedding.go (1)
10-12: Refine the comment phrasing and spacing for clarity.The function’s comment lines could be slightly restated and spaced to enhance readability. Also, consider rephrasing “creates a middleware handler enforces” to maintain proper English grammar.
-// New creates a middleware handler enforces a timeout on request processing to manage server load. -// If a request exceeds the specified timeout, a custom load-shedding handler is executed. +// New creates a middleware handler that enforces a timeout on request processing to help manage server load. +// If a request exceeds the specified timeout, a custom load-shedding handler is invoked.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
middleware/loadshedding/loadshedding.go(1 hunks)middleware/loadshedding/loadshedding_test.go(1 hunks)
🔇 Additional comments (9)
middleware/loadshedding/loadshedding.go (5)
13-17: Check the return value flow for excluded requests.
The exclusion check is handled correctly, ensuring that excluded routes bypass load shedding. Just confirm all necessary route patterns are accounted for in your exclude function.
19-21: Appropriate use of context for the timeout.
Creating a context with the specified timeout is done well here. This approach cleanly provides a timeout boundary for processing the request.
23-25: Double-check concurrency safety for the channel usage.
The buffered channel with capacity 1 is a good approach to avoid potential goroutine leaks. Also, ensure that no other paths close or reuse this channel.
26-29: Potential concurrency concern with fiber.Ctx usage.
Fiber’s Ctx implementation is generally not concurrency-safe. Calling c.Next() in a separate goroutine might pose a risk if the fiber.Ctx gets accessed by multiple goroutines simultaneously. Although this pattern often works in practice, be sure to confirm with Fiber documentation or tests that no concurrency safety issues arise.
31-38: Clear and effective use of select for timeout vs. request completion.
The logic to invoke loadSheddingHandler on timeout vs. returning the normal handler response is correct and straightforward.
middleware/loadshedding/loadshedding_test.go (4)
13-21: Adequate clarity in helper handlers implementation.
The division into discreet handlers (success, timeout, load-shedding) enhances readability. The 2-second sleep for timeout is ingenious for simulating a long-running request.
31-50: Comprehensive test for excluded routes.
Verifies that the middleware works as intended when routes are excluded. This is an essential scenario to ensure safe bypass logic.
51-73: Excellent timeout test approach.
The test methodically confirms that a request exceeding the 1-second cap triggers load-shedding. This thorough check ensures the middleware’s main functionality.
75-92: Valid test for successful requests within the timeout.
This covers the happy path, ensuring the request completes successfully within the allotted time. Including this case rounds out the coverage.
|
@ErfanMomeniii You should be able to replicate the CI using the provided Makefile. Run this: |
|
I believe that bug 1 is the cause of the tests failing. We should also add the improvement 1 from the below review. This review was generated using OpenAI Overall ImpressionThe middleware is straightforward and mostly does what it promises:
Its behavior is reasonably clear, and the accompanying tests show various scenarios (excluded routes, timeouts, and successful requests). However, there are a few issues and potential improvements worth pointing out. Bugs / Potential Issues
Potential Improvements
Missing Features / Wishlist
|
|
Hi @gaby, it should be fixed now. |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
middleware/loadshedding/loadshedding.go (2)
10-11: Clarify the documentation header.The comment does not explicitly mention that the timeout is enforced on a per-request basis. Consider clarifying that this middleware applies a deadline to each incoming request.
-// New creates a middleware handler enforces a timeout on request processing to manage server load. -// If a request exceeds the specified timeout, a custom load-shedding handler is executed. +// New creates a middleware handler that enforces a per-request timeout to manage server load. +// If a request exceeds the specified timeout, the custom load-shedding handler is executed.
26-36: Finalize or remove thedonechannel logic.Because
c.Next()blocks until completion, theerr := c.Next()line has already captured any error before the goroutine sends it intodone. Consequently, the channel-based approach offers little concurrency benefit. If you wish to enforce the timeout dynamically, consider running the entire operation in a separate goroutine and listening on the channel in real time. Otherwise, remove the channel to simplify.middleware/loadshedding/loadshedding_test.go (3)
1-2: Use a consistent naming scheme for the test package.Currently, the package is named
loadshedding_test. If your convention is to place middleware tests in the same package, rename the package toloadshedding. Otherwise, ensure the_testsuffix is applied consistently across your test packages for clarity.
23-25: Extend the load-shedding handler for partial data.The
Service Overloadedmessage is concise and predictable. For advanced scenarios (e.g., streaming or partial request completions), consider including more diagnostic information, such as expected wait times or retry intervals, to help consumers handle overload situations.
51-77: Double-check concurrency test coverage.The
Test_LoadSheddingTimeoutensures that a 2-second simulated delay triggers a 503 with a 1-second middleware timeout. However, it relies on Fiber’s internal mechanism for context cancellation. Confirm that all critical concurrency edge cases (e.g., partial content, streaming) are covered. If additional concurrency scenarios exist, create separate tests to avoid conflating multiple concerns in a single test.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
middleware/loadshedding/loadshedding.go(1 hunks)middleware/loadshedding/loadshedding_test.go(1 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
middleware/loadshedding/loadshedding_test.go
18-18: func timeoutHandler is unused
(unused)
🔇 Additional comments (5)
middleware/loadshedding/loadshedding.go (3)
14-17: Check the exclusion function contract.
When exclude is set, the middleware bypasses timeout enforcement without returning any annotation or log for the excluded path. If debugging load shedding is needed, consider adding optional logging or instrumentation to confirm that certain routes are excluded as intended.
37-45: Note potential partial response issues.
If data has already been written to the response before the context expires, clients might receive partial results without a clear error or an HTTP status code update. Ensure that the load-shedding handler or subsequent logic can convey a proper 503 or alternative response if partial data streaming is possible.
19-25:
Revisit context usage for concurrency and cancellation.
c.Next() is called synchronously, which means the middleware will not reach the select statement until after the handler completes (successfully or otherwise). Ensure that downstream handlers actually respect the context’s cancellation, or this approach will not explicitly interrupt a long-running or blocking operation. Fiber might handle context internally, but confirm that partial responses or streaming are properly aborted when the context is canceled.
Suggestion:
- Spawn
c.Next()in a separate goroutine, and use aselectto cancel if the context times out before completion. - Confirm that your handlers check
c.Context().Err()or similar to exit early.
middleware/loadshedding/loadshedding_test.go (2)
18-21: Confirm necessity of timeoutHandler.
Static analysis warns that timeoutHandler may be unused. Verify whether it is still required, or remove it to keep the code tidy.
🧰 Tools
🪛 golangci-lint (1.62.2)
18-18: func timeoutHandler is unused
(unused)
79-96: Consider adding negative scenarios.
While Test_LoadSheddingSuccessfulRequest verifies normal operation within the timeout, consider including scenarios for boundary conditions—e.g., requests that finish exactly at the timeout boundary or near it—to ensure robust behavior in borderline cases.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3264 +/- ##
==========================================
- Coverage 84.11% 84.08% -0.04%
==========================================
Files 116 117 +1
Lines 11522 11549 +27
==========================================
+ Hits 9692 9711 +19
- Misses 1401 1407 +6
- Partials 429 431 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Hi again @gaby , I’ve resolved the lint CI issue as well, so I believe everything is ready for merging now. |
|
@ErfanMomeniii Code and Tests look fine to me. Three things missing:
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
docs/whats_new.md (2)
814-822: Enhance the load shedding section formatting and content.The section needs better markdown formatting and more detailed explanations:
- Add a blank line before the "Load shedding" heading.
- Expand the bullet points to better explain each feature:
- Explain what types of timeouts are supported
- Describe what makes a good load-shedding response
- Provide examples of when to use request exclusion
Apply this diff to improve the section:
+ ### Load shedding -We've added **Load Shedding Middleware**.It ensures system stability under high load by enforcing timeouts on request processing. This mechanism allows the application to shed excessive load gracefully and maintain responsiveness. +We've added **Load Shedding Middleware** to ensure system stability under high load by enforcing timeouts on request processing. This mechanism allows the application to shed excessive load gracefully and maintain responsiveness. **Functionality** -**Timeout Enforcement**: Automatically terminates requests exceeding a specified processing time. -**Custom Response**: Uses a configurable load-shedding handler to define the response for shed requests. -**Request Exclusion**: Allows certain requests to bypass load-shedding logic through an exclusion filter. +- **Timeout Enforcement**: Automatically terminates requests exceeding a specified processing time, preventing resource exhaustion. +- **Custom Response**: Uses a configurable load-shedding handler to define the response for shed requests (e.g., 503 Service Unavailable with retry hints). +- **Request Exclusion**: Allows critical requests (e.g., health checks, admin operations) to bypass load-shedding logic through an exclusion filter.🧰 Tools
🪛 Markdownlint (0.37.0)
814-814: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
818-818: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
817-817: null
Emphasis used instead of a heading(MD036, no-emphasis-as-heading)
1374-1401: Enhance the implementation details section with better examples and descriptions.The section needs better formatting and more comprehensive examples:
- Add a blank line before the "Load shedding" heading.
- Enhance parameter descriptions with typical values and use cases.
- Provide a more comprehensive example showing integration with existing error handling.
Apply this diff to improve the section:
+ #### Load shedding -This middleware uses `context.WithTimeout` to manage the lifecycle of requests. If a request exceeds the specified timeout, the custom load-shedding handler is triggered, ensuring the system remains stable under stress. +This middleware uses `context.WithTimeout` to manage request lifecycles. If a request exceeds the specified timeout, the custom load-shedding handler is triggered, ensuring system stability during high load. **Key Parameters** -`timeout` (`time.Duration`): The maximum time a request is allowed to process. Requests exceeding this time are terminated. +`timeout` (`time.Duration`): The maximum time allowed for request processing (e.g., `5*time.Second` for API endpoints, `30*time.Second` for file uploads). -`loadSheddingHandler` (`fiber.Handler`): A custom handler that executes when a request exceeds the timeout. Typically used to return a `503 Service Unavailable` response or a custom message. +`loadSheddingHandler` (`fiber.Handler`): A custom handler for timed-out requests. Typically returns a `503 Service Unavailable` response with a Retry-After header and a user-friendly message. -`exclude` (`func(fiber.Ctx) bool`): A filter function to exclude specific requests from being subjected to the load-shedding logic (optional). +`exclude` (`func(fiber.Ctx) bool`): Optional filter function to exempt specific requests from load-shedding (e.g., health checks, critical admin operations). **Usage Example** ```go import "github.com/gofiber/fiber/v3/middleware/loadshedding +// Configure load shedding with different timeouts per route app.Use(loadshedding.New( - 10*time.Second, // Timeout duration + 5*time.Second, // Default timeout for most requests func(c fiber.Ctx) error { // Load shedding response return c.Status(fiber.StatusServiceUnavailable). - SendString("Service overloaded, try again later.") + Set("Retry-After", "30"). // Suggest retry after 30 seconds + JSON(fiber.Map{ + "error": "Service is temporarily overloaded", + "message": "Please retry after 30 seconds", + "code": "LOAD_SHED", + }) }, func(c fiber.Ctx) bool { // Exclude health checks - return c.Path() == "/health" + return c.Path() == "/health" || c.Path() == "/metrics" || strings.HasPrefix(c.Path(), "/admin") }, )) + +// Use longer timeout for file upload endpoints +app.Post("/upload", loadshedding.New( + 30*time.Second, + func(c fiber.Ctx) error { + return c.Status(fiber.StatusServiceUnavailable). + JSON(fiber.Map{ + "error": "Upload service is temporarily overloaded", + "message": "Please try uploading later", + }) + }, +), func(c fiber.Ctx) error { + // Handle file upload + return nil +})<details> <summary>🧰 Tools</summary> <details> <summary>🪛 LanguageTool</summary> [style] ~1375-~1375: ‘under stress’ might be wordy. Consider a shorter alternative. Context: ...red, ensuring the system remains stable under stress. **Key Parameters** `timeout` (`time.... (EN_WORDINESS_PREMIUM_UNDER_STRESS) --- [uncategorized] ~1381-~1381: A comma may be missing after the conjunctive/linking adverb ‘Typically’. Context: ...tes when a request exceeds the timeout. Typically used to return a `503 Service Unavailab... (SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA) </details> <details> <summary>🪛 Markdownlint (0.37.0)</summary> 1374-1374: Expected: 1; Actual: 0; Below Headings should be surrounded by blank lines (MD022, blanks-around-headings) --- 1377-1377: null Emphasis used instead of a heading (MD036, no-emphasis-as-heading) --- 1385-1385: null Emphasis used instead of a heading (MD036, no-emphasis-as-heading) </details> </details> </blockquote></details> </blockquote></details> <details> <summary>📜 Review details</summary> **Configuration used: CodeRabbit UI** **Review profile: CHILL** **Plan: Pro** <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between 6d7fbc2e6f804aae31d0ee58deaf225dff5fc3b3 and d0607b500dc2121dfec1ee6931289c6ec1ef01a8. </details> <details> <summary>📒 Files selected for processing (1)</summary> * `docs/whats_new.md` (3 hunks) </details> <details> <summary>🧰 Additional context used</summary> <details> <summary>🪛 Markdownlint (0.37.0)</summary> <details> <summary>docs/whats_new.md</summary> 34-34: null Link fragments should be valid (MD051, link-fragments) --- 814-814: Expected: 1; Actual: 0; Below Headings should be surrounded by blank lines (MD022, blanks-around-headings) --- 818-818: null Lists should be surrounded by blank lines (MD032, blanks-around-lists) --- 817-817: null Emphasis used instead of a heading (MD036, no-emphasis-as-heading) --- 1374-1374: Expected: 1; Actual: 0; Below Headings should be surrounded by blank lines (MD022, blanks-around-headings) --- 1377-1377: null Emphasis used instead of a heading (MD036, no-emphasis-as-heading) --- 1385-1385: null Emphasis used instead of a heading (MD036, no-emphasis-as-heading) </details> </details> <details> <summary>🪛 LanguageTool</summary> <details> <summary>docs/whats_new.md</summary> [style] ~1375-~1375: ‘under stress’ might be wordy. Consider a shorter alternative. Context: ...red, ensuring the system remains stable under stress. **Key Parameters** `timeout` (`time.... (EN_WORDINESS_PREMIUM_UNDER_STRESS) --- [uncategorized] ~1381-~1381: A comma may be missing after the conjunctive/linking adverb ‘Typically’. Context: ...tes when a request exceeds the timeout. Typically used to return a `503 Service Unavailab... (SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA) </details> </details> </details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
docs/middleware/loadshedding.md (4)
12-18: Fix heading hierarchy.The "Features" section should use h2 (##) instead of h3 (###) to maintain proper heading hierarchy.
-### Features +## Features🧰 Tools
🪛 Markdownlint (0.37.0)
12-12: Expected: h2; Actual: h3
Heading levels should only increment by one level at a time(MD001, heading-increment)
35-39: Enhance configuration documentation.The config table should provide more detailed information about:
- The behavior when timeout occurs (context cancellation)
- Requirements and constraints for the loadSheddingHandler
- Best practices for timeout values
Expand the config table:
| Property | Type | Description | Default | |:---------------------|:------------------|:-----------------------------------------------------------------------------------------------|:---------| -| `timeout` | `time.Duration` | Maximum allowed processing time for a request. | Required | -| `loadSheddingHandler`| `fiber.Handler` | Handler invoked for requests that exceed the timeout. | Required | +| `timeout` | `time.Duration` | Maximum allowed processing time for a request. When exceeded, the request context is canceled and the loadSheddingHandler is triggered. Choose values based on your application's performance characteristics and SLAs. | Required | +| `loadSheddingHandler`| `fiber.Handler` | Handler invoked for requests that exceed the timeout. This handler should complete quickly and avoid starting new goroutines or long-running operations, as it's called during high-load scenarios. | Required | | `exclude` | `func(fiber.Ctx) bool` | Optional function to exclude specific requests from load-shedding logic. | `nil` |
51-72: Enhance example usage with best practices and common scenarios.The current examples could be improved to demonstrate:
- Proper error handling with context cancellation
- Handling of streaming responses
- Dynamic timeout configuration
- Graceful shutdown integration
Here's an expanded example:
func main() { app := fiber.New() // Example 1: Basic usage with proper error handling app.Use(loadshedding.New(5*time.Second, func(c fiber.Ctx) error { return c.Status(fiber.StatusServiceUnavailable).JSON(fiber.Map{ "error": "Service temporarily unavailable", "retry_after": "5s", }) }, nil)) // Example 2: Dynamic timeout based on route complexity app.Use(loadshedding.New(3*time.Second, func(c fiber.Ctx) error { return c.Status(fiber.StatusServiceUnavailable).SendString("Request timed out") }, func(c fiber.Ctx) bool { // Exclude health checks and metrics endpoints return c.Path() == "/health" || c.Path() == "/metrics" })) // Example 3: Handling streaming responses app.Get("/stream", func(c fiber.Ctx) error { ctx := c.Context() c.Set("Content-Type", "text/event-stream") c.Set("Cache-Control", "no-cache") c.Set("Connection", "keep-alive") c.Set("Transfer-Encoding", "chunked") for i := 0; i < 10; i++ { select { case <-ctx.Done(): return nil // Context canceled, stop streaming default: if err := c.WriteString(fmt.Sprintf("data: Message %d\n\n", i)); err != nil { return err } c.Push() time.Sleep(500 * time.Millisecond) } } return nil }) // Graceful shutdown go func() { sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, syscall.SIGTERM, syscall.SIGINT) <-sigChan // During shutdown, adjust timeout to respond quickly app.Use(loadshedding.New(100*time.Millisecond, func(c fiber.Ctx) error { return c.Status(fiber.StatusServiceUnavailable).SendString("Server is shutting down") }, nil)) if err := app.Shutdown(); err != nil { log.Printf("Shutdown error: %v\n", err) } }() app.Listen(":3000") }
73-73: Add newline at end of file.Add a newline character at the end of the file to comply with markdown formatting standards.
🧰 Tools
🪛 Markdownlint (0.37.0)
73-73: null
Files should end with a single newline character(MD047, single-trailing-newline)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/README.md(1 hunks)docs/middleware/loadshedding.md(1 hunks)
🧰 Additional context used
🪛 Markdownlint (0.37.0)
docs/middleware/loadshedding.md
12-12: Expected: h2; Actual: h3
Heading levels should only increment by one level at a time
(MD001, heading-increment)
41-41: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines
(MD022, blanks-around-headings)
73-73: null
Files should end with a single newline character
(MD047, single-trailing-newline)
🔇 Additional comments (1)
.github/README.md (1)
691-691: LGTM! Documentation addition is clear and consistent.The new load-shedding middleware entry is well-documented and follows the established format of the Internal Middleware table. The description effectively communicates the middleware's purpose of managing server load through request timeouts and resource handling.
|
it's done. @gaby |
|
The lint problem should be fixed now. @gaby |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
docs/whats_new.md (1)
1380-1406: Improve code example documentation.The code example is correct, but could benefit from minor improvements:
- Add a period after "under stress" for consistency
- Add a comma after "Typically" in the handler description
- Consider adding an example response status code in the comment for the exclude function
Apply these changes to improve readability:
This middleware uses `context.WithTimeout` to manage the lifecycle of requests. If a request exceeds the specified timeout, the custom load-shedding handler is triggered, ensuring the system remains stable under stress. + ##### Key Parameters `timeout` (`time.Duration`): The maximum time a request is allowed to process. Requests exceeding this time are terminated. -`loadSheddingHandler` (`fiber.Handler`): A custom handler that executes when a request exceeds the timeout. Typically used to return a `503 Service Unavailable` response or a custom message. +`loadSheddingHandler` (`fiber.Handler`): A custom handler that executes when a request exceeds the timeout. Typically, used to return a `503 Service Unavailable` response or a custom message. `exclude` (`func(fiber.Ctx) bool`): A filter function to exclude specific requests from being subjected to the load-shedding logic (optional). ##### Usage Example ```go import "github.com/gofiber/fiber/v3/middleware/loadshedding app.Use(loadshedding.New( 10*time.Second, // Timeout duration func(c fiber.Ctx) error { // Load shedding response return c.Status(fiber.StatusServiceUnavailable). SendString("Service overloaded, try again later.") }, - func(c fiber.Ctx) bool { // Exclude health checks + func(c fiber.Ctx) bool { // Exclude health checks (returns 200 OK) return c.Path() == "/health" }, ))<details> <summary>🧰 Tools</summary> <details> <summary>🪛 LanguageTool</summary> [style] ~1381-~1381: ‘under stress’ might be wordy. Consider a shorter alternative. Context: ...red, ensuring the system remains stable under stress. ##### Key Parameters `timeout` (`tim... (EN_WORDINESS_PREMIUM_UNDER_STRESS) --- [uncategorized] ~1387-~1387: A comma may be missing after the conjunctive/linking adverb ‘Typically’. Context: ...tes when a request exceeds the timeout. Typically used to return a `503 Service Unavailab... (SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA) </details> </details> </blockquote></details> </blockquote></details> <details> <summary>📜 Review details</summary> **Configuration used: CodeRabbit UI** **Review profile: CHILL** **Plan: Pro** <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between f4682bd976ca9fbf0fbfc9fcedaf50a1a9f2c224 and a192bd126d91e6fea644650b88013404a33463a5. </details> <details> <summary>📒 Files selected for processing (2)</summary> * `docs/middleware/loadshedding.md` (1 hunks) * `docs/whats_new.md` (3 hunks) </details> <details> <summary>🚧 Files skipped from review as they are similar to previous changes (1)</summary> * docs/middleware/loadshedding.md </details> <details> <summary>🧰 Additional context used</summary> <details> <summary>🪛 LanguageTool</summary> <details> <summary>docs/whats_new.md</summary> [style] ~1381-~1381: ‘under stress’ might be wordy. Consider a shorter alternative. Context: ...red, ensuring the system remains stable under stress. ##### Key Parameters `timeout` (`tim... (EN_WORDINESS_PREMIUM_UNDER_STRESS) --- [uncategorized] ~1387-~1387: A comma may be missing after the conjunctive/linking adverb ‘Typically’. Context: ...tes when a request exceeds the timeout. Typically used to return a `503 Service Unavailab... (SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA) </details> </details> </details> <details> <summary>🔇 Additional comments (2)</summary><blockquote> <details> <summary>docs/whats_new.md (2)</summary> `814-817`: **LGTM! Clear and concise introduction.** The introduction effectively explains the purpose and benefits of the Load Shedding Middleware. --- `818-826`: **LGTM! Well-structured functionality section.** The bullet points clearly outline the key features of the middleware. </details> </blockquote></details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
|
The lint problem should be fixed now and it's ready to merge. @gaby |
gaby
changed the title
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
docs/middleware/loadshedding.md (1)
16-21: Enhance use cases section with additional scenarios.Consider adding these important use cases:
- Graceful shutdown handling
- Request prioritization during high load
- Integration with monitoring systems
## Use Cases - **High-Traffic Scenarios**: Safeguard critical resources by rejecting overly long or resource-intensive requests. - **Health Check Protection**: Exclude monitoring endpoints (e.g., `/health`) to ensure uninterrupted external checks. - **Dynamic Load Management**: Utilize exclusion logic to adjust load-shedding behavior for specific routes or request types. +- **Graceful Shutdown**: Manage in-flight requests during server shutdown. +- **Request Prioritization**: Implement different timeout thresholds based on request priority. +- **Monitoring Integration**: Track and alert on load-shedding events for system health monitoring.docs/whats_new.md (1)
1391-1404: Improve code example formatting and completeness.The example could be enhanced with better formatting and additional scenarios:
import "github.com/gofiber/fiber/v3/middleware/loadshedding app.Use(loadshedding.New( 10*time.Second, // Timeout duration func(c fiber.Ctx) error { // Load shedding response return c.Status(fiber.StatusServiceUnavailable). SendString("Service overloaded, try again later.") }, func(c fiber.Ctx) bool { // Exclude health checks - return c.Path() == "/health" + // Exclude both health check and metrics endpoints + path := c.Path() + return path == "/health" || path == "/metrics" }, )) + +// Example with monitoring integration +app.Use(loadshedding.New( + 5*time.Second, + func(c fiber.Ctx) error { + // Track load shedding events + metrics.LoadShedEvents.Inc() + return c.Status(fiber.StatusServiceUnavailable). + JSON(fiber.Map{ + "error": "Service temporarily unavailable", + "retry_after": "5s", + }) + }, + nil, +))
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docs/middleware/loadshedding.md(1 hunks)docs/whats_new.md(3 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/whats_new.md
[style] ~1379-~1379: ‘under stress’ might be wordy. Consider a shorter alternative.
Context: ...red, ensuring the system remains stable under stress. ##### Key Parameters timeout (`tim...
(EN_WORDINESS_PREMIUM_UNDER_STRESS)
[uncategorized] ~1385-~1385: A comma may be missing after the conjunctive/linking adverb ‘Typically’.
Context: ...tes when a request exceeds the timeout. Typically used to return a `503 Service Unavailab...
(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
🔇 Additional comments (3)
docs/middleware/loadshedding.md (2)
1-15: Well-structured introduction and features section!The documentation clearly explains the middleware's purpose and key features, providing a good overview for users.
81-81: Add a "Limitations and Considerations" section.As suggested in previous reviews, add a section documenting known limitations:
## Limitations and Considerations - **Context Cancellation**: Ensure your handlers properly respect context cancellation to prevent goroutine leaks. - **Streaming Responses**: Be cautious when using this middleware with streaming responses, as timeout cancellation might result in partial data being sent. - **Graceful Shutdown**: During server shutdown, requests might experience immediate timeouts. Consider implementing appropriate shutdown hooks.docs/whats_new.md (1)
814-824: Well-structured introduction of the new middleware!The introduction effectively explains the purpose and key features of the Load Shedding middleware.
| |-----------------------|--------------------|----------------------------------------------------------------------------------|-----------| | ||
| | `timeout` | `time.Duration` | The maximum allowed processing time for a request. | Required | | ||
| | `loadSheddingHandler`| `fiber.Handler` | The handler invoked for requests that exceed the `timeout`. | Required | | ||
| | `exclude` | `func(fiber.Ctx) bool` | Optional function to exclude certain requests from load-shedding logic. | `nil` | |
There was a problem hiding this comment.
we have similar behavior in all other middlewares in the core repo and also in the contrib repository
unless it is completely wrong i would like to get the wording of this parameter
the users know the parameter as Next function for skipping
https://docs.gofiber.io/search/?q=Next&version=next%2F
| ## Signature | ||
|
|
||
| ```go | ||
| func New(timeout time.Duration, loadSheddingHandler fiber.Handler, exclude func(fiber.Ctx) bool) fiber.Handler |
There was a problem hiding this comment.
in order to ensure expandability in the future and not have to make breaking changes, i would like to stick with the configuration concept, which we use in all other middlewares and which has proven to be very easy to expand
| @@ -688,6 +688,7 @@ Here is a list of middleware that are included within the Fiber framework. | |||
| | [idempotency](https://github.com/gofiber/fiber/tree/main/middleware/idempotency) | Allows for fault-tolerant APIs where duplicate requests do not erroneously cause the same action performed multiple times on the server-side. | | |||
| | [keyauth](https://github.com/gofiber/fiber/tree/main/middleware/keyauth) | Adds support for key based authentication. | | |||
| | [limiter](https://github.com/gofiber/fiber/tree/main/middleware/limiter) | Adds Rate-limiting support to Fiber. Use to limit repeated requests to public APIs and/or endpoints such as password reset. | | |||
| | [loadshedding](https://github.com/gofiber/fiber/tree/main/middleware/loadshedding) | Gracefully manages server load by enforcing request timeouts and handling resource-intensive requests during high-traffic periods. | | |||
There was a problem hiding this comment.
what is the difference to the timeout middleware? could you also store a handler in the config that handles these cases?
|
@ErfanMomeniii thank you for your contribution and the time you have invested to support us we have analyzed the middleware again and also found another similar one i also can't say 100% right now whether i would rather provide the reaction to a timeout in a loadshed middleware or as an option in the timeout middleware the current options would be to extend the timeout middleware with a handler for reacting to processes that run into the timeout or to extend the existing loadshred middleware in contrib other examples for loadshed |
Description
This pull request introduces a new Load-Shedding Middleware. This middleware enforces a configurable timeout on request processing to effectively manage server load. If a request exceeds the specified timeout, a custom load-shedding handler is invoked to gracefully handle the overloaded state.
Purpose of the Change:
Changes introduced
This pull request adds the following features and updates:
Load-Shedding Middleware (
loadshedding.New):Unit Tests:
Test_LoadSheddingExcluded: Verifies that excluded routes bypass the middleware.Test_LoadSheddingTimeout: Tests behavior when requests exceed the timeout.Test_LoadSheddingSuccessfulRequest: Confirms successful processing of requests completed within the timeout.Documentation Update: Detailed usage examples for the middleware are added.
Benchmarks: Basic performance benchmarks have been conducted to ensure minimal overhead.
Changelog/What's New: This middleware addition will be documented in the changelog for the next release.
API Longevity: Designed with consistency in mind to align with Fiber's existing middleware and ensure backward compatibility.
Type of change
Checklist
/docs/directory for Fiber's documentation.Commit formatting
Commits follow the recommended format with appropriate emojis for better identification: