-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Retry with backoff #30
Open
minkimcello
wants to merge
14
commits into
thefrontside:main
Choose a base branch
from
minkimcello:mk/retry
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+184
−7
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
2f481b3
Make small edits to root README
minkimcello 00c6fe8
Initialize new retry-backoff package
minkimcello 82ceae6
Remove duplicate entry in deno.json
minkimcello e6a9012
Write useRetryWithBackoff
minkimcello e813d8d
Add website URL to root readme
minkimcello 044d299
Add context and take operation name from passed in fn
minkimcello 1a5f5e2
Write README
minkimcello 149e9d3
Write some tests
minkimcello 4c0cb88
Export initRetryWithBackoff
minkimcello c81df37
Update readme with more practical examples
minkimcello b247beb
Rename RetryWithBackoffContext to RetryBackoffContext
minkimcello ff4d72d
Change how the function is passed into call in README
minkimcello 62a71c6
Add documentation for jsdoc
minkimcello 64af38c
Export context and remove init function
minkimcello File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,7 +3,6 @@ | |
"dev": "deno run -A --watch www/main.ts" | ||
}, | ||
"imports": { | ||
"effection": "npm:[email protected]", | ||
"effection-www/": "https://raw.githubusercontent.com/thefrontside/effection/4982887c1677b847d256402e8709f2b1d49437e6/www/", | ||
"revolution": "https://deno.land/x/[email protected]/mod.ts", | ||
"revolution/jsx-runtime": "https://deno.land/x/[email protected]/jsx-runtime.ts", | ||
|
@@ -38,7 +37,8 @@ | |
"./deno-deploy", | ||
"./task-buffer", | ||
"./tinyexec", | ||
"./websocket" | ||
"./websocket", | ||
"./retry-backoff" | ||
], | ||
"deploy": { | ||
"project": "aa1dbfaa-d7c1-49d7-b514-69e1d8344f95", | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# retry-backoff | ||
|
||
Retry operations with incremental backoff. | ||
|
||
--- | ||
|
||
There's a default timeout set to 30 seconds. If you'd like to set a different | ||
timeout, you'll need to either pass in options as a second argument to | ||
`useRetryWithBackoff`: | ||
|
||
```js | ||
import { main } from "effection"; | ||
import { useRetryWithBackoff } from "@effection-contrib/retry-backoff"; | ||
|
||
await main(function* () { | ||
yield* useRetryWithBackoff(function* () { | ||
yield* call(() => fetch("https://foo.bar/")); | ||
}, { timeout: 45_000 }); | ||
}); | ||
``` | ||
|
||
Or set the timeout via the context so that the same timeout can be applied to all | ||
of your retry operations: | ||
|
||
```js | ||
import { main } from "effection"; | ||
import { | ||
RetryBackoffContext, | ||
useRetryWithBackoff, | ||
} from "@effection-contrib/retry-backoff"; | ||
|
||
await main(function* () { | ||
yield* RetryBackoffContext.set({ timeout: 45_000 }); | ||
yield* retryWithBackoff(function* () { | ||
yield* call(() => fetch("https://foo.bar/")); | ||
}); | ||
}); | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"name": "@effection-contrib/retry-backoff", | ||
"version": "0.1.0", | ||
"exports": "./mod.ts", | ||
"license": "MIT" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
export { | ||
useRetryWithBackoff, | ||
RetryBackoffContext, | ||
} from './retry-backoff.ts'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
import { run } from "npm:[email protected]"; | ||
import { describe, it } from "bdd"; | ||
import { expect } from "expect"; | ||
import { useRetryWithBackoff } from "./retry-backoff.ts" | ||
|
||
describe("RetryBackoff", () => { | ||
it("retries operation and returns output if operation finishes on time", async () => { | ||
await run(function* () { | ||
let attempts = 0; | ||
let result = 0; | ||
yield* useRetryWithBackoff(function* () { | ||
if (attempts < 2) { | ||
attempts++; | ||
throw new Error("operation failed"); | ||
} else { | ||
result = 1; | ||
} | ||
}, { timeout: 2_000 }); | ||
expect(attempts).toBe(2); | ||
expect(result).toBe(1); | ||
}) | ||
}); | ||
|
||
it("retries operation and handles timeout when operation exceeds limit", async () => { | ||
await run(function* () { | ||
let attempts = 0; | ||
let result = 0; | ||
yield* useRetryWithBackoff(function* () { | ||
if (attempts < 2) { | ||
attempts++; | ||
throw new Error("operation failed"); | ||
} else { | ||
result = 1; | ||
} | ||
}, { timeout: 500 }); | ||
expect(result).toBe(0); | ||
}) | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
import { | ||
createContext, | ||
type Operation, | ||
race, | ||
sleep, | ||
} from "npm:[email protected]"; | ||
import prettyMilliseconds from "npm:[email protected]"; | ||
|
||
interface UseRetryBackoffOptions { | ||
timeout?: number; | ||
} | ||
|
||
interface RetryWithContextDefaults { | ||
timeout: number; | ||
} | ||
|
||
export const RetryBackoffContext = createContext<RetryWithContextDefaults>( | ||
"retry-with-context", | ||
{ | ||
timeout: 30_000, | ||
} | ||
); | ||
|
||
/** | ||
* Retry an operation with incremental cooldown until it exceeds | ||
* the configured timeout value. The default timeout is 30 seconds. | ||
* | ||
* ```js | ||
* import { main } from "effection"; | ||
* import { useRetryWithBackoff } from "@effection-contrib/retry-backoff"; | ||
* | ||
* await main(function* () { | ||
* yield* useRetryWithBackoff(function* () { | ||
* yield* call(() => fetch("https://foo.bar/")); | ||
* }, { timeout: 45_000 }); | ||
* }); | ||
* ``` | ||
* | ||
* @param {Object} [options] - The options object | ||
* @param {number} [options.timeout] - Timeout value in milliseconds | ||
*/ | ||
export function* useRetryWithBackoff<T> ( | ||
fn: () => Operation<T>, | ||
options?: UseRetryBackoffOptions, | ||
) { | ||
const defaults = yield* RetryBackoffContext.expect(); | ||
const _options = { | ||
...defaults, | ||
...options, | ||
}; | ||
|
||
let attempt = -1; | ||
let name = fn.name || "unknown"; | ||
|
||
function* body() { | ||
while (true) { | ||
try { | ||
const result = yield* fn(); | ||
if (attempt !== -1) { | ||
console.log(`Operation[${name}] succeeded after ${attempt + 2} retry.`); | ||
} | ||
return result; | ||
} catch { | ||
// https://aws.amazon.com/ru/blogs/architecture/exponential-backoff-and-jitter/ | ||
const backoff = Math.pow(2, attempt) * 1000; | ||
const delayMs = Math.round((backoff * (1 + Math.random())) / 2); | ||
console.log(`Operation[${name}] failed, will retry in ${prettyMilliseconds(delayMs)}.`); | ||
yield* sleep(delayMs); | ||
attempt++; | ||
} | ||
} | ||
} | ||
|
||
function* timeout() { | ||
yield* sleep(_options.timeout); | ||
console.log(`Operation[${name}] timedout after ${attempt + 2}`); | ||
} | ||
|
||
yield* race([ | ||
body(), | ||
timeout(), | ||
]); | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need docs for this function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would be the right way to write docs for this context?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's right. You don't need to describe the parameters and properties because those will be generated from typescript