Simplified handling of idle connections #4306
Labels
tracking-issue
Issues which are the entry point to bigger projects.
Comments
thomaseizinger
changed the title
thomaseizinger
changed the title
thomaseizinger
added
tracking-issue
This was referenced Sep 20, 2023
|
All done! 🥳 |
4 tasks
2 tasks
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this issue
Dec 12, 2024
# Description Fixes #5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]>
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this issue
Dec 12, 2024
# Description Fixes #5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]>
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this issue
Dec 12, 2024
# Description Fixes #5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]>
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this issue
Dec 16, 2024
# Description Fixes #5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]>
nazar-pc
added a commit
to autonomys/polkadot-sdk
that referenced
this issue
Dec 20, 2024
# Description Fixes paritytech#5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]> (cherry picked from commit c881288)
dudo50
pushed a commit
to paraspell-research/polkadot-sdk
that referenced
this issue
Jan 4, 2025
# Description Fixes paritytech#5996 https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.53.0 https://github.com/libp2p/rust-libp2p/blob/master/CHANGELOG.md ## Integration Nothing special is needed, just note that `yamux_window_size` is no longer applicable to libp2p (litep2p seems to still have it though). ## Review Notes There are a few simplifications and improvements done in libp2p 0.53 regarding swarm interface, I'll list a few key/applicable here. libp2p/rust-libp2p#4788 removed `write_length_prefixed` function, so I inlined its code instead. libp2p/rust-libp2p#4120 introduced new `libp2p::SwarmBuilder` instead of now deprecated `libp2p::swarm::SwarmBuilder`, the transition is straightforward and quite ergonomic (can be seen in tests). libp2p/rust-libp2p#4581 is the most annoying change I have seen that basically makes many enums `#[non_exhaustive]`. I mapped some, but those that couldn't be mapped I dealt with by printing log messages once they are hit (the best solution I could come up with, at least with stable Rust). libp2p/rust-libp2p#4306 makes connection close as soon as there are no handler using it, so I had to replace `KeepAlive::Until` with an explicit future that flips internal boolean after timeout, achieving the old behavior, though it should ideally be removed completely at some point. `yamux_window_size` is no longer used by libp2p thanks to libp2p/rust-libp2p#4970 and generally Yamux should have a higher performance now. I have resolved and cleaned up all deprecations related to libp2p except `BandwidthSinks`. Libp2p deprecated it (though it is still present in 0.54.1, which is why I didn't handle it just yet). Ideally Substrate would finally [switch to the official Prometheus client](paritytech/substrate#12699), in which case we'd get metrics for free. Otherwise a bit of code will need to be copy-pasted to maintain current behavior with `BandwidthSinks` gone, which I left a TODO about. The biggest change in 0.54.0 is libp2p/rust-libp2p#4568 that changed transport APIs and enabled unconditional potential port reuse, which can lead to very confusing errors if running two Substrate nodes on the same machine without changing listening port explicitly. Overall nothing scary here, but testing is always appreciated. # Checklist * [x] My PR includes a detailed description as outlined in the "Description" and its two subsections above. * [x] My PR follows the [labeling requirements]( https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process ) of this project (at minimum one label for `T` required) * External contributors: ask maintainers to put the right label on your PR. --- Polkadot Address: 1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd --------- Co-authored-by: Dmitry Markin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
For a longer discussion see #3353 (comment).
With #3844, we are deprecating
KeepAlive::Until. With #4121, users can configure aSwarm-wide idle timeout that activates once allConnectionHandlers reportKeepAlive::No.Tasks
KeepAlive::Until#3844idle_connection_timeoutconfig option #4121Motivation
Easier idle connection management for custom protocols. In particular, most users should not need to deal with idle connections and keep alive tracking but it should just work out of the box.
The text was updated successfully, but these errors were encountered: