Tirpc Fixes to enabled NFS over RDMA #296
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@ffilz please review and add the reviewers. |
dang
reviewed
May 10, 2024
1>Connection management.
2>Scalable per connection buffer management to use/reuse rdma registered
buffers for NFS request/responses/data buffers.
3>Granular buffer sizes for headers and data.
4>Rdma credit management based on outstanding recv buffers.
5>Async rdma_send/rdma_write callbacks.
6>Fixes to make rdma_read and rdma_write work.
7>Fixes to make reply_list and write_list work.
8>Limit rdma connections.
9>Connection disconnect handling to cleanup posted buffers.
10>cbc ref management.
11>rdma xprt ref management.
We have supporting patch in Ganesha to use rdma registered buffers for read and
readdir responses.
With this patch and supporting Ganesha patch, we should be able to use
NFSv3 and NFSv4.0 over RDMA.
Delegation has to be disabled for NFSv4.0, since callback channel is not
yet supported over RDMA.
We tested NFSoRDMA with both hardware and SoftROCE
Hardware:
Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function
Rocky Linux release 8.9 (Green Obsidian)
Linux kernel version 5.10
SoftROCE:
Rocky Linux release 8.9 (Green Obsidian)
Linux kernel version 5.4.275-1.el8.elrepo.x86_64
Tested with fsal=MEM
Signed-off-by: Gaurav Gangalwar <[email protected]>
|
Thanks. Waiting on Frank. |
dang
approved these changes
May 13, 2024
|
Looks good to me. |
|
Gaurav - could you take a look at this issue: #220 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch Enable NFSoRDMA with following fixes
We have supporting patch in Ganesha to use rdma registered buffers for read and readdir responses.
With this patch and supporting Ganesha patch, we should be able to use NFSv3 and NFSv4.0 over RDMA.
Delegation has to be disabled for NFSv4.0, since callback channel is not yet supported over RDMA.
We tested NFSoRDMA with both hardware and SoftROCE
Hardware:
Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function Rocky Linux release 8.9 (Green Obsidian)
Linux kernel version 5.10
SoftROCE:
Rocky Linux release 8.9 (Green Obsidian)
Linux kernel version 5.4.275-1.el8.elrepo.x86_64
Tested with fsal=MEM