issue-539: add tablet-side two-stage writes support for scaling throughput #707
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
debnatkh
changed the title
debnatkh
added
the
filestore
added 2 commits
March 14, 2024 16:17
debnatkh
changed the title
qkrorlqr
requested changes
Mar 20, 2024
There was a problem hiding this comment.
Надо вынести подготовительные правки в отдельный PR, иначе будем очень долго это ревьюить. Подготовительные правки - это, например, вынос TWriteDataActor в tablet/actors/, это вынос кода валидации запроса из HandleWriteData в отдельную функцию и другие подобные правки
cloud/filestore/libs/storage/tablet/tablet_actor_writedata_actor.cpp
Outdated
Show resolved
Hide resolved
added 2 commits
March 21, 2024 16:57
qkrorlqr
reviewed
Mar 27, 2024
cloud/filestore/libs/storage/tablet/actors/tablet_writedata.cpp
Outdated
Show resolved
Hide resolved
cloud/filestore/libs/storage/tablet/actors/tablet_writedata.cpp
Outdated
Show resolved
Hide resolved
cloud/filestore/libs/storage/tablet/actors/tablet_writedata.cpp
Outdated
Show resolved
Hide resolved
cloud/filestore/libs/storage/tablet/actors/tablet_writedata.cpp
Outdated
Show resolved
Hide resolved
qkrorlqr
approved these changes
Mar 28, 2024
yegorskii
approved these changes
Mar 28, 2024
debnatkh
added a commit
that referenced
this pull request
Apr 2, 2024
…ghput (#707) issue-539: add two-stage writes for scaling throughput
debnatkh
added a commit
that referenced
this pull request
Apr 3, 2024
* issue-539: extract writedata actor into separate location + add TABLET_VERIFY for ReleaseCollectBarrier (#790) issue-539: extract writedata actor into separate location + add TABLET_VERIFY for ReleaseCollectBarrier * fix arcadia build: trailing EOL (#808) * issue-539: add tablet-side two-stage writes support for scaling throughput (#707) issue-539: add two-stage writes for scaling throughput * issue-539: add service-side two-stage writes support for scaling throughput (#807) issue-539: add service-side two-stage writes support * fix cmake build
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#539
Introduce 2 private methods for index tablet:
IssueBlob– issue blob IDs for the data to be written to directly from the client. This method also acquires a collect barrier for the commit ID of the issued blob. This is done to prevent the barrier from moving beyond the commit ID of the issued blob. Also, the release of this barrier is scheduled after the IssueBlobReleaseCollectBarrierTimeout. This is done as a failsafe for clients that do not go through with the two-stage write path.MarkWriteComplete– after the data has been written to an issued blob, this method works in a similar manner as an ordinary WriteData method but with the difference that it does not write the data to the blob storage as it is supposed to be already written by the client.In the next PR, we will implement the client side of the two-stage write path.
Created new TX:
AddData. It is basically the same asWriteDataexcept it does not use fresh bytes/blocks.Both of these TXs use
TWriteDataActoron their completion, so its code was extracted totablet_actor_writedata_actor.(cpp|h)