Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement validation for buffered partial messages #829

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

masih
Copy link
Member

@masih masih commented Jan 16, 2025

When messages arrive with yet to be discovered EC chain we want to do as much validation as possible before buffering the message for future use.

Because the key of the EC chain is included in the partial messages we are able to validate basically everything about the message except the chain itself. The changes here implement this ability.

The implementation introduces a new validator implementation specifically written to handle partial messages. The validation rules are similar to the full GMesssage validator but then are adopted to infer the state of a message from chain key instead. A separate issue is captured to reduce duplicate rules across the two validators. This refactor is postponed on purpose as it touches a lot of the core implementations. Instead, the implementation here aims to provide a correct partial validation mechanism to progress efforts.

The new validation makes a final validation attempt once the chain is discovered to check the validity of chain, its consistency with key and justification.

The partial validator maintains its own cache, similar to the full validator, where messages are evicted as soon as GPBFT progresses. The caches between full and partial validators are independent and do not have an overlap: a message that can immediately be completed is never cached by the partial validator and vise versa. This should keep the total memory footprint across the both validators basically the same as before.

Fixes #813

When messages arrive with yet to be discovered EC chain we want to do as
much validation as possible before buffering the message for future use.

Because the key of the EC chain is included in the partial messages we
are able to validate basically everything about the message except the
chain itself. The changes here implement this ability.

The implementation introduces a new validator implementation
specifically written to handle partial messages. The validation rules
are similar to the full GMesssage validator but then are adopted to
infer the state of a message from chain key instead. A separate issue
is captured to reduce duplicate rules across the two validators. This
refactor is postponed on purpose as it touches a lot of the core
implementations. Instead, the implementation here aims to provide a
correct partial validation mechanism to progress efforts.

The new validation makes a final validation attempt once the chain is
discovered to check the validity of chain, its consistency with key and
justification.

The partial validator maintains its own cache, similar to the full
validator, where messages are evicted as soon as GPBFT progresses. The
caches between full and partial validators are independent and do not
have an overlap: a message that can immediately be completed is never
cached by the partial validator and vise versa. This should keep the
total memory footprint across the both validators basically the same as
before.

Fixes #813
@masih masih marked this pull request as ready for review January 16, 2025 10:42
@masih masih requested a review from Kubuxu January 16, 2025 10:42
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 27.44479% with 230 lines in your changes missing coverage. Please review.

Project coverage is 67.32%. Comparing base (d3d8077) to head (df14ff4).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
partial_validator.go 22.14% 205 Missing and 13 partials ⚠️
host.go 74.07% 6 Missing and 1 partial ⚠️
partial_msg.go 50.00% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #829      +/-   ##
==========================================
- Coverage   69.42%   67.32%   -2.11%     
==========================================
  Files          83       84       +1     
  Lines        8662     8942     +280     
==========================================
+ Hits         6014     6020       +6     
- Misses       2128     2392     +264     
- Partials      520      530      +10     
Files with missing lines Coverage Δ
merkle/merkle.go 92.30% <ø> (ø)
partial_msg.go 59.80% <50.00%> (-22.97%) ⬇️
host.go 64.58% <74.07%> (-0.34%) ⬇️
partial_validator.go 22.14% <22.14%> (ø)

... and 8 files with indirect coverage changes

@masih masih self-assigned this Jan 16, 2025
@masih masih added this to the M2: Mainnet Passive Testing milestone Jan 16, 2025
@masih masih added this pull request to the merge queue Jan 16, 2025
Merged via the queue into main with commit 79a7008 Jan 16, 2025
13 of 14 checks passed
@masih masih deleted the masih/partial-validation branch January 16, 2025 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Implement the ability to partially validate messages without knowing the ec chain
2 participants