Implement the ability to partially validate messages without knowing the ec chain #813
Assignees
Milestone
Comments
masih
added a commit
that referenced
this issue
Jan 16, 2025
When messages arrive with yet to be discovered EC chain we want to do as much validation as possible before buffering the message for future use. Because the key of the EC chain is included in the partial messages we are able to validate basically everything about the message except the chain itself. The changes here implement this ability. The implementation introduces a new validator implementation specifically written to handle partial messages. The validation rules are similar to the full GMesssage validator but then are adopted to infer the state of a message from chain key instead. A separate issue is captured to reduce duplicate rules across the two validators. This refactor is postponed on purpose as it touches a lot of the core implementations. Instead, the implementation here aims to provide a correct partial validation mechanism to progress efforts. The new validation makes a final validation attempt once the chain is discovered to check the validity of chain, its consistency with key and justification. The partial validator maintains its own cache, similar to the full validator, where messages are evicted as soon as GPBFT progresses. The caches between full and partial validators are independent and do not have an overlap: a message that can immediately be completed is never cached by the partial validator and vise versa. This should keep the total memory footprint across the both validators basically the same as before. Fixes #813
masih
added a commit
that referenced
this issue
Jan 16, 2025
When messages arrive with yet to be discovered EC chain we want to do as much validation as possible before buffering the message for future use. Because the key of the EC chain is included in the partial messages we are able to validate basically everything about the message except the chain itself. The changes here implement this ability. The implementation introduces a new validator implementation specifically written to handle partial messages. The validation rules are similar to the full GMesssage validator but then are adopted to infer the state of a message from chain key instead. A separate issue is captured to reduce duplicate rules across the two validators. This refactor is postponed on purpose as it touches a lot of the core implementations. Instead, the implementation here aims to provide a correct partial validation mechanism to progress efforts. The new validation makes a final validation attempt once the chain is discovered to check the validity of chain, its consistency with key and justification. The partial validator maintains its own cache, similar to the full validator, where messages are evicted as soon as GPBFT progresses. The caches between full and partial validators are independent and do not have an overlap: a message that can immediately be completed is never cached by the partial validator and vise versa. This should keep the total memory footprint across the both validators basically the same as before. Fixes #813
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 16, 2025
When messages arrive with yet to be discovered EC chain we want to do as much validation as possible before buffering the message for future use. Because the key of the EC chain is included in the partial messages we are able to validate basically everything about the message except the chain itself. The changes here implement this ability. The implementation introduces a new validator implementation specifically written to handle partial messages. The validation rules are similar to the full GMesssage validator but then are adopted to infer the state of a message from chain key instead. A separate issue is captured to reduce duplicate rules across the two validators. This refactor is postponed on purpose as it touches a lot of the core implementations. Instead, the implementation here aims to provide a correct partial validation mechanism to progress efforts. The new validation makes a final validation attempt once the chain is discovered to check the validity of chain, its consistency with key and justification. The partial validator maintains its own cache, similar to the full validator, where messages are evicted as soon as GPBFT progresses. The caches between full and partial validators are independent and do not have an overlap: a message that can immediately be completed is never cached by the partial validator and vise versa. This should keep the total memory footprint across the both validators basically the same as before. Fixes #813
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As part of #792 the ec chain (or Vote Value) may arrive in a different time than the message that references it. We will need to buffer such messages until the chain they reference is known. But we want to avoid buffering invalid messages for security reasons. For example, we can validate:
ErrValidationNotRelevant,ErrValidationTooOld)To make this possible, the current validation logic needs to be refactored to allow partial validation. We need to dig out the validation rules and apply them in a more pluggable way. An alternative is to duplicate the validation logic and harden it by tests to avoid divergence. The latter approach is probably faster but is a tech debt.
The text was updated successfully, but these errors were encountered: