You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the case where a one-block arrives late (late defined as after merger has already created a merged-block), the merger should evaluate the one-block as to if the one-block is already included in the merged-blocks or if it is a "forked block" and should be written to forked blocks.
When this happens: readers might be sending blocks to live firehose users but those blocks are delayed being send to the merger due to network issues. When you run different firehose components in different parts of the planet, weird things can happen.
If the one-block file is older than the "retain forked blocks" time period, then the one-block should simply be discarded (as today).
If not, read the one-block, read the merged-block, then take the decision on what to do (throw away or write to forked blocks)
The text was updated successfully, but these errors were encountered:
matthewdarwin
changed the title
Bug: merger should process late arriving one-block files
Bug: merger should process late arriving one-block files (to copy to forked blocks)
Feb 4, 2025
Copy from: streamingfast/merger#31
In the case where a one-block arrives late (late defined as after merger has already created a merged-block), the merger should evaluate the one-block as to if the one-block is already included in the merged-blocks or if it is a "forked block" and should be written to forked blocks.
When this happens: readers might be sending blocks to live firehose users but those blocks are delayed being send to the merger due to network issues. When you run different firehose components in different parts of the planet, weird things can happen.
If the one-block file is older than the "retain forked blocks" time period, then the one-block should simply be discarded (as today).
If not, read the one-block, read the merged-block, then take the decision on what to do (throw away or write to forked blocks)
The text was updated successfully, but these errors were encountered: