fix: circular deps finalized head

[parketh]

Summary

This PR addresses various circular dependency issues with the current FG implementation.

Problems

• FG currently only processes blocks above the finalized head. However, the finalized head must be both btc+eth finalized. This means the FG never advances past the eth finalized block (it never achieves fast finality).
• Basing the block processing logic on the RPC also introduces potential circular dependencies
• block processing is currently very slow as it is sequential. if the FG falls behind for any reason (e.g. crash in another system service), it may never catch up (we observe this in our devnet)

Proposed fixes

1. Decouple the startup loop from the block processing loop. Only use the eth finalized block on startup, not anywhere else in the code.
2. Implement batch insert by parallelising calls
3. Update QueryBlockRangeBabylonFinalized to query the internal DB rather than call the Babylon node

Test plan

make lint
make test

[lesterli]

for this change, we need to update the FP e2e test.
I just ran make test-e2e-op with this commit and new rpc QueryIsBlockBabylonFinalizedFromBabylon, got this error:

2024-11-02T12:58:34.164+0800    ERROR   finalitygadget/finalitygadget.go:275    Failed to get activated timestamp from database {"error": "database not open"}
github.com/babylonlabs-io/finality-gadget/finalitygadget.(*FinalityGadget).QueryBtcStakingActivatedTimestamp
        /Users/lyy/go/pkg/mod/github.com/babylonlabs-io/[email protected]/finalitygadget/finalitygadget.go:275
github.com/babylonlabs-io/finality-gadget/server.(*rpcServer).QueryBtcStakingActivatedTimestamp
        /Users/lyy/go/pkg/mod/github.com/babylonlabs-io/[email protected]/server/rpcserver.go:80
github.com/babylonlabs-io/finality-gadget/proto._FinalityGadget_QueryBtcStakingActivatedTimestamp_Handler
        /Users/lyy/go/pkg/mod/github.com/babylonlabs-io/[email protected]/proto/finalitygadget_grpc.pb.go:218
google.golang.org/grpc.(*Server).processUnaryRPC
        /Users/lyy/go/pkg/mod/google.golang.org/[email protected]/server.go:1379
google.golang.org/grpc.(*Server).handleStream
        /Users/lyy/go/pkg/mod/google.golang.org/[email protected]/server.go:1790
google.golang.org/grpc.(*Server).serveStreams.func2.1
        /Users/lyy/go/pkg/mod/google.golang.org/[email protected]/server.go:1029

[bap2pecs]

I just ran make test-e2e-op with this commit and new rpc QueryIsBlockBabylonFinalizedFromBabylon, got this error:

I also met with this error testing my PR. the problem happens without this PR. i think it's likely caused by one of the commit here:

I guess this also means whenever we have a FG PR, we should test it with FP or it will be hard to bisect :)

[bap2pecs]

[bap2pecs]

why is this removed? should we add back?

[bap2pecs]

lgtm but will let @SebastianElvis take another look

[SebastianElvis]

Great work! Some non-blocking comments

[SebastianElvis]

Would it be possible that block h is finalised but block h+1 is not? I think it's not possiible as BTC staking enforces consecutive finalisation. So wdyt of maintaining the last finalised height in DB and accessing that value, instead of iterating over all these blocks?

[bap2pecs]

Would it be possible that block h is finalised but block h+1 is not?

no it's not possible (we can enforce it in bbolt.go as well but can go into another pr)

So wdyt of maintaining the last finalised height in DB and accessing that value, instead of iterating over all these blocks?

good point. we already maintain fist and last finalized heights. we can use them

const (
	blocksBucket          = "blocks"
	blockHeightsBucket    = "block_heights"
	indexerBucket         = "indexer"
	earliestBlockKey      = "earliest"
	latestBlockKey        = "latest"
	activatedTimestampKey = "activated_timestamp"
)

[SebastianElvis]

Yes, if using last finalised height then the logic can be greatly simplified. Can be done in a separate PR though

[parketh]

we tested it with a new devnet and it works! will merge to main