Sync in-memory trees using DB (DB is source of truth now). #842

piohei · 2025-01-07T09:46:03Z

Motivation

For Highly Available sequencer we will use database as source of truth and synchronization point.

Changes:

new task introduced to sync with database
for tree now there is an option to move forward and backward in history
added e2e tests for offchain mode
processing insertions/deletions is now consolidated into single task to give deletions precedence over insertions and also simplify logic with mutexes
identity tree refactored
for offchain mode transactions are being finalized based on database state instead of internal in-memory queue

e2e_tests/scenarios/tests/common/docker_compose.rs

src/ethereum/write_provider/openzeppelin.rs

src/identity_tree/db_sync.rs

Dzejkop · 2025-01-08T15:28:37Z

src/identity_tree/db_sync.rs

+    };
+
+    // And then update trees
+    update_latest_tree(tx, latest_tree, &latest_pending_tree_update, || {


very nice! Clever approach

docs/images/flows-Identity flow.png

src/identity/processor.rs

src/task_monitor/tasks/modify_tree.rs

src/task_monitor/mod.rs

src/task_monitor/tasks/modify_tree.rs

src/ethereum/write_provider/tx_sitter.rs

tests/tree_restore_with_root_back_to_init.rs

piohei marked this pull request as ready for review January 7, 2025 14:06

piohei requested a review from a team as a code owner January 7, 2025 14:06

Dzejkop reviewed Jan 8, 2025

View reviewed changes

e2e_tests/scenarios/tests/common/docker_compose.rs Show resolved Hide resolved

Dzejkop reviewed Jan 8, 2025

View reviewed changes

src/ethereum/write_provider/openzeppelin.rs Show resolved Hide resolved

Dzejkop reviewed Jan 8, 2025

View reviewed changes

src/identity_tree/db_sync.rs Show resolved Hide resolved

piohei force-pushed the piohei/sync_over_db branch 2 times, most recently from bb6f06c to d4c3f15 Compare January 8, 2025 13:52

Sync in-memory trees using DB (DB is source of truth now).

236227c

piohei force-pushed the piohei/sync_over_db branch from d4c3f15 to 236227c Compare January 8, 2025 14:36

Dzejkop reviewed Jan 8, 2025

View reviewed changes

Add docs images.

a98af42