Topdown finality proposal and execution (#310)

* migrate top down crate * top down crate * new ipc changes * export proxy mod * fix tests * update cargo * update convert * stashed * stashed * initial impl * update cargo * lint * update tests * execute top down * format code * fm-303: update ipc sdk deps * ipc-308: address review * ipc-308: add additional check for reorgs * ipc-308: revert additional reorg check and add todo * detect chain reorg * temp changes * update reorg * fix tests * fix review * move get finality * more logging * fix getting genesis epoch * update config * more logging * more logs * update cargo lock * update config * remove network name * more logs * update cargo * fix top down error * more logs * fix prev hash * revert toolchain * implicit execution * remove context * implicit execution * more logging * more logging * address review * Fix topdown (#321) * fix lotus querying * query range * fix decoding * Revert typechain * log of parsing * fix order * more logs * more logs * more logs * disable change set * debug * more code * more code * more code * more logs * mint to gateway * debug code * working version with clean up * update comments * remove testing code * fix lint * adding topdown module * remove unused code * Test staking: Part 2 (#319) * TEST: Use EthAddress * TEST: Update closure * TEST: Generate join and leave * TEST: Separate configuration number for current and next * TEST: Take &mut self in StakingState * TEST: Test join * TEST: Test collateral and balance * TEST: Test stake command; fix check of has_joined * TEST: Test leave command; fix total collateral * TEST: Test checkpoint command; signature fails * TEST: Build checkpoint during command run * TEST: Higher default random size * TEST: Unit test for ABI encoding * TEST: Hash checkpoint as tuple * TEST: Try sending the signature with +27 recovery ID * TEST: Use non-masked ethereum address * TEST: Do not increment config when staking 0 * TEST: Tweak token amount generation so there aren't that many zeroes * TEST: Choose min_collateral so the last joiner activates * TEST: Bootstrap tests * TEST: Claim * TEST: Show error data in genesis * TEST: min_collateral > 0 * TEST: use local actors * TEST: Ranking * TEST: Subnet deactivation * TEST: Trying to debug the active validator set * TEST: See doesn't need bytes * TEST: Do not fail test if minimum collateral disagrees * TEST: Maybe we should fail * TEST: Assert active limit * TEST: Fix repeatability * TEST: Example of going over the limit of 2 active validators * TEST: Debugged the over-the-limit validators * TEST: Fix the active collateral sum * TEST: Quit testing if we hit the situation of differently ordered minimum collaterals * TEST: Check the hash of cross messages * TEST: Point at the integration testing branch of ABIs * TEST: Fix clippy * TEST: Update ABI git reference * FIX: Only take the /out * FIX: Install openssl in docker * TEST: Update Rust version in docker * FIX: Remove debug * FIX: 27 shift only needed by Solidity * TEST: Add unstake * FIX: SMT unit tests * FIX: Update RocksDB code * TEST: Use IPC actors dev * Update fendermint/vm/interpreter/src/fvm/state/ipc.rs Co-authored-by: Akosh Farkash <[email protected]> * update cargo * format code * log parent sync messages * sort staking change requests * sort in ascending order * fmt code --------- Co-authored-by: Akosh Farkash <[email protected]> --------- Signed-off-by: Alfonso de la Rocha <[email protected]> Co-authored-by: Alfonso de la Rocha <[email protected]> Co-authored-by: Akosh Farkash <[email protected]>
consensus-shipyard · Oct 25, 2023 · 69295d6 · 69295d6
1 parent 66ca42d
commit 69295d6
Show file tree

Hide file tree

Showing 29 changed files with 920 additions and 176 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/docs/running.md b/docs/running.md
@@ -26,14 +26,19 @@ In the following sections we will create a Genesis file for a network named `tes
 mkdir test-network
 ```
 
+If you are running in test network, define the network using env variable.
+```shell
+export FM_NETWORK=test
+```
+
 ### Create a new Genesis file
 
 First, create a new `genesis.json` file devoid of accounts and validators. The `--base-fee` here is completely arbitrary.
 The `--power-scale` value of `0` means we'll grant 1 voting power per 1 FIL; to use more precision, we can set it to `3`
 to use milliFIL for example.
 
 ```shell
-cargo run -p fendermint_app -- \
+cargo run -p fendermint_app --release -- \
   genesis --genesis-file test-network/genesis.json \
   new \
   --chain-name test \
@@ -64,7 +69,7 @@ Next, let's create some cryptographic key pairs we want want to use either for a
 ```shell
 mkdir test-network/keys
 for NAME in alice bob charlie dave; do
-  cargo run -p fendermint_app -- key gen --out-dir test-network/keys --name $NAME;
+  cargo run -p fendermint_app --release -- key gen --out-dir test-network/keys --name $NAME;
 done
 ```
 
@@ -83,7 +88,7 @@ Ak5Juk793ZAg/7Ojj4bzOmIFGpwLhET1vg2ROihUJFkq
 Add one of the keys we created to the Genesis file as a stand-alone account:
 
 ```shell
- cargo run -p fendermint_app -- \
+ cargo run -p fendermint_app --release -- \
         genesis --genesis-file test-network/genesis.json \
         add-account --public-key test-network/keys/alice.pk --balance 10
 ```
@@ -110,7 +115,7 @@ but it has to be one based on a public key, otherwise we would not be able to va
 Let's add an example of the other possible account type, a multi-sig account:
 
 ```shell
-cargo run -p fendermint_app -- \
+cargo run -p fendermint_app --release -- \
         genesis --genesis-file test-network/genesis.json \
         add-multisig --public-key test-network/keys/bob.pk --public-key test-network/keys/charlie.pk --public-key test-network/keys/dave.pk \
           --threshold 2 --vesting-start 0 --vesting-duration 1000000 --balance 30
@@ -142,7 +147,7 @@ $ cat test-network/genesis.json | jq .accounts[1]
 Finally, let's add one validator to the Genesis, with a monopoly on voting power, so we can run a standalone node:
 
 ```shell
-cargo run -p fendermint_app -- \
+cargo run -p fendermint_app --release -- \
       genesis --genesis-file test-network/genesis.json \
       add-validator --public-key test-network/keys/bob.pk --power 1;
 ```
@@ -166,6 +171,32 @@ The public key was spliced in as it was, in base64 format, which is how it would
 own genesis file format. Note that here we don't have the option to use `Address`, because we have to return
 these as actual `PublicKey` types to Tendermint through ABCI, not as a hash of a key.
 
+### (Optional) Add ipc to the Genesis file
+
+If you need ipc related function, let's add the subnet info to the Genesis with deployed subnet id: /r31415926
+
+```shell
+cargo run -p fendermint_app --release -- \
+      genesis --genesis-file test-network/genesis.json \
+      ipc \
+      gateway --subnet-id /r31415926 \
+      --bottom-up-check-period 10 \
+      --msg-fee 1 --majority-percentage 65 --min-collateral 1
+```
+Check the result:
+```console
+$ cat test-network/genesis.json | jq .ipc
+{
+  "gateway": {
+    "subnet_id": "/r31415926",
+    "bottom_up_check_period": 10,
+    "top_down_check_period": 10,
+    "msg_fee": "1",
+    "majority_percentage": 65
+  }
+}
+```
+
 ### Configure CometBFT
 
 First, follow the instructions in [getting started with CometBFT](./tendermint.md) to install the binary,
@@ -191,7 +222,7 @@ file we created earlier to the format CometBFT accepts. Start with the genesis f
 
 ```shell
 mv ~/.cometbft/config/genesis.json ~/.cometbft/config/genesis.json.orig
-cargo run -p fendermint_app -- \
+cargo run -p fendermint_app --release -- \
   genesis --genesis-file test-network/genesis.json \
   into-tendermint --out ~/.cometbft/config/genesis.json
 ```
@@ -281,7 +312,7 @@ one of the validators we created.
 
 ```shell
 mv ~/.cometbft/config/priv_validator_key.json ~/.cometbft/config/priv_validator_key.json.orig
-cargo run -p fendermint_app -- \
+cargo run -p fendermint_app --release -- \
   key into-tendermint --secret-key test-network/keys/bob.sk --out ~/.cometbft/config/priv_validator_key.json
 ```
 

diff --git a/fendermint/app/Cargo.toml b/fendermint/app/Cargo.toml
@@ -48,15 +48,16 @@ fendermint_vm_genesis = { path = "../vm/genesis" }
 fendermint_vm_interpreter = { path = "../vm/interpreter", features = ["bundle"] }
 fendermint_vm_message = { path = "../vm/message" }
 fendermint_vm_resolver = { path = "../vm/resolver" }
+fendermint_vm_topdown = { path = "../vm/topdown" }
 
 fvm = { workspace = true }
 fvm_ipld_blockstore = { workspace = true }
 fvm_ipld_car = { workspace = true }
 fvm_ipld_encoding = { workspace = true }
 fvm_shared = { workspace = true }
 ipc-sdk = { workspace = true }
-ipc_ipld_resolver = { workspace = true }
 ipc-provider = { workspace = true }
+ipc_ipld_resolver = { workspace = true }
 
 [dev-dependencies]
 tempfile = { workspace = true }

diff --git a/fendermint/app/config/default.toml b/fendermint/app/config/default.toml
@@ -168,3 +168,7 @@ event_buffer_capacity = 100
 rate_limit_bytes = 0
 # Length of the time period at which the consumption limit fills. 0 means no limit.
 rate_limit_period = 0
+
+# IPC related configuration parameters
+[ipc]
+subnet_id = "/r0"
diff --git a/fendermint/app/settings/Cargo.toml b/fendermint/app/settings/Cargo.toml
@@ -21,5 +21,7 @@ tendermint-rpc = { workspace = true }
 fvm_shared = { workspace = true }
 fvm_ipld_encoding = { workspace = true }
 ipc-sdk = { workspace = true }
+ipc-provider = { workspace = true }
 
 fendermint_vm_encoding = { path = "../../vm/encoding" }
+fendermint_vm_topdown = { path = "../../vm/topdown" }
diff --git a/fendermint/app/settings/src/lib.rs b/fendermint/app/settings/src/lib.rs
@@ -1,19 +1,24 @@
 // Copyright 2022-2023 Protocol Labs
 // SPDX-License-Identifier: Apache-2.0, MIT
 
-use anyhow::Context;
+use anyhow::{anyhow, Context};
 use config::{Config, ConfigError, Environment, File};
+use fvm_shared::address::Address;
 use fvm_shared::econ::TokenAmount;
 use ipc_sdk::subnet_id::SubnetID;
 use serde::Deserialize;
+use serde_with::{serde_as, DurationSeconds};
 use std::path::{Path, PathBuf};
+use std::time::Duration;
 use tendermint_rpc::Url;
 
 use fendermint_vm_encoding::{human_readable_delegate, human_readable_str};
+use fendermint_vm_topdown::BlockHeight;
 
 use self::eth::EthSettings;
 use self::fvm::FvmSettings;
 use self::resolver::ResolverSettings;
+use ipc_provider::config::deserialize::deserialize_eth_address_from_str;
 
 pub mod eth;
 pub mod fvm;
@@ -93,6 +98,54 @@ pub struct BroadcastSettings {
     pub max_retries: u8,
 }
 
+#[serde_as]
+#[derive(Debug, Deserialize, Clone)]
+pub struct TopDownConfig {
+    /// The number of blocks to delay before reporting a height as final on the parent chain.
+    /// To propose a certain number of epochs delayed from the latest height, we see to be
+    /// conservative and avoid other from rejecting the proposal because they don't see the
+    /// height as final yet.
+    pub chain_head_delay: BlockHeight,
+    /// Parent syncing cron period, in seconds
+    #[serde_as(as = "DurationSeconds<u64>")]
+    pub polling_interval: Duration,
+    /// Top down exponential back off retry base
+    #[serde_as(as = "DurationSeconds<u64>")]
+    pub exponential_back_off: Duration,
+    /// The max number of retries for exponential backoff before giving up
+    pub exponential_retry_limit: usize,
+    /// The parent rpc http endpoint
+    pub parent_http_endpoint: Url,
+    /// The parent registry address
+    #[serde(deserialize_with = "deserialize_eth_address_from_str")]
+    pub parent_registry: Address,
+    /// The parent gateway address
+    #[serde(deserialize_with = "deserialize_eth_address_from_str")]
+    pub parent_gateway: Address,
+}
+
+#[serde_as]
+#[derive(Debug, Deserialize, Clone)]
+pub struct IpcSettings {
+    #[serde_as(as = "IsHumanReadable")]
+    pub subnet_id: SubnetID,
+    /// The config for top down checkpoint. It's None if subnet id is root or not activating
+    /// any top down checkpoint related operations
+    pub topdown: Option<TopDownConfig>,
+}
+
+impl IpcSettings {
+    pub fn is_topdown_enabled(&self) -> bool {
+        !self.subnet_id.is_root() && self.topdown.is_some()
+    }
+
+    pub fn topdown_config(&self) -> anyhow::Result<&TopDownConfig> {
+        self.topdown
+            .as_ref()
+            .ok_or_else(|| anyhow!("top down config missing"))
+    }
+}
+
 #[derive(Debug, Deserialize, Clone)]
 pub struct Settings {
     /// Home directory configured on the CLI, to which all paths in settings can be set relative.
@@ -115,6 +168,7 @@ pub struct Settings {
     pub fvm: FvmSettings,
     pub resolver: ResolverSettings,
     pub broadcast: BroadcastSettings,
+    pub ipc: IpcSettings,
 }
 
 #[macro_export]