Bake stake into genesis

Cargo.toml

@@ -20,6 +20,8 @@ openssl = "0.10.66"
 rand = "0.8.5"
 reqwest = { version = "0.11.23", features = ["blocking", "brotli", "deflate", "gzip", "rustls-tls", "json"] }
 rustls = { version = "0.21.11", default-features = false, features = ["quic"] }
+serde = "1.0.208"
+serde_yaml = "0.9.34"
 solana-accounts-db = "1.18.20"
 solana-clap-v3-utils = "1.18.20"
 solana-core = "1.18.20"

PROGRESS.md

@@ -79,16 +79,23 @@ By here:
 - We can deploy bootstrap, N validators, M RPC nodes, and C clients with various command line configurations
 - We can control the how and where we deploy kubernetes pods
 
-- [ ] Other Features
+- [x] Other Features
     - [x] Heterogeneous Clusters (i.e. multiple validator versions)
     - [x] Deploy specific commit
     - [x] Generic Clients
-    - [ ] Deploy with user-defined stake distribution
+    - [x] Deploy with user-defined stake distribution
 
 By here:
 - We can deploy bootstrap, N validators, M RPC nodes, and C clients with various command line configurations
 - We can control the how and where we deploy kubernetes pods
 - We can deploy multiple cluster versions and have them interact with each other
 - We can define a stake distribution for our cluster
 
+Features TODO
+- [ ] Heterogenous Agave/Firedancer clusters
+- [ ] Latency and packet drop simulation
+- [ ] Feature gate activation
+- [ ] High Level: Usage scheduling
+  - based on a user's deployment scale, need time-based user multiplexing of infrastructure
+
 DONE

README.md

@@ -9,6 +9,17 @@ In Validator Lab we can deploy and test new validator features quickly and easil
 
 ## How to run
 
+### Requirements
+1) Docker. Create `docker` group add user to `docker` group
+```
+sudo usermod -aG docker $USER
+newgrp docker
+```
+2) jq
+```
+sudo apt install jq
+```
+
 ### Setup
 Ensure you have the proper permissions to connect to the Monogon Kubernetes endpoint. Reach out to Leo on slack if you need the key (you do if you haven't asked him in the past).
 
@@ -79,13 +90,15 @@ cargo run --bin cluster --
     --base-image <base-image>           # e.g. ubuntu:20.04
     --image-name <docker-image-name>    # e.g. cluster-image
     # validator config
+    --skip-primordial-accounts
     --full-rpc
-    --internal-node-sol <Sol>
-    --internal-node-stake-sol <Sol>
+    --internal-node-sol <sol>
+    --internal-node-stake-sol <sol>
     # kubernetes config
     --cpu-requests <cores>
     --memory-requests <memory>
     # deploy with clients
+    bench-tps
     -c <num-clients>
     --client-type <client-type e.g. tpu-client>
     --client-to-run <type-of-client e.g. bench-tps>
@@ -107,6 +120,41 @@ For client Version >= 2.0.0
 --bench-tps-args 'tx-count=5000 keypair-multiplier=4 threads=16 num-lamports-per-account=200000000 sustained tpu-connection-pool-size=8 thread-batch-sleep-ms=0 commitment-config=processed'
 ```
 
+## Baking Validator Stakes into Genesis
+- You can bake validator accounts and delegated stakes into genesis creation by passing in `--validator-balances-file <file-path-to-validator-balances-yml>`. This way when the cluster boots up, all validators will consistently be in the leader schedule and no need to wait for stake to warm up. In the validator balances file, you can set specific validator balances and stake amounts. 
+The validator balances file has the following yaml format:
+```
+---
+v0:
+  balances_lamports: <balance0>
+  stake_lamports: <stake0>
+v1: 
+  balances_lamports: <balance1>
+  stake_lamports: <stake1>
+...
+vN:
+  balances_lamports: <balanceN>
+  stake_lamports: <stakeN>
+```
+^ Note, the file must have the `v0`, `v1`, ..., `vN` format. The number of validators in this file must match `--num-validators <number-of-validators>`
+
+For example, we could create: `validator-balances.yml` and have it look like:
+```
+---
+v0:
+  balance_lamports: 400000000000
+  stake_lamports: 40000000000
+v1:
+  balance_lamports: 200000000000
+  stake_lamports: 20000000000
+v2:
+  balance_lamports: 300000000000
+  stake_lamports: 30000000000
+```
+
+- If you do not want to bake stakes into genesis and instead want the stake to warm up after deplyoyment, pass in the flag `--skip-primordial-stakes` and leave out `--validator-balances`
+- `--internal-node-sol`, `--internal-node-stake-sol`, are `--comission` are only valid with `--skip-primordial-stakes`
+
 ## Metrics
 1) Setup metrics database:
 ```
@@ -128,7 +176,7 @@ You can add in RPC nodes. These sit behind a load balancer. Load balancer distri
 --num-rpc-nodes <num-nodes>
 ```
 
-## Heterogeneous Clusters
+## Heterogeneous Agave Clusters
 You can deploy a cluster with heterogeneous validator versions
 For example, say you want to deploy a cluster with the following nodes:
 * 1 bootstrap, 3 validators, 1 rpc-node, and 1 client running some agave-repo local commit
@@ -155,7 +203,9 @@ cargo run --bin cluster -- -n <namespace> --registry <registry> --release-channe
 
 For steps (2) and (3), when using `--no-bootstrap`, we assume that the directory at `--cluster-data-path <directory>` has the correct genesis, bootstrap identity, and faucet account stored. These are all created in step (1).
 
-Note: We can't deploy heterogeneous clusters across v1.17 and v1.18 due to feature differences. Hope to fix this in the future. Have something where we can specifically define which features to enable.
+Notes: 
+1) We can't deploy heterogeneous clusters across v1.17 and v1.18 due to feature differences. Hope to fix this in the future. Have something where we can specifically define which features to enable.
+2) Heterogenous clusters with primordial stakes baked into genesis is not supported yet
 
 ## Querying the RPC from outside the cluster
 The cluster now has an external IP/port that can be queried to reach the cluster RPC. The external RPC port will be logged during cluster boot, e.g.: