Merge pull request #106 from skip-mev/mergify/bp/release/v2.x.x/pr-88

feat(docs): add docs for packages (backport #88)

contrib/digitalocean/petri_docker.pkr.hcl.example

@@ -14,7 +14,7 @@ source "digitalocean" "petri" {
     size = "s-1vcpu-1gb"
     ssh_username = "root"
     snapshot_name = "petri-ubuntu-23-10-x64"
-    snapshot_regions = ["ams3", "nyc3"]
+    snapshot_regions = ["<REGIONS>"]
 }
 
 build {

contrib/docs/chain.md

@@ -0,0 +1,67 @@
+## Using the Petri chain package
+
+The `chain` package is the main entrypoint for creating Cosmos chains using Petri.
+A chain is a collection of nodes that are either validators or full nodes and are fully connected to each other (for now).
+
+### Creating a chain
+
+The main way to create a chain is by using the `NewChain` function. 
+The function accepts a `ChainConfig` struct that defines the chain's configuration.
+
+The basic gist on how to use the CreateChain function is as such:
+```go
+var infraProvider provider.Provider
+
+// create the chain
+chain, err := chain.CreateChain(ctx, logger, provider, chainConfig)
+
+if err != nil {
+    panic(err)
+}
+
+err = s.chain.Init(ctx)
+
+if err != nil {
+    panic(err)
+}
+```
+
+The CreateChain function only creates the nodes and their underlying workloads using the Provider. It does not
+create the genesis file, start the chain or anything else. All of the initial configuration is done in the `Init` function.
+
+### Waiting for a chain to go live
+
+After creating a chain, you can wait for it to go live by using the `WaitForBlocks` function.
+
+```go
+err = chain.WaitForBlocks(ctx, 1) // this will wait for the chain to produce 1 block
+```
+
+### Funding wallets
+
+To fund a wallet, you can use the `cosmosutil` package to send a `bank/send` transaction to the chain.
+
+```go
+encodingConfig := cosmosutil.EncodingConfig{
+    InterfaceRegistry: encodingConfig.InterfaceRegistry,
+    Codec:             encodingConfig.Codec,
+    TxConfig:          encodingConfig.TxConfig,
+}
+
+interactingFaucet := cosmosutil.NewInteractingWallet(chain, chain.GetFaucetWallet(), encodingConfig)
+
+user, err := wallet.NewGeneratedWallet("user", chainConfig.WalletConfig)
+
+sendGasSettings := petritypes.GasSettings{
+    Gas:         100000,
+    PricePerGas: int64(0),
+    GasDenom:    chainConfig.Denom,
+}
+
+// this will block until the bankSend transaction lands on the chain
+txResp, err := s.chainClient.BankSend(ctx, *interactingFaucet, user.Address(), sdk.NewCoins(sdk.NewCoin(chain.GetConfig().Denom, sdkmath.NewInt(1000000000))), sendGasSettings, true)
+
+if err != nil {
+    panic(err)
+}
+```

contrib/docs/digitalocean.md

@@ -0,0 +1,75 @@
+## DigitalOcean provider setup
+
+The DigitalOcean provider is a provider that uses the DigitalOcean API to create and manage droplets.
+It's different from the Docker provider in that it requires a one-time set-up.
+
+Petri includes a Packer definition file for an Ubuntu image that already has Docker set up and remotely exposed
+to the world. This is done for optimization reasons - creating an image one time is much faster than installing
+Docker and other dependencies on every created instance (1 minute vs >5 minutes).
+
+This only needs to be done once on your DigitalOcean account. After that, you can use the DigitalOcean provider
+as you would use the Docker provider.
+
+### Prerequisites
+
+- A DigitalOcean API token
+- [Packer](https://developer.hashicorp.com/packer/tutorials/docker-get-started/get-started-install-cli) 
+
+### Creating the Packer image
+
+1. Rename the `contrib/digitalocean/petri_docker.pkr.hcl.example` file to `contrib/digitalocean/petri_docker.pkr.hcl`
+2. Replace `<DO_API_TOKEN>` with your DigitalOcean API token 
+3. Include the regions you're going to run Petri on in the "snapshot_regions" variable.
+4. Run `packer build petri_docker.pkr.hcl`
+
+### Finding the image ID of your snapshot
+
+You can use the `doctl` (DigitalOcean CLI tool) to find the image ID of your snapshot.
+
+```bash
+doctl compute snapshot list
+```
+
+Denote the ID of the `petri-ubuntu-xxx` image.
+
+### Using the Petri Docker image
+
+**Using a provider directly**
+
+When using a provider directly to create a task, you can just modify the `TaskDefinition` to include a DigitalOcean
+specific configuration that includes the image ID.
+
+```go
+provider.TaskDefinition {
+		Name: "petri_example",
+		Image: provider.ImageDefinition{
+			Image: "nginx",
+		},
+		DataDir: "/test",
+		ProviderSpecific: provider.ProviderSpecific{
+			"image_id": <IMAGE_ID>,
+		},
+}
+```
+
+**Using a Chain**
+
+When using a Chain to create nodes, you have to include a `NodeDefinitionModifier` in the `ChainConfig` that
+includes the Image ID in the provider specific configuration
+
+```go
+spec = petritypes.ChainConfig{
+	// other configuration removed
+	NodeDefinitionModifier: func(def provider.TaskDefinition, nodeConfig petritypes.NodeConfig) provider.TaskDefinition {
+		doConfig := digitalocean.DigitalOceanTaskConfig{
+			Size: "s-1vcpu-2gb",
+			Region: "ams3", // only choose regions that the snapshot is available in
+			ImageID: <IMAGE_ID>,
+		}
+
+		def.ProviderSpecificConfig = doConfig
+
+		return def
+	},
+}
+```

contrib/docs/monitoring.md

@@ -0,0 +1,100 @@
+## Monitoring your tasks with the monitoring package (batteries included)
+
+The `monitoring` package in `petri/core` allows you to easily set up Prometheus and Grafana to ingest metrics
+from your tasks.
+
+### Setting up Prometheus
+
+The monitoring package has a function `SetupPrometheusTask` that handles the setting up and configuration of a
+Prometheus task.
+
+```go
+prometheusTask, err := monitoring.SetupPrometheusTask(ctx, logger, provider, monitoring.PrometheusOptions{
+	Targets: endpoints, // these endpoints are in the format host:port, make sure they are reachable from the Prometheus task
+	ProviderSpecificConfig: struct{}{} // if any apply
+})
+
+if err != nil {
+	panic(err)
+}
+
+err = prometheusTask.Start(ctx, false)
+
+if err != nil {
+	panic(err)
+}
+```
+
+### Setting up Prometheus for your chain
+
+The set up for ingesting metrics from the chain nodes is pretty similar, with one caveat of getting the node
+metrics endpoints. The Prometheus task has to be created after the chain is started, as only after that
+you can be sure that you can receive the correct external IP for the nodes.
+
+```go
+var endpoints []string
+
+for _, node := range append(chain.GetValidators(), chain.GetNodes()...) {
+	endpoint, err := node.GetTask().GetIP(ctx)
+	if err != nil {
+		panic(err)
+	}
+
+	endpoints = append(endpoints, fmt.Sprintf("%s:26660", endpoint))
+}
+```
+
+### Setting up Grafana
+
+The monitoring package has a similar function for setting up a Grafana task called `SetupGrafanaTask`.
+The difference between Prometheus and Grafana setup is that Grafana additionally requires a snapshot of a Grafana dashboard
+exported in JSON. 
+
+You can look up how to export a dashboard in JSON format [here](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/).
+You need to note down the dashboard ID if you want to take a snapshot and replace the `version` in the JSON model with `1`.
+Additionally, you need to replace all mentions of your Prometheus data source with `petri_prometheus`.
+
+```go
+import _ "embed"
+
+//go:embed files/dashboard.json
+var dashboardJSON string
+
+grafanaTask, err := monitoring.SetupGrafanaTask(ctx, logger, provider, monitoring.GrafanaOptions{
+    DashboardJSON: dashboardJSON,
+    PrometheusURL: fmt.Sprintf("http://%s", prometheusIP),
+    ProviderSpecificConfig: struct{}{} // if any apply
+})
+
+if err != nil {
+    panic(err)
+}
+
+err = grafanaTask.Start(ctx, false)
+
+if err != nil {
+    panic(err)
+}
+
+grafanaIP, err := s.grafanaTask.GetExternalAddress(ctx, "3000")
+
+if err != nil {
+    panic(err)
+}
+
+fmt.Printf("Visit Grafana at http://%s\n", grafanaIP)
+```
+
+### Taking a public snapshot of a Grafana dashboard
+
+You can take a public snapshot of a Grafana dashboard by using the `TakeSnapshot` function.
+
+```go
+snapshotURL, err := monitoring.TakeSnapshot(ctx, "<dashboard_uid>", "<grafana_ip>")
+
+if err != nil {
+    panic(err)
+}
+
+fmt.Printf("Visit the snapshot at %s\n", snapshotURL)
+```

contrib/docs/provider.md

@@ -0,0 +1,125 @@
+# Petri providers
+
+In Petri, providers are meant to abstract the implementation details
+of creating workloads (containers) from the downstream dependencies.
+At the current state, Petri supports two providers: `docker` and `digitalocean`.
+
+In most cases, you won't interact with the providers directly, but rather use the 
+`chain`, `node`, etc. packages to create a chain. If you still want to use the `provider`
+package directly to create custom workloads, feel free to continue reading this doc.
+
+## Using a provider
+
+Instead of interacting with providers implementing the `Provider` interface directly,
+you should use the `NewTask` function to create tasks. Tasks are a higher level
+abstraction that allows you to create workloads in a more declarative way.
+
+A working example on how to create a task:
+```go
+package main
+
+import (
+	"context"
+	"fmt"
+	"github.com/skip-mev/petri/general/v2/provider"
+	"github.com/skip-mev/petri/general/v2/provider/digitalocean"
+	"go.uber.org/zap"
+)
+
+func main() {
+	logger, _ := zap.NewDevelopment()
+	doProvider, err := docker.NewDockerProvider(context.Background(), logger, "petri_docker")
+	if err != nil {
+		panic(err)
+	}
+
+	task, err := provider.CreateTask(context.Background(), logger, doProvider, provider.TaskDefinition{
+		Name: "petri_example",
+		Image: provider.ImageDefinition{
+			Image: "nginx",
+		},
+		DataDir: "/test",
+	})
+
+	if err != nil {
+		panic(err)
+	}
+
+	err = task.Start(context.Background(), false)
+
+	if err != nil {
+		panic(err)
+	}
+}
+```
+
+## Interacting with tasks
+
+Once you have a task, you can interact with it using the following methods:
+- `Start`: starts the task and its sidecars
+- `Stop`: stops the task and its sidecars
+- `(Write/Read)File` : writes/reads a file from the container
+- `RunCommand`: runs a command inside the container
+- `GetExternalAddress`: given a port, it returns the external address 
+                        of the container for that specific port
+                        (useful for API-based interaction)
+
+## Hooking into the lifecycle of a task
+
+You can hook into the lifecycle of a task or its sidecars (since they're also a Task)
+by using the `SetPreStart` / `SetPostStop` methods. By default, a Task does not have
+any pre-start or post-stop hooks.
+
+Example of using a pre-start hook:
+
+```go
+package main
+
+package main
+
+import (
+	"context"
+	"fmt"
+	"github.com/skip-mev/petri/general/v2/provider"
+	"github.com/skip-mev/petri/general/v2/provider/digitalocean"
+	"go.uber.org/zap"
+)
+
+func main() {
+	logger, _ := zap.NewDevelopment()
+	doProvider, err := docker.NewDockerProvider(context.Background(), logger, "petri_docker")
+	if err != nil {
+		panic(err)
+	}
+
+	task, err := provider.CreateTask(context.Background(), logger, doProvider, provider.TaskDefinition{
+		Name: "petri_example",
+		Image: provider.ImageDefinition{
+			Image: "nginx",
+		},
+		DataDir: "/test",
+	})
+
+	if err != nil {
+		panic(err)
+	}
+
+	err = task.Start(context.Background(), false)
+
+	if err != nil {
+		panic(err)
+	}
+
+    task.SetPreStart(func(ctx context.Context, task *provider.Task) error {
+		_, _, _, err := task.RunCommand(ctx, []string{"mv",  "/mnt/index.html", "/usr/share/nginx/html"})
+
+        return err
+    })
+}
+```
+
+## Provider specific configuration
+
+Every task can have its own provider-specific configuration.
+The provider-specific configuration has a type of `interface{}` and each provider
+is responsible for verifying that the configuration is of the correct type.