Skip to content

Commit

Permalink
rewrite wasm manager documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
Zwiterrion committed Nov 20, 2023
1 parent 26d8d10 commit cec9508
Show file tree
Hide file tree
Showing 20 changed files with 12,159 additions and 129 deletions.
4 changes: 2 additions & 2 deletions manual/src/main/paradox/how-to-s/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
in this section, we will explain some mainstream Otoroshi usage scenario's

* @ref:[Otoroshi and WASM](./wasm-usage.md)
* @ref:[WASM Manager](./wasm-manager-installation.md)
* @ref:[Wasmo](./wasmo-installation.md)
* @ref:[Tailscale integration](./tailscale-integration.md)
* @ref:[End-to-end mTLS](./end-to-end-mtls.md)
* @ref:[Send alerts by emails](./export-alerts-using-mailgun.md)
Expand Down Expand Up @@ -32,7 +32,7 @@ in this section, we will explain some mainstream Otoroshi usage scenario's


* [WASM usage](./wasm-usage.md)
* [WASM Manager](./wasm-manager-installation.md)
* [wasmo](./wasmo-installation.md)
* [Tailscale integration](./tailscale-integration.md)
* [End-to-end mTLS](./end-to-end-mtls.md)
* [Send alerts by emails](./export-alerts-using-mailgun.md)
Expand Down
24 changes: 12 additions & 12 deletions manual/src/main/paradox/how-to-s/wasm-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ WebAssembly (WASM) is a simple machine model and executable format with an exten
To simplify the process of WASM creation and usage, Otoroshi provides:

- otoroshi ui integration: a full set of plugins that let you pick which WASM function to runtime at any point in a route
- otoroshi `wasm-manager`: a code editor in the browser that let you write your plugin in `Rust`, `TinyGo`, `Javascript` or `Assembly Script` without having to think about compiling it to WASM (you can find a complete tutorial about it @ref:[here](../how-to-s/wasm-manager-installation.md))
- otoroshi `wasmo`: a code editor in the browser that let you write your plugin in `Rust`, `TinyGo`, `Javascript` or `Assembly Script` without having to think about compiling it to WASM (you can find a complete tutorial about it @ref:[here](../how-to-s/wasmo-installation.md))

@@@ div { .centered-img }
<img src="../imgs/otoroshi-wasm-manager-1.png" title="screenshot of a wasm manager instance" />
<img src="../imgs/otoroshi-wasmo-1.png" title="screenshot of a wasmo instance" />
@@@

## Tutorial
Expand All @@ -32,19 +32,19 @@ For this tutorial, we will start with an existing wasm file. The main function o
The main function of this validator, written in rust, should look like:

validator.rs
: @@snip [validator.rs](../snippets/wasm-manager/validator.rs)
: @@snip [validator.rs](../snippets/wasmo/validator.rs)

validator.js
: @@snip [validator.js](../snippets/wasm-manager/validator.js)
: @@snip [validator.js](../snippets/wasmo/validator.js)

validator.ts
: @@snip [validator.ts](../snippets/wasm-manager/validator.ts)
: @@snip [validator.ts](../snippets/wasmo/validator.ts)

validator.js
: @@snip [validator.js](../snippets/wasm-manager/validator.js)
: @@snip [validator.js](../snippets/wasmo/validator.js)

validator.go
: @@snip [validator.js](../snippets/wasm-manager/validator.go)
: @@snip [validator.js](../snippets/wasmo/validator.go)

The plugin receives the request context from Otoroshi (the matching route, the api key if present, the headers, etc) as `WasmAccessValidatorContext` object.
Then it applies a check on the headers, and responds with an error or success depending on the content of the foo header.
Expand Down Expand Up @@ -137,19 +137,19 @@ The next step in this tutorial is to use a WASM file as backend of the route. W
The content of this plugin, called `wasm-target.wasm`, looks like:

target.rs
: @@snip [target.rs](../snippets/wasm-manager/target.rs)
: @@snip [target.rs](../snippets/wasmo/target.rs)

target.js
: @@snip [target.js](../snippets/wasm-manager/target.js)
: @@snip [target.js](../snippets/wasmo/target.js)

target.ts
: @@snip [target.ts](../snippets/wasm-manager/target.ts)
: @@snip [target.ts](../snippets/wasmo/target.ts)

target.js
: @@snip [target.js](../snippets/wasm-manager/target.js)
: @@snip [target.js](../snippets/wasmo/target.js)

target.go
: @@snip [target.js](../snippets/wasm-manager/target.go)
: @@snip [target.js](../snippets/wasmo/target.go)

Let's explain this snippet. The purpose of this type of plugin is to respond an HTTP response with http status, body and headers map.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,112 +1,33 @@
# Deploy your own WASM Manager
# Deploy your own Wasmo

@@@ div { .centered-img }
<img src="../imgs/otoroshi-wasm-manager-1.png" />
@@@

## Manager's configuration

In the @ref:[WASM tutorial](./wasm-usage.md) we used existing WASM files. These files has been generated with the WASM Manager solution provided by the Otoroshi team.

The wasm manager is a code editor in the browser that will help you to write and compile your plugin to WASM using Rust or Assembly Script.
You can install your own man ager instance using a docker image.

```sh
docker run -p 5001:5001 maif/otoroshi-wasm-manager
```

This should download and run the latest version of the manager. Once launched, you can navigate [http://localhost:5001](http://localhost:5001) (or any other binding port).

This should show an authentication error. The manager can run with or without authentication, and you can confige it using the `AUTH_MODE` environment variable (`AUTH` or `NO_AUTH` values).

The manager is configurable by environment variables. The manager uses an object storage (S3 compatible) as storage solution.
You can configure your S3 with the four variables `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `S3_ENDPOINT` and `S3_BUCKET`.

Feel free to change the following variables:


| NAME | DEFAULT VALUE | DESCRIPTION |
| ------------------------- | ------------------ | -------------------------------------------------------------------------- |
| MANAGER_PORT | 5001 | The manager will be exposed on this port |
| MANAGER_ALLOWED_DOMAINS | otoroshi.oto.tools | Array of origins, separated by comma, which is allowed to call the manager |
| MANAGER_MAX_PARALLEL_JOBS | 2 | Number of parallel jobs to compile plugins |

The following variables are useful to bind the manager with Otoroshi and to run it behind (we will use them in the next section of this tutorial).

| NAME | DEFAULT VALUE | DESCRIPTION |
| ---------------------- | ----------------------- | ------------------------------------------------------ |
| OTOROSHI_USER_HEADER | Otoroshi-User | Header used to extract the user from Otoroshi request |
| OTOROSHI_TOKEN_SECRET | veryverysecret | the secret used to sign the user token |
Installing Wasmo can be done by following the [Getting Started](https://ubiquitous-adventure-yrognwg.pages.github.io/builder/getting-started) in Wasmo documentation.

## Tutorial

- [Deploy your own WASM Manager](#deploy-your-own-wasm-manager)
- [Manager's configuration](#managers-configuration)
- [Deploy your own Wasmo](#deploy-your-own-wasmo)
- [Tutorial](#tutorial)
- [Before your start](#before-your-start)
- [Deploy the manager using Docker](#deploy-the-manager-using-docker)
- [Create a route to expose and protect the manager with authentication](#create-a-route-to-expose-and-protect-the-manager-with-authentication)
- [Create a first validator plugin using the manager](#create-a-first-validator-plugin-using-the-manager)
- [Configure the danger zone of Otoroshi to bind Otoroshi and the manager](#configure-the-danger-zone-of-otoroshi-to-bind-otoroshi-and-the-manager)
- [Create a route to expose and protect Wasmo with authentication](#create-a-route-to-expose-and-protect-wasmo-with-authentication)
- [Create a first validator plugin using Wasmo](#create-a-first-validator-plugin-using-wasmo)
- [Pairing Otoroshi and Wasmo](#pairing-otoroshi-with-wasmo)
- [Create a route using the generated wasm file](#create-a-route-using-the-generated-wasm-file)
- [Test your route](#test-your-route)

After completing these steps you will have a running Otoroshi instance and our owm WASM manager linked together.
After completing these steps you will have a running Otoroshi instance and our owm Wasmo linked together.

### Before your start

@@include[initialize.md](../includes/initialize.md) { #initialize-otoroshi }

### Deploy the manager using Docker

Let's start by deploying an instance of S3. If you already have an instance you can skip the next section.

```sh
docker network create manager-network
docker run --name s3Server -p 8000:8000 -e SCALITY_ACCESS_KEY_ID=access_key -e SCALITY_SECRET_ACCESS_KEY=secret --net manager-network scality/s3server
```

Once launched, we can run a manager instance.

```sh
docker run -d --net manager-network \
--name wasm-manager \
-p 5001:5001 \
-e "MANAGER_PORT=5001" \
-e "AUTH_MODE=AUTH" \
-e "MANAGER_MAX_PARALLEL_JOBS=2" \
-e "MANAGER_ALLOWED_DOMAINS=otoroshi.oto.tools,wasm-manager.oto.tools,localhost:5001" \
-e "OTOROSHI_USER_HEADER=Otoroshi-User" \
-e "OTOROSHI_TOKEN_SECRET=veryverysecret" \
-e "S3_ACCESS_KEY_ID=access_key" \
-e "S3_SECRET_ACCESS_KEY=secret" \
-e "S3_FORCE_PATH_STYLE=true" \
-e "S3_ENDPOINT=http://host.docker.internal:8000" \
-e "S3_BUCKET=wasm-manager" \
-e "DOCKER_USAGE=true" \
maif/otoroshi-wasm-manager
```

Once launched, go to [http://localhost:5001](http://localhost:5001). If everything is working as intended,
you should see, at the bottom right of your screen the following error

```
You're not authorized to access to manager
```

This error indicates that the manager could not authorize the request.
Actually, the manager expects to be only reachable through Otoroshi (this is the definition of the `AUTH_MODE=AUTH`).
So we need to create a route in Otoroshi to properly expose our manager to the rest of the world.

### Create a route to expose and protect the manager with authentication
### Create a route to expose and protect Wasmo with authentication

We are going to use the admin API of Otoroshi to create the route. The configuration of the route is:

* `wasm-manager` as name
* `wasm-manager.oto.tools` as exposed domain
* `wasmo` as name
* `wasmo.oto.tools` as exposed domain
* `localhost:5001` as target without TLS option enabled

We need to add two more plugins to require the authentication from users and to pass the logged in user to the manager.
We need to add two more plugins to require the authentication from users and to pass the logged in user to Wasmo.
These plugins are named `Authentication` and `Otoroshi Info. token`.
The Authentication plugin will use an in-memory authentication with one default user ([email protected]/password).
The second plugin will be configured with the value of the `OTOROSHI_USER_HEADER` environment variable.
Expand All @@ -121,7 +42,7 @@ curl -X POST "http://otoroshi-api.oto.tools:8080/api/auths" \
-H 'Content-Type: application/json; charset=utf-8' \
-d @- <<'EOF'
{
"id": "wasm_manager_in_memory",
"id": "wasmo_in_memory",
"type": "basic",
"name": "In memory authentication",
"desc": "Group of static users",
Expand All @@ -140,18 +61,18 @@ curl -X POST "http://otoroshi-api.oto.tools:8080/api/auths" \
EOF
```

Once created, you can create our route to expose the manager.
Once created, you can create our route to expose Wasmo.

```sh
curl -X POST "http://otoroshi-api.oto.tools:8080/api/routes" \
-H "Content-type: application/json" \
-u "admin-api-apikey-id:admin-api-apikey-secret" \
-d @- <<'EOF'
{
"id": "wasm-manager",
"name": "wasm-manager",
"id": "wasmo",
"name": "wasmo",
"frontend": {
"domains": ["wasm-manager.oto.tools"]
"domains": ["wasmo.oto.tools"]
},
"backend": {
"targets": [
Expand All @@ -176,7 +97,7 @@ curl -X POST "http://otoroshi-api.oto.tools:8080/api/routes" \
"config": {
"pass_with_apikey": false,
"auth_module": null,
"module": "wasm_manager_in_memory"
"module": "wasmo_in_memory"
}
},
{
Expand Down Expand Up @@ -207,13 +128,13 @@ curl -X POST "http://otoroshi-api.oto.tools:8080/api/routes" \
EOF
```

Try to access to the manager with the new domain: http://wasm-manager.oto.tools:8080.
Try to access to Wasmo with the new domain: http://wasmo.oto.tools:8080.
This should redirect you to the login page of Otoroshi. Enter the credentials of the user: [email protected]/password
Congratulations, you now have a secure manager.
Congratulations, you now have secured Wasmo.

### Create a first validator plugin using the manager
### Create a first validator plugin using Wasmo

In the previous part, we secured the manager. Now, is the time to create your first simple plugin, written in Rust.
In the previous part, we secured the access to Wasmo. Now, is the time to create your first simple plugin, written in Rust.
This plugin will apply a check on the request and ensure that the headers contains the key-value foo:bar.

1. On the right top of the screen, click on the plus icon to create a new plugin
Expand Down Expand Up @@ -344,16 +265,17 @@ Let's edit the fake input context by adding the exepected foo Header.

Resubmit the command. It should pass.

### Configure the danger zone of Otoroshi to bind Otoroshi and the manager
### Pairing Otoroshi and Wasmo

Now that we have our compiled plugin, we have to connect Otoroshi with the manager. Let's navigate to the danger zone, and add the following values in the WASM manager section:
Now that we have our compiled plugin, we have to connect Otoroshi with Wasmo. Let's navigate to the danger zone, and add the following values in the Wasmo section:

* `URL`: admin-api-apikey-id
* `Apikey id`: admin-api-apikey-secret
* `Apikey secret`: http://localhost:5001
* `User(s)`: *
* `Token secret`:

The User(s) property is used by the manager to filter the list of returned plugins (example: [email protected] will only return the list of plugins created by this user).
The User(s) property is used by Wasmo to filter the list of returned plugins (example: [email protected] will only return the list of plugins created by this user).

Don't forget to save the configuration.

Expand Down Expand Up @@ -414,4 +336,4 @@ and
curl "http://wasm-route.oto.tools:8080" -H "foo:bar"
```

Congratulations, you have successfully written your first validator using your own manager.
Congratulations, you have successfully written your first validator using your own Wasmo.
22 changes: 11 additions & 11 deletions manual/src/main/paradox/topics/wasm-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,18 @@ WebAssembly (WASM) is a simple machine model and executable format with an exten
To simplify the process of WASM creation and usage, Otoroshi provides:

- otoroshi ui integration: a full set of plugins that let you pick which WASM function to runtime at any point in a route
- otoroshi `wasm-manager`: a code editor in the browser that let you write your plugin in `Rust`, `TinyGo`, `Javascript` or `Assembly Script` without having to think about compiling it to WASM (you can find a complete tutorial about it @ref:[here](../how-to-s/wasm-manager-installation.md))
- otoroshi `wasmo`: a code editor in the browser that let you write your plugin in `Rust`, `TinyGo`, `Javascript` or `Assembly Script` without having to think about compiling it to WASM (you can find a complete tutorial about it @ref:[here](../how-to-s/wasmo-installation.md))

@@@ div { .centered-img }
<img src="../imgs/otoroshi-wasm-manager-1.png" title="screenshot of a wasm manager instance" />
<img src="../imgs/otoroshi-wasmo-1.png" title="screenshot of a wasmo instance" />
@@@

## Available tutorials

here is the list of available tutorials about wasm in Otoroshi

1. @ref:[install a wasm manager](../how-to-s/wasm-manager-installation.md)
2. @ref:[use a wasm plugin](../how-to-s/wasm-usage.md)
1. @ref:[Install a Wasmo](../how-to-s/wasmo-installation.md)
2. @ref:[Use a WASM plugin](../how-to-s/wasm-usage.md)

## Wasm plugins entities

Expand All @@ -39,7 +39,7 @@ In a wasm plugin entity, you can define the source of your wasm plugin. You can
- `base64`: a base64 encoded wasm script
- `file`: the path to a wasm script file
- `http`: the url to a wasm script file
- `wasm-manager`: the name of a wasm script compiled by a wasm manager instance
- `wasmo`: the name of a wasm script compiled by a Wasmo instance

then you can define the number of memory pages available for each plugin instanciation, the name of the function you want to invoke, the config. map of the VM and if you want to keep a wasm vm alive during the request lifecycle to be able to reuse it in different plugin steps

Expand Down Expand Up @@ -87,13 +87,13 @@ the following examples are written in rust. the rust macros provided by extism m
do not forget to add the extism pdk library to your project to make it compile

Cargo.toml
: @@snip [Cargo.toml](../snippets/wasm-manager/Cargo.toml)
: @@snip [Cargo.toml](../snippets/wasmo/Cargo.toml)

go.mod
: @@snip [go.mod](../snippets/wasm-manager/go.mod)
: @@snip [go.mod](../snippets/wasmo/go.mod)

package.json
: @@snip [package.json](../snippets/wasm-manager/package.json)
: @@snip [package.json](../snippets/wasmo/package.json)

### WasmRouteMatcher

Expand Down Expand Up @@ -599,10 +599,10 @@ extern "C" {
}
```

right know, when using the wasm manager, a default idiomatic implementation is provided for `TinyGo` and `Rust`
right know, when using the Wasmo, a default idiomatic implementation is provided for `TinyGo` and `Rust`

host.rs
: @@snip [host.rs](../snippets/wasm-manager/host.rs)
: @@snip [host.rs](../snippets/wasmo/host.rs)

host.go
: @@snip [host.go](../snippets/wasm-manager/host.go)
: @@snip [host.go](../snippets/wasmo/host.go)
28 changes: 28 additions & 0 deletions tools/otoroshi-wasm-manager/server/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
MANAGER_PORT=5001
MANAGER_ALLOWED_DOMAINS=localhost:5001,otoroshi.oto.tools
MANAGER_MAX_PARALLEL_JOBS=2

MANAGER_TEMPLATES=file://templates
# MANAGER_TEMPLATES=https://api.github.com/repos/maif/otoroshi/contents/tools/otoroshi-wasm-manager/server/templates

WAPM_REGISTRY_TOKEN=wap_36003b6d9ffb8a9b05966601d23bcfe72ccc71b9eb406dd107dc6b362d65440d

GITHUB_MAX_REPO_SIZE=250000000 # bytes
GITHUB_PERSONAL_TOKEN=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

AUTH_MODE=NO_AUTH

OTOROSHI_USER_HEADER=Otoroshi-User
OTOROSHI_CLIENT_ID=admin-api-apikey-id
OTOROSHI_CLIENT_SECRET=admin-api-apikey-secret

S3_ACCESS_KEY_ID=J11Q131JBRSOXFEOIHR8
S3_SECRET_ACCESS_KEY=JYIcOAYq3CGAUDx4cup5yxljEtEvMYWHbTJggDDF
S3_ENDPOINT=cellar-c2.services.clever-cloud.com
S3_BUCKET=wasm-manager

# S3_ACCESS_KEY_ID=access_key
# S3_SECRET_ACCESS_KEY=secret
# S3_ENDPOINT=http://localhost:8000
# S3_FORCE_PATH_STYLE=true
# S3_BUCKET=wasm-manager
Loading

0 comments on commit cec9508

Please sign in to comment.