Skip to content

Commit

Permalink
update subql config and readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@shunjizhan
shunjizhan committed Nov 14, 2024
1 parent d2ca281 commit 7fb1720
Show file tree
Hide file tree
Showing 9 changed files with 106 additions and 228 deletions.
218 changes: 85 additions & 133 deletions packages/evm-subql/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,116 +1,31 @@
# @acala-network/evm-subql

Subquery services that index and query Acala EVM+ transactions and logs.

## Run

### Prepare

- install dependencies

```
yarn
```

- generate the required types from the GraphQL schema, and build code

```shell
yarn build
```

### Run all of the services using Docker

This will run all 4 services together:
1. A Mandala node with `--instant-sealing` in development mode
2. A Postgres database
3. A SubQl indexer that indexes data into Postgres
4. A SubQl query that wraps the data with GraphQl interface

```shell
docker-compose down -v # clean docker volume (optional)

docker-compose up
```

Please note that the indexer won't start until we [feed some transactions to the node](#feed-evm-transactions-to-node) if the node is running with `--instant-sealing`

### Run each service in the CLI seperately

Optionally, if you don't want to use the Docker, you can also run each service separately in the CLI ([official documentation](https://academy.subquery.network/run_publish/run.html#running-an-indexer-subql-node)).

NOTE: using CLI is for **local testing** only, so you can get familiar with each service. [For production](#for-production) please use docker setup instead.

1. Install SubQl globally

```shell
npm i -g @subql/[email protected] @subql/[email protected]
```

2. Run an [Acala](https://github.com/AcalaNetwork/Acala) node locally and listen to port number `9944` (in the first terminal)

If you already have a node running elsewhere, you can skip this step.

```shell
docker run -it --rm -p 9944:9944 ghcr.io/acalanetwork/mandala-node:sha-16f147e --dev --rpc-external --rpc-cors=all --rpc-methods=unsafe -levm=debug --pruning=archive --instant-sealing
```

3. Run a Postgres service and listen to port number 5432 (in the second terminal)

```shell
docker run -it -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:12-alpine
```

4. [Feed some transactions to node](#feeding-evm-transactions-to-the-node) if the node is running with `--instant-sealing`

5. Run a subquery indexer (in the third terminal)

```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

yarn index
```

6. Run the Query service (in the fourth terminal)

### Run with Docker
```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

yarn query
docker compose down -v # clean docker volume
docker compose up
```

### Feeding EVM transactions to the node
This will run all 3 services together:
- a postgres database
- a subql node indexer that indexes data into postgres
- a subql query service that wraps the data with GraphQl interface, so we can query the data via GraphQl API

If the acala node is running with `--instant-sealing`, it won't start producing blocks without transactions coming in, and subquery indexer won't start either without any new blocks. So we need to feed some transactions to it.
Note that docker compose is for demo purpose, and in production we usually need to start each of the `{ postgres, indexer, query }` seperately, so it's more flexible and controllable.

For example we can run any of the [evm examples](https://github.com/AcalaNetwork/bodhi.js/tree/master/examples).

```shell
cd ../../examples/waffle/dex
yarn build
yarn test
subql node should output something like this
```

If there are any transactions in the node, we should see the subquery indexer log something similar to this:

```shell
<BlockDispatcherService> INFO Enqueing blocks 3...12, total 10 blocks
<BlockDispatcherService> INFO fetch block [3,12], total 10 blocks
...
INFO Node started on port: 3000
INFO Enqueueing blocks 1102550...1102570, total 20 blocks
INFO Enqueueing blocks 1102570...1102590, total 20 blocks
```

## Query Data

Now we can explore the GraphQl data at [`http://localhost:3001/`](http://localhost:3001/) 🎉🎉
We start indexing from block 1102550 for Acala as defined in the [config file](./project-acala.ts), since the first acala evm tx is at block 1102550.

For example we can query:
#### query the data
We now check if the whole stack works as expected by querying the data via subql query service at `http://localhost:3001`

```graphql
query {
Expand Down Expand Up @@ -163,59 +78,97 @@ query {
}
```

## For Production
### Run with npm locally
For production it's recommended to use [docker setup](#run-with-docker), but if you don't want to use Docker, you can still run each service locally ([official documentation](https://academy.subquery.network/run_publish/run.html#running-an-indexer-subql-node)).

- install deps and build subql types and code locally
```
yarn
yarn build
```

install subql cli globally
```shell
npm i -g @subql/[email protected] @subql/[email protected]
```

Previous examples are examples of the **local development setup**, which uses the [example configuration](./project.yaml) that is tailored to local development node.
run a Postgres db and listen to port number 5432 (in the second terminal)
```shell
docker run -it -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:12-alpine
```

For production deployment, there are a couple differences:
run a subql node indexer
```shell
export TZ=UTC
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

#### services
In local setup we can run all of the services together with one single [docker compose](./docker-compose.yml). However, in prod we usually need to start each of the `{ node, postgres, indexer, query }` seperately with Docker or k8s.
subql-node \
-f ./project-acala.yaml \
--network-endpoint wss://acala-rpc.aca-api.network \
--db-schema evm-acala \
--batch-size 20 \
--workers 2 \
--store-flush-interval 10 \
--log-level info \
--disable-historical \
--scale-batch-size \
--unsafe
```

#### image
In the local example, we use `subquerynetwork/subql-node-substrate:v3.10.1` as indexer image, which requires **local mounted project path**. For prod we should use [acala/evm-subql](https://hub.docker.com/r/acala/evm-subql/tags) instead, which already has all the required files encapsulated, so we don't need to mount local files anymore.
run a subql query service
```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

An example is [here](../docker-compose-example.yml#L27)
subql-query \
--name evm-acala \
--playground \
--indexer http://localhost:3000
```

Latest stable versions:
- `acala/eth-rpc-adapter:2.8.1`
- `acala/evm-subql:2.8.1`
- `subquerynetwork/subql-query:v2.10.0`
now we should be able to [query the data](#query-the-data) via `http://localhost:3001`

#### config
One trick is that we don't have to start indexing from block 0, since Acala and Karura didn't enable EVM+ until a certain block. In particular we can use these two configs for production (change the `endpoint` value to your custom one if needed):
## Latest stable versions:
- eth rpc: `acala/eth-rpc-adapter:2.9.4`
- subql node: `acala/evm-subql:2.9.4` (or `@subql/[email protected]`)
- subql query: `subquerynetwork/subql-query:v2.10.0` (or `@subql/[email protected]`)

- [Acala production](./project-acala-840000.yaml), by setting `-f=/app/project-acala-840000.yaml`
- [Karura production](./project-karura-1780000.yaml), by setting `-f=/app/project-karura-1780000.yaml`
[release page](https://github.com/AcalaNetwork/bodhi.js/releases)

## Upgrade subquery version
To upgrade the production subql, we usually **do not** need to reindex from the beginning. Just upgrade the `acala/evm-subql` image version, and it should just continue indexing from the last indexed block. We also do not need to touch subql query service.

It usually takes 1 to 3 days to index all of the data, depending on the node latency and performance.
If you are running subql node locally, just pull the latest code and run `yarn build`, then run `yarn index` again.

### Upgrade Production Subquery
To upgrade the production subql, we usually need to do a full re-index. In order not to interrupt the currnetly running subql, we can run another indexer in parallel to the old one, and hot replace the old one once the full re-index is finished.
In rare cases, we might need to do a full re-index (we will explicitly state in the release note when this is needed). In order not to interrupt the currnetly running subql, we can run another indexer in parallel to the old one, and hot replace the old one once the full re-index is finished.

Below are the detailed steps:
1) start a new indexer service `acala/evm-subql` that uses a difference `--db-schema`, for example, `--db-schema=evm-acala`. It can share the same DB with the old indexer
1) start a new indexer service `acala/evm-subql` that uses a difference `--db-schema`, for example, `--db-schema=evm-acala-2`. It can share the same DB with the old indexer
2) wait until the new indexer finish indexing
3) update the config of graphql service `subquerynetwork/subql-query` to use the new indexer. In particular, change the `--name` command, such as `--name=evm-acala`, and `--indexer=<new indexer url>`
3) update the config of graphql service `subquerynetwork/subql-query` to use the new indexer. In particular, change the `--name` command to `--name=evm-acala-2`, and `--indexer=<new subql node url>`
4) delete the old indexer service, as well as the old db schema
5) upgrade is finished! No need to modify `eth-rpc-adapter`

Note: for `acala/evm-subql:v2.5.9` please add `--disable-historical` command. ([example](https://github.com/AcalaNetwork/bodhi.js/blob/d763bc588a4a90e4421d65ebfe1d95ba581c6d37/evm-subql/docker-compose.yml#L52))

## Dump and Restore Database
Sometimes it's useful to take a snapshot of the database, so that we can restore it when needed. We can also pass it along to others, so that they can quickly setup a copy of the same evm subql project, without needing to index from the beginning.

Below are CLI commands to do it, you can also use pgAdmin GUI to achieve the same thing.

### install postgres CLI
make sure you have `pg_dump` and `pg_restore` commands available.
make sure you have `pg_dump` and `pg_restore` commands available.

- for Mac: `brew install libpq`
- for other OS: `you are on your own`

### dump database
suppose we have an `evm-karura` schema in `postgres` db, and we want to dump it to a tar file `evm-karura.tgz`.
suppose we have an `evm-acala` schema in `postgres` db, and we want to dump it to a tar file `evm-acala.tgz`.
```
export PGPASSWORD=<password>
pg_dump \
Expand All @@ -224,13 +177,13 @@ pg_dump \
--dbname postgres \
--username postgres \
--format tar \
--file ./evm-karura.tgz \
--schema evm-karura \
--file ./evm-acala.tgz \
--schema evm-acala \
--verbose
```

### restore database
in previous step we dumped data from `evm-karura` schema in database `postgres`, so when restoring data, we need to first make sure a db called `postgres` exists, and it does **NOT** have `evm-karura` schema. Then we can restore the database with the following command
in previous step we dumped data from `evm-acala` schema in database `postgres`, so when restoring data, we need to first make sure a db called `postgres` exists, and it does **NOT** have `evm-acala` schema. Then we can restore the database with the following command
```
export PGPASSWORD=<password>
pg_restore \
Expand All @@ -239,20 +192,19 @@ pg_restore \
--dbname postgres \
--username postgres \
--verbose \
./evm-karura.tgz
./evm-acala.tgz
```

### (optional) rename schema
Since we dumped `evm-karura` schema, the restore process will create a new schema with the same name. If you want to use a different name, you can simply rename `evm-karura` schema to the desired name after the restore process.
### rename schema (optional)
Since we dumped `evm-acala` schema, the restore process will create a new schema with the same name. If you want to use a different name, you can simply rename `evm-acala` schema to the desired name after the restore process.

This can be done with pgAdmin by:
- right click the schema name
- select "properties"
- enter a new name and save

## More References

- [SubQuery official documentation](https://doc.subquery.network/)
- [About the unsafe flag](https://academy.subquery.network/run_publish/references.html#unsafe-node-service)
- [Acala EVM+ documentation](https://evmdocs.acala.network/network/network-setup/local-development-network)
- [Acala EVM+ documentation](https://evmdocs.acala.network/)
- [Subql Quick Migration Guide](https://hackmd.io/Z3ka28y4Tky6sHPsQVt2lw)
2 changes: 0 additions & 2 deletions packages/evm-subql/codegen.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@

projects=(
project-acala.ts
project-acala-840000.ts
project-acala-testnet.ts
project-karura.ts
project-karura-1780000.ts
project-karura-testnet.ts
project-tc9.ts
project.ts
Expand Down
46 changes: 12 additions & 34 deletions packages/evm-subql/docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,26 +1,6 @@
version: '3.9'

services:
mandala-node:
image: ghcr.io/acalanetwork/mandala-node:sha-16f147e
ports:
- 9944:9944
healthcheck:
test: "curl --fail -X POST -H \"Content-Type: application/json\" http://localhost:9944 -d \"{\"jsonrpc\": \"2.0\" }\" || exit 1"
interval: 2s
retries: 100
start_period: 3s
command:
- --dev
# - -lruntime=debug
# - -levm=debug
- --rpc-external
- --rpc-cors=all
- --rpc-methods=unsafe
- --pruning=archive
- --instant-sealing
- --tmp

postgres:
image: postgres:12-alpine
container_name: postgres
Expand All @@ -35,11 +15,9 @@ services:
retries: 5

subquery-node:
image: onfinality/subql-node:v5.2.9
image: acala/evm-subql:2.9.3 # this is built from onfinality/subql-node:v5.2.9
depends_on:
postgres:
condition: service_healthy
mandala-node:
postgres:
condition: service_healthy
ports:
- 3000:3000
Expand All @@ -50,17 +28,17 @@ services:
DB_DATABASE: postgres
DB_HOST: postgres
DB_PORT: 5432
volumes:
- ./:/app
command:
- ${SUB_COMMAND:-} # set SUB_COMMAND env variable to "test" to run tests
- -f=/app/project.yaml
- --network-endpoint=ws://mandala-node:9944
- --db-schema=acala_evm
- --log-level=debug
- --unsafe
- -f=./project-acala.yaml
- --network-endpoint=wss://acala-rpc.aca-api.network # acala node url
- --db-schema=evm-acala
- --batch-size=20
- --workers=2
- --store-flush-interval=10
- --log-level=info
- --disable-historical
- --batch-size=30
- --scale-batch-size
- --unsafe
healthcheck:
test: ["CMD", "curl", "-f", "http://subquery-node:3000/ready"]
interval: 3s
Expand All @@ -85,7 +63,7 @@ services:
DB_PORT: 5432
PORT: 3001
command:
- --name=acala_evm
- --name=evm-acala
- --playground
- --indexer=http://subquery-node:3000

Expand Down
4 changes: 1 addition & 3 deletions packages/evm-subql/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,7 @@
"clean": "rm -rf dist",
"codegen": "./codegen.sh",
"prepack": "yarn build",
"build": "yarn codegen; ncc build src/index.ts -o dist/ -t --target es2017; ncc build src/chain-types.ts -o dist/chain-types -t --target es2017",
"index": "subql-node -f ./project.yaml --batch-size 30 --db-schema acala_evm --log-level debug --port 3000 --unsafe --disable-historical",
"query": "PORT=3001 subql-query --name acala_evm --playground --indexer=http://localhost:3000 --debug"
"build": "yarn codegen; ncc build src/index.ts -o dist/ -t --target es2017; ncc build src/chain-types.ts -o dist/chain-types -t --target es2017"
},
"dependencies": {
"@acala-network/eth-providers": "workspace:*"
Expand Down
Loading

0 comments on commit 7fb1720

Please sign in to comment.