Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

Commit

Permalink
fix: re-enable default indexer ci (#1251)
Browse files Browse the repository at this point in the history
* fix: re-enable default indexer ci

* cargo fmt

* fix bad merge from #1211

* update docs link

---------

Co-authored-by: Rashad Alston <[email protected]>
  • Loading branch information
ra0x3 and Rashad Alston authored Aug 10, 2023
1 parent e9ece3a commit 085e722
Show file tree
Hide file tree
Showing 12 changed files with 107 additions and 104 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -267,8 +267,7 @@ jobs:
run: cargo test --locked --all-targets --features e2e,postgres -p
fuel-indexer-tests -- --test-threads=1
build-default-indexer:
# if: needs.set-env-vars.outputs.IS_RELEASE_OR_MASTER_OR_DEVELOP_OR_SEMVER != 'true'
if: false
if: needs.set-env-vars.outputs.IS_RELEASE_OR_MASTER_OR_DEVELOP_OR_SEMVER != 'true'
needs:
- cargo-toml-fmt-check
- set-env-vars
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ There are a few points that Fuel indexer users should know when using WASM:

2. Developers should be aware of what things may not work off-the-shelf in a module: file I/O, thread spawning, and anything that depends on system libraries. This is due to the technological limitations of WASM as a whole; more information can be found [here](https://rustwasm.github.io/docs/book/reference/which-crates-work-with-wasm.html).

3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/stripper.bash).
3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/stripper.bash).

> IMPORTANT: Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables:
>
Expand Down
4 changes: 2 additions & 2 deletions docs/src/getting-started/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ To run the Fuel indexer, you'll need to install a few dependencies on your syste
- The [`wasm32-unknown-unknown`](#wasm) `rustup` target
- [`wasm-snip`](#wasm), a utility for stripping symbols from WebAssemly binaries.

If you don't want to install a database directly onto your system, you can use Docker to run it as an isolated container. You can install it by following the [install instructions](https://docs.docker.com/get-docker/). For reference purposes, we provide a [`docker compose` file](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/docker-compose.yaml) that runs a Postgres database and the Fuel indexer service.
If you don't want to install a database directly onto your system, you can use Docker to run it as an isolated container. You can install it by following the [install instructions](https://docs.docker.com/get-docker/). For reference purposes, we provide a [`docker compose` file](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/docker-compose.yaml) that runs a Postgres database and the Fuel indexer service.

> IMPORTANT: Note for Apple Silicon macOS users:
>
Expand Down Expand Up @@ -46,7 +46,7 @@ Once installed, you can add PostgreSQL to your system by running `brew install p

Two additonal cargo components will be required to build your indexers: `wasm-snip` and the `wasm32-unknown-unknown` target.

> As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due an errant upstream dependency. For now, you can use `wasm-snip` to remove the errant symbols from the WASM module. An example can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/stripper.bash).
> As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due an errant upstream dependency. For now, you can use `wasm-snip` to remove the errant symbols from the WASM module. An example can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/stripper.bash).
### `wasm-snip`

Expand Down
2 changes: 1 addition & 1 deletion docs/src/graphql/api-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ To run the standalone Fuel indexer GraphQL API server using a configuration file
fuel-indexer-api-server run --config config.yaml
```

In the above example, `config.yaml` is based on [the default service configuration file](https://github.com/FuelLabs/fuel-indexer/blob/master/config.yaml).
In the above example, `config.yaml` is based on [the default service configuration file](https://github.com/FuelLabs/fuel-indexer/blob/develop/config.yaml).

## Options

Expand Down
2 changes: 1 addition & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ The Fuel indexer is a standalone service that can be used to index various compo

By using a combination of Fuel-flavored GraphQL schema, a SQL backend, and indexers written in Rust, users of the Fuel indexer can get started creating production-ready backends for their dApps, meant to go fast 🚗💨.

For those wanting to build dApp backends right away, feel free to check out the [Quickstart](./getting-started/quickstart.md). And for those willing to contribute to the Fuel indexer project, please feel free to read [our contributor guidelines](https://github.com/FuelLabs/fuel-indexer/blob/master/docs/CONTRIBUTING.md) as well as the [For Contributors](for-contributors/index.md) chapter of the book.
For those wanting to build dApp backends right away, feel free to check out the [Quickstart](./getting-started/quickstart.md). And for those willing to contribute to the Fuel indexer project, please feel free to read [our contributor guidelines](https://github.com/FuelLabs/fuel-indexer/blob/develop/docs/CONTRIBUTING.md) as well as the [For Contributors](for-contributors/index.md) chapter of the book.
<!-- KEEP THIS IN CASE WE WANT TO MAKE DIAGRAM CHANGES [![fo](https://mermaid.ink/img/pako:eNp9kWFrwjAQhv_KcZ8U9A-UIajZhrDBZkUYjR9uzamFNqlpAhbrf1-6TFCY5r6EN-9zb7g7YW4UY4I7S_UeVkJqCGeavXguYW4sb2A8nnTrd0gdOe5gNvh9WmjFR7bD6J9l12JEXvuOn2-wamtuOhADQY6-qeE_Zn7DwMU-_VhEPi12msrmTuIjGp56fskHz437L1pkF2EThefsy3h7-_-tsTkUvQKK69K03aNQqa8LR1ixrahQYbSnPkOi23PFEpNwVbwlXzqJUp-DlbwzaatzTJz1PEJfqzBqUVBYShXF8w9dz4xz?type=png)](https://mermaid.live/edit#pako:eNp9kWFrwjAQhv_KcZ8U9A-UIajZhrDBZkUYjR9uzamFNqlpAhbrf1-6TFCY5r6EN-9zb7g7YW4UY4I7S_UeVkJqCGeavXguYW4sb2A8nnTrd0gdOe5gNvh9WmjFR7bD6J9l12JEXvuOn2-wamtuOhADQY6-qeE_Zn7DwMU-_VhEPi12msrmTuIjGp56fskHz437L1pkF2EThefsy3h7-_-tsTkUvQKK69K03aNQqa8LR1ixrahQYbSnPkOi23PFEpNwVbwlXzqJUp-DlbwzaatzTJz1PEJfqzBqUVBYShXF8w9dz4xz) -->

## Architecture
Expand Down
2 changes: 1 addition & 1 deletion docs/src/project-components/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ There are a few points that Fuel indexer users should know when using WASM:

2. Developers should be aware of what things may not work off-the-shelf in a module: file I/O, thread spawning, and anything that depends on system libraries. This is due to the technological limitations of WASM as a whole; more information can be found [here](https://rustwasm.github.io/docs/book/reference/which-crates-work-with-wasm.html).

3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/master/scripts/stripper.bash).
3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/stripper.bash).

4. Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables:

Expand Down
2 changes: 1 addition & 1 deletion packages/fuel-indexer/src/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -904,4 +904,4 @@ impl Executor for WasmIndexExecutor {

Ok(())
}
}
}
7 changes: 4 additions & 3 deletions plugins/forc-index/src/ops/forc_index_build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -78,14 +78,15 @@ pub fn init(command: BuildCommand) -> anyhow::Result<()> {

let manifest_schema_file = {
let workspace_root: std::path::PathBuf =
crate::ops::utils::cargo_workspace_root_dir(path.as_path()).unwrap();
crate::utils::cargo_workspace_root_dir(path.as_path()).unwrap();
Path::new(&workspace_root).join(manifest.graphql_schema())
};

// Rebuild the WASM module even if only the schema has changed.
crate::ops::utils::ensure_rebuild_if_schema_changed(
crate::utils::ensure_rebuild_if_schema_changed(
root_dir.as_path(),
Path::new(manifest_schema_file.as_path()),
manifest.execution_source(),
)?;

// Construct our build command
Expand Down Expand Up @@ -187,7 +188,7 @@ pub fn init(command: BuildCommand) -> anyhow::Result<()> {
let profile = if release { "release" } else { "debug" };

let target_dir: std::path::PathBuf =
crate::ops::utils::cargo_target_dir(path.as_path()).unwrap();
crate::utils::cargo_target_dir(path.as_path()).unwrap();

let abs_artifact_path = target_dir
.join(defaults::WASM_TARGET)
Expand Down
3 changes: 1 addition & 2 deletions plugins/forc-index/src/ops/forc_index_deploy.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,7 @@ pub async fn init(command: DeployCommand) -> anyhow::Result<()> {

let path = path.unwrap_or(current_dir);

let workspace_root =
crate::ops::utils::cargo_workspace_root_dir(path.as_path()).unwrap();
let workspace_root = crate::utils::cargo_workspace_root_dir(path.as_path()).unwrap();

let manifest_schema_file = Path::new(&workspace_root)
.join(manifest.graphql_schema())
Expand Down
1 change: 0 additions & 1 deletion plugins/forc-index/src/ops/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,3 @@ pub mod forc_index_new;
pub mod forc_index_remove;
pub mod forc_index_start;
pub mod forc_index_status;
pub mod utils;
85 changes: 0 additions & 85 deletions plugins/forc-index/src/ops/utils.rs

This file was deleted.

98 changes: 94 additions & 4 deletions plugins/forc-index/src/utils.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
use reqwest::{multipart::Part, Body};
use tokio::fs::File;
use tokio::io;

use crate::{defaults, defaults::manifest_name};
use fuel_indexer_lib::ExecutionSource;
use reqwest::{multipart::Part, Body};
use std::{
fs::canonicalize,
path::{Path, PathBuf},
process::Command,
};
use tokio::fs::File;
use tokio::io;

pub fn dasherize_to_underscore(s: &str) -> String {
str::replace(s, "-", "_")
Expand Down Expand Up @@ -93,3 +93,93 @@ pub async fn file_part<T: AsRef<Path>>(path: T) -> io::Result<Part> {
field
})
}

/// Given a path to a directory in which `Cargo.toml` is located, extract Cargo
/// metadata.
pub fn cargo_metadata(cargo_manifest_dir: &Path) -> anyhow::Result<serde_json::Value> {
let output = std::process::Command::new("cargo")
.arg("metadata")
.arg("--manifest-path")
.arg(cargo_manifest_dir.join("Cargo.toml").as_path())
.output()
.expect("Failed to execute command");

if !output.status.success() {
anyhow::bail!("cargo metadata execution failed");
}

let output_str = String::from_utf8_lossy(&output.stdout);

Ok(serde_json::from_str(&output_str).expect("Failed to parse JSON"))
}

/// Given a path to a directory in which `Cargo.toml` is located, find the
/// `target` directory using `cargo metadata`.
pub fn cargo_target_dir(cargo_manifest_dir: &Path) -> anyhow::Result<std::path::PathBuf> {
let metadata_json = cargo_metadata(cargo_manifest_dir)?;

// Use serde to extract the "target_directory" field
let target_directory = metadata_json["target_directory"]
.as_str()
.expect("target_directory not found or invalid");

Ok(target_directory.into())
}

/// Given a path to a directory in which `Cargo.toml` is located, find the
/// `workspace_root` directory using `cargo metadata`.
pub fn cargo_workspace_root_dir(
cargo_manifest_dir: &Path,
) -> anyhow::Result<std::path::PathBuf> {
let metadata_json = cargo_metadata(cargo_manifest_dir)?;

// Use serde to extract the "target_directory" field
let target_directory = metadata_json["workspace_root"]
.as_str()
.expect("target_directory not found or invalid");

Ok(target_directory.into())
}

/// Set file's atime and mtime to now.
pub fn touch_file(path: &Path) -> std::io::Result<()> {
let time = filetime::FileTime::now();
filetime::set_file_times(path, time, time)?;
Ok(())
}

/// Set src/lib.rs' atime and mtime to now and thus ensure the WASM module is
/// rebuilt if schema file has changed.
pub fn ensure_rebuild_if_schema_changed(
project_dir: &Path,
schema: &Path,
exec_source: ExecutionSource,
) -> std::io::Result<()> {
let schema_mtime = {
let metadata = std::fs::metadata(schema).unwrap();
filetime::FileTime::from_last_modification_time(&metadata)
};

let sourcefile = match exec_source {
ExecutionSource::Native => "main.rs",
ExecutionSource::Wasm => "lib.rs",
};

let lib_rs = {
let mut path = project_dir.to_owned();
path.push("src");
path.push(sourcefile);
path
};

let lib_rs_mtime = {
let metadata = std::fs::metadata(lib_rs.as_path()).unwrap();
filetime::FileTime::from_last_modification_time(&metadata)
};

if schema_mtime > lib_rs_mtime {
touch_file(lib_rs.as_path())?;
}

Ok(())
}

0 comments on commit 085e722

Please sign in to comment.