View: POI Divergence Report

[azf20]

Provide investigators information to debug POI divergences for a subgraph.

A given subgraph has some characteristics which might be of interest to an investigator, for example the network being indexed, the apiVersion, and any subgraph features (e.g. file data sources).

As an investigator, I am interested in understanding the characteristics of the subgraph under investigation

A Divergence Report compares two POIs (which may involve multiple indexers). The report will be for the specific block where there is a divergence (noteworthy edge case: subgraphs where there are no blocks with a matching POI, i.e. where there has been divergence since the startBlock).

As an investigator, I am interested in knowing the block at which two POIs diverged.

For a given subgraph block, we can fetch the following data from an indexer (given #15):

• Block data
• Call cache
• Entity changes

Can we assume that indexers with a matching POI will definitely have the same data for these, and as such we only need to fetch from one indexer per POI group?

As an investigator, I am interested in knowing which of the available datasets differs across the POIs.

The indexers may be running different software versions, which might cause a divergence.

As an investigator, I am interested in what version of Graph Node an indexer is running

There is other information which may not be automatically available in the POI Divergence Report, but could be relevant for an investigation. For example:

• If a given block was processed using a Firehose or an RPC
• The specific client software (e.g. Erigon, geth)

This information is out of scope for the divergence report.

[azf20]

Sharing a proposal from @leoyvens:

The idea is to store and serve the divergence reports not in a relational format in the DB, but to treat each report as a blob of data, possibly a big json. This can be stored in Postgres as a jsonb, and also on IPFS. This has a few benefits for the different users of divergence reports:
Core devs looking for bugs in indexer software will like the flexibility of downloading the file and then using a CLI or ad-hoc scripts to explore and diff it as desired.
Arbitrators may like the verifiability of having the dump content addressed on IPFS, so anyone can reproduce the steps of their investigation.
This approach is not the easiest to use for less technical consumers or for at-a-glance analysis, but after we complete a few investigations with this ad-hoc scripting approach, we will have a better understanding of what data would be useful for a more human-readable report format or a nice web frontend.