Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydra explorer supporting multiple hydra versions #1282

Open
2 of 4 tasks
ch1bo opened this issue Feb 5, 2024 · 9 comments · Fixed by #1815, cardano-scaling/hydra-explorer#18 or #1828 · May be fixed by cardano-scaling/hydra-explorer#20
Open
2 of 4 tasks

Hydra explorer supporting multiple hydra versions #1282

ch1bo opened this issue Feb 5, 2024 · 9 comments · Fixed by #1815, cardano-scaling/hydra-explorer#18 or #1828 · May be fixed by cardano-scaling/hydra-explorer#20
Assignees
Labels
green 💚 Low complexity or well understood feature 💬 feature A feature on our roadmap

Comments

@ch1bo
Copy link
Member

ch1bo commented Feb 5, 2024

Why

The Hydra explorer should be able to report Hydra heads from multiple different protocol versions on all public networks. The explorer can be used to give usage information about each version.

What

  • The hydra-explorer lists heads from all networks and supported versions according to networks.json

  • The hydra-explorer does not need to be updated on every script change (e.g. when we switch plutus-tx/GHC versions)

  • The hydra-explorer reports a protocol version per head.

  • Can run multiple "(chain) observers", one for each network and hydra protocol version, feeding information into one hydra-explorer instance.

    • The hydra-explorer is decoupled from a specific hydra version
    • The interface between observers and explorer is specified and tests will ensure we don't break it accidentally
  • The hydra-explorer should be deployed from https://github.com/cardano-scaling/hydra-explorer using multiple versions of the main hydra repository

    • We need to do this because, say, the script hashes are not enough to distinguish the version; i.e. we can't make hydra-node/src/Hydra/Chain/Direct/Tx.hs generic across versions; the easiest way is just to depend on the entire codebase 🥲
  • We should delete the hydra-explorer folder from the main hydra repo

How

  • The endpoint is a POST to /v1/observations/<network>/<version> with request body of some JSON object enumerating the observations

  • Add an observer-side API definition to hydra-explorer OpenAPI schema

  • The interface should be JSON-based simpler HTTP requests; so that it's easy reason about backwards compatibility and debug, and test

  • hydra-chain-observers are HTTP clients sending requests to hydra-explorer observer-side API whenever a head transition is observed

  • Observer-side API is exposed on separate port so we can host and port configurable on the command-line of hydra-explorer

  • Both hydra-chain-observer and hydra-explorer are tested agains the observer-side API schema

TBD

  • Is specifying an interface + testing against enough to keep this maintainable?
  • Make the explorer understand any "old versions" of observer data?
    • Eventually drop too old observer versions
    • For how long should be support older versions? Will we have some separate notion of LTS versions for Hydra?
    • Might be older versions that we don't support but keep an eye on
  • Can this support protocol changes like
    • additional transitions (incremental commit)
    • not having a commit phase anymore
  • Protocol version is basically:
    • scripts, their hashes and addresses
    • datums and redeemers as contained in transactions (their format)
    • structure of protocol transactions
@ch1bo ch1bo converted this from a draft issue Feb 5, 2024
@ch1bo ch1bo added the 💭 idea An idea or feature request label Feb 5, 2024
@ch1bo ch1bo mentioned this issue Feb 5, 2024
4 tasks
@ch1bo ch1bo moved this from Later to Next in Hydra Head Roadmap Apr 10, 2024
@github-project-automation github-project-automation bot moved this to In Progress 🕐 in ☕ Hydra Team Work Jul 22, 2024
@ch1bo ch1bo moved this from In Progress 🕐 to Later in ☕ Hydra Team Work Jul 22, 2024
@ch1bo ch1bo moved this from Next to Later in 🚢 Hydra Head Roadmap Jul 23, 2024
@noonio noonio added green 💚 Low complexity or well understood feature 💬 feature A feature on our roadmap and removed 💭 idea An idea or feature request labels Aug 13, 2024
@noonio noonio added this to the 1.0.0 milestone Aug 13, 2024
@noonio
Copy link
Contributor

noonio commented Nov 7, 2024

Having looked into this a bit as part of cardano-scaling/hydra-explorer#5 I think it's worth making some notes:

  1. I don't think it's best to invoke hydra-explorer multiple times; let's just make an option for the chain-observer to look for multiple "versions"
  2. A version is defined by the Tx Id in networks.json and the script outputs from there; from this a ScriptInfo datatype can be reconstructed
  3. This can be used by the chain-observer to send ticks

I.e. the real limitation is with the chain-observer; so we just need to make that parameterisable by a specific ScriptInfo parameter; then maybe we just run multiple chain-observers, or (probably better) run a single one that attempts to find all the ScriptInfo things that it is provided.

-- Edit:

As discussed with @ffakenz and @ch1bo , it is not possible to "de-version" the relevant parts of the chain-observer code (i.e. to parameterise it by the version), because all the version information is essentially hard-coded in the code itself; so it's easier just to depend on different revisions of the code and adapt them to write to a generic (?JSON) format consumable by the hydra-explorer.

@ch1bo
Copy link
Member Author

ch1bo commented Jan 9, 2025

We discussed again the intended architecture of this and drew this picture today. It shows multiple versions of the chain observing code (in hydra-chain-observer) and also running on multiple networks / against different cardano-nodes, all aggregating data into a single hydra-explorer instance.

hydra-explorer multiple versions - Frame 1

@noonio noonio moved this from In Progress 🕐 to Todo 📋 in ☕ Hydra Team Work Jan 13, 2025
@noonio noonio assigned ch1bo and unassigned locallycompact and ffakenz Jan 20, 2025
@noonio noonio modified the milestones: 1.0.0, 0.20.0 Jan 20, 2025
@noonio noonio removed this from the 0.20.0 milestone Jan 20, 2025
@ch1bo ch1bo moved this from Todo 📋 to In Progress 🕐 in ☕ Hydra Team Work Jan 21, 2025
@ch1bo ch1bo linked a pull request Jan 31, 2025 that will close this issue
5 tasks
@ch1bo ch1bo moved this from In Progress 🕐 to In review 👀 in ☕ Hydra Team Work Jan 31, 2025
@ch1bo
Copy link
Member Author

ch1bo commented Feb 1, 2025

When working on this I realized that a lot of our dependencies could be improved. For example:

  • The hydra-explorer now depends on hydra-tx (which depends on hydra-plutus scripts), only for the "interface" types like HeadId, Snapshot etc. Having a hydra-api, that contains Hydra specific entities and (hydra-)cardano-api that contains cardano specific entities would be more suitable.
  • Similarly, the hydra-explorer depends on hydra-node mainly because of the OnChainTx type in Hydra.Chain. Either, we deliberately sever that dependency by replicating a compatible type into hydra-explorer, or we could move that (and other chain interface things) into a hydra-chain package. This would be used by all applications, the hydra-chain-observer, hydra-explorer and hydra-node.

@noonio noonio moved this from In review 👀 to In Progress 🕐 in ☕ Hydra Team Work Feb 4, 2025
@noonio noonio linked a pull request Feb 4, 2025 that will close this issue
1 task
@github-project-automation github-project-automation bot moved this from In Progress 🕐 to Done ✔ in ☕ Hydra Team Work Feb 4, 2025
@ch1bo ch1bo reopened this Feb 4, 2025
@github-project-automation github-project-automation bot moved this from Done ✔ to Triage 🏥 in ☕ Hydra Team Work Feb 4, 2025
@ch1bo ch1bo linked a pull request Feb 4, 2025 that will close this issue
@ch1bo ch1bo moved this from Triage 🏥 to In Progress 🕐 in ☕ Hydra Team Work Feb 4, 2025
@locallycompact
Copy link
Contributor

locallycompact commented Feb 4, 2025

I dont think just a hydra-api library could carry the acceptance criteria needed to know that those types are well-defined for the task, i.e it needs hydra-tx to know itself, so I would probably want to keep those types in hydra-tx, and pull the types from there knowing that they are book kept with the contract tests.

@ch1bo ch1bo linked a pull request Feb 4, 2025 that will close this issue
4 tasks
@ch1bo
Copy link
Member Author

ch1bo commented Feb 5, 2025

I dont think just a hydra-api library could carry the acceptance criteria needed to know that those types are well-defined for the task, i.e it needs hydra-tx to know itself, so I would probably want to keep those types in hydra-tx, and pull the types from there knowing that they are book kept with the contract tests.

@locallycompact What acceptance criteria? I don't understand "i.e it needs hydra-tx to know itself". This is about the explorer, which just wants to use those types when communicating with hydra-chain-observer and offering API to its clients. Which is quite a similar use case than (Haskell) clients to the hydra-node, who just want to understand the API provided to them. Hence, my thought that would be a hydra-api package.

@github-project-automation github-project-automation bot moved this from In Progress 🕐 to Done ✔ in ☕ Hydra Team Work Feb 5, 2025
@locallycompact
Copy link
Contributor

Having a hydra-api package that just contains interface types (Snapshot, HeadId) for use by downstream applications like hydra-explorer is dubious in my opinion because the evidence that those types are sound are the hydra-tx tests. I can't imagine hydra-api alone having any tests (acceptance criteria) that would be any smaller than the hydra-tx contract tests.

@ch1bo
Copy link
Member Author

ch1bo commented Feb 5, 2025

Having a hydra-api package that just contains interface types (Snapshot, HeadId) for use by downstream applications like hydra-explorer is dubious in my opinion because the evidence that those types are sound are the hydra-tx tests. I can't imagine hydra-api alone having any tests (acceptance criteria) that would be any smaller than the hydra-tx contract tests.

I see what you mean and respectfully disagree. The types themselves can have semantics that is not necessarily tied to how they are used in L1 hydra protocol transactions (= hydra-tx using them), for example any conversion between them, serialization to binary (currently not tested, but should be), serialization to json, law-fulness of instances etc.

@locallycompact
Copy link
Contributor

OK. I'm inclined towards more library separation than less. But is hydra-api the correct name for it? It sounds more like hydra-type-definitions maybe?

@ch1bo ch1bo reopened this Feb 5, 2025
@github-project-automation github-project-automation bot moved this from Done ✔ to Triage 🏥 in ☕ Hydra Team Work Feb 5, 2025
@ch1bo
Copy link
Member Author

ch1bo commented Feb 5, 2025

OK. I'm inclined towards more library separation than less. But is hydra-api the correct name for it? It sounds more like hydra-type-definitions maybe?

@locallycompact see also #432 (comment)

@noonio noonio moved this from Triage 🏥 to In progress 🕐 in ☕ Hydra Team Work Feb 5, 2025
@ffakenz ffakenz moved this from In progress 🕐 to In review 👀 in ☕ Hydra Team Work Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
green 💚 Low complexity or well understood feature 💬 feature A feature on our roadmap
Projects
Status: In review 👀
Status: No status
4 participants