Skip to content

Commit

Permalink
docs: Rewrite README (& add echo.rs example) (#2960)
Browse files Browse the repository at this point in the history
## Description

<!-- A summary of what this pull request achieves and a rough list of
changes. -->
- Rewrote the README
- Renamed `iroh-router/examples/custom-protocol.rs` to `search.rs` (It's
not a "custom" protocol. Just a protocol.) Removed "custom" wording
- Added a simpler `echo.rs` example (used in the README)


## Notes & open questions

<!-- Any notes, remarks or open questions you have to make about the PR.
-->
- In the "repository structure" section, I describe the purpose of
*some* crates, but not all. Since stuff will be moving around in the
coming weeks, I *think* that's, fine, but someone please tell me if we
should do sth differently there.

## Change checklist

- [x] Self-review.
- [x] Documentation updates following the [style
guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text),
if relevant.
- [x] Tests if relevant.
- [x] All breaking changes documented.
  • Loading branch information
matheus23 authored Nov 26, 2024
1 parent 4e3b431 commit 4abfd61
Show file tree
Hide file tree
Showing 4 changed files with 242 additions and 27 deletions.
138 changes: 121 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<h1 align="center"><a href="https://iroh.computer"><img alt="iroh" src="./.img/iroh_wordmark.svg" width="100" /></a></h1>

<h3 align="center">
A toolkit for building distributed applications
less net work for networks
</h3>

[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg?style=flat-square)](https://docs.rs/iroh/)
Expand All @@ -22,38 +22,125 @@ A toolkit for building distributed applications
<a href="https://docs.rs/iroh">
Rust Docs
</a>
<span> | </span>
<a href="https://github.com/n0-computer/iroh/releases">
Releases
</a>
</h3>
</div>
<br/>

## Features
## What is iroh?

Iroh gives you an API for dialing by public key.
You say “connect to that phone”, iroh will find & maintain the fastest connection for you, regardless of where it is.

### Hole-punching

The fastest route is a direct connection, so if necessary, iroh tries to hole-punch.
Should this fail, it can fall back to an open ecosystem of public relay servers.
To ensure these connections are as fast as possible, we [continuously measure iroh][iroh-perf].

### Built on [QUIC]

- Documents - Authors create and join documents: mutable key-value stores that multiple users read from, write to, and sync with, subscribing to live updates in real time.
- Blobs - Iroh works with content-addressed blobs of opaque data, which are often the bytes of a file.
- Networking - At the core of iroh is the ability to connect any two devices, no matter where they are.
Iroh uses [Quinn] to establish [QUIC] connections between nodes.
This way you get authenticated encryption, concurrent streams with stream prioirities, a datagram transport and avoid head-of-line-blocking out of the box.

## Overview
## Compose Protocols

Iroh is a protocol for syncing & moving bytes. Bytes of any size, on any device. At its core, it's a peer-2-peer network built on a _magic socket_ that establishes [QUIC](https://en.wikipedia.org/wiki/QUIC) connections between peers. Peers request and provide _blobs_ of opaque bytes that are incrementally verified by their BLAKE3 hash during transfer.
Use pre-existing protocols built on iroh instead of writing your own:
- [iroh-blobs] for [BLAKE3]-based content-addressed blob transfer scaling from kilobytes to terrabytes
- [iroh-gossip] for establishing publish-subscribe overlay networks that scale, requiring only resources that your average phone can handle
- [iroh-docs] for an eventually-consistent key-value store of [iroh-blobs] blobs
- [iroh-willow] for an (in-construction) implementation of the [willow protocol]

## Getting Started

Iroh is delivered as a Rust library and a CLI.
### Rust Library

It's easiest to use iroh from rust.
Install it using `cargo add iroh`, then on the connecting side:

```rs
const ALPN: &[u8] = b"iroh-example/echo/0";

let endpoint = Endpoint::builder().discovery_n0().bind().await?;

// Open a connection to the accepting node
let conn = endpoint.connect(addr, ALPN).await?;

// Open a bidirectional QUIC stream
let (mut send, mut recv) = conn.open_bi().await?;

// Send some data to be echoed
send.write_all(b"Hello, world!").await?;
send.finish()?;

### Library
// Receive the echo
let response = recv.read_to_end(1000).await?;
assert_eq!(&response, b"Hello, world!");

Run `cargo add iroh`, to add `iroh` to your project.
// Close the endpoint and all its connections
endpoint.close(0u32.into(), b"bye!").await?;
```

And on the accepting side:
```rs
let endpoint = Endpoint::builder().discovery_n0().bind().await?;

let router = Router::builder(endpoint)
.accept(ALPN.to_vec(), Arc::new(Echo))
.spawn()
.await?;

// The protocol definition:
#[derive(Debug, Clone)]
struct Echo;

impl ProtocolHandler for Echo {
fn accept(self: Arc<Self>, connecting: Connecting) -> BoxedFuture<Result<()>> {
Box::pin(async move {
let connection = connecting.await?;
let (mut send, mut recv) = connection.accept_bi().await?;

// Echo any bytes received back directly.
let bytes_sent = tokio::io::copy(&mut recv, &mut send).await?;

send.finish()?;
connection.closed().await;

Ok(())
})
}
}
```

The full example code with more comments can be found at [`echo.rs`][echo-rs].

Or use one of the pre-existing protocols, e.g. [iroh-blobs] or [iroh-gossip].

### Other Languages

If you want to use iroh from other languages, make sure to check out [iroh-ffi], the repository for FFI bindings.

### Links

- [Introducing Iroh (video)](https://www.youtube.com/watch?v=RwAt36Xe3UI_)
- [Iroh Examples](https://github.com/n0-computer/iroh-examples)
- [Introducing Iroh (video)][iroh-yt-video]
- [Iroh Documentation][docs]
- [Iroh Examples]
- [Iroh Experiments]

## Repository Structure

This repository contains a workspace of crates:
- `iroh`: Re-exports of stables APIs like `iroh-router` and `iroh-net`.
- `iroh-router`: APIs to compose multiple protocols in a node.
- `iroh-net`: The core networking code for hole-punching & connections to relays.
- `iroh-relay`: The relay server implementation. This is the code we run in production (and you can, too!).
- `iroh-base`: Common types like `Hash`, key types or `RelayUrl`.
- `iroh-metrics`: Helper library for adding metrics support to crates.
- `iroh-test`: Test utilities.
- `iroh-dns-server`: DNS server implementation powering the `n0_discovery` for NodeIds, running at dns.iroh.link.
- `iroh-net-report`: Analyzes your host's networking ability & NAT.
- `net-tools/*`: Networking utility crates

The main entry point for anyone interested in the inner workings should be `iroh-net`.

## License

Expand All @@ -68,6 +155,23 @@ This project is licensed under either of

at your option.

### Contribution
## Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[QUIC]: https://en.wikipedia.org/wiki/QUIC
[BLAKE3]: https://github.com/BLAKE3-team/BLAKE3
[Quinn]: https://github.com/quinn-rs/quinn
[iroh-blobs]: https://github.com/n0-computer/iroh-blobs
[iroh-gossip]: https://github.com/n0-computer/iroh-gossip
[iroh-docs]: https://github.com/n0-computer/iroh-docs
[iroh-willow]: github.com/n0-computer/iroh-willow
[iroh-doctor]: github.com/n0-computer/iroh-doctor
[willow protocol]: https://willowprotocol.org
[iroh-ffi]: https://github.com/n0-computer/iroh-ffi
[iroh-yt-video]: https://www.youtube.com/watch?v=RwAt36Xe3UI_
[Iroh Examples]: https://github.com/n0-computer/iroh-examples
[Iroh Experiments]: https://github.com/n0-computer/iroh-experiments
[echo-rs]: /iroh-router/examples/echo.rs
[iroh-perf]: https://perf.iroh.computer
[docs]: https://iroh.computer/docs
6 changes: 5 additions & 1 deletion iroh-router/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,9 @@ default = []
examples = ["dep:clap", "dep:tracing-subscriber"]

[[example]]
name = "custom-protocol"
name = "search"
required-features = ["examples"]

[[example]]
name = "echo"
required-features = ["examples"]
107 changes: 107 additions & 0 deletions iroh-router/examples/echo.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
//! Very basic example to showcase how to use iroh's APIs.
//!
//! This example implements a simple protocol that echos any data sent to it in the first stream.
//!
//! ## Usage
//!
//! cargo run --example echo --features=examples
use std::sync::Arc;

use anyhow::Result;
use futures_lite::future::Boxed as BoxedFuture;
use iroh_net::{endpoint::Connecting, Endpoint, NodeAddr};
use iroh_router::{ProtocolHandler, Router};

/// Each protocol is identified by its ALPN string.
///
/// The ALPN, or application-layer protocol negotiation, is exchanged in the connection handshake,
/// and the connection is aborted unless both nodes pass the same bytestring.
const ALPN: &[u8] = b"iroh-example/echo/0";

#[tokio::main]
async fn main() -> Result<()> {
let router = accept_side().await?;
let node_addr = router.endpoint().node_addr().await?;

connect_side(node_addr).await?;

router.shutdown().await?;

Ok(())
}

async fn connect_side(addr: NodeAddr) -> Result<()> {
let endpoint = Endpoint::builder().discovery_n0().bind().await?;

// Open a connection to the accepting node
let conn = endpoint.connect(addr, ALPN).await?;

// Open a bidirectional QUIC stream
let (mut send, mut recv) = conn.open_bi().await?;

// Send some data to be echoed
send.write_all(b"Hello, world!").await?;

// Signal the end of transfer.
send.finish()?;

// Receive the echo
let response = recv.read_to_end(1000).await?;
assert_eq!(&response, b"Hello, world!");

// Close the endpoint (and all its connections) in one:
endpoint.close(0u32.into(), b"bye!").await?;

Ok(())
}

async fn accept_side() -> Result<Router> {
let endpoint = Endpoint::builder().discovery_n0().bind().await?;

// Build our protocol handler and add our protocol, identified by its ALPN, and spawn the node.
let router = Router::builder(endpoint)
.accept(ALPN, Arc::new(Echo))
.spawn()
.await?;

Ok(router)
}

#[derive(Debug, Clone)]
struct Echo;

impl ProtocolHandler for Echo {
/// The `accept` method is called for each incoming connection for our ALPN.
///
/// The returned future runs on a newly spawned tokio task, so it can run as long as
/// the connection lasts.
fn accept(self: Arc<Self>, connecting: Connecting) -> BoxedFuture<Result<()>> {
// We have to return a boxed future from the handler.
Box::pin(async move {
// Wait for the connection to be fully established.
let connection = connecting.await?;
// We can get the remote's node id from the connection.
let node_id = iroh_net::endpoint::get_remote_node_id(&connection)?;
println!("accepted connection from {node_id}");

// Our protocol is a simple request-response protocol, so we expect the
// connecting peer to open a single bi-directional stream.
let (mut send, mut recv) = connection.accept_bi().await?;

// Echo any bytes received back directly.
let bytes_sent = tokio::io::copy(&mut recv, &mut send).await?;
println!("Copied over {bytes_sent} byte(s)");

// By calling `finish` on the send stream we signal that we will not send anything
// further, which makes the receive stream on the other end terminate.
send.finish()?;

// Wait until the remote closes the connection, which it does once it
// received the response.
connection.closed().await;

Ok(())
})
}
}
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
//! Example for adding a custom protocol.
//! Example protocol for running search on a remote node.
//!
//! We are building a very simple custom protocol here.
//! We are building a very simple protocol here.
//!
//! Our custom protocol allows querying the text stored on the other node.
//! Our protocol allows querying the text stored on the other node.
//!
//! The example is contrived - we only use memory nodes, and our database is a hashmap in a mutex,
//! and our queries just match if the query string appears as-is.
Expand All @@ -11,16 +11,16 @@
//!
//! In one terminal, run
//!
//! cargo run --example custom-protocol --features=examples -- listen "hello-world" "foo-bar" "hello-moon"
//! cargo run --example search --features=examples -- listen "hello-world" "foo-bar" "hello-moon"
//!
//! This spawns an iroh endpoint with three blobs. It will print the node's node id.
//!
//! In another terminal, run
//!
//! cargo run --example custom-protocol --features=examples -- query <node-id> hello
//! cargo run --example search --features=examples -- query <node-id> hello
//!
//! Replace <node-id> with the node id from above. This will connect to the listening node with our
//! custom protocol and query for the string `hello`. The listening node will return a number of how many
//! protocol and query for the string `hello`. The listening node will return a number of how many
//! strings match the query.
//!
//! For this example, this will print:
Expand Down Expand Up @@ -64,7 +64,7 @@ pub enum Command {
},
}

/// Each custom protocol is identified by its ALPN string.
/// Each protocol is identified by its ALPN string.
///
/// The ALPN, or application-layer protocol negotiation, is exchanged in the connection handshake,
/// and the connection is aborted unless both nodes pass the same bytestring.
Expand All @@ -78,7 +78,7 @@ async fn main() -> Result<()> {
// Build an endpoint
let endpoint = Endpoint::builder().discovery_n0().bind().await?;

// Build our custom protocol handler. The `builder` exposes access to various subsystems in the
// Build our protocol handler. The `builder` exposes access to various subsystems in the
// iroh node. In our case, we need a blobs client and the endpoint.
let proto = BlobSearch::new(endpoint.clone());

Expand All @@ -102,7 +102,7 @@ async fn main() -> Result<()> {
}
Command::Query { node_id, query } => {
// Query the remote node.
// This will send the query over our custom protocol, read hashes on the reply stream,
// This will send the query over our protocol, read hashes on the reply stream,
// and download each hash over iroh-blobs.
let num_matches = proto.query_remote(node_id, &query).await?;

Expand Down

0 comments on commit 4abfd61

Please sign in to comment.