Merge #793: bdk v1.0.0-alpha.0

bc3e05c Doc fixes (Daniela Brozzoni)
352f95f [ci] misc fixes (Daniela Brozzoni)
2fcf9c4 Make async esplora futures Send (LLFourn)
5dd4ce7 More documentation improvements (志宇)
ae9b19d `bdk` README improvements (志宇)
def0c9e Add `wallet_esplora_async` example and various fixes (志宇)
26ab2e2 Implement EsploraExt for Async client (Vladimir Fomene)
ab9242d Replace current caching solution with Rust Cache (Vladimir Fomene)
0aaf420 examples: exit if balance < SEND_AMOUNT (Daniela Brozzoni)
47faa88 Fix cargo clippy (Daniela Brozzoni)
9d26121 Improve stdout output for wallet esplora/electrum examples (志宇)
eddd748 Add `wallet_electrum` example (志宇)
0505cd7 Remove transaction-based type parameters and traits (志宇)
de9457f Changed `inflate_update` logic to not depend on `Cow` (志宇)
69cf6d7 Use os-specific temp dir for `wallet_esplora` example (志宇)
b3836cb Use hardcoded send amount for `wallet_esplora` example (志宇)
b082932 Add the wallet_esplora example (Daniela Brozzoni)
d267517 NewError implements StdError (fix typo) (Daniela Brozzoni)
0c7a0ab Disable code coverage (Daniela Brozzoni)
dfcbafd Use action-rs/toolchain in CI (Daniela Brozzoni)
0ba41c5 Make bdk_esplora wasm compatible again (Daniela Brozzoni)
a38f633 Make bdk_file_store use bincode v1 (LLFourn)
38ef170 Make bdk and bdk_chain work under 1.57.0 (Steve Myers)
3a5d727 Update workspace dependencies to be relative paths (Steve Myers)
96d932c Add clippy.toml (Daniela Brozzoni)
5708bf0 Fix docs (Daniela Brozzoni)
5acee82 Update rust stable in CI (1.67.0) (Daniela Brozzoni)
8c9bceb Fix the "repository" field in Cargo.toml (Daniela Brozzoni)
c61b360 Fix cargo clippy warnings (Daniela Brozzoni)
1805bd3 Fix clippy (Daniela Brozzoni)
3f5a78a Disable test-hardware-signer (Daniela Brozzoni)
303a170 Rust fmt (Daniela Brozzoni)
b555976 Rename the stub wallet examples (LLFourn)
2e82cd8 Use tempfile for file_store tests (LLFourn)
c069b0f Move everything else over 🎉 (LLFourn)
949608a Move bdk_electrum into the bdk repo 🎉 (Daniela Brozzoni)
03deafb Move bdk_file_store into the bdk repo 🎉 (Daniela Brozzoni)
37dfa77 Move bdk_chain into the bdk repo 🎉 (Daniela Brozzoni)
f2188f9 Make lib.rs's docs be the README.md (LLFourn)
1c970a9 Fix code coverage CI (Daniela Brozzoni)
94a084a Fix doc links (LLFourn)
9edbdf5 [ci] Fix feature flags (LLFourn)
20e45b7 Add back test-hardware-signer feature (LLFourn)
6d05598 Remove test-readme-examples (LLFourn)
b60820a Remove authors.workspace because older cargo don't like (LLFourn)
22bec6d Delete unused errors (LLFourn)
8a6de3a Convert to workspace (LLFourn)
fdfc9b9 Delete unused things (LLFourn)
e1eb025 Make doctests work (LLFourn)
3baf972 Use bdk_chain Balance (LLFourn)
b310a7a Add Wallet::cancel_tx (LLFourn)
5985706 Add wallet persistence (LLFourn)
57538e5 Move tests to /tests (LLFourn)
a40da9b Make bdk no_std (LLFourn)
aab2b12 bdk_core integration initial commit 🔥 (LLFourn)
544c397 Modified build, clippy and test steps (志宇)

Pull request description:

  We prepare the BDK repo for a major restructuring 🔥. This PR maintains the existing wallet API as much as possible and adds very little.

  ## Things Done

  - database modules removed
  - blockchain gutted but new esplora syncing code added (this will be gone soon hopefully).
  - minimal API changes.
  - Many macros removed.
  - no longer applicable examples removed.
  - Much conditional compilation removed. Can compile with `--all-features` now.
  - All wallet tests passing
  - TestClient moved into its own repo
  - Example using `esplora`

  ## APIs changed

  - wallet no longer has a `sync` method. This is replaced with `apply_wallet_scan`.
  - address "caching" is gone. You can just change the derivation index with `ensure_derived_up_to` which sets your derivation to at least the argument. Unlike `ensure_addresses_cached` used to do this will alter what getting a new address gives you.
  - `AddressIndex::Reset` is gone. This thing didn't make much sense and is hard to do with the more sane internals we've established. Changing the derivation index changes what script pubkeys the wallet will search so this is dangerous. We plan to add method like `trim_unused` which lowers the derivation index to the highest unused index. Applications must handle giving out old addresses manually now (which I think is good).

  ## Unfinished work

  - [x] esplora example doesn't work for mempool transactions yet (seems like our esplora in testclient doesn't index mempool??).
  - [x] we need to figure out a way to retrieve and store transaction timestamps (we're currently just setting them to `u64::MAX`). In `bdk_core` we never got around to doing this but it needs to be done.
  - [x] A few insights we got from doing this PR should be applied to bdk_core first.
  - [x] doctests not working.

  ### Notes to the reviewers

  Try not to review the actual changes. This PR will be forced pushed a bit so it will be likely wasted.
  I think I did a faithful job of translating the tests. A bit of review here would be helpful.

  I *do* think it would be good to merge this PR soon into the v1 branch so we have something to work off once unfinished work is done.
  Checking out the branch and poke around and give feedback would be the most helpful thing.

  Run the (sort of) working example:

  ```
  cargo run --example esplora --features="bdk_test_client/bitcoind_22_0 bdk_test_client/esplora esplora"
  ```
  ### Checklists

  #### All Submissions:

  * [x] I've signed all my commits
  * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
  * [x] I ran `cargo fmt` and `cargo clippy` before committing

  #### Bugfixes:

  * [x] This pull request breaks the existing API
  * [ ] I'm linking the issue being fixed by this PR (there's too many!)

ACKs for top commit:
  danielabrozzoni:
    ACK bc3e05c

Tree-SHA512: b5fbfd2263bb2a7cbf7b486530e639683775209bea944b266efd9c0447fb632295f7f5ddbd2e8d79796338415eaca26f1876617c2b66523485167b22e75e449e

.github/workflows/code_coverage.yml

@@ -16,44 +16,37 @@ jobs:
         uses: actions/checkout@v2
       - name: Install lcov tools
         run: sudo apt-get install lcov -y
-      - name: Install rustup
-        run: curl https://sh.rustup.rs -sSf | sh -s -- -y
-      - name: Set default toolchain
-        run: rustup default nightly
-      - name: Set profile
-        run: rustup set profile minimal
-      - name: Add llvm tools
-        run: rustup component add llvm-tools-preview
-      - name: Update toolchain
-        run: rustup update
-      - name: Cache cargo
-        uses: actions/cache@v3
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
         with:
-          path: |
-            ~/.cargo/bin/
-            ~/.cargo/registry/index/
-            ~/.cargo/registry/cache/
-            ~/.cargo/git/db/
-          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
+            toolchain: "1.65.0"
+            override: true
+            profile: minimal
+            components: llvm-tools-preview
+      - name: Rust Cache
+        uses: Swatinem/[email protected]
       - name: Install grcov
         run: if [[ ! -e ~/.cargo/bin/grcov ]]; then cargo install grcov; fi
+      - name: Build simulator image
+        run: docker build -t hwi/ledger_emulator ./ci -f ci/Dockerfile.ledger
+      - name: Run simulator image
+        run: docker run --name simulator --network=host hwi/ledger_emulator &
+      - name: Install Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+      - name: Install python dependencies
+        run: pip install hwi==2.1.1 protobuf==3.20.1
       - name: Test
-        # WARNING: this is not testing the following features: test-esplora, test-hardware-signer, async-interface
-        # This is because some of our features are mutually exclusive, and generating various reports and
-        # merging them doesn't seem to be working very well.
-        # For more info, see:
-        # - https://github.com/bitcoindevkit/bdk/issues/696
-        # - https://github.com/bitcoindevkit/bdk/pull/748#issuecomment-1242721040
-        run: cargo test --features all-keys,compact_filters,compiler,key-value-db,sqlite,sqlite-bundled,test-electrum,test-rpc,verify
+        run: cargo test --all-features
       - name: Run grcov
         run: mkdir coverage; grcov . --binary-path ./target/debug/ -s . -t lcov --branch --ignore-not-existing --ignore '/*' -o ./coverage/lcov.info
       - name: Generate HTML coverage report
         run: genhtml -o coverage-report.html ./coverage/lcov.info
-
-      - name: Coveralls upload
-        uses: coverallsapp/github-action@master
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
+          #      - name: Coveralls upload
+          #        uses: coverallsapp/github-action@master
+          #        with:
+          #          github-token: ${{ secrets.GITHUB_TOKEN }}
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with:

.github/workflows/cont_integration.yml

@@ -10,118 +10,27 @@ jobs:
     strategy:
       matrix:
         rust:
-          - version: 1.65.0 # STABLE
+          - version: stable
             clippy: true
           - version: 1.57.0 # MSRV
         features:
-          - default
-          - minimal
-          - all-keys
-          - minimal,use-esplora-blocking
-          - key-value-db
-          - electrum
-          - compact_filters
-          - use-esplora-blocking,key-value-db,electrum
-          - compiler
-          - rpc
-          - verify
-          - async-interface
-          - use-esplora-async
-          - sqlite
-          - sqlite-bundled
+          - --no-default-features
+          - --all-features
     steps:
       - name: checkout
         uses: actions/checkout@v2
-      - name: Generate cache key
-        run: echo "${{ matrix.rust.version }} ${{ matrix.features }}" | tee .cache_key
-      - name: cache
-        uses: actions/cache@v2
-        with:
-          path: |
-            ~/.cargo/registry
-            ~/.cargo/git
-            target
-          key: ${{ runner.os }}-cargo-${{ hashFiles('.cache_key') }}-${{ hashFiles('**/Cargo.toml','**/Cargo.lock') }}
-      - name: Set default toolchain
-        run: rustup default ${{ matrix.rust.version }}
-      - name: Set profile
-        run: rustup set profile minimal
-      - name: Add clippy
-        if: ${{ matrix.rust.clippy }}
-        run: rustup component add clippy
-      - name: Update toolchain
-        run: rustup update
-      - name: Build
-        run: cargo build --features ${{ matrix.features }} --no-default-features
-      - name: Clippy
-        if: ${{ matrix.rust.clippy }}
-        run: cargo clippy --all-targets --features ${{ matrix.features }} --no-default-features -- -D warnings
-      - name: Test
-        run: cargo test --features ${{ matrix.features }} --no-default-features
-
-  test-readme-examples:
-    name: Test README.md examples
-    runs-on: ubuntu-latest
-    steps:
-      - name: checkout
-        uses: actions/checkout@v2
-      - name: cache
-        uses: actions/cache@v2
-        with:
-          path: |
-            ~/.cargo/registry
-            ~/.cargo/git
-            target
-          key: ${{ runner.os }}-cargo-test-md-docs-${{ hashFiles('**/Cargo.toml','**/Cargo.lock') }}
-      - name: Set default toolchain
-        run: rustup default nightly
-      - name: Set profile
-        run: rustup set profile minimal
-      - name: Update toolchain
-        run: rustup update
-      - name: Test
-        run: cargo test --features test-md-docs --no-default-features -- doctest::ReadmeDoctests
-
-  test-blockchains:
-    name: Blockchain ${{ matrix.blockchain.features }}
-    runs-on: ubuntu-20.04
-    strategy:
-      fail-fast: false
-      matrix:
-        blockchain:
-          - name: electrum
-            testprefix: blockchain::electrum::test
-            features: test-electrum,verify
-          - name: rpc
-            testprefix: blockchain::rpc::test
-            features: test-rpc
-          - name: rpc-legacy
-            testprefix: blockchain::rpc::test
-            features: test-rpc-legacy
-          - name: esplora
-            testprefix: esplora
-            features: test-esplora,use-esplora-async,verify
-          - name: esplora
-            testprefix: esplora
-            features: test-esplora,use-esplora-blocking,verify
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v2
-      - name: Cache
-        uses: actions/cache@v2
-        with:
-          path: |
-            ~/.cargo/registry
-            ~/.cargo/git
-            target
-          key: ${{ runner.os }}-cargo-${{ github.job }}-${{ hashFiles('**/Cargo.toml','**/Cargo.lock') }}
-      - name: Setup rust toolchain
+      - name: Install Rust toolchain
         uses: actions-rs/toolchain@v1
         with:
-          toolchain: stable
-          override: true
+            toolchain: ${{ matrix.rust.version }}
+            override: true
+            profile: minimal
+      - name: Rust Cache
+        uses: Swatinem/[email protected]
+      - name: Build
+        run: cargo build ${{ matrix.features }}
       - name: Test
-        run: cargo test --no-default-features --features ${{ matrix.blockchain.features }} ${{ matrix.blockchain.testprefix }}::bdk_blockchain_tests
+        run: cargo test ${{ matrix.features }}
 
   check-wasm:
     name: Check WASM
@@ -132,72 +41,57 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v2
-      - name: Cache
-        uses: actions/cache@v2
-        with:
-          path: |
-            ~/.cargo/registry
-            ~/.cargo/git
-            target
-          key: ${{ runner.os }}-cargo-${{ github.job }}-${{ hashFiles('**/Cargo.toml','**/Cargo.lock') }}
         # Install a recent version of clang that supports wasm32
       - run: wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - || exit 1
       - run: sudo apt-add-repository "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main" || exit 1
       - run: sudo apt-get update || exit 1
       - run: sudo apt-get install -y libclang-common-10-dev clang-10 libc6-dev-i386 || exit 1
-      - name: Set default toolchain
-        run: rustup default 1.65.0 # STABLE
-      - name: Set profile
-        run: rustup set profile minimal
-      - name: Add target wasm32
-        run: rustup target add wasm32-unknown-unknown
-      - name: Update toolchain
-        run: rustup update
-      - name: Check
-        run: cargo check --target wasm32-unknown-unknown --features async-interface,use-esplora-async,dev-getrandom-wasm --no-default-features
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: stable
+            override: true
+            profile: minimal
+            target: "wasm32-unknown-unknown"
+      - name: Rust Cache
+        uses: Swatinem/[email protected]
+      - name: Check bdk
+        working-directory: ./crates/bdk
+        run: cargo check --target wasm32-unknown-unknown --features dev-getrandom-wasm
+      - name: Check esplora
+        working-directory: ./crates/esplora
+        run: cargo check --target wasm32-unknown-unknown --features async --no-default-features
 
   fmt:
     name: Rust fmt
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
         uses: actions/checkout@v2
-      - name: Set default toolchain
-        run: rustup default nightly
-      - name: Set profile
-        run: rustup set profile minimal
-      - name: Add rustfmt
-        run: rustup component add rustfmt
-      - name: Update toolchain
-        run: rustup update
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: stable
+            override: true
+            profile: minimal
+            components: rustfmt
       - name: Check fmt
         run: cargo fmt --all -- --config format_code_in_doc_comments=true --check
 
-  test_hardware_wallet:
-    runs-on: ubuntu-20.04
-    strategy:
-      matrix:
-        rust:
-          - version: 1.65.0 # STABLE
-          - version: 1.57.0 # MSRV
+  clippy_check:
+    runs-on: ubuntu-latest
     steps:
-    - name: Checkout
-      uses: actions/checkout@v3
-    - name: Build simulator image
-      run: docker build -t hwi/ledger_emulator ./ci -f ci/Dockerfile.ledger
-    - name: Run simulator image
-      run: docker run --name simulator --network=host hwi/ledger_emulator &
-    - name: Install Python
-      uses: actions/setup-python@v4
-      with:
-        python-version: '3.9'
-    - name: Install python dependencies
-      run: pip install hwi==2.1.1 protobuf==3.20.1
-    - name: Set default toolchain
-      run: rustup default ${{ matrix.rust.version }}
-    - name: Set profile
-      run: rustup set profile minimal
-    - name: Update toolchain
-      run: rustup update
-    - name: Test
-      run: cargo test --features test-hardware-signer
+      - uses: actions/checkout@v1
+      - uses: actions-rs/toolchain@v1
+        with:
+            # we pin clippy instead of using "stable" so that our CI doesn't break
+            # at each new cargo release
+            toolchain: "1.67.0"
+            components: clippy
+            override: true
+      - name: Rust Cache
+        uses: Swatinem/[email protected]
+      - uses: actions-rs/clippy-check@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          args: --all-features --all-targets -- -D warnings

.github/workflows/nightly_docs.yml

@@ -9,22 +9,18 @@ jobs:
     steps:
       - name: Checkout sources
         uses: actions/checkout@v2
-      - name: Setup cache
-        uses: actions/cache@v2
-        with:
-          path: |
-            ~/.cargo/registry
-            ~/.cargo/git
-            target
-          key: nightly-docs-${{ hashFiles('**/Cargo.toml','**/Cargo.lock') }}
       - name: Set default toolchain
         run: rustup default nightly-2022-12-14
       - name: Set profile
         run: rustup set profile minimal
       - name: Update toolchain
         run: rustup update
+      - name: Rust Cache
+        uses: Swatinem/[email protected]
       - name: Build docs
-        run: cargo rustdoc --verbose --features=compiler,electrum,esplora,use-esplora-blocking,compact_filters,rpc,key-value-db,sqlite,all-keys,verify,hardware-signer -- --cfg docsrs -Dwarnings
+        run: cargo doc --no-deps
+        env:
+          RUSTDOCFLAGS: '--cfg docsrs -Dwarnings'
       - name: Upload artifact
         uses: actions/upload-artifact@v2
         with: