From c8a7f89e11967724b9c0efec70bcd1515908e520 Mon Sep 17 00:00:00 2001 From: Johnny Matthews Date: Thu, 21 May 2020 11:17:11 -0400 Subject: [PATCH 001/144] Updates docs link to non-beta site. --- .github/ISSUE_TEMPLATE/config.yml | 2 +- docs/config.md | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 594bbf8b898..e5afd8ffa56 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -10,7 +10,7 @@ contact_links: url: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md about: Documentation on Private Networks, Filestore and other experimental features. - name: HTTP API Reference - url: https://docs-beta.ipfs.io/reference/http/api/#api-v0-add + url: https://docs.ipfs.io/reference/http/api/#api-v0-add about: Documentation of all go-ipfs HTTP API endpoints. - name: IPFS Official Forum url: https://discuss.ipfs.io diff --git a/docs/config.md b/docs/config.md index 2a64275bd26..c7773cac446 100644 --- a/docs/config.md +++ b/docs/config.md @@ -505,7 +505,7 @@ Default: `[]` A boolean to configure whether the gateway at the hostname provides [Origin isolation](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) between content roots. -- `true` - enables [subdomain gateway](#https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://*.{hostname}/` +- `true` - enables [subdomain gateway](#https://docs.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://*.{hostname}/` - **Requires whitelist:** make sure respective `Paths` are set. For example, `Paths: ["/ipfs", "/ipns"]` are required for `http://{cid}.ipfs.{hostname}` and `http://{foo}.ipns.{hostname}` to work: ```json @@ -521,7 +521,7 @@ between content roots. - **Backward-compatible:** requests for content paths such as `http://{hostname}/ipfs/{cid}` produce redirect to `http://{cid}.ipfs.{hostname}` - **API:** if `/api` is on the `Paths` whitelist, `http://{hostname}/api/{cmd}` produces redirect to `http://api.{hostname}/api/{cmd}` -- `false` - enables [path gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://{hostname}/*` +- `false` - enables [path gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://{hostname}/*` - Example: ```json "Gateway": { @@ -575,7 +575,7 @@ $ ipfs config --json Gateway.PublicGateways '{"localhost": null }' Below is a list of the most common public gateway setups. -* Public [subdomain gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://{cid}.ipfs.dweb.link` (each content root gets its own Origin) +* Public [subdomain gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://{cid}.ipfs.dweb.link` (each content root gets its own Origin) ```console $ ipfs config --json Gateway.PublicGateways '{ "dweb.link": { @@ -589,7 +589,7 @@ Below is a list of the most common public gateway setups. **Note II:** if you run go-ipfs behind a reverse proxy that provides TLS, make it adds a `X-Forwarded-Proto: https` HTTP header to ensure users are redirected to `https://`, not `http://`. The NGINX directive is `proxy_set_header X-Forwarded-Proto "https";`.: `https://dweb.link/ipfs/{cid}` → `https://{cid}.ipfs.dweb.link` -* Public [path gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://ipfs.io/ipfs/{cid}` (no Origin separation) +* Public [path gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://ipfs.io/ipfs/{cid}` (no Origin separation) ```console $ ipfs config --json Gateway.PublicGateways '{ "ipfs.io": { @@ -605,7 +605,7 @@ Below is a list of the most common public gateway setups. ``` * Note that `NoDNSLink: false` is the default (it works out of the box unless set to `true` manually) -* Hardened, site-specific [DNSLink gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#dnslink-gateway). +* Hardened, site-specific [DNSLink gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#dnslink-gateway). Disable fetching of remote data (`NoFetch: true`) and resolving DNSLink at unknown hostnames (`NoDNSLink: true`). Then, enable DNSLink gateway only for the specific hostname (for which data From 25a2e31b63a77f66d7c0c37881fcb6aab02ac9fe Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Tue, 26 May 2020 10:25:32 +0200 Subject: [PATCH 002/144] systemd: add a service file with systemd hardening features --- misc/systemd/ipfs-hardened.service | 70 ++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 misc/systemd/ipfs-hardened.service diff --git a/misc/systemd/ipfs-hardened.service b/misc/systemd/ipfs-hardened.service new file mode 100644 index 00000000000..dbb35c5978b --- /dev/null +++ b/misc/systemd/ipfs-hardened.service @@ -0,0 +1,70 @@ +# This file will be overwritten on package upgrades, avoid customizations here. +# +# To make persistant changes, create file in +# "/etc/systemd/system/ipfs.service.d/overwrite.conf" with +# `systemctl edit ipfs.service`. This file will be parsed after this +# file has been parsed. +# +# To overwrite a variable, like ExecStart you have to specify it once +# blank and a second time with a new value, like: +# ExecStart= +# ExecStart=/usr/bin/ipfs daemon --flag1 --flag2 +# +# For more info about custom unit files see systemd.unit(5). + +# This service file enables systemd-hardening features compatible with IPFS, +# while breaking compability with the fuse-mount function. Use this one only +# if you don't need the fuse-mount functionality. + +[Unit] +Description=InterPlanetary File System (IPFS) daemon +Documentation=https://docs.ipfs.io/ +After=network.target + +[Service] +# hardening +ReadWritePaths="/var/lib/ipfs/" +NoNewPrivileges=true +ProtectSystem=strict +ProtectKernelTunables=true +ProtectKernelModules=true +ProtectKernelLogs=true +PrivateDevices=true +DevicePolicy=closed +ProtectControlGroups=true +RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK +ProtectHostname=true +PrivateTmp=true +ProtectClock=true +LockPersonality=true +RestrictNamespaces=true +RestrictRealtime=true +MemoryDenyWriteExecute=true +SystemCallArchitectures=native +SystemCallFilter=@system-service +SystemCallFilter=~@privileged +ProtectHome=true +RemoveIPC=true +RestrictSUIDSGID=true +CapabilityBoundingSet=CAP_NET_BIND_SERVICE + +# enable for 1-1024 port listening +#AmbientCapabilities=CAP_NET_BIND_SERVICE +# enable to specify a custom path see docs/environment-variables.md for further documentations +#Environment=IPFS_PATH=/custom/ipfs/path +# enable to specify a higher limit for open files/connections +#LimitNOFILE=1000000 + +#don't use swap +MemorySwapMax=0 + +Type=notify +User=ipfs +Group=ipfs +StateDirectory=ipfs +ExecStart=/usr/bin/ipfs daemon --init --migrate +Restart=on-failure +KillSignal=SIGINT + +[Install] +WantedBy=default.target From 4ea43cb53ad220ecd8469c5069754781661d68ae Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Tue, 26 May 2020 16:07:36 +0200 Subject: [PATCH 003/144] docs: add pacman.store (@RubenKelevra) to the early testers --- docs/EARLY_TESTERS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/EARLY_TESTERS.md b/docs/EARLY_TESTERS.md index f2332165211..16d80f1fc82 100644 --- a/docs/EARLY_TESTERS.md +++ b/docs/EARLY_TESTERS.md @@ -22,6 +22,7 @@ We will ask early testers to participate at two points in the process: ## Who has signed up? +- [ ] pacman.store ([@RubenKelevra](https://github.com/RubenKelevra)) - [ ] Infura (@MichaelMure) - [ ] Textile (@sanderpick) - [ ] Pinata (@obo20) From e0ea0b369573c009862557770d0e02bb9b267259 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 26 May 2020 14:22:28 -0700 Subject: [PATCH 004/144] doc: update the release template for git flow changes --- docs/RELEASE_ISSUE_TEMPLATE.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/RELEASE_ISSUE_TEMPLATE.md b/docs/RELEASE_ISSUE_TEMPLATE.md index 33fb815d015..2e8774eaf4f 100644 --- a/docs/RELEASE_ISSUE_TEMPLATE.md +++ b/docs/RELEASE_ISSUE_TEMPLATE.md @@ -24,8 +24,8 @@ We're happy to announce go-ipfs X.Y.Z, bla bla... For each RC published in each stage: -- version string in `version.go` has been updated -- tag commit with vX.Y.Z-rcN +- version string in `version.go` has been updated (in the `release-vX.Y.Z` branch). +- tag commit with `vX.Y.Z-rcN` - upload to dist.ipfs.io 1. Build: https://github.com/ipfs/distributions#usage. 2. Pin the resulting release. @@ -39,7 +39,9 @@ For each RC published in each stage: Checklist: - [ ] **Stage 0 - Automated Testing** - - [ ] Feature freeze. If any "non-trivial" changes (see the footnotes of [docs/releases.md](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md) for a definition) get added to the release, uncheck all the checkboxes and return to this stage. + - [ ] Fork a new branch (`release-vX.Y.Z`) from `master` and make any further release related changes to this branch. If any "non-trivial" changes (see the footnotes of [docs/releases.md](https://github.com/ipfs/go-ipfs/tree/master/docs/releases.md) for a definition) get added to the release, uncheck all the checkboxes and return to this stage. + - [ ] Follow the RC release process to cut the first RC. + - [ ] Bump the version in `version.go` in the `master` branch to `vX.(Y+1).0-dev`. - [ ] Automated Testing (already tested in CI) - Ensure that all tests are passing, this includes: - [ ] unit, sharness, cross-build, etc (`make test`) - [ ] lint (`make test_go_lint`) @@ -79,9 +81,9 @@ Checklist: - [ ] IRC - [ ] **Stage 4 - Release** - [ ] Final preparation - - [ ] Verify that version string in [`version.go`](https://github.com/ipfs/go-ipfs/tree/master/version.go) has been updated - - [ ] tag commit with vX.Y.Z - - [ ] update release branch to point to release commit (`git merge vX.Y.Z`). + - [ ] Verify that version string in [`version.go`](https://github.com/ipfs/go-ipfs/tree/master/version.go) has been updated. + - [ ] Merge `release-vX.Y.Z` into the `release` branch. + - [ ] Tag this merge commit (on the `release` branch) with `vX.Y.Z`. - [ ] Release published - [ ] to [dist.ipfs.io](https://dist.ipfs.io) - [ ] to [npm-go-ipfs](https://github.com/ipfs/npm-go-ipfs) @@ -98,7 +100,7 @@ Checklist: - [ ] [discuss.ipfs.io](https://discuss.ipfs.io/c/announcements) - [ ] Announce it on the [IPFS Users Mailing List](https://groups.google.com/forum/#!forum/ipfs-users) - [ ] **Post-Release** - - [ ] Bump the version in `version.go` to `vX.(Y+1).0-dev`. + - [ ] Merge the `release` branch back into `master`, ignoring the changes to `version.go` (keep the `-dev` version from master). - [ ] Create an issue using this release issue template for the _next_ release. - [ ] Make sure any last-minute changelog updates from the blog post make it back into the CHANGELOG. From 7c563660041b232d10019fa591b5cd35219092ff Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 26 May 2020 21:46:16 -0700 Subject: [PATCH 005/144] feat: add a mailmap file This will allow us to dedup contributors when generating release notes. --- .mailmap | 290 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 290 insertions(+) create mode 100644 .mailmap diff --git a/.mailmap b/.mailmap new file mode 100644 index 00000000000..ef596933c29 --- /dev/null +++ b/.mailmap @@ -0,0 +1,290 @@ +@RubenKelevra +Aaron Hill +Adam Gashlin +Adam Uhlir +Adin Schmahmann +Adrian Lanzafame +Adrian Ulrich +Alan Shaw +Alec Brickner +Alex +Alfie John +Alfonso Montero +Ali Mirlou +Andres Buritica +Andrew Chin +Andrew Nesbitt +Andy Leap +Antti Kaihola +Artem Andreenko +Arthur Elliott +Bamvor Zhang +Baptiste Jonglez +Bernhard M. Wiedemann +Boris Mann +Brendan Mc +Brendan McMillion +Brendan O'Brien +Brian Tiger Chow +Brian Tiger Chow +Caian +Caio Alonso +Carlos Cobo +Casey Chance +Cayman Nava +Chas Leichner +Chris Boddy +Chris Chinchilla +Chris Grimmett +Chris P +Christopher Sasarak +Christian Couder +Christian Kniep +Christopher Buesser +Cole Brown +Corbin Page +Cornelius Toole +Dan <35669742+NukeManDan@users.noreply.github.com> +Daniel Aleksandersen +Daniel Grossmann-Kavanagh +Daniel Mack +David +David Braun +David Brennan +David Dias +David Wagner +Devin +Dimitris Apostolou +Diogo Silva +Dirk McCormick +Djalil Dreamski <32184973+dreamski21@users.noreply.github.com> +Dominic Della Valle +Dominic Tarr +Ian Preston +Dylan Powers +Edison Lee +Elias Gabrielsson +Emery Hemingway +Enrique Erne +Eoghan Ó Carragáin +Erik Ingenito +Esteban +Ethan Buchman +Etienne Laurin +Forrest Weston +Francesco Canessa +Frank Sachsenheim +Frederik Riedel +Friedel Ziegelmayer +George Antoniadis +George Masgras +Giulitti Salvatore +Giuseppe Bertone +Gowtham G +Harald Nordgren +Harlan T Wood +Hector Sanjuan +Henrique Dias +Henry +Herman Junge +Hlib +Ho-Sheng Hsiao +Hucg <41573766+hcg1314@users.noreply.github.com> +Iaroslav Gridin +Igor Velkov +Ivan +JP Hastings-Spital +Jack Loughran <30052269+jackloughran@users.noreply.github.com> +Jakub Sztandera +Jakub Kaczmarzyk +James Stanley +Jamie Wilkinson +Jan Winkelmann +Jason Carver +Jeff Thompson +Jeromy Johnson +Jesse Weinstein +Jessica Schilling +Jim McDonald +John Reed +Johnny <9611008+johnnymatthews@users.noreply.github.com> +Jon Choi +Jonathan Dahan +Jordan Danford +Jorropo +Juan Batiz-Benet +Justin Drake +Kacper Łukawski +Karthik Bala +Kejie Zhang <601172892@qq.com> +Kerem +Kevin Atkinson +Kevin Simper +Kevin Wallace +Kirill Goncharov +Kishan Mohanbhai Sagathiya +Knut Ahlers +Konstantin Koroviev +Koushik Roy +Kristoffer Ström +Kuro1 <412681778@qq.com> +Lars Gierth +Leo Arias +Li Zheng +Lorenzo Manacorda +Lorenzo Setale +Louis Thibault +Lucas Garron +Lucas Molas +Marcin Janczyk +Marcin Rataj +Markus Amalthea Magnuson +Marten Seemann +Masashi Salvador Mitsuzawa +Massino Tinovan +Mat Kelly +Mathijs de Bruin +Matouš Skála +Matt Bell +Matt Joiner +Max Chechel +Max Kerp +Mib Kd743naq +Michael Avila +Michael Lovci +Michael Muré +Michael Pfister +Michelle Lee +Miguel Torres +Mikaela Suomalainen +Mildred Ki'Lya +Molly +Muneeb Ali +Mykola Nikishov +Nathan Musoke +Nick Hamann +Oli Evans +Or Rikon +Overbool +Patrick Connolly +Pavol Rusnak +Peter Borzov +Peter Rabbitson +Peter Wu +Philip Nelson +Pierre-Alain TORET +PoorPockets McNewHold +Pretty Please Mark Darkly <55382229+pleasemarkdarkly@users.noreply.github.com> +Péter Szilágyi +Quantomic +Quinn Slack +Raúl Kripalani +ReadmeCritic +Remco Bloemen +Richard Littauer +RideWindX +Rob Brackett +Robert Carlsen +Rod Vagg +Roerick Sweeney +Roman Khafizianov +Roman Proskuryakov +Ronsor +RubenKelevra +Ryan Carver +Ryan Morey +SH +Sag0Sag0 +Sander Pick +Scott Bigelow +Sean Lang +Shanti Bouchez-Mongardé +Shaun Bruce +Sherod Taylor +Simon Kirkby +Simon Menke +Siraj Ravel +Siva Chandran +Spartucus +Stephan Kulla +Stephan Seidt +Stephen Sugden +Stephen Whitmore +Steve Recio +Steven Allen +Steven Vandevelde +Sönke Hahn +TUSF +Tarnay Kálmán +Thomas Gardner +Tiger +Tim Groeneveld +Tim Stahel +Timothy Hobbs +Tom O'Donnell +Tom Swindell +Tommi Virtanen +Tonis Tiigi +Tor Arne Vestbø +Travis Person +Tylar +John Reed +Vasil Dimov +Vijayee Kulkaa +Vikram +Vitor Baptista +W. Trevor King +Wes Morgan +Will Scott +Willi Butz +Xiaoyi Wang +Yuval Langer +Zander Mackie +ZenGround0 +achingbrain +adamliesko +anarcat +bbenshoof +camelmasa +chenminjian <727180553@qq.com> +devedge +dgrisham +drathir +epitron +eric wu +flowed +forstmeier +fyrchik +gatesvp +hannahhoward +hikerpig +hoenirvili +hucg +ivan386 +klauspost +kpcyrd +kvm2116 +mateon1 +matrushka +michael +myself659 +nmalhotra +palkeo +requilence +rht +rob-deutsch +slothbag +sroerick +swedneck <40505480+swedneck@users.noreply.github.com> +tarekbadr +tcme +tg +theswitch +verokarhu +vitzli +vyzo +wzhd +zramsay +Łukasz Magiera +ᴍᴀᴛᴛ ʙᴇʟʟ +ᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ From 30dcc26be848389a6d6f0af43a1a0b140f7b9954 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 26 May 2020 21:46:45 -0700 Subject: [PATCH 006/144] feat: support the mailmap file in mkreleasenotes This will use either the mailmap file in the current repo, or the mailmap file in the target repo, when generating the contributor stats. --- bin/mkreleaselog | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/bin/mkreleaselog b/bin/mkreleaselog index c30ee7051df..0588adaee24 100755 --- a/bin/mkreleaselog +++ b/bin/mkreleaselog @@ -33,15 +33,22 @@ AUTHORS=( NL=$'\n' +ROOT_DIR="$(git rev-parse --show-toplevel)" + msg() { echo "$*" >&2 } statlog() { - rpath="$GOPATH/src/$1" - start="${2:-}" - end="${3:-HEAD}" - git -C "$rpath" log --shortstat --no-merges --pretty="tformat:%H%n%aN%n%aE" "$start..$end" | while + local rpath="$GOPATH/src/$1" + local start="${2:-}" + local end="${3:-HEAD}" + local mailmap_file="$rpath/.mailmap" + if ! [[ -e "$mailmap_file" ]]; then + mailmap_file="$ROOT_DIR/.mailmap" + fi + + git -C "$rpath" -c mailmap.file="$mailmap_file" log --use-mailmap --shortstat --no-merges --pretty="tformat:%H%n%aN%n%aE" "$start..$end" | while read hash read name read email From 3849c0fb80ba39de9dca0620d8870c73e687779c Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Wed, 27 May 2020 10:22:28 -0700 Subject: [PATCH 007/144] chore(mailcap): change ribasushi addresses Co-authored-by: Peter Rabbitson --- .mailmap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.mailmap b/.mailmap index ef596933c29..e0ba27dd523 100644 --- a/.mailmap +++ b/.mailmap @@ -169,7 +169,7 @@ Overbool Patrick Connolly Pavol Rusnak Peter Borzov -Peter Rabbitson +Peter Rabbitson Peter Wu Philip Nelson Pierre-Alain TORET From 1a161c2351e812b81f77b6c0628476ac5d6c5949 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Wed, 27 May 2020 10:22:44 -0700 Subject: [PATCH 008/144] chore(mailcap): document rational Co-authored-by: Peter Rabbitson --- .mailmap | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.mailmap b/.mailmap index e0ba27dd523..b8713c1839f 100644 --- a/.mailmap +++ b/.mailmap @@ -1,3 +1,9 @@ +# This file allows re-mapping author names/emails +# while maintaining the integrity of the repository +# +# Spec: https://www.kernel.org/pub/software/scm/git/docs/git-shortlog.html#_mapping_authors +# + @RubenKelevra Aaron Hill Adam Gashlin From 58aac04a53d60d683cbe64fc26f6b0abac4e855b Mon Sep 17 00:00:00 2001 From: Shotaro Yamada Date: Thu, 28 May 2020 10:53:25 +0900 Subject: [PATCH 009/144] chore: update WebUI to 2.8.0 --- core/corehttp/webui.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/corehttp/webui.go b/core/corehttp/webui.go index 02f2da73ad9..22143e49a41 100644 --- a/core/corehttp/webui.go +++ b/core/corehttp/webui.go @@ -1,11 +1,12 @@ package corehttp // TODO: move to IPNS -const WebUIPath = "/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq" +const WebUIPath = "/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i" // this is a list of all past webUI paths. var WebUIPaths = []string{ WebUIPath, + "/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq", "/ipfs/QmfQkD8pBSBCBxWEwFSu4XaDVSWK6bjnNuaWZjMyQbyDub", "/ipfs/QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ", "/ipfs/QmenEBWcAk3tN94fSKpKFtUMwty1qNwSYw3DMDFV6cPBXA", From e925313d6cef2a98115bacac5ddb500090947dca Mon Sep 17 00:00:00 2001 From: "dependabot-preview[bot]" <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Thu, 28 May 2020 11:28:27 +0000 Subject: [PATCH 010/144] chore(deps): bump github.com/libp2p/go-libp2p-kad-dht Bumps [github.com/libp2p/go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) from 0.8.0 to 0.8.1. - [Release notes](https://github.com/libp2p/go-libp2p-kad-dht/releases) - [Commits](https://github.com/libp2p/go-libp2p-kad-dht/compare/v0.8.0...v0.8.1) Signed-off-by: dependabot-preview[bot] --- go.mod | 2 +- go.sum | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/go.mod b/go.mod index f30d8b217e9..9f220d4ffc9 100644 --- a/go.mod +++ b/go.mod @@ -66,7 +66,7 @@ require ( github.com/libp2p/go-libp2p-core v0.5.6 github.com/libp2p/go-libp2p-discovery v0.4.0 github.com/libp2p/go-libp2p-http v0.1.5 - github.com/libp2p/go-libp2p-kad-dht v0.8.0 + github.com/libp2p/go-libp2p-kad-dht v0.8.1 github.com/libp2p/go-libp2p-kbucket v0.4.2 github.com/libp2p/go-libp2p-loggables v0.1.0 github.com/libp2p/go-libp2p-mplex v0.2.3 diff --git a/go.sum b/go.sum index 8ef94f4f697..c25ecb2cd68 100644 --- a/go.sum +++ b/go.sum @@ -637,6 +637,8 @@ github.com/libp2p/go-libp2p-interface-connmgr v0.0.5/go.mod h1:GarlRLH0LdeWcLnYM github.com/libp2p/go-libp2p-interface-pnet v0.0.1/go.mod h1:el9jHpQAXK5dnTpKA4yfCNBZXvrzdOU75zz+C6ryp3k= github.com/libp2p/go-libp2p-kad-dht v0.8.0 h1:vfqDTa/dUlNVlK3nL1b9pTMV9/hhIr4sDjUVk0VEtFQ= github.com/libp2p/go-libp2p-kad-dht v0.8.0/go.mod h1:u3rbYbp3CSraAHD5s81CJ3hHozKTud/UOXfAgh93Gek= +github.com/libp2p/go-libp2p-kad-dht v0.8.1 h1:PS/mgLSzFqH5lS3PnnxcqsIrHy+qbQ5GkhzcrT12LyA= +github.com/libp2p/go-libp2p-kad-dht v0.8.1/go.mod h1:u3rbYbp3CSraAHD5s81CJ3hHozKTud/UOXfAgh93Gek= github.com/libp2p/go-libp2p-kbucket v0.4.2 h1:wg+VPpCtY61bCasGRexCuXOmEmdKjN+k1w+JtTwu9gA= github.com/libp2p/go-libp2p-kbucket v0.4.2/go.mod h1:7sCeZx2GkNK1S6lQnGUW5JYZCFPnXzAZCCBBS70lytY= github.com/libp2p/go-libp2p-loggables v0.0.1/go.mod h1:lDipDlBNYbpyqyPX/KcoO+eq0sJYEVR2JgOexcivchg= From e456462414f4f8b50decda3d48643e77409bb92b Mon Sep 17 00:00:00 2001 From: Jessica Schilling Date: Thu, 28 May 2020 11:10:39 -0600 Subject: [PATCH 011/144] Change docs-beta link to its final path --- .github/ISSUE_TEMPLATE/config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 594bbf8b898..e5afd8ffa56 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -10,7 +10,7 @@ contact_links: url: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md about: Documentation on Private Networks, Filestore and other experimental features. - name: HTTP API Reference - url: https://docs-beta.ipfs.io/reference/http/api/#api-v0-add + url: https://docs.ipfs.io/reference/http/api/#api-v0-add about: Documentation of all go-ipfs HTTP API endpoints. - name: IPFS Official Forum url: https://discuss.ipfs.io From 766023311a7084d56cc336f0a65add4a0457b8e6 Mon Sep 17 00:00:00 2001 From: Jessica Schilling Date: Thu, 28 May 2020 11:14:00 -0600 Subject: [PATCH 012/144] Update docs-beta links to final URLs --- docs/config.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/config.md b/docs/config.md index ac5e01682c1..36560f8a0ef 100644 --- a/docs/config.md +++ b/docs/config.md @@ -510,7 +510,7 @@ Default: `[]` A boolean to configure whether the gateway at the hostname provides [Origin isolation](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) between content roots. -- `true` - enables [subdomain gateway](#https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://*.{hostname}/` +- `true` - enables [subdomain gateway](#https://docs.ipfs.io/how-to/address-ipfs-on-web/#http-gateways) at `http://*.{hostname}/` - **Requires whitelist:** make sure respective `Paths` are set. For example, `Paths: ["/ipfs", "/ipns"]` are required for `http://{cid}.ipfs.{hostname}` and `http://{foo}.ipns.{hostname}` to work: ```json @@ -526,7 +526,7 @@ between content roots. - **Backward-compatible:** requests for content paths such as `http://{hostname}/ipfs/{cid}` produce redirect to `http://{cid}.ipfs.{hostname}` - **API:** if `/api` is on the `Paths` whitelist, `http://{hostname}/api/{cmd}` produces redirect to `http://api.{hostname}/api/{cmd}` -- `false` - enables [path gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://{hostname}/*` +- `false` - enables [path gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://{hostname}/*` - Example: ```json "Gateway": { @@ -580,7 +580,7 @@ $ ipfs config --json Gateway.PublicGateways '{"localhost": null }' Below is a list of the most common public gateway setups. -* Public [subdomain gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://{cid}.ipfs.dweb.link` (each content root gets its own Origin) +* Public [subdomain gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://{cid}.ipfs.dweb.link` (each content root gets its own Origin) ```console $ ipfs config --json Gateway.PublicGateways '{ "dweb.link": { @@ -594,7 +594,7 @@ Below is a list of the most common public gateway setups. **Note II:** if you run go-ipfs behind a reverse proxy that provides TLS, make it adds a `X-Forwarded-Proto: https` HTTP header to ensure users are redirected to `https://`, not `http://`. The NGINX directive is `proxy_set_header X-Forwarded-Proto "https";`.: `https://dweb.link/ipfs/{cid}` → `https://{cid}.ipfs.dweb.link` -* Public [path gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://ipfs.io/ipfs/{cid}` (no Origin separation) +* Public [path gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#path-gateway) at `http://ipfs.io/ipfs/{cid}` (no Origin separation) ```console $ ipfs config --json Gateway.PublicGateways '{ "ipfs.io": { @@ -610,7 +610,7 @@ Below is a list of the most common public gateway setups. ``` * Note that `NoDNSLink: false` is the default (it works out of the box unless set to `true` manually) -* Hardened, site-specific [DNSLink gateway](https://docs-beta.ipfs.io/how-to/address-ipfs-on-web/#dnslink-gateway). +* Hardened, site-specific [DNSLink gateway](https://docs.ipfs.io/how-to/address-ipfs-on-web/#dnslink-gateway). Disable fetching of remote data (`NoFetch: true`) and resolving DNSLink at unknown hostnames (`NoDNSLink: true`). Then, enable DNSLink gateway only for the specific hostname (for which data From c86244ecf5a403e565800fa2607a7f391ce55492 Mon Sep 17 00:00:00 2001 From: Jessica Schilling Date: Thu, 28 May 2020 11:15:09 -0600 Subject: [PATCH 013/144] Update docs/config.md --- docs/config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/config.md b/docs/config.md index 36560f8a0ef..05d26c63f9c 100644 --- a/docs/config.md +++ b/docs/config.md @@ -510,7 +510,7 @@ Default: `[]` A boolean to configure whether the gateway at the hostname provides [Origin isolation](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) between content roots. -- `true` - enables [subdomain gateway](#https://docs.ipfs.io/how-to/address-ipfs-on-web/#http-gateways) at `http://*.{hostname}/` +- `true` - enables [subdomain gateway](#https://docs.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway) at `http://*.{hostname}/` - **Requires whitelist:** make sure respective `Paths` are set. For example, `Paths: ["/ipfs", "/ipns"]` are required for `http://{cid}.ipfs.{hostname}` and `http://{foo}.ipns.{hostname}` to work: ```json From 0f76ed79f97de7d2c886385ada0030b2e8ee42c8 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Thu, 28 May 2020 23:04:35 +0200 Subject: [PATCH 014/144] feat: webui v2.9.0 --- core/corehttp/webui.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/corehttp/webui.go b/core/corehttp/webui.go index 22143e49a41..f94df7850f0 100644 --- a/core/corehttp/webui.go +++ b/core/corehttp/webui.go @@ -1,11 +1,12 @@ package corehttp // TODO: move to IPNS -const WebUIPath = "/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i" +const WebUIPath = "/ipfs/bafybeigkbbjnltbd4ewfj7elajsbnjwinyk6tiilczkqsibf3o7dcr6nn4" // v2.9.0 // this is a list of all past webUI paths. var WebUIPaths = []string{ WebUIPath, + "/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i", "/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq", "/ipfs/QmfQkD8pBSBCBxWEwFSu4XaDVSWK6bjnNuaWZjMyQbyDub", "/ipfs/QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ", From 62f61c588d75ceafa1974f0584ffbf35950be2ca Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Thu, 28 May 2020 17:12:00 -0700 Subject: [PATCH 015/144] fix: use bitswap sessions for ipfs refs This isn't perfect (we only use sessions after resolving the root cid) but it's better than what we have. The real solution is #7198 so we can use sessions everywhere. --- core/commands/refs.go | 22 ++++++++++++++-------- core/coreapi/dag.go | 10 ++++++++++ 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/core/commands/refs.go b/core/commands/refs.go index 4a7285cc95c..828853e38f0 100644 --- a/core/commands/refs.go +++ b/core/commands/refs.go @@ -13,6 +13,7 @@ import ( cidenc "github.com/ipfs/go-cidutil/cidenc" cmds "github.com/ipfs/go-ipfs-cmds" ipld "github.com/ipfs/go-ipld-format" + merkledag "github.com/ipfs/go-merkledag" iface "github.com/ipfs/interface-go-ipfs-core" path "github.com/ipfs/interface-go-ipfs-core/path" ) @@ -102,6 +103,7 @@ NOTE: List all references recursively by using the flag '-r'. format = " -> " } + // TODO: use session for resolving as well. objs, err := objectsForPaths(ctx, api, req.Arguments) if err != nil { return err @@ -109,7 +111,7 @@ NOTE: List all references recursively by using the flag '-r'. rw := RefWriter{ res: res, - DAG: api.Dag(), + DAG: merkledag.NewSession(ctx, api.Dag()), Ctx: ctx, Unique: unique, PrintFmt: format, @@ -164,16 +166,16 @@ Displays the hashes of all local objects. Type: RefWrapper{}, } -func objectsForPaths(ctx context.Context, n iface.CoreAPI, paths []string) ([]ipld.Node, error) { - objects := make([]ipld.Node, len(paths)) +func objectsForPaths(ctx context.Context, n iface.CoreAPI, paths []string) ([]cid.Cid, error) { + roots := make([]cid.Cid, len(paths)) for i, sp := range paths { - o, err := n.ResolveNode(ctx, path.New(sp)) + o, err := n.ResolvePath(ctx, path.New(sp)) if err != nil { return nil, err } - objects[i] = o + roots[i] = o.Cid() } - return objects, nil + return roots, nil } type RefWrapper struct { @@ -183,7 +185,7 @@ type RefWrapper struct { type RefWriter struct { res cmds.ResponseEmitter - DAG ipld.DAGService + DAG ipld.NodeGetter Ctx context.Context Unique bool @@ -194,7 +196,11 @@ type RefWriter struct { } // WriteRefs writes refs of the given object to the underlying writer. -func (rw *RefWriter) WriteRefs(n ipld.Node, enc cidenc.Encoder) (int, error) { +func (rw *RefWriter) WriteRefs(c cid.Cid, enc cidenc.Encoder) (int, error) { + n, err := rw.DAG.Get(rw.Ctx, c) + if err != nil { + return 0, err + } return rw.writeRefsRecursive(n, 0, enc) } diff --git a/core/coreapi/dag.go b/core/coreapi/dag.go index ecc4a7408bb..7994842b8e5 100644 --- a/core/coreapi/dag.go +++ b/core/coreapi/dag.go @@ -6,6 +6,7 @@ import ( cid "github.com/ipfs/go-cid" "github.com/ipfs/go-ipfs-pinner" ipld "github.com/ipfs/go-ipld-format" + dag "github.com/ipfs/go-merkledag" ) type dagAPI struct { @@ -50,3 +51,12 @@ func (adder *pinningAdder) AddMany(ctx context.Context, nds []ipld.Node) error { func (api *dagAPI) Pinning() ipld.NodeAdder { return (*pinningAdder)(api.core) } + +func (api *dagAPI) Session(ctx context.Context) ipld.NodeGetter { + return dag.NewSession(ctx, api.DAGService) +} + +var ( + _ ipld.DAGService = (*dagAPI)(nil) + _ dag.SessionMaker = (*dagAPI)(nil) +) From a773e890d8f1dad2d9d4f3d33fe36a7b599dc477 Mon Sep 17 00:00:00 2001 From: "dependabot-preview[bot]" <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Fri, 29 May 2020 11:27:31 +0000 Subject: [PATCH 016/144] chore(deps): bump github.com/stretchr/testify from 1.5.1 to 1.6.0 Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.5.1 to 1.6.0. - [Release notes](https://github.com/stretchr/testify/releases) - [Commits](https://github.com/stretchr/testify/compare/v1.5.1...v1.6.0) Signed-off-by: dependabot-preview[bot] --- go.mod | 2 +- go.sum | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/go.mod b/go.mod index f30d8b217e9..13bfd56c3d7 100644 --- a/go.mod +++ b/go.mod @@ -95,7 +95,7 @@ require ( github.com/opentracing/opentracing-go v1.1.0 github.com/pkg/errors v0.9.1 github.com/prometheus/client_golang v1.6.0 - github.com/stretchr/testify v1.5.1 + github.com/stretchr/testify v1.6.0 github.com/syndtr/goleveldb v1.0.0 github.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc github.com/whyrusleeping/go-sysinfo v0.0.0-20190219211824-4a357d4b90b1 diff --git a/go.sum b/go.sum index 8ef94f4f697..ddabebf7ee4 100644 --- a/go.sum +++ b/go.sum @@ -1090,6 +1090,7 @@ github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnIn github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s= github.com/src-d/envconfig v1.0.0/go.mod h1:Q9YQZ7BKITldTBnoxsE5gOeB5y66RyPXeue/R4aaNBc= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= @@ -1098,6 +1099,8 @@ github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJy github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= +github.com/stretchr/testify v1.6.0 h1:jlIyCplCJFULU/01vCkhKuTyc3OorI3bJFuw6obfgho= +github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE= github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ= github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA= @@ -1477,6 +1480,8 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.5 h1:ymVxjfMaHvXD8RqPRmzHHsB3VvucivSkIAvJFDI5O3c= gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= grpc.go4.org v0.0.0-20170609214715-11d0a25b4919/go.mod h1:77eQGdRu53HpSqPFJFmuJdjuHRquDANNeA4x7B8WQ9o= honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= From 4e14582c56f6ef90f4b6abcf5234a0ca6a72a064 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Fri, 29 May 2020 16:10:14 -0700 Subject: [PATCH 017/144] fix: update node to fix mocha tests --- .circleci/config.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 754bf98e602..ae52998aafd 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -44,7 +44,7 @@ executors: TEST_VERBOSE: 1 node: docker: - - image: circleci/node:10 + - image: circleci/node:12 working_directory: ~/ipfs/go-ipfs environment: <<: *default_environment @@ -191,15 +191,14 @@ jobs: - run: name: Installing reporting tools command: | - npm install --save-dev mocha-junit-reporter@1.23.1 mocha-multi-reporters@1.1.0 + npm install --save-dev mocha-circleci-reporter@0.0.3 working_directory: ~/ipfs/go-ipfs/interop - run: name: Running tests command: | mkdir -p /tmp/test-results/interop/ export MOCHA_FILE="$(mktemp /tmp/test-results/interop/unit.XXXXXX.xml)" - echo '{"reporterEnabled": "mocha-junit-reporter, spec"}' > mocha-conf.json - node_modules/.bin/mocha -R mocha-multi-reporters --reporter-options "configFile=mocha-conf.json" \ + node_modules/.bin/mocha --reporter mocha-circleci-reporter \ $(sed -n -e "s|^require('\(.*\)')$|test/\1|p" test/node.js | circleci tests split) working_directory: ~/ipfs/go-ipfs/interop environment: From ac8a88d054e38f396a781cd938d724b074d6312b Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Fri, 29 May 2020 15:52:59 -0700 Subject: [PATCH 018/144] fix(pubsub): flood publish When publishing a value, flood. This ensures that our messages make it to the network even if some of our peers are bad and/or overloaded. --- core/node/libp2p/pubsub.go | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/core/node/libp2p/pubsub.go b/core/node/libp2p/pubsub.go index 001a787da71..e8b3e57fbda 100644 --- a/core/node/libp2p/pubsub.go +++ b/core/node/libp2p/pubsub.go @@ -17,6 +17,10 @@ func FloodSub(pubsubOptions ...pubsub.Option) interface{} { func GossipSub(pubsubOptions ...pubsub.Option) interface{} { return func(mctx helpers.MetricsCtx, lc fx.Lifecycle, host host.Host, disc discovery.Discovery) (service *pubsub.PubSub, err error) { - return pubsub.NewGossipSub(helpers.LifecycleCtx(mctx, lc), host, append(pubsubOptions, pubsub.WithDiscovery(disc))...) + return pubsub.NewGossipSub(helpers.LifecycleCtx(mctx, lc), host, append( + pubsubOptions, + pubsub.WithDiscovery(disc), + pubsub.WithFloodPublish(true))..., + ) } } From c58e3e4c7d400c238846b42376e52c555b44229b Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Fri, 29 May 2020 15:54:30 -0700 Subject: [PATCH 019/144] fix: remove pubsub discovery hack Pubsub handles this internally now. --- core/commands/pubsub.go | 6 ++--- core/coreapi/coreapi.go | 3 --- core/coreapi/pubsub.go | 59 +++++------------------------------------ 3 files changed, 9 insertions(+), 59 deletions(-) diff --git a/core/commands/pubsub.go b/core/commands/pubsub.go index aaf01672b26..10bddcd57be 100644 --- a/core/commands/pubsub.go +++ b/core/commands/pubsub.go @@ -74,7 +74,7 @@ This command outputs data in the following encodings: cmds.StringArg("topic", true, false, "String name of topic to subscribe to."), }, Options: []cmds.Option{ - cmds.BoolOption(pubsubDiscoverOptionName, "try to discover other peers subscribed to the same topic"), + cmds.BoolOption(pubsubDiscoverOptionName, "Deprecated option to instruct pubsub to discovery peers for the topic. Discovery is now built into pubsub."), }, Run: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error { api, err := cmdenv.GetApi(env, req) @@ -83,9 +83,7 @@ This command outputs data in the following encodings: } topic := req.Arguments[0] - discover, _ := req.Options[pubsubDiscoverOptionName].(bool) - - sub, err := api.PubSub().Subscribe(req.Context, topic, options.PubSub.Discover(discover)) + sub, err := api.PubSub().Subscribe(req.Context, topic) if err != nil { return err } diff --git a/core/coreapi/coreapi.go b/core/coreapi/coreapi.go index 3235b7d3966..5b638826b45 100644 --- a/core/coreapi/coreapi.go +++ b/core/coreapi/coreapi.go @@ -26,7 +26,6 @@ import ( "github.com/ipfs/go-ipfs-provider" offlineroute "github.com/ipfs/go-ipfs-routing/offline" ipld "github.com/ipfs/go-ipld-format" - logging "github.com/ipfs/go-log" dag "github.com/ipfs/go-merkledag" coreiface "github.com/ipfs/interface-go-ipfs-core" "github.com/ipfs/interface-go-ipfs-core/options" @@ -44,8 +43,6 @@ import ( "github.com/ipfs/go-ipfs/repo" ) -var log = logging.Logger("core/coreapi") - type CoreAPI struct { nctx context.Context diff --git a/core/coreapi/pubsub.go b/core/coreapi/pubsub.go index de2ce467293..e84d1503481 100644 --- a/core/coreapi/pubsub.go +++ b/core/coreapi/pubsub.go @@ -3,14 +3,9 @@ package coreapi import ( "context" "errors" - "strings" - "sync" - "time" - cid "github.com/ipfs/go-cid" coreiface "github.com/ipfs/interface-go-ipfs-core" caopts "github.com/ipfs/interface-go-ipfs-core/options" - p2phost "github.com/libp2p/go-libp2p-core/host" peer "github.com/libp2p/go-libp2p-core/peer" routing "github.com/libp2p/go-libp2p-core/routing" pubsub "github.com/libp2p/go-libp2p-pubsub" @@ -19,7 +14,6 @@ import ( type PubSubAPI CoreAPI type pubSubSubscription struct { - cancel context.CancelFunc subscription *pubsub.Subscription } @@ -61,12 +55,16 @@ func (api *PubSubAPI) Publish(ctx context.Context, topic string, data []byte) er } func (api *PubSubAPI) Subscribe(ctx context.Context, topic string, opts ...caopts.PubSubSubscribeOption) (coreiface.PubSubSubscription, error) { - options, err := caopts.PubSubSubscribeOptions(opts...) + // Parse the options to avoid introducing silent failures for invalid + // options. However, we don't currently have any use for them. The only + // subscription option, discovery, is now a no-op as it's handled by + // pubsub itself. + _, err := caopts.PubSubSubscribeOptions(opts...) if err != nil { return nil, err } - r, err := api.checkNode() + _, err = api.checkNode() if err != nil { return nil, err } @@ -77,45 +75,7 @@ func (api *PubSubAPI) Subscribe(ctx context.Context, topic string, opts ...caopt return nil, err } - pubctx, cancel := context.WithCancel(api.nctx) - - if options.Discover { - go func() { - blk, err := api.core().Block().Put(pubctx, strings.NewReader("floodsub:"+topic)) - if err != nil { - log.Error("pubsub discovery: ", err) - return - } - - connectToPubSubPeers(pubctx, r, api.peerHost, blk.Path().Cid()) - }() - } - - return &pubSubSubscription{cancel, sub}, nil -} - -func connectToPubSubPeers(ctx context.Context, r routing.Routing, ph p2phost.Host, cid cid.Cid) { - ctx, cancel := context.WithCancel(ctx) - defer cancel() - - provs := r.FindProvidersAsync(ctx, cid, 10) - var wg sync.WaitGroup - for p := range provs { - wg.Add(1) - go func(pi peer.AddrInfo) { - defer wg.Done() - ctx, cancel := context.WithTimeout(ctx, time.Second*10) - defer cancel() - err := ph.Connect(ctx, pi) - if err != nil { - log.Info("pubsub discover: ", err) - return - } - log.Info("connected to pubsub peer:", pi.ID) - }(p) - } - - wg.Wait() + return &pubSubSubscription{sub}, nil } func (api *PubSubAPI) checkNode() (routing.Routing, error) { @@ -132,7 +92,6 @@ func (api *PubSubAPI) checkNode() (routing.Routing, error) { } func (sub *pubSubSubscription) Close() error { - sub.cancel() sub.subscription.Cancel() return nil } @@ -161,7 +120,3 @@ func (msg *pubSubMessage) Seq() []byte { func (msg *pubSubMessage) Topics() []string { return msg.msg.TopicIDs } - -func (api *PubSubAPI) core() coreiface.CoreAPI { - return (*CoreAPI)(api) -} From 3aa86f8ed9602b86ed65ae4b9ddca3a97e1471e7 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Fri, 29 May 2020 18:47:21 -0700 Subject: [PATCH 020/144] fix(commands): print consistent addresses in ipfs id Consistently append `/p2p/QmMyId` to addresses when calling `ipfs id Me` and `ipfs id NotMe`. Fixes https://github.com/ipfs/go-ipfs/issues/7378. --- core/commands/id.go | 8 +++++++- test/sharness/t0140-swarm.sh | 11 +++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/core/commands/id.go b/core/commands/id.go index dc4363e64c9..ae9d97d52b3 100644 --- a/core/commands/id.go +++ b/core/commands/id.go @@ -154,7 +154,13 @@ func printPeer(ps pstore.Peerstore, p peer.ID) (interface{}, error) { info.PublicKey = base64.StdEncoding.EncodeToString(pkb) } - for _, a := range ps.Addrs(p) { + addrInfo := ps.PeerInfo(p) + addrs, err := peer.AddrInfoToP2pAddrs(&addrInfo) + if err != nil { + return nil, err + } + + for _, a := range addrs { info.Addresses = append(info.Addresses, a.String()) } diff --git a/test/sharness/t0140-swarm.sh b/test/sharness/t0140-swarm.sh index 77e2af1b00f..1468c560abf 100755 --- a/test/sharness/t0140-swarm.sh +++ b/test/sharness/t0140-swarm.sh @@ -123,6 +123,17 @@ test_expect_success "/p2p addresses work" ' [ $(ipfsi 0 swarm peers | wc -l) -eq 1 ] ' +test_expect_success "ipfs id is consistent for node 0" ' + ipfsi 1 id "$(iptb attr get 0 id)" > 1see0 && + ipfsi 0 id > 0see0 && + test_cmp 1see0 0see0 +' +test_expect_success "ipfs id is consistent for node 1" ' + ipfsi 0 id "$(iptb attr get 1 id)" > 0see1 && + ipfsi 1 id > 1see1 && + test_cmp 0see1 1see1 +' + test_expect_success "stopping cluster" ' iptb stop ' From 0865634f7535db3585739c69a848ba231af98c0d Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Mon, 1 Jun 2020 14:46:35 -0700 Subject: [PATCH 021/144] chore: update quic transport This is a _protocol breaking_ change for the QUIC transport, the last one before we stabilize it. --- go.mod | 2 +- go.sum | 10 ++++------ 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/go.mod b/go.mod index 9f220d4ffc9..55fe682a7ac 100644 --- a/go.mod +++ b/go.mod @@ -74,7 +74,7 @@ require ( github.com/libp2p/go-libp2p-peerstore v0.2.4 github.com/libp2p/go-libp2p-pubsub v0.3.1 github.com/libp2p/go-libp2p-pubsub-router v0.3.0 - github.com/libp2p/go-libp2p-quic-transport v0.4.1 + github.com/libp2p/go-libp2p-quic-transport v0.5.0 github.com/libp2p/go-libp2p-record v0.1.3 github.com/libp2p/go-libp2p-routing-helpers v0.2.3 github.com/libp2p/go-libp2p-secio v0.2.2 diff --git a/go.sum b/go.sum index c25ecb2cd68..08de80934a3 100644 --- a/go.sum +++ b/go.sum @@ -635,8 +635,6 @@ github.com/libp2p/go-libp2p-interface-connmgr v0.0.1/go.mod h1:GarlRLH0LdeWcLnYM github.com/libp2p/go-libp2p-interface-connmgr v0.0.4/go.mod h1:GarlRLH0LdeWcLnYM/SaBykKFl9U5JFnbBGruAk/D5k= github.com/libp2p/go-libp2p-interface-connmgr v0.0.5/go.mod h1:GarlRLH0LdeWcLnYM/SaBykKFl9U5JFnbBGruAk/D5k= github.com/libp2p/go-libp2p-interface-pnet v0.0.1/go.mod h1:el9jHpQAXK5dnTpKA4yfCNBZXvrzdOU75zz+C6ryp3k= -github.com/libp2p/go-libp2p-kad-dht v0.8.0 h1:vfqDTa/dUlNVlK3nL1b9pTMV9/hhIr4sDjUVk0VEtFQ= -github.com/libp2p/go-libp2p-kad-dht v0.8.0/go.mod h1:u3rbYbp3CSraAHD5s81CJ3hHozKTud/UOXfAgh93Gek= github.com/libp2p/go-libp2p-kad-dht v0.8.1 h1:PS/mgLSzFqH5lS3PnnxcqsIrHy+qbQ5GkhzcrT12LyA= github.com/libp2p/go-libp2p-kad-dht v0.8.1/go.mod h1:u3rbYbp3CSraAHD5s81CJ3hHozKTud/UOXfAgh93Gek= github.com/libp2p/go-libp2p-kbucket v0.4.2 h1:wg+VPpCtY61bCasGRexCuXOmEmdKjN+k1w+JtTwu9gA= @@ -700,8 +698,8 @@ github.com/libp2p/go-libp2p-pubsub v0.3.1/go.mod h1:TxPOBuo1FPdsTjFnv+FGZbNbWYsp github.com/libp2p/go-libp2p-pubsub-router v0.3.0 h1:ghpHApTMXN+aZ+InYvpJa/ckBW4orypzNI0aWQDth3s= github.com/libp2p/go-libp2p-pubsub-router v0.3.0/go.mod h1:6kZb1gGV1yGzXTfyNsi4p+hyt1JnA1OMGHeExTOJR3A= github.com/libp2p/go-libp2p-quic-transport v0.3.7/go.mod h1:Kr4aDtnfHHNeENn5J+sZIVc+t8HpQn9W6BOxhVGHbgI= -github.com/libp2p/go-libp2p-quic-transport v0.4.1 h1:RyBy+0doNHYBgK8lKe7w9rXOx23mLiGJJNUp8GDGVkw= -github.com/libp2p/go-libp2p-quic-transport v0.4.1/go.mod h1:uDr5c1JgdqAfczV1lgiREnud0cr+k3rJcI3Eov/d7C8= +github.com/libp2p/go-libp2p-quic-transport v0.5.0 h1:BUN1lgYNUrtv4WLLQ5rQmC9MCJ6uEXusezGvYRNoJXE= +github.com/libp2p/go-libp2p-quic-transport v0.5.0/go.mod h1:IEcuC5MLxvZ5KuHKjRu+dr3LjCT1Be3rcD/4d8JrX8M= github.com/libp2p/go-libp2p-record v0.0.1/go.mod h1:grzqg263Rug/sRex85QrDOLntdFAymLDLm7lxMgU79Q= github.com/libp2p/go-libp2p-record v0.1.0/go.mod h1:ujNc8iuE5dlKWVy6wuL6dd58t0n7xI4hAIl8pE6wu5Q= github.com/libp2p/go-libp2p-record v0.1.2 h1:M50VKzWnmUrk/M5/Dz99qO9Xh4vs8ijsK+7HkJvRP+0= @@ -855,8 +853,8 @@ github.com/libp2p/go-yamux v1.3.7 h1:v40A1eSPJDIZwz2AvrV3cxpTZEGDP11QJbukmEhYyQI github.com/libp2p/go-yamux v1.3.7/go.mod h1:fr7aVgmdNGJK+N1g+b6DW6VxzbRCjCOejR/hkmpooHE= github.com/lucas-clemente/quic-go v0.15.7 h1:Pu7To5/G9JoP1mwlrcIvfV8ByPBlCzif3MCl8+1W83I= github.com/lucas-clemente/quic-go v0.15.7/go.mod h1:Myi1OyS0FOjL3not4BxT7KN29bRkcMUV5JVVFLKtDp8= -github.com/lucas-clemente/quic-go v0.15.8 h1:LkU5Fi+RswEOWIoHFDGkMRmmp4JLBZNGVGxjePso66s= -github.com/lucas-clemente/quic-go v0.15.8/go.mod h1:I0+fcNTdb9eS1ZcjQZbDVPGchJ86chcIxPALn9lEJqE= +github.com/lucas-clemente/quic-go v0.16.0 h1:jJw36wfzGJhmOhAOaOC2lS36WgeqXQszH47A7spo1LI= +github.com/lucas-clemente/quic-go v0.16.0/go.mod h1:I0+fcNTdb9eS1ZcjQZbDVPGchJ86chcIxPALn9lEJqE= github.com/lunixbochs/vtclean v1.0.0/go.mod h1:pHhQNgMf3btfWnGBVipUOjRYhoOsdGqdm/+2c2E2WMI= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329 h1:2gxZ0XQIU/5z3Z3bUBu+FXuk2pFbkN6tcwi/pjyaDic= From 8bd2b36fe1a66c63c81ba4f77c77ec5398c8c2f1 Mon Sep 17 00:00:00 2001 From: "dependabot-preview[bot]" <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2020 11:27:23 +0000 Subject: [PATCH 022/144] chore(deps): bump github.com/coreos/go-systemd/v22 from 22.0.0 to 22.1.0 Bumps [github.com/coreos/go-systemd/v22](https://github.com/coreos/go-systemd) from 22.0.0 to 22.1.0. - [Release notes](https://github.com/coreos/go-systemd/releases) - [Commits](https://github.com/coreos/go-systemd/compare/v22.0.0...v22.1.0) Signed-off-by: dependabot-preview[bot] --- go.mod | 2 +- go.sum | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/go.mod b/go.mod index 55fe682a7ac..ad928abb7f5 100644 --- a/go.mod +++ b/go.mod @@ -5,7 +5,7 @@ require ( github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect github.com/blang/semver v3.5.1+incompatible github.com/bren2010/proquint v0.0.0-20160323162903-38337c27106d - github.com/coreos/go-systemd/v22 v22.0.0 + github.com/coreos/go-systemd/v22 v22.1.0 github.com/dustin/go-humanize v1.0.0 github.com/elgris/jsondiff v0.0.0-20160530203242-765b5c24c302 github.com/fatih/color v1.9.0 // indirect diff --git a/go.sum b/go.sum index 08de80934a3..364f65efc97 100644 --- a/go.sum +++ b/go.sum @@ -100,6 +100,8 @@ github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d h1:t5Wuyh53qYyg9 github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd/v22 v22.0.0 h1:XJIw/+VlJ+87J+doOxznsAWIdmWuViOVhkQamW5YV28= github.com/coreos/go-systemd/v22 v22.0.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= +github.com/coreos/go-systemd/v22 v22.1.0 h1:kq/SbG2BCKLkDKkjQf5OWwKWUKj1lgs3lFI4PxnR5lg= +github.com/coreos/go-systemd/v22 v22.1.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE= github.com/crackcomm/go-gitignore v0.0.0-20170627025303-887ab5e44cc3 h1:HVTnpeuvF6Owjd5mniCL8DEXo7uYXdQEmOP4FJbV5tg= github.com/crackcomm/go-gitignore v0.0.0-20170627025303-887ab5e44cc3/go.mod h1:p1d6YEZWvFzEh4KLyvBcVSnrfNDDvK2zfK/4x2v/4pE= From 86c30d6b5913519c442163572d42ae44fb659fe3 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 2 Jun 2020 09:26:06 -0700 Subject: [PATCH 023/144] test(sharness): ensure addresses end in p2p addr --- test/sharness/t0140-swarm.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/test/sharness/t0140-swarm.sh b/test/sharness/t0140-swarm.sh index 1468c560abf..352467136df 100755 --- a/test/sharness/t0140-swarm.sh +++ b/test/sharness/t0140-swarm.sh @@ -128,12 +128,20 @@ test_expect_success "ipfs id is consistent for node 0" ' ipfsi 0 id > 0see0 && test_cmp 1see0 0see0 ' + test_expect_success "ipfs id is consistent for node 1" ' ipfsi 0 id "$(iptb attr get 1 id)" > 0see1 && ipfsi 1 id > 1see1 && test_cmp 0see1 1see1 ' +test_expect_success "addresses contain /p2p/..." ' + test_should_contain "/p2p/$(iptb attr get 1 id)\"" 0see1 && + test_should_contain "/p2p/$(iptb attr get 1 id)\"" 1see1 && + test_should_contain "/p2p/$(iptb attr get 0 id)\"" 1see0 && + test_should_contain "/p2p/$(iptb attr get 0 id)\"" 0see0 +' + test_expect_success "stopping cluster" ' iptb stop ' From 179bed8f606cfed950ef5322792fcc72ddb8b590 Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Fri, 5 Jun 2020 03:15:57 +0200 Subject: [PATCH 024/144] go: bump minimal dependency to 1.14.4 due to major bugfixes in the .4 release --- .circleci/config.yml | 4 ++-- Dockerfile | 2 +- mk/golang.mk | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index ae52998aafd..cfeeb88f99c 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -33,7 +33,7 @@ default_environment: &default_environment executors: golang: docker: - - image: circleci/golang:1.14.2 + - image: circleci/golang:1.14.4 working_directory: ~/ipfs/go-ipfs environment: <<: *default_environment @@ -60,7 +60,7 @@ executors: E2E_IPFSD_TYPE: go dockerizer: docker: - - image: circleci/golang:1.14.2 + - image: circleci/golang:1.14.4 environment: IMAGE_NAME: ipfs/go-ipfs WIP_IMAGE_TAG: wip diff --git a/Dockerfile b/Dockerfile index 5cd7d54b12d..b210764e327 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM golang:1.14.2-buster +FROM golang:1.14.4-buster LABEL maintainer="Steven Allen " # Install deps diff --git a/mk/golang.mk b/mk/golang.mk index 36d40ea6f24..73073370194 100644 --- a/mk/golang.mk +++ b/mk/golang.mk @@ -1,5 +1,5 @@ # golang utilities -GO_MIN_VERSION = 1.14.2 +GO_MIN_VERSION = 1.14.4 export GO111MODULE=on From cb196d24955ee89519fcb7d2878bccddfff1c8e3 Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Fri, 5 Jun 2020 03:16:49 +0200 Subject: [PATCH 025/144] README.md: documentation update, requires at least go 1.14.4 now --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 00548dbb0e4..68398327b11 100644 --- a/README.md +++ b/README.md @@ -170,7 +170,7 @@ To build without GCC, build with `CGO_ENABLED=0` (e.g., `make build CGO_ENABLED= #### Install Go -The build process for ipfs requires Go 1.14.2 or higher. If you don't have it: [Download Go 1.14+](https://golang.org/dl/). +The build process for ipfs requires Go 1.14.4 or higher. If you don't have it: [Download Go 1.14+](https://golang.org/dl/). You'll need to add Go's bin directories to your `$PATH` environment variable e.g., by adding these lines to your `/etc/profile` (for a system-wide installation) or `$HOME/.profile`: From 3119f85e72f0532565605f4b64cd1c284ded54c7 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Mon, 8 Jun 2020 09:20:38 -0700 Subject: [PATCH 026/144] chore: add migration to listen on QUIC by default --- repo/fsrepo/fsrepo.go | 2 +- repo/fsrepo/migrations/migrations.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/repo/fsrepo/fsrepo.go b/repo/fsrepo/fsrepo.go index 103597feac1..b4292708a59 100644 --- a/repo/fsrepo/fsrepo.go +++ b/repo/fsrepo/fsrepo.go @@ -36,7 +36,7 @@ const LockFile = "repo.lock" var log = logging.Logger("fsrepo") // version number that we are currently expecting to see -var RepoVersion = 9 +var RepoVersion = 10 var migrationInstructions = `See https://github.com/ipfs/fs-repo-migrations/blob/master/run.md Sorry for the inconvenience. In the future, these will run automatically.` diff --git a/repo/fsrepo/migrations/migrations.go b/repo/fsrepo/migrations/migrations.go index c7cec945ea1..890c8ae6821 100644 --- a/repo/fsrepo/migrations/migrations.go +++ b/repo/fsrepo/migrations/migrations.go @@ -15,7 +15,7 @@ import ( "strings" ) -var DistPath = "https://ipfs.io/ipfs/QmXuKUDChMXkcq36CbaggvY3UaAtgpi7yyoAiNy5VD9Dfr" +var DistPath = "https://ipfs.io/ipfs/QmRnvRrRwto4QAMuwWeEU9GpNaHKqQPdvdrNeGaXg7yruC" func init() { if dist := os.Getenv("IPFS_DIST_PATH"); dist != "" { From f01c9625f337661b8de9b4e8db03bb5171e7d920 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 9 Jun 2020 08:53:50 -0700 Subject: [PATCH 027/144] fix(migration): correctly migrate quic addresses 1. Don't add quic addresses when they're already present (correctly detect quic addresses). 2. Correctly distinguish between quic addresses and quic + relay addresses. 3. Only add quic addresses to mirror tcp addresses, not tcp/ws addresses. fixes #7444 --- repo/fsrepo/migrations/migrations.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repo/fsrepo/migrations/migrations.go b/repo/fsrepo/migrations/migrations.go index 890c8ae6821..6bd4ae3a10c 100644 --- a/repo/fsrepo/migrations/migrations.go +++ b/repo/fsrepo/migrations/migrations.go @@ -15,7 +15,7 @@ import ( "strings" ) -var DistPath = "https://ipfs.io/ipfs/QmRnvRrRwto4QAMuwWeEU9GpNaHKqQPdvdrNeGaXg7yruC" +var DistPath = "https://ipfs.io/ipfs/QmQWvUrSwGHWKaoJnUrLQoBy9ikfiiyRrXKBFSHUTcS6aM" func init() { if dist := os.Getenv("IPFS_DIST_PATH"); dist != "" { From b0c359d53853a00f63f8a70b18334349401ce65c Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Tue, 9 Jun 2020 18:20:23 +0200 Subject: [PATCH 028/144] readme: update go-version --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 68398327b11..b2ccb56d745 100644 --- a/README.md +++ b/README.md @@ -160,7 +160,7 @@ PS> scoop install go-ipfs ### Build from Source -go-ipfs's build system requires Go 1.13 and some standard POSIX build tools: +go-ipfs's build system requires Go 1.14.4 and some standard POSIX build tools: * GNU make * Git From ed1e54ebf8e4b52f34471de4ce7f40d0ba9a3240 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 9 Jun 2020 11:05:59 -0700 Subject: [PATCH 029/144] fix(migration): migrate /ipfs/ bootstrappers to /p2p/ 1. We should have done this in 7-to-8 anyways, but didn't in some cases. 2. We assume these addresses use /p2p/ in the 9-to-10 migration. --- repo/fsrepo/migrations/migrations.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repo/fsrepo/migrations/migrations.go b/repo/fsrepo/migrations/migrations.go index 6bd4ae3a10c..b099216217c 100644 --- a/repo/fsrepo/migrations/migrations.go +++ b/repo/fsrepo/migrations/migrations.go @@ -15,7 +15,7 @@ import ( "strings" ) -var DistPath = "https://ipfs.io/ipfs/QmQWvUrSwGHWKaoJnUrLQoBy9ikfiiyRrXKBFSHUTcS6aM" +var DistPath = "https://ipfs.io/ipfs/Qmdo5m6bpQXCayzfGghyvgXJdVHSsXsCKDUo9vWktDKq3K" func init() { if dist := os.Getenv("IPFS_DIST_PATH"); dist != "" { From 643699e2e81123b4287b558ad53a0bea02dea2fd Mon Sep 17 00:00:00 2001 From: Mark Gaiser Date: Thu, 11 Jun 2020 17:11:10 +0200 Subject: [PATCH 030/144] Use the -p option which makes parents as needed. --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index b210764e327..f8a823b9574 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,7 +23,7 @@ ARG IPFS_PLUGINS # Build the thing. # Also: fix getting HEAD commit hash via git rev-parse. RUN cd $SRC_DIR \ - && mkdir .git/objects \ + && mkdir -p .git/objects \ && make build GOTAGS=openssl IPFS_PLUGINS=$IPFS_PLUGINS # Get su-exec, a very minimal tool for dropping privileges, From ea55e747f253de3223ecc3b0eea5413b2198f7ed Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Thu, 11 Jun 2020 17:58:17 -0700 Subject: [PATCH 031/144] doc(config): expand peering docs --- docs/config.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/config.md b/docs/config.md index 05d26c63f9c..aa49a77de52 100644 --- a/docs/config.md +++ b/docs/config.md @@ -745,15 +745,28 @@ Peering can be asymmetric or symmetric: #### `Peering.Peers` -The set of peers with which to peer. Each entry is of the form: +The set of peers with which to peer. -```js +```json { - "ID": "QmSomePeerID", # The peers ID. - "Addrs": ["/ip4/1.2.3.4/tcp/1234"] # Known addresses for the peer. If none are specified, the DHT will be queried. + "Peering": { + "Peers": [ + { + "ID": "QmPeerID1", + "Addrs": ["/ip4/18.1.1.1/tcp/4001"] + }, + { + "ID": "QmPeerID2", + "Addrs": ["/ip4/18.1.1.2/tcp/4001", "/ip4/18.1.1.2/udp/4001/quic"] + } + ] + } + ... } ``` +Where `ID` is the peer ID and `Addrs` is a set of known addresses for the peer. If no addresses are specified, the DHT will be queried. + Additional fields may be added in the future. ## `Reprovider` From 91de6c972e8c4ccd70af13836522f7fde219c1d8 Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Fri, 12 Jun 2020 12:13:29 -0700 Subject: [PATCH 032/144] doc(prod): start documenting production stuff Hopefully, we can get the community to start pooling knowledge of how to run go-ipfs in production. --- docs/production/reverse-proxy.md | 93 ++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/production/reverse-proxy.md diff --git a/docs/production/reverse-proxy.md b/docs/production/reverse-proxy.md new file mode 100644 index 00000000000..dfb5359fa50 --- /dev/null +++ b/docs/production/reverse-proxy.md @@ -0,0 +1,93 @@ +# IPFS & Reverse HTTP Proxies + +When run in production environments, go-ipfs should generally be run behind a +reverse HTTP proxy (usually NGINX). You may need a reverse proxy to: + +* Load balance requests across multiple go-ipfs daemons. +* Cache responses. +* Buffer requests, only releasing them to go-ipfs when complete. This can help + protect go-ipfs from the + [slowloris](https://en.wikipedia.org/wiki/Slowloris_(computer_security) + attack. +* Block content. +* Rate limit and timeout requests. +* Apply QoS rules (e.g., prioritize traffic for certain important IPFS resources). +* Expose a limited subset of the HTTP API. + +This document contains a collection of tips, tricks, and pitfalls when running a +go-ipfs node behind a reverse HTTP proxy. + +## Peering + +Go-ipfs gateways behind a single load balancing reverse proxy should use the +[peering](../config.md#peering) subsystem to peer with each other. That way, as +long as one go-ipfs daemon has the content being requested, the others will be +able to serve it. + +# Garbage Collection + +Gateways rarely store content permanently. However, running garbage collection +can slow down a go-ipfs node significantly. If you've noticed this issue in +production, consider "garbage collecting" by resetting the go-ipfs repo whenever +you run out of space, instead of garbage collecting. + +1. Initialize your gateways repo to some known-good state (possibly pre-seeding + it with some content, a config, etc.). +2. When you start running low on space, for each load-balanced go-ipfs node: + 1. Use the nginx API to set one of the upstream go-ipfs node's to "down". + 2. Wait a minute to let go-ipfs finish processing any in-progress requests + (or the short-lived ones, at least). + 3. Take the go-ipfs node down. + 4. Rollback the go-ipfs repo to the seed state. + 5. Restart the go-ipfs daemon. + 6. Update the nginx config, removing the "down" status from the node. + +This will effectively "garbage collect" without actually running the garbage +collector. + +# Buffering Requests & Responses + +In general, requests to the gateway should be buffered by the reverse proxy for +the best performance. This is usually enabled by default (`proxy_request_buffering`). + +## API + +The go-ipfs HTTP API (`/api/...`) starts sending a response before it's done +reading the request. This allows it to, e.g., send back progress updates while +adding a file to go-ipfs. + +However, these progress updates won't work if the HTTP reverse proxy is +configured to buffer requests. While requests to the go-ipfs _gateway_ should +usually be buffered for better performance, requests to the go-ipfs API should +generally not be buffered. + +In NGINX, you can turn off buffering for the API with: + +```nginx +server { + ... + location /api { + ... + proxy_request_buffering off; + proxy_buffering off; + proxy_http_version 1.1; + } +} +``` + +See: https://github.com/ipfs/go-ipfs/issues/6402#issuecomment-643025868 + +# Content Blocking + +TODO: + +* Filtering requests +* Checking the X-IPFS-Path header in responses to filter again after resolving. + +# Subdomain Gateway + +TODO: Reverse proxies and the subdomain gateway. + +# Load balancing + +TODO: discuss load balancing based on the CID versus the source IP. From 5c4f5adc85f77b3382e8c2430ca77d47a2491c9e Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Sat, 13 Jun 2020 07:50:28 +0200 Subject: [PATCH 033/144] README.md: remove link for init-systems to ipfs/website repo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b2ccb56d745..86d9205b911 100644 --- a/README.md +++ b/README.md @@ -224,7 +224,7 @@ dependencies as well. (See https://github.com/ipfs/go-ipfs/issues/177) - For more details on setting up FUSE (so that you can mount the filesystem), see the docs folder. - Shell command completion is available in `misc/completion/ipfs-completion.bash`. Read [docs/command-completion.md](docs/command-completion.md) to learn how to install it. -- See the [init examples](https://github.com/ipfs/website/tree/master/static/docs/examples/init) for how to connect IPFS to systemd or whatever init system your distro uses. +- See the [misc folder](https://github.com/ipfs/go-ipfs/tree/master/misc) for how to connect IPFS to systemd or whatever init system your distro uses. ### Updating go-ipfs From 77f2b04ccef8511731c7ec55fe89fc4de56b00fb Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Sat, 13 Jun 2020 07:54:40 +0200 Subject: [PATCH 034/144] misc/README.md: import readme from ipfs/website repo Authors: @lgierth @CameronNemo @jessicaschilling URL: https://github.com/ipfs/website/blob/master/static/docs/examples/init/README.md License: MIT --- misc/README.md | 249 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 249 insertions(+) create mode 100644 misc/README.md diff --git a/misc/README.md b/misc/README.md new file mode 100644 index 00000000000..8462532f618 --- /dev/null +++ b/misc/README.md @@ -0,0 +1,249 @@ +## init system integration + +go-ipfs can be started by your operating system's native init system. + +- [systemd](#systemd) +- [LSB init script](#initd) +- [Upstart/startup job](#upstart) +- [launchd](#launchd) + +### systemd + +For `systemd`, the best approach is to run the daemon in a user session. Here is a sample service file: + +```systemd +[Unit] +Description=IPFS daemon + +[Service] +# Environment="IPFS_PATH=/data/ipfs" # optional path to ipfs init directory if not default ($HOME/.ipfs) +ExecStart=/usr/bin/ipfs daemon +Restart=on-failure + +[Install] +WantedBy=default.target +``` + +To run this in your user session, save it as `~/.config/systemd/user/ipfs.service` (creating directories as necessary). Once you run `ipfs init` to create your IPFS settings, you can control the daemon using the following commands: + +* `systemctl --user start ipfs` - start the daemon +* `systemctl --user stop ipfs` - stop the daemon +* `systemctl --user status ipfs` - get status of the daemon +* `systemctl --user enable ipfs` - enable starting the daemon at boot +* `systemctl --user disable ipfs` - disable starting the daemon at boot + +*Note:* If you want this `--user` service to run at system boot, you must [`enable-linger`](http://www.freedesktop.org/software/systemd/man/loginctl.html) on the account that runs the service: + +``` +# loginctl enable-linger [user] +``` +Read more about `--user` services here: [wiki.archlinux.org:Systemd ](https://wiki.archlinux.org/index.php/Systemd/User#Automatic_start-up_of_systemd_user_instances) + +### initd + +- Here is a full-featured sample service file: https://github.com/dylanPowers/ipfs-linux-service/blob/master/init.d/ipfs +- Use `service` or your distribution's equivalent to control the service. + +## upstart + +- And below is a very basic sample upstart job. **Note the username jbenet**. + +``` +cat /etc/init/ipfs.conf +``` +``` +description "ipfs: interplanetary filesystem" + +start on (local-filesystems and net-device-up IFACE!=lo) +stop on runlevel [!2345] + +limit nofile 524288 1048576 +limit nproc 524288 1048576 +setuid jbenet +chdir /home/jbenet +respawn +exec ipfs daemon +``` + +Another version is available here: + +```sh +ipfs cat /ipfs/QmbYCwVeA23vz6mzAiVQhJNa2JSiRH4ebef1v2e5EkDEZS/ipfs.conf >/etc/init/ipfs.conf +``` + +For both, edit to replace occurrences of `jbenet` with whatever user you want it to run as: + +```sh +sed -i s/jbenet// /etc/init/ipfs.conf +``` + +Once you run `ipfs init` to create your IPFS settings, you can control the daemon using the `init.d` commands: + +```sh +sudo service ipfs start +sudo service ipfs stop +sudo service ipfs restart +... +``` + +## launchd + +Similar to `systemd`, on macOS you can run `go-ipfs` via a user LaunchAgent. + +- Create `~/Library/LaunchAgents/io.ipfs.go-ipfs.plist`: + +```xml + + + + + KeepAlive + + Label + io.ipfs.go-ipfs + ProcessType + Background + ProgramArguments + + /bin/sh + -c + ~/go/bin/ipfs daemon + + RunAtLoad + + + +``` +The reason for running `ipfs` under a shell is to avoid needing to hard-code the user's home directory in the job. + +- To start the job, run `launchctl load ~/Library/LaunchAgents/io.ipfs.go-ipfs.plist` + +Notes: + +- To check that the job is running, run `launchctl list | grep ipfs`. +- IPFS should now start whenever you log in (and exit when you log out). +- [LaunchControl](http://www.soma-zone.com/LaunchControl/) is a GUI tool which simplifies management of LaunchAgents.## init system integration + +go-ipfs can be started by your operating system's native init system. + +- [systemd](#systemd) +- [LSB init script](#initd) +- [Upstart/startup job](#upstart) +- [launchd](#launchd) + +### systemd + +For `systemd`, the best approach is to run the daemon in a user session. Here is a sample service file: + +```systemd +[Unit] +Description=IPFS daemon + +[Service] +# Environment="IPFS_PATH=/data/ipfs" # optional path to ipfs init directory if not default ($HOME/.ipfs) +ExecStart=/usr/bin/ipfs daemon +Restart=on-failure + +[Install] +WantedBy=default.target +``` + +To run this in your user session, save it as `~/.config/systemd/user/ipfs.service` (creating directories as necessary). Once you run `ipfs init` to create your IPFS settings, you can control the daemon using the following commands: + +* `systemctl --user start ipfs` - start the daemon +* `systemctl --user stop ipfs` - stop the daemon +* `systemctl --user status ipfs` - get status of the daemon +* `systemctl --user enable ipfs` - enable starting the daemon at boot +* `systemctl --user disable ipfs` - disable starting the daemon at boot + +*Note:* If you want this `--user` service to run at system boot, you must [`enable-linger`](http://www.freedesktop.org/software/systemd/man/loginctl.html) on the account that runs the service: + +``` +# loginctl enable-linger [user] +``` +Read more about `--user` services here: [wiki.archlinux.org:Systemd ](https://wiki.archlinux.org/index.php/Systemd/User#Automatic_start-up_of_systemd_user_instances) + +### initd + +- Here is a full-featured sample service file: https://github.com/dylanPowers/ipfs-linux-service/blob/master/init.d/ipfs +- Use `service` or your distribution's equivalent to control the service. + +## upstart + +- And below is a very basic sample upstart job. **Note the username jbenet**. + +``` +cat /etc/init/ipfs.conf +``` +``` +description "ipfs: interplanetary filesystem" + +start on (local-filesystems and net-device-up IFACE!=lo) +stop on runlevel [!2345] + +limit nofile 524288 1048576 +limit nproc 524288 1048576 +setuid jbenet +chdir /home/jbenet +respawn +exec ipfs daemon +``` + +Another version is available here: + +```sh +ipfs cat /ipfs/QmbYCwVeA23vz6mzAiVQhJNa2JSiRH4ebef1v2e5EkDEZS/ipfs.conf >/etc/init/ipfs.conf +``` + +For both, edit to replace occurrences of `jbenet` with whatever user you want it to run as: + +```sh +sed -i s/jbenet// /etc/init/ipfs.conf +``` + +Once you run `ipfs init` to create your IPFS settings, you can control the daemon using the `init.d` commands: + +```sh +sudo service ipfs start +sudo service ipfs stop +sudo service ipfs restart +... +``` + +## launchd + +Similar to `systemd`, on macOS you can run `go-ipfs` via a user LaunchAgent. + +- Create `~/Library/LaunchAgents/io.ipfs.go-ipfs.plist`: + +```xml + + + + + KeepAlive + + Label + io.ipfs.go-ipfs + ProcessType + Background + ProgramArguments + + /bin/sh + -c + ~/go/bin/ipfs daemon + + RunAtLoad + + + +``` +The reason for running `ipfs` under a shell is to avoid needing to hard-code the user's home directory in the job. + +- To start the job, run `launchctl load ~/Library/LaunchAgents/io.ipfs.go-ipfs.plist` + +Notes: + +- To check that the job is running, run `launchctl list | grep ipfs`. +- IPFS should now start whenever you log in (and exit when you log out). +- [LaunchControl](http://www.soma-zone.com/LaunchControl/) is a GUI tool which simplifies management of LaunchAgents. From b47ecd0cba85905f18b85de9dd771008adf60fd0 Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Sat, 13 Jun 2020 04:12:05 +0200 Subject: [PATCH 035/144] systemd: specify repo path, to avoid unnecessary subdirectory --- misc/systemd/ipfs-hardened.service | 1 + misc/systemd/ipfs.service | 1 + 2 files changed, 2 insertions(+) diff --git a/misc/systemd/ipfs-hardened.service b/misc/systemd/ipfs-hardened.service index dbb35c5978b..bf202218f84 100644 --- a/misc/systemd/ipfs-hardened.service +++ b/misc/systemd/ipfs-hardened.service @@ -62,6 +62,7 @@ Type=notify User=ipfs Group=ipfs StateDirectory=ipfs +Environment=IPFS_PATH="${HOME}" ExecStart=/usr/bin/ipfs daemon --init --migrate Restart=on-failure KillSignal=SIGINT diff --git a/misc/systemd/ipfs.service b/misc/systemd/ipfs.service index b9bd4a2e232..cd689d3bafd 100644 --- a/misc/systemd/ipfs.service +++ b/misc/systemd/ipfs.service @@ -33,6 +33,7 @@ Type=notify User=ipfs Group=ipfs StateDirectory=ipfs +Environment=IPFS_PATH="${HOME}" ExecStart=/usr/bin/ipfs daemon --init --migrate Restart=on-failure KillSignal=SIGINT From 406d70ff7cfe9eb4e1fd2dc3a5f35be6d37a3ccd Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Mon, 15 Jun 2020 17:01:40 -0700 Subject: [PATCH 036/144] doc: document reverse proxy bug see https://github.com/ipfs/go-ipfs/issues/6402 --- docs/production/reverse-proxy.md | 39 +++++--------------------------- 1 file changed, 6 insertions(+), 33 deletions(-) diff --git a/docs/production/reverse-proxy.md b/docs/production/reverse-proxy.md index dfb5359fa50..f73efad0a11 100644 --- a/docs/production/reverse-proxy.md +++ b/docs/production/reverse-proxy.md @@ -12,11 +12,16 @@ reverse HTTP proxy (usually NGINX). You may need a reverse proxy to: * Block content. * Rate limit and timeout requests. * Apply QoS rules (e.g., prioritize traffic for certain important IPFS resources). -* Expose a limited subset of the HTTP API. This document contains a collection of tips, tricks, and pitfalls when running a go-ipfs node behind a reverse HTTP proxy. +**WARNING:** Due to +[nginx#1293](https://trac.nginx.org/nginx/ticket/1293)/[go-ipfs#6402](https://github.com/ipfs/go-ipfs/issues/6402), +parts of the go-ipfs API will not work correctly behind an NGINX reverse proxy +as go-ipfs starts sending back a response before it finishes reading the request +body. The gateway itself is unaffected. + ## Peering Go-ipfs gateways behind a single load balancing reverse proxy should use the @@ -45,38 +50,6 @@ you run out of space, instead of garbage collecting. This will effectively "garbage collect" without actually running the garbage collector. -# Buffering Requests & Responses - -In general, requests to the gateway should be buffered by the reverse proxy for -the best performance. This is usually enabled by default (`proxy_request_buffering`). - -## API - -The go-ipfs HTTP API (`/api/...`) starts sending a response before it's done -reading the request. This allows it to, e.g., send back progress updates while -adding a file to go-ipfs. - -However, these progress updates won't work if the HTTP reverse proxy is -configured to buffer requests. While requests to the go-ipfs _gateway_ should -usually be buffered for better performance, requests to the go-ipfs API should -generally not be buffered. - -In NGINX, you can turn off buffering for the API with: - -```nginx -server { - ... - location /api { - ... - proxy_request_buffering off; - proxy_buffering off; - proxy_http_version 1.1; - } -} -``` - -See: https://github.com/ipfs/go-ipfs/issues/6402#issuecomment-643025868 - # Content Blocking TODO: From b86e93045aaa0ac357d939884e1c7996b0fcdbd4 Mon Sep 17 00:00:00 2001 From: Adin Schmahmann Date: Fri, 19 Jun 2020 20:09:15 -0400 Subject: [PATCH 037/144] update version to 0.7.0-dev --- version.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/version.go b/version.go index 1cecf68d627..db77a366406 100644 --- a/version.go +++ b/version.go @@ -4,7 +4,7 @@ package ipfs var CurrentCommit string // CurrentVersionNumber is the current application's version literal -const CurrentVersionNumber = "0.6.0" +const CurrentVersionNumber = "0.7.0-dev" const ApiVersion = "/go-ipfs/" + CurrentVersionNumber + "/" From cc4a136360697ec7633fe19db80555de2af5e58b Mon Sep 17 00:00:00 2001 From: Rafael Ramalho Date: Mon, 22 Jun 2020 17:39:30 +0100 Subject: [PATCH 038/144] chore: bump webui version --- core/corehttp/webui.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/corehttp/webui.go b/core/corehttp/webui.go index f94df7850f0..e00cf788fb8 100644 --- a/core/corehttp/webui.go +++ b/core/corehttp/webui.go @@ -1,11 +1,12 @@ package corehttp // TODO: move to IPNS -const WebUIPath = "/ipfs/bafybeigkbbjnltbd4ewfj7elajsbnjwinyk6tiilczkqsibf3o7dcr6nn4" // v2.9.0 +const WebUIPath = "/ipfs/bafybeid6luolenf4fcsuaw5rgdwpqbyerce4x3mi3hxfdtp5pwco7h7qyq" // v2.10.0 // this is a list of all past webUI paths. var WebUIPaths = []string{ WebUIPath, + "/ipfs/bafybeigkbbjnltbd4ewfj7elajsbnjwinyk6tiilczkqsibf3o7dcr6nn4", "/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i", "/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq", "/ipfs/QmfQkD8pBSBCBxWEwFSu4XaDVSWK6bjnNuaWZjMyQbyDub", From 4c38ea748a6d7f6d5b9d4900dc77f976d049d427 Mon Sep 17 00:00:00 2001 From: Rafael Ramalho Date: Tue, 23 Jun 2020 16:28:38 +0100 Subject: [PATCH 039/144] chore:bump webui version to 2.10.1 --- core/corehttp/webui.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/corehttp/webui.go b/core/corehttp/webui.go index e00cf788fb8..2728bc7b38d 100644 --- a/core/corehttp/webui.go +++ b/core/corehttp/webui.go @@ -1,11 +1,12 @@ package corehttp // TODO: move to IPNS -const WebUIPath = "/ipfs/bafybeid6luolenf4fcsuaw5rgdwpqbyerce4x3mi3hxfdtp5pwco7h7qyq" // v2.10.0 +const WebUIPath = "/ipfs/bafybeibnnxd4etu4tq5fuhu3z5p4rfu3buabfkeyr3o3s4h6wtesvvw6mu" // v2.10.1 // this is a list of all past webUI paths. var WebUIPaths = []string{ WebUIPath, + "/ipfs/bafybeid6luolenf4fcsuaw5rgdwpqbyerce4x3mi3hxfdtp5pwco7h7qyq", "/ipfs/bafybeigkbbjnltbd4ewfj7elajsbnjwinyk6tiilczkqsibf3o7dcr6nn4", "/ipfs/bafybeicp23nbcxtt2k2twyfivcbrc6kr3l5lnaiv3ozvwbemtrb7v52r6i", "/ipfs/bafybeidatpz2hli6fgu3zul5woi27ujesdf5o5a7bu622qj6ugharciwjq", From 1034724ca075e5208ad62a63032693b79d07aeca Mon Sep 17 00:00:00 2001 From: Steven Allen Date: Tue, 23 Jun 2020 14:03:23 -0700 Subject: [PATCH 040/144] fix: use static binaries in docker container This is more robust cross-platform/architecture. fixes #7422 --- Dockerfile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Dockerfile b/Dockerfile index f8a823b9574..ea28fbf8cb5 100644 --- a/Dockerfile +++ b/Dockerfile @@ -33,14 +33,14 @@ ENV TINI_VERSION v0.19.0 RUN set -eux; \ dpkgArch="$(dpkg --print-architecture)"; \ case "${dpkgArch##*-}" in \ - "amd64" | "armhf" | "arm64") tiniArch="tini-$dpkgArch" ;;\ + "amd64" | "armhf" | "arm64") tiniArch="tini-static-$dpkgArch" ;;\ *) echo >&2 "unsupported architecture: ${dpkgArch}"; exit 1 ;; \ esac; \ cd /tmp \ && git clone https://github.com/ncopa/su-exec.git \ && cd su-exec \ && git checkout -q $SUEXEC_VERSION \ - && make \ + && make su-exec-static \ && cd /tmp \ && wget -q -O tini https://github.com/krallin/tini/releases/download/$TINI_VERSION/$tiniArch \ && chmod +x tini @@ -53,7 +53,7 @@ LABEL maintainer="Steven Allen " ENV SRC_DIR /go-ipfs COPY --from=0 $SRC_DIR/cmd/ipfs/ipfs /usr/local/bin/ipfs COPY --from=0 $SRC_DIR/bin/container_daemon /usr/local/bin/start_ipfs -COPY --from=0 /tmp/su-exec/su-exec /sbin/su-exec +COPY --from=0 /tmp/su-exec/su-exec-static /sbin/su-exec COPY --from=0 /tmp/tini /sbin/tini COPY --from=0 /bin/fusermount /usr/local/bin/fusermount COPY --from=0 /etc/ssl/certs /etc/ssl/certs From c532cb4d5ae7b49f644a7f438d9fedd007b83ee0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mart=C3=ADn=20Triay?= Date: Sat, 27 Jun 2020 15:57:18 -0300 Subject: [PATCH 041/144] Document add behavior when the daemon is not running --- core/commands/add.go | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/core/commands/add.go b/core/commands/add.go index c66994b3726..df8dd5f9bbb 100644 --- a/core/commands/add.go +++ b/core/commands/add.go @@ -60,6 +60,10 @@ Adds contents of to ipfs. Use -r to add directories. Note that directories are added recursively, to form the ipfs MerkleDAG. +If the daemon is not running, it will just add locally. +If the daemon is started later, it will be advertised after a few +seconds when the reprovider runs. + The wrap option, '-w', wraps the file (or files, if using the recursive option) in a directory. This directory contains only the files which have been added, and means that the file retains From 8b235cf47ac5d6e2db099288b938a929e23b6b68 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Thu, 2 Jul 2020 14:57:34 +0700 Subject: [PATCH 042/144] update QUIC --- go.mod | 2 +- go.sum | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/go.mod b/go.mod index 7ecf7614f85..25b5b6da722 100644 --- a/go.mod +++ b/go.mod @@ -74,7 +74,7 @@ require ( github.com/libp2p/go-libp2p-peerstore v0.2.6 github.com/libp2p/go-libp2p-pubsub v0.3.1 github.com/libp2p/go-libp2p-pubsub-router v0.3.0 - github.com/libp2p/go-libp2p-quic-transport v0.6.0 + github.com/libp2p/go-libp2p-quic-transport v0.7.0 github.com/libp2p/go-libp2p-record v0.1.3 github.com/libp2p/go-libp2p-routing-helpers v0.2.3 github.com/libp2p/go-libp2p-secio v0.2.2 diff --git a/go.sum b/go.sum index 5ccff0543c6..11a11901ee1 100644 --- a/go.sum +++ b/go.sum @@ -712,8 +712,8 @@ github.com/libp2p/go-libp2p-pubsub v0.3.1/go.mod h1:TxPOBuo1FPdsTjFnv+FGZbNbWYsp github.com/libp2p/go-libp2p-pubsub-router v0.3.0 h1:ghpHApTMXN+aZ+InYvpJa/ckBW4orypzNI0aWQDth3s= github.com/libp2p/go-libp2p-pubsub-router v0.3.0/go.mod h1:6kZb1gGV1yGzXTfyNsi4p+hyt1JnA1OMGHeExTOJR3A= github.com/libp2p/go-libp2p-quic-transport v0.3.7/go.mod h1:Kr4aDtnfHHNeENn5J+sZIVc+t8HpQn9W6BOxhVGHbgI= -github.com/libp2p/go-libp2p-quic-transport v0.6.0 h1:d5bcq7y+t6IiumD9Ib0S4oHgWu66rRjQ1Y8ligii6G8= -github.com/libp2p/go-libp2p-quic-transport v0.6.0/go.mod h1:HR435saAZhTrFabI+adf3tVBY7ZJg5rKNoJ+CrIIg8c= +github.com/libp2p/go-libp2p-quic-transport v0.7.0 h1:sDfgsgxfQIHfjE7GjmasoNxqfYYEtERLqBxt9J7pocQ= +github.com/libp2p/go-libp2p-quic-transport v0.7.0/go.mod h1:HK9llw8/wvidGF+KyVVvdumhJ3wE/0jlewfHd13YF6Q= github.com/libp2p/go-libp2p-record v0.0.1/go.mod h1:grzqg263Rug/sRex85QrDOLntdFAymLDLm7lxMgU79Q= github.com/libp2p/go-libp2p-record v0.1.0/go.mod h1:ujNc8iuE5dlKWVy6wuL6dd58t0n7xI4hAIl8pE6wu5Q= github.com/libp2p/go-libp2p-record v0.1.2 h1:M50VKzWnmUrk/M5/Dz99qO9Xh4vs8ijsK+7HkJvRP+0= @@ -865,8 +865,8 @@ github.com/libp2p/go-yamux v1.3.7 h1:v40A1eSPJDIZwz2AvrV3cxpTZEGDP11QJbukmEhYyQI github.com/libp2p/go-yamux v1.3.7/go.mod h1:fr7aVgmdNGJK+N1g+b6DW6VxzbRCjCOejR/hkmpooHE= github.com/lucas-clemente/quic-go v0.15.7 h1:Pu7To5/G9JoP1mwlrcIvfV8ByPBlCzif3MCl8+1W83I= github.com/lucas-clemente/quic-go v0.15.7/go.mod h1:Myi1OyS0FOjL3not4BxT7KN29bRkcMUV5JVVFLKtDp8= -github.com/lucas-clemente/quic-go v0.16.2 h1:A27xKWQtPTeOcIUF4EymmROkKlF/RbKBiMbflwv6RK0= -github.com/lucas-clemente/quic-go v0.16.2/go.mod h1:I0+fcNTdb9eS1ZcjQZbDVPGchJ86chcIxPALn9lEJqE= +github.com/lucas-clemente/quic-go v0.17.2 h1:4iQInIuNQkPNZmsy9rCnwuOzpH0qGnDo4jn0QfI/qE4= +github.com/lucas-clemente/quic-go v0.17.2/go.mod h1:I0+fcNTdb9eS1ZcjQZbDVPGchJ86chcIxPALn9lEJqE= github.com/lunixbochs/vtclean v1.0.0/go.mod h1:pHhQNgMf3btfWnGBVipUOjRYhoOsdGqdm/+2c2E2WMI= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329 h1:2gxZ0XQIU/5z3Z3bUBu+FXuk2pFbkN6tcwi/pjyaDic= From 32724b6e33a651649861ae2e9dd03084e97da7e0 Mon Sep 17 00:00:00 2001 From: "@RubenKelevra" Date: Thu, 2 Jul 2020 15:57:48 +0200 Subject: [PATCH 043/144] github issues: add info about the security email address --- .github/ISSUE_TEMPLATE/bug-report.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md index 23eab547c51..72d014c8fda 100644 --- a/.github/ISSUE_TEMPLATE/bug-report.md +++ b/.github/ISSUE_TEMPLATE/bug-report.md @@ -4,6 +4,8 @@ about: 'Report a bug in go-ipfs.' labels: kind/bug, need/triage --- + + #### Version information: