-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
ca9c323
commit 0f8f807
Showing
9 changed files
with
1,780 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,102 @@ | ||
| # k-bucket | ||
| # KBucket | ||
|
|
||
| [](https://goreportcard.com/report/github.com/attilabuti/k-bucket) | ||
| [](https://pkg.go.dev/github.com/attilabuti/k-bucket) | ||
| [](https://raw.githubusercontent.com/attilabuti/k-bucket/main/LICENSE) | ||
|
|
||
| Kademlia DHT K-bucket implementation as a binary tree. | ||
| Ported from Tristan Slominski's [k-bucket](https://github.com/tristanls/k-bucket). | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| $ go get github.com/attilabuti/k-bucket@latest | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| For more information, please see the [Package Docs](https://pkg.go.dev/github.com/attilabuti/k-bucket). | ||
|
|
||
| ### Overview | ||
|
|
||
| A [*Distributed Hash Table (DHT)*](http://en.wikipedia.org/wiki/Distributed_hash_table) is a decentralized distributed system that provides a lookup table similar to a hash table. | ||
|
|
||
| *k-bucket* is an implementation of a storage mechanism for keys within a DHT. It stores `contact` objects which represent locations and addresses of nodes in the decentralized distributed system. `contact` objects are typically identified by a SHA-1 hash, however this restriction is lifted in this implementation. Additionally, node ids of different lengths can be compared. | ||
|
|
||
| This Kademlia DHT k-bucket implementation is meant to be as minimal as possible. It assumes that `contact` objects consist only of `id`. It is useful, and necessary, to attach other properties to a `contact`. For example, one may want to attach `ip` and `port` properties, which allow an application to send IP traffic to the `contact`. However, this information is extraneous and irrelevant to the operation of a k-bucket. | ||
|
|
||
| ### arbiter function | ||
|
|
||
| This *k-bucket* implementation implements a conflict resolution mechanism using an `arbiter` function. The purpose of the `arbiter` is to choose between two `contact` objects with the same `id` but perhaps different properties and determine which one should be stored. As the `arbiter` function returns the actual object to be stored, it does not need to make an either/or choice, but instead could perform some sort of operation and return the result as a new object that would then be stored. See [kBucket.update(node, index, contact)](https://github.com/attilabuti/k-bucket/blob/main/kbucket.go#L173) for detailed semantics of which `contact` (`incumbent` or `candidate`) is selected. | ||
|
|
||
| For example, an `arbiter` function implementing a `VectorClock` mechanism would look something like: | ||
|
|
||
| ```go | ||
| // Contact example | ||
| contact := Contact{ | ||
| Id: []byte("contactId"), | ||
| VectorClock: 0 | ||
| }; | ||
|
|
||
| func arbiter(incumbent Contact, candidate Contact) Contact { | ||
| if incumbent.VectorClock > candidate.VectorClock { | ||
| return incumbent | ||
| } | ||
|
|
||
| return candidate | ||
| } | ||
| ``` | ||
|
|
||
| ### Documentation | ||
|
|
||
| For more information, please see the [Package Docs](https://pkg.go.dev/github.com/attilabuti/k-bucket#KBucket). | ||
|
|
||
| Implementation of a Kademlia DHT k-bucket used for storing contact (peer node) information. | ||
|
|
||
| For a step by step example of k-bucket operation you may find the following slideshow useful: [Distribute All The Things](https://docs.google.com/presentation/d/11qGZlPWu6vEAhA7p3qsQaQtWH7KofEC9dMeBFZ1gYeA/edit#slide=id.g1718cc2bc_0661). | ||
|
|
||
| KBucket starts off as a single k-bucket with capacity of _k_. As contacts are added, once the _k+1_ contact is added, the k-bucket is split into two k-buckets. The split happens according to the first bit of the contact node id. The k-bucket that would contain the local node id is the "near" k-bucket, and the other one is the "far" k-bucket. The "far" k-bucket is marked as _don't split_ in order to prevent further splitting. The contact nodes that existed are then redistributed along the two new k-buckets and the old k-bucket becomes an inner node within a tree data structure. | ||
|
|
||
| As even more contacts are added to the "near" k-bucket, the "near" k-bucket will split again as it becomes full. However, this time it is split along the second bit of the contact node id. Again, the two newly created k-buckets are marked "near" and "far" and the "far" k-bucket is marked as _don't split_. Again, the contact nodes that existed in the old bucket are redistributed. This continues as long as nodes are being added to the "near" k-bucket, until the number of splits reaches the length of the local node id. | ||
|
|
||
| As more contacts are added to the "far" k-bucket and it reaches its capacity, it does not split. Instead, the k-bucket emits a "ping" event (register a listener: `emitter.On("kbucket.ping", function (old Contacts, new Contact) {...});` and includes a slice of old contact nodes that it hasn't heard from in a while and requires you to confirm that those contact nodes still respond (literally respond to a PING RPC). If an old contact node still responds, it should be re-added (`kBucket.Add(old Contact)`) back to the k-bucket. This puts the old contact on the "recently heard from" end of the list of nodes in the k-bucket. If the old contact does not respond, it should be removed (`kBucket.Remove(oldContact.Id []byte)`) and the new contact being added now has room to be stored (`kBucket.Add(new Contact)`). | ||
|
|
||
| #### Events | ||
|
|
||
| * kbucket.added | ||
| * `newContact Contact`: The new contact that was added. | ||
| * Emitted only when "newContact" was added to bucket and it was not stored in the bucket before. | ||
|
|
||
| * kbucket.ping | ||
| * `old Contacts`: The slice of contacts to ping. | ||
| * `new Contact`: The new contact to be added if one of old contacts does not respond. | ||
| * Emitted every time a contact is added that would exceed the capacity of a "don't split" k-bucket it belongs to. | ||
|
|
||
| * kbucket.removed | ||
| * `contact Contact`: The contact that was removed. | ||
| * Emitted when "contact" was removed from the bucket. | ||
|
|
||
| * kbucket.updated | ||
| * `old Contact`: The contact that was stored prior to the update. | ||
| * `new Contact`: The new contact that is now stored after the update. | ||
| * Emitted when a previously existing ("previously existing" means "oldContact.id" equals "newContact.id") contact was added to the bucket and it was replaced with "newContact". | ||
|
|
||
| ## Further reading | ||
|
|
||
| - [Distributed Hash Table (DHT)](http://en.wikipedia.org/wiki/Distributed_hash_table) | ||
| - [A formal specification of the Kademlia distributed hash table](http://maude.sip.ucm.es/kademlia/files/pita_kademlia.pdf) | ||
| - [Distributed Hash Tables (part 2)](https://web.archive.org/web/20140217064545/http://offthelip.org/?p=157) | ||
| - [DHT Walkthrough Notes](https://gist.github.com/gubatron/cd9cfa66839e18e49846) | ||
| - [Distribute All The Things](https://docs.google.com/presentation/d/11qGZlPWu6vEAhA7p3qsQaQtWH7KofEC9dMeBFZ1gYeA/edit#slide=id.g1718cc2bc_0661) | ||
|
|
||
| ## Issues | ||
|
|
||
| Submit the [issues](https://github.com/attilabuti/k-bucket/issues) if you find any bug or have any suggestion. | ||
|
|
||
| ## Contribution | ||
|
|
||
| Fork the [repo](https://github.com/attilabuti/k-bucket) and submit pull requests. | ||
|
|
||
| ## License | ||
|
|
||
| This extension is licensed under the [MIT License](https://github.com/attilabuti/k-bucket/blob/main/LICENSE). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| /* | ||
| # KBucket | ||
| Kademlia DHT K-bucket implementation as a binary tree. | ||
| KBucket was ported from Tristan Slominski's k-bucket: github.com/tristanls/k-bucket | ||
| A Distributed Hash Table (DHT) is a decentralized distributed system that | ||
| provides a lookup table similar to a hash table. | ||
| KBucket is an implementation of a storage mechanism for keys within a DHT. | ||
| It stores Contact objects which represent locations and addresses of nodes in | ||
| the decentralized distributed system. Contact objects are typically identified | ||
| by a SHA-1 hash, however this restriction is lifted in this implementation. | ||
| Additionally, node ids of different lengths can be compared. | ||
| This Kademlia DHT k-bucket implementation is meant to be as minimal as possible. | ||
| It assumes that Contact objects consist only of Id. It is useful, and necessary, | ||
| to attach other properties to a Contact. For example, one may want to attach | ||
| ip and port properties, which allow an application to send IP traffic to the | ||
| Contact. However, this information is extraneous and irrelevant to the operation | ||
| of a k-bucket. | ||
| KBucket events: | ||
| kbucket.added | ||
| newContact Contact: The new contact that was added. | ||
| Emitted only when "newContact" was added to bucket and it was not stored | ||
| in the bucket before. | ||
| kbucket.ping | ||
| old Contacts: The slice of contacts to ping. | ||
| new Contact: The new contact to be added if one of old contacts does not respond. | ||
| Emitted every time a contact is added that would exceed the capacity of a | ||
| "don't split" k-bucket it belongs to. | ||
| kbucket.removed | ||
| contact Contact: The contact that was removed. | ||
| Emitted when "contact" was removed from the bucket. | ||
| kbucket.updated | ||
| old Contact: The contact that was stored prior to the update. | ||
| new Contact: The new contact that is now stored after the update. | ||
| Emitted when a previously existing ("previously existing" means "oldContact.id" | ||
| equals "newContact.id") contact was added to the bucket and it was replaced with | ||
| "newContact". | ||
| */ | ||
| package kbucket |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| module github.com/attilabuti/k-bucket | ||
|
|
||
| go 1.19 | ||
|
|
||
| require ( | ||
| github.com/attilabuti/eventemitter/v2 v2.1.0 | ||
| github.com/stretchr/testify v1.8.1 | ||
| ) | ||
|
|
||
| require ( | ||
| github.com/davecgh/go-spew v1.1.1 // indirect | ||
| github.com/kr/pretty v0.3.1 // indirect | ||
| github.com/pmezard/go-difflib v1.0.0 // indirect | ||
| gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect | ||
| gopkg.in/yaml.v3 v3.0.1 // indirect | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| github.com/attilabuti/eventemitter/v2 v2.1.0 h1:5zi46VhbiBc9v6b6C5dazyP77fgciMlgNLuJRT6Y2cs= | ||
| github.com/attilabuti/eventemitter/v2 v2.1.0/go.mod h1:m7Ri5O38OMWY7rdrhXUXCVjGT65u7So9lhyQMQ5DDGE= | ||
| github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= | ||
| github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= | ||
| github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= | ||
| github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= | ||
| github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= | ||
| github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= | ||
| github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= | ||
| github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= | ||
| github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= | ||
| github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= | ||
| github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= | ||
| github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8= | ||
| github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= | ||
| github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= | ||
| github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= | ||
| github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= | ||
| github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= | ||
| github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= | ||
| github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= | ||
| github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= | ||
| gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= | ||
| gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= | ||
| gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= | ||
| gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= | ||
| gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= | ||
| gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= |
Oops, something went wrong.