Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Transaction queuing and proof generation node #1

Open
wants to merge 117 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
5c03a59
initial commit
vaibhavchellani Apr 18, 2019
6d83fb0
setup tx receiver
vaibhavchellani Apr 20, 2019
34bd8d4
added transaction class
vaibhavchellani Apr 20, 2019
ace3a6a
Merge branch 'master' of https://github.com/barryWhiteHat/rollupNC_co…
vaibhavchellani Apr 20, 2019
f47857b
using promise based amqp
vaibhavchellani Apr 21, 2019
551cbd7
added readme
vaibhavchellani Apr 21, 2019
0bf4eb5
updated readme
vaibhavchellani Apr 21, 2019
5998768
added utils
vaibhavchellani Apr 21, 2019
4702a73
added fetch txs
vaibhavchellani Apr 22, 2019
126bec0
added rate limit to consumption
vaibhavchellani Apr 22, 2019
5d18a17
added max transactions in config
vaibhavchellani Apr 22, 2019
a4a4416
added getTx
vaibhavchellani May 6, 2019
5770075
working additions to queue
vaibhavchellani May 6, 2019
710bbcd
integrated babel
vaibhavchellani May 7, 2019
03b8da2
added processor
vaibhavchellani May 7, 2019
0e5714d
working prefetch for queue
vaibhavchellani May 12, 2019
7bab991
ack on successfull message consumption
vaibhavchellani May 12, 2019
0bd430a
updated readme
vaibhavchellani May 12, 2019
a2b54cc
added circuit
vaibhavchellani May 15, 2019
897f83e
WIP: snark inputs
vaibhavchellani May 23, 2019
6f9e402
Changing sig structure
vaibhavchellani May 23, 2019
233887b
adding queue integration
vaibhavchellani May 23, 2019
0401bf9
WIP: integrating mysql and knex
vaibhavchellani May 23, 2019
06c512c
added DB
vaibhavchellani May 25, 2019
e53fa21
updated readme
vaibhavchellani May 25, 2019
00a5998
WIP fixing merkel proof issue
vaibhavchellani May 26, 2019
8be15ca
WIP fixing merkel proof issue
vaibhavchellani May 27, 2019
7396580
working Merkle proof
May 28, 2019
930f22a
working input
May 29, 2019
5a8427c
getAllAccounts() orderBy 'id' in src/db.js
May 30, 2019
993554c
Merge branch 'development'
May 30, 2019
9bbbb7d
add index to Account class
May 30, 2019
fe009ad
implement tx sign
ChihChengLiang Jun 11, 2019
cb216ff
implement poll
ChihChengLiang Jun 11, 2019
9aafc9d
add commend
ChihChengLiang Jun 11, 2019
465f50c
Merge pull request #1 from ChihChengLiang/test-submit-tx
vaibhavchellani Jun 11, 2019
e42563f
pad tx leaves
ChihChengLiang Jun 11, 2019
e46eae9
define unique accounts
ChihChengLiang Jun 11, 2019
30ad8f4
Use correct public-privkey pair
ChihChengLiang Jun 11, 2019
6d55019
saving deposits off of contract event
stubocka Jun 11, 2019
5e8958d
Merge pull request #3 from ChihChengLiang/fix-pubkey
vaibhavchellani Jun 12, 2019
8ae4f50
Merge pull request #4 from stuarth1/feature/deposits
vaibhavchellani Jun 12, 2019
2962ede
add circle
ChihChengLiang Jun 11, 2019
38c6e7f
setupDB
ChihChengLiang Jun 12, 2019
2f9de11
test sentTx
ChihChengLiang Jun 12, 2019
90e5599
add test dependencies
ChihChengLiang Jun 12, 2019
3f741aa
add npm run test
ChihChengLiang Jun 12, 2019
51d5ce3
add submodule
ChihChengLiang Jun 12, 2019
2e79c3a
trivial
ChihChengLiang Jun 12, 2019
59ddaa3
update dependency
ChihChengLiang Jun 12, 2019
7c2998a
fix events
ChihChengLiang Jun 12, 2019
634bdf8
test-prepTx
ChihChengLiang Jun 12, 2019
03ed53d
Merge pull request #5 from ChihChengLiang/setup-ci
Jun 12, 2019
1ae983f
Merge pull request #6 from ChihChengLiang/test-prepTx
Jun 12, 2019
7fd7eeb
cosmetic a little bit
ChihChengLiang Jun 13, 2019
6245789
fix prepTx
ChihChengLiang Jun 13, 2019
b695e7b
fix toMultiHash
ChihChengLiang Jun 13, 2019
f102ce9
async works better
ChihChengLiang Jun 13, 2019
e6504ae
setup DB
ChihChengLiang Jun 13, 2019
1a9d929
add migration
ChihChengLiang Jun 13, 2019
ac3436e
fix submodule
ChihChengLiang Jun 13, 2019
c1f50b8
Merge pull request #7 from ChihChengLiang/fix-prepTx
Jun 13, 2019
28d96e2
hotfix
ChihChengLiang Jun 13, 2019
b6bc2be
Merge pull request #8 from ChihChengLiang/hotfix
Jun 13, 2019
832a9f2
failing asyncToGenerator
Jun 12, 2019
1c7c231
backup JIC
Jun 13, 2019
d4de814
WIP :-))))))
Jun 13, 2019
152d93f
working test prepare txs
Jun 14, 2019
cc95298
fix everything
ChihChengLiang Jun 14, 2019
ae52c2b
cosmetic
ChihChengLiang Jun 14, 2019
8d221df
Merge pull request #9 from ChihChengLiang/WIP-nonce
Jun 14, 2019
e5b02d2
updated migration path
vaibhavchellani Jul 6, 2019
534724a
added new migrations
vaibhavchellani Jul 6, 2019
00547e2
refactor
vaibhavchellani Jul 6, 2019
55a85c9
added modifiedBy for account
vaibhavchellani Jul 7, 2019
1582cca
Merge pull request #10 from vaibhavchellani/migration
Jul 7, 2019
747be67
add secret.js to set up coordinator prvkey
Jul 13, 2019
5d24b6c
add secret.js to set up coordinator prvkey
Jul 13, 2019
3fbffa0
'npm run setup' for 'node config/secret.js'
Jul 13, 2019
ba3d8bc
Update README.md
Jul 13, 2019
14db240
transation insertion and selection
vaibhavchellani Jul 14, 2019
23eec7b
generate genesis.json
Jul 14, 2019
31bb3e6
fixed async issue on send dummy tx
vaibhavchellani Jul 14, 2019
22075f6
working transaction DB save and update
vaibhavchellani Jul 14, 2019
08b078a
Merge pull request #11 from vaibhavchellani/secret
vaibhavchellani Jul 14, 2019
fa6b24a
fixed conflicts
vaibhavchellani Jul 14, 2019
5aa5561
Merge pull request #12 from vaibhavchellani/migration
Jul 14, 2019
8bf9aa6
fix relative paths in tests
Jul 14, 2019
e634be1
initialise zeroCache with npm run setup
Jul 15, 2019
738dd58
move circuitInput.js into src/helpers
Jul 15, 2019
c4ae114
list zeroCache in ascending order (0 is root, 12 is leaf)
Jul 15, 2019
5ccc9ce
Merge pull request #13 from vaibhavchellani/zeroCache
vaibhavchellani Jul 16, 2019
6f08dbe
Update README.md
Jul 17, 2019
6808b8b
added padding function
vaibhavchellani Jul 18, 2019
b36d341
Merge pull request #14 from vaibhavchellani/therealyingtong-patch-1
Jul 18, 2019
283c6cd
Merge pull request #15 from vaibhavchellani/padding
Jul 19, 2019
d5384ab
WIP integrating rollupnc utils
Jul 19, 2019
5915b4e
working validation of txs at /submitTx
Jul 20, 2019
57a8869
WIP padding, passing signature checks
Jul 21, 2019
f19052d
automate generation of preset-genesis.json
Jul 21, 2019
a4a5cf6
change test_e2e.js to send test txs from Alice acct
Jul 21, 2019
7699512
remove sendTestRequests.sh
Jul 21, 2019
44b09cc
implement lock for padding txs
Jul 21, 2019
c09321e
update status of selected txs from 0 to 1
Jul 21, 2019
3144237
persist test tx counts across processes
Jul 21, 2019
513a3fc
fix lock to only toggle when txArray > 0
Jul 21, 2019
5aa2a7f
implement padding, txTree, accountTree, sparseProof in processTxs()
Jul 21, 2019
b770a94
implement accountTree.save()
Jul 21, 2019
0697a78
fix off by 1 error in accountTree/getSparseProof
Jul 22, 2019
1a5c73f
successfully producing input.json
Jul 22, 2019
d6cde74
change config-sample.json settings
Jul 22, 2019
ab56141
fix fromIndex in test_e2e.js
Jul 23, 2019
2badca3
save emptyTxs to txTable
Jul 23, 2019
5f78d17
Merge pull request #16 from vaibhavchellani/refactor
vaibhavchellani Jul 25, 2019
f2917a2
Merge branch 'processTxs'
Jul 26, 2019
24c9da5
modify gitignore
Jul 26, 2019
e7733d4
Merge pull request #17 from vaibhavchellani/processTxs
vaibhavchellani Jul 26, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .babelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"presets": ["env"],
"plugins": [
"babel-plugin-add-module-exports",
"transform-class-properties",
"transform-decorators-legacy",
"transform-object-rest-spread"
]
}

40 changes: 40 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
version: 2.0
jobs:
core_test:
working_directory: ~/repo
docker:
- image: circleci/node:10.16.0
- image: mysql:5
environment:
MYSQL_ROOT_PASSWORD: "123"
- image: rabbitmq:3.7
steps:
- checkout
- run:
name: "Pull Submodules"
command: 'git submodule update --init --recursive'
- run:
name: update-npm
command: 'sudo npm install -g npm@latest'
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: install-npm-wee
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- ./node_modules
- run:
name: setup-config
command: 'cp ./config/config-sample.json ./config/config.json'
- run:
name: create-DB
command: 'npm run create-DB'
- run:
command: 'npm run test'
workflows:
version: 2
workflow:
jobs:
- core_test
8 changes: 8 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
module.exports = {
parser: 'babel-eslint',
extends: 'standard',
rules: {
'space-before-function-paren': ['error', 'never']
}
}

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node_modules
config/config.json
test/testCount.json
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "circomlib"]
path = circomlib
url = https://github.com/iden3/circomlib.git
23 changes: 23 additions & 0 deletions DB/createDB.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
'use strict';
const config = require('../config/config.js');
const {attachOnDuplicateUpdate} = require('knex-on-duplicate-update');

async function createDatabase() {
const knexConfig = require(process.cwd() + '/DB/knexfile');
knexConfig.connection.database = null;
const knex = require('knex')(knexConfig);
attachOnDuplicateUpdate();
console.log("Creating database ", global.gConfig.db_name);
try {
await knex.raw('CREATE DATABASE ' + global.gConfig.db_name);
} catch (e) {
if (e.errno === 1007) {
console.log("DB already exits")
} else {
console.log("Error while creating database", e)
}
}
await knex.destroy();
}

createDatabase();
7 changes: 7 additions & 0 deletions DB/dbClient.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
const {attachOnDuplicateUpdate} = require('knex-on-duplicate-update');

var config = require('./knexfile.js');
var knex = require('knex')(config);
attachOnDuplicateUpdate();

export default knex
16 changes: 16 additions & 0 deletions DB/knexfile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
var config = require('../config/config.js')

module.exports = {
client: 'mysql',
connection: {
user: global.gConfig.user,
password: global.gConfig.password,
database: global.gConfig.db_name
},
pool: {
max: 1000
},
migrations: {
directory: './migrations',
},
}
39 changes: 39 additions & 0 deletions DB/migrations/20190524003152_create_tx_table.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
exports.up = function(knex, Promise) {
return knex.schema.createTable("tx", function(t) {
t.string("hash")
.notNullable()
.primary()
.unique();
t.string("fromX").notNullable();
t.string("fromY").notNullable();
t.integer("fromIndex").notNullable();
t.string("toX").notNullable();
t.string("toY").notNullable();
t.integer("toIndex").notNullable();
t.integer("nonce").notNullable();
t.integer("amount").notNullable();
t.integer("tokenType").notNullable();
t.string("R8x").notNullable();
t.string("R8y").notNullable();
t.string("S").notNullable();
t.datetime("timestamp")
.notNullable()
.defaultTo(knex.fn.now());
// status definitions to later be included in a table like token type
// 0: processing
// 1: awaiting transfer inclusion in proof,
// 2: transfer included in proof,
// 3: awaiting withdraw inclusion in proof,
// TODO remove usage of 3
// 4: withdraw included in proof,
// 5: withdrawn on smart contract
t.integer("status")
.notNullable()
.defaultTo(0);
t.string("txRoot");
});
};

exports.down = function(knex, Promise) {
return knex.schema.dropTableIfExists("tx");
};
23 changes: 23 additions & 0 deletions DB/migrations/20190525210231_create_account.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
exports.up = function(knex, Promise) {
return knex.schema.createTable("accounts", function(t) {
t.integer("index")
.primary()
.unique();
t.string("pubkeyX").notNullable();
t.string("pubkeyY").notNullable();
t.integer("balance").notNullable();
t.integer("nonce").defaultTo(0);
t.integer("tokenType").notNullable();
t.string("modifiedBy");
t.datetime("createdAt")
.defaultTo(knex.fn.now())
.notNullable();
t.datetime("updatedAt")
.defaultTo(knex.fn.now())
.notNullable();
});
};

exports.down = function(knex, Promise) {
return knex.schema.dropTableIfExists("accounts");
};
13 changes: 13 additions & 0 deletions DB/migrations/20190525210232_create_deposits.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module.exports = {
up: knex =>
knex.schema.createTable("deposits", table => {
table.string("txHash").notNullable().unique().primary();
table.integer("blockNumber").notNullable();
table.string("pubkeyX").notNullable();
table.string("pubkeyY").notNullable();
table.integer("amount").notNullable();
table.integer("tokenType").notNullable();
table.string("depositHash");
}),
down: knex => knex.schema.dropTable("deposits")
};
18 changes: 18 additions & 0 deletions DB/migrations/20190706235129_account_tree.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
module.exports = {
up: knex =>
knex.schema.createTable("account_tree", table => {
table.integer("depth").notNullable();
table.integer("index").notNullable();
table.unique(["depth", "index"])
table.string("hash").notNullable();
table
.datetime("createdAt")
.notNullable()
.defaultTo(knex.fn.now());
table
.datetime("modifiedAt")
.notNullable()
.defaultTo(knex.fn.now());
}),
down: knex => knex.schema.dropTable("account_tree")
};
18 changes: 18 additions & 0 deletions DB/migrations/20190706235549_create_state_transition_table.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
exports.up = function(knex, Promise) {
return knex.schema.createTable("state_transitions", function(t) {
t.string("txRoot")
.primary()
.unique()
.notNullable();
t.string("toIndexes").notNullable();
t.string("txHashes").notNullable();
t.json("proof").notNullable();
t.datetime("createdAt")
.notNullable()
.defaultTo(knex.fn.now());
});
};

exports.down = function(knex, Promise) {
return knex.schema.dropTableIfExists("state_transitions");
};
84 changes: 84 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# RollupNC co-ordinator

Acts as a co-ordinator between users for RollupNC

## Install

> Please ensure you have `rabbitmq-server` running in background

Install node version 10.16.0, possibly using nvm

```bash
npm i
```
## Setup

- randomly generate coordinator prvkey and pubkey,
- set DB password, and
- initialise `zeroCache`.

(from root directory)
```bash
cp config/config-sample.json config/config.json
npm run setup
```

NB: you can manually change the secrets in `config.json` instead of running `npm run setup`.

## Create Migration
```bash
knex migrate:make account_tree --knexfile=DB/knexfile.js
```
## Run Migrations

```bash
npm run migrations
```

> Note mysql needs to be installed

## Reset DB and Rerun Migrations

```bash
npm run redo
```

## Alter genesis balance state

To alter the genesis state, change config/genesis.json

## Run

> Please set configration as per requirement in config/config.json

```bash
npm run coordinator
```

## Send test transactions

```bash
bash sendTestRequests.sh
```

## Send test transaction

```bash
curl -X POST \
http://localhost:3000/submitTx \
-H 'Content-Type: application/json' \
-H 'Postman-Token: 6a5d7f6b-2189-4b7b-81c8-dd62067217a4' \
-H 'cache-control: no-cache' \
-d '{
"fromX":"5686635804472582232015924858874568287077998278299757444567424097636989354076",
"fromY":"20652491795398389193695348132128927424105970377868038232787590371122242422611",
"toX":"5188413625993601883297433934250988745151922355819390722918528461123462745458",
"toY":"12688531930957923993246507021135702202363596171614725698211865710242486568828",
"amount":"500",
"tokenType":"10",
"signature":{
"R8": "16983338799297102739311784762032344870039222281986241143318530383875079624912,17974713114491039907234265242193070110554943394288196512948620624137364193552",
"S": "1343828434385488942743107335063404632386867645513272609057729341930827645964"
}
}'
```
1 change: 1 addition & 0 deletions circomlib
Submodule circomlib added at 779288
48 changes: 48 additions & 0 deletions config/config-sample.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"development": {
"amqp_url": "amqp://localhost",
"tx_queue": "transfer",
"txs_per_snark": 8,
"log_level": "debug",
"poll_interval": "15000",
"contractAddress": "",
"RPC": "http://localhost:8545",
"tx_depth": 3,
"balance_depth": 12,
"host": "localhost",
"user": null,
"password": null,
"pubkey": null,
"prvkey": null,
"alicePubkey": null,
"alicePrvkey": null,
"bobPubkey": null,
"bobPrvkey": null,
"db_name": "rollupNC",
"port": 3000
},
"staging": {
"amqp_url": "amqp://localhost",
"tx_queue": "tx",
"txs_per_snark": 4,
"log_level": "info",
"poll_interval": "1000",
"contractAddress": "",
"RPC": "https://ropsten.etherscan.io",
"tx_depth": 2,
"balance_depth": 2,
"port": 3000
},
"production": {
"amqp_url": "amqp://<insert ip>",
"tx_queue": "tx",
"txs_per_snark": 4,
"log_level": "info",
"poll_interval": "1000",
"contractAddress": "",
"RPC": "https://mainnet.etherscan.io",
"tx_depth": 2,
"balance_depth": 2,
"port": 3000
}
}
13 changes: 13 additions & 0 deletions config/config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// requires
const _ = require("lodash");

// module variables
const config = require("./config.json");
const defaultConfig = config.development;
const environment = process.env.NODE_ENV || "development";
const environmentConfig = config[environment];
const finalConfig = _.merge(defaultConfig, environmentConfig);

global.gConfig = finalConfig;

console.log("Loaded config....");
Loading