Skip to content

Commit

Permalink
Deliver 0.7.0
Browse files Browse the repository at this point in the history
  • Loading branch information
jcwangxp committed Aug 26, 2024
1 parent a331dd1 commit 54fda38
Show file tree
Hide file tree
Showing 39 changed files with 1,881 additions and 522 deletions.
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Change Log

## 0.7.0 <small>(2024-08-26)</small>

**Features**

- `UPDATE` SET clause supports simple expression, ex: `SET val=val+5` `SET val=a-b`
- `UPDATE` SET supports prepared statement
- `INSERT` supports prepared statement
- New APIs: `xdb_bexec`, `xdb_vbexec`, `xdb_stmt_bexec`, `xdb_stmt_vexec`, `xdb_clear_bindings`

**Improvements**

- `INSERT` parser avoids malloc
- `UPDATE` only updates affected indexes
- Optimize `INSERT` `UPDATE` `DELETE` auto-commit performance for `IMDB`

**Test**

- Improve benchmark test
- Add `SQLite` benchmark test

**Bug Fixes**

- Fix hash index infinite loop issue
- Fix bench test time unit `ns` to `us`


## 0.6.0 <small>(2024-08-15)</small>

- **Initial refactor release**
Expand Down
22 changes: 12 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,35 +5,34 @@ help:
@echo "make clean Clean build result"
@echo "make install Install crossdb(lib&tool&header) to Linux/FreeBSD"
@echo "make uninstall Uninstall crossdb from Linux/FreeBSD"
@echo "make installmacos Install crossdb(lib&tool&header) to MacOS"
@echo "make uninstallmacos Uninstall crossdb from MacOS"
@echo "make installmac Install crossdb(lib&tool&header) to MacOS"
@echo "make uninstallmac Uninstall crossdb from MacOS"
@echo "make example Build and run example (need to install crossdb first)"
@echo "make bench Build and run bench test (need to install crossdb first)"
@echo "make bench-sqlite Build and run sqlite bench test (need to install sqlite3 first)"

.PHONY: build
build:
mkdir -p build
$(CC) -o build/libcrossdb.so -fPIC -shared -lpthread -O2 src/crossdb.c
$(CC) -o build/crossdb src/main.c -lpthread -O2
cp src/crossdb.h build/
cp include/crossdb.h build/

debug:
$(CC) -o build/libcrossdb.so -fPIC -lpthread -shared -g src/crossdb.c
$(CC) -o build/crossdb src/main.c -lpthread -g
cp src/crossdb.h build/
cp include/crossdb.h build/

run:
build/crossdb

clean:
rm -rf build/*
make -C examples/c/ clean
make -C bench/c/ clean
make -C bench/basic/ clean

wall:
$(CC) -o build/libcrossdb.so -fPIC -shared -lpthread -O2 -Wall src/crossdb.c
$(CC) -o build/crossdb src/main.c -lpthread -O2 -Wall
cp src/crossdb.h build/

gdb:
$(CC) -o build/crossdb src/main.c -lpthread -g
Expand All @@ -49,13 +48,13 @@ uninstall:
rm -rf /usr/include/crossdb.h
rm -rf /usr/bin/crossdb

installmacos:
installmac:
$(CC) -o build/libcrossdb.so -dynamiclib -lpthread -O2 src/crossdb.c
install -c build/libcrossdb.so /usr/local/lib/
install -c build/crossdb.h /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
install -c build/crossdb /usr/local/bin/

uninstallmacos:
uninstallmac:
rm -rf /usr/local/lib/libcrossdb.so
rm -rf /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/crossdb.h
rm -rf /usr/local/bin/crossdb
Expand All @@ -65,4 +64,7 @@ example:

.PHONY: bench
bench:
make -C bench/c/
make -C bench/basic/

bench-sqlite:
make -C bench/basic/ sqlite
31 changes: 22 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,22 @@
</a>
</p>
<p align="center">
<strong>Super High-performance Embedded and Server RDBMS</strong>
<strong>Super High-performance Lightweight Embedded and Server OLTP RDBMS</strong>
</p>

> **NOTE**
> This project was redesigned and rewritten from scratch.
> It's still in early development stage, so please DO NOT use in your project now.
# Introduction

**CrossDB** is a super high-performance lightweight embedded and server SQL RDBMS.
**CrossDB** is a super high-performance lightweight embedded and server OLTP RDBMS.
It's developed for high performance scenarios with main memory can hold whole DB.


## Features

- Support Multiple OS Platforms: Linux/Windows/MacOS/FreeBSD etc
- Support Multiple CPU ARCH: X86/ARM/PPC/MIPS etc
- Support OnDisk/In-memory/RamDisk/Hybrid Storage
- Support OnDisk/In-memory(IMDB)/RamDisk/Hybrid Storage
- Support Standard RDBMS model
- Support Standard SQL and many extensions from MySQL
- Support Multiple databases
Expand All @@ -41,6 +40,7 @@ It's developed for high performance scenarios with main memory can hold whole DB
- Very Simple: Simple header and library file
- Zero Config: no complex config, real out-of-the-box


## Use Cases

- High-frenquency trade (OLTP)
Expand All @@ -50,6 +50,7 @@ It's developed for high performance scenarios with main memory can hold whole DB
- You can use CrossDB RamDisk DB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
- You can use CrossDB to work as a super fast cache DB.


## Build and Install

### Linux/FreeBSD
Expand All @@ -63,7 +64,7 @@ sudo make install

```bash
make build
sudo make installmacos
sudo make installmac
```

### Windows
Expand All @@ -72,7 +73,7 @@ You need to install [MINGW64](https://www.mingw-w64.org/) to build.
Then set `gcc` path to `system environment variables` `Path` and make sure `gcc` can run.

```
build.bat
winbuild.bat
```

## Contribution
Expand All @@ -82,5 +83,17 @@ Following contributions are welcome:
- Language bindings: `Python`, `Java`, `Go`, `CSharp`, `Javascript`, `PHP`, etc
- Test and report bugs

## Want to Lean More?
https://crossdb.org

## Reference

### SQL Statements

https://crossdb.org/sql/statements/

### APIs

https://crossdb.org/client/api-c/

### Tutorial

https://crossdb.org/get-started/tutorial/
18 changes: 18 additions & 0 deletions bench/basic/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
all:
$(CC) -o bench-crossdb bench-crossdb.c -O2 -lcrossdb -lpthread
./bench-crossdb

debug:
$(CC) -o bench-crossdb bench-crossdb.c ../../src/crossdb.c -g -fsanitize=address
./bench-crossdb

sqlite:
$(CC) -o bench-sqlite bench-sqlite.c -O2 -lsqlite3
./bench-sqlite

fast:
$(CC) -o bench-crossdb bench-crossdb.c ../../src/crossdb.c -O3 -march=native
./bench-crossdb

clean:
rm -f a.out bench-crossdb bench-sqlite
160 changes: 160 additions & 0 deletions bench/basic/bench-crossdb.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@

#define BENCH_DBNAME "CrossDB"

#define LKUP_COUNT 10000000
#define SQL_LKUP_COUNT LKUP_COUNT/5
#define UPD_COUNT LKUP_COUNT/10

#include "bench.h"
#include <crossdb.h>

void* bench_create ()
{
xdb_conn_t* pConn = xdb_open (":memory:");
XDB_CHECK (NULL != pConn, printf ("Can't open connection:\n"); return NULL;);

xdb_res_t* pRes = xdb_exec (pConn, "CREATE TABLE student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score INT)");
XDB_RESCHK (pRes, printf ("Can't create table student\n"); xdb_close(pConn); return NULL;);

return pConn;
}

void bench_close (void *pConn)
{
xdb_close (pConn);
}

void bench_sql_test (void *pConn, int STU_COUNT, bool bRand, bench_result_t *pResult)
{
xdb_res_t* pRes;
xdb_row_t *pRow;

bench_print ("\n[============= SQL Test =============]\n\n");


bench_print ("------------ INSERT %d ------------\n", STU_COUNT);
bench_ts_beg();
for (int i = 0; i < STU_COUNT; ++i) {
pRes = xdb_bexec (pConn, "INSERT INTO student (id,name,age,class,score) VALUES (?,?,?,?,?)",
i, STU_NAME(i), STU_AGE(i), STU_CLASS(i), STU_SCORE(i));
XDB_RESCHK (pRes, bench_print ("Can't insert table student id=%d\n", i); return;);
}
pResult->insert_qps += bench_ts_end (STU_COUNT);


bench_print ("------------ %s LKUP %d ------------\n", ORDER_STR(bRand), SQL_LKUP_COUNT);
uint64_t qps_sum = 0;
for (int t = 0; t < 5; ++t) {
int count = 0;
bench_ts_beg();
for (int i = 0; i < SQL_LKUP_COUNT; ++i) {
pRes = xdb_bexec (pConn, "SELECT * FROM student WHERE id=?", STU_ID(i));
pRow = xdb_fetch_row (pRes);
if (NULL != pRow) {
int id = *(int*)pRow[0];
const char *name = (char*)pRow[1];
int age = *(int*)pRow[2];
const char *class = (char*)pRow[3];
int score = *(int*)pRow[4];
(void)id; (void)name; (void)age; (void)class; (void)score;
count++;
}
xdb_free_result (pRes);
}
qps_sum += bench_ts_end (SQL_LKUP_COUNT);
XDB_CHECK (count == SQL_LKUP_COUNT, bench_print ("OK %d != LKUP %d\n", count, SQL_LKUP_COUNT););
}
pResult->query_qps += qps_sum / 5;


bench_print ("------------ %s UPDATE %d ------------\n", ORDER_STR(bRand), UPD_COUNT);
bench_ts_beg();
for (int i = 0; i < UPD_COUNT; ++i) {
pRes = xdb_bexec (pConn, "UPDATE student SET age=age+? WHERE id=?", 1, STU_ID(i));
XDB_RESCHK (pRes, bench_print ("Can't update table student id%d\n", STU_ID(i)); return;);
}
pResult->update_qps += bench_ts_end (UPD_COUNT);


bench_print ("------------ %s DELETE %d ------------\n", ORDER_STR(bRand), STU_COUNT);
bench_ts_beg();
for (int i = 0; i < STU_COUNT; ++i) {
pRes = xdb_bexec (pConn, "DELETE FROM student WHERE id=?", STU_ID(i));
XDB_RESCHK (pRes, bench_print ("Can't delete table student id=%d\n", STU_ID(i)); return;);
}
pResult->delete_qps += bench_ts_end (STU_COUNT);
}

void bench_pstmt_test (void *pConn, int STU_COUNT, bool bRand, bench_result_t *pResult)
{
xdb_res_t* pRes;
xdb_row_t *pRow;
xdb_stmt_t *pStmt = NULL;

bench_print ("\n[============= Prepared STMT Test =============]\n");


bench_print ("\n------------ INSERT %d ------------\n", STU_COUNT);
pStmt = xdb_stmt_prepare (pConn, "INSERT INTO student (id,name,age,class,score) VALUES (?,?,?,?,?)");
bench_ts_beg();
for (int i = 0; i < STU_COUNT; ++i) {
pRes = xdb_stmt_bexec (pStmt, i, STU_NAME(i), STU_AGE(i), STU_CLASS(i), STU_SCORE(i));
XDB_RESCHK (pRes, bench_print ("Can't insert table student id%d\n", i); goto error;);
}
pResult->insert_qps += bench_ts_end (STU_COUNT);
xdb_stmt_close (pStmt);


bench_print ("------------ %s LKUP %d ------------\n", ORDER_STR(bRand), LKUP_COUNT);
pStmt = xdb_stmt_prepare (pConn, "SELECT * FROM student WHERE id=?");
XDB_CHECK (pStmt != NULL, goto error);
uint64_t qps_sum = 0;
for (int t = 0; t < 5; ++t) {
int count = 0;
bench_ts_beg();
for (int i = 0; i < LKUP_COUNT; ++i) {
pRes = xdb_stmt_bexec (pStmt, STU_ID(i));
pRow = xdb_fetch_row (pRes);
if (NULL != pRow) {
int id = *(int*)pRow[0];
const char *name = (char*)pRow[1];
int age = *(int*)pRow[2];
const char *class = (char*)pRow[3];
int score = *(int*)pRow[4];
(void)id; (void)name; (void)age; (void)class; (void)score;
count++;
}
xdb_free_result (pRes);
}
qps_sum += bench_ts_end (LKUP_COUNT);
XDB_CHECK (count == LKUP_COUNT, bench_print ("OK %d != LKUP %d\n", count, LKUP_COUNT););
}
xdb_stmt_close (pStmt);
pResult->query_qps += qps_sum / 5;


bench_print ("------------ %s UPDATE %d ------------\n", ORDER_STR(bRand), UPD_COUNT);
pStmt = xdb_stmt_prepare (pConn, "UPDATE student SET age=age+? WHERE id=?");
XDB_CHECK (pStmt != NULL, goto error);
bench_ts_beg();
for (int i = 0; i < UPD_COUNT; ++i) {
pRes = xdb_stmt_bexec (pStmt, 1, STU_ID(i));
XDB_RESCHK (pRes, bench_print ("Can't update table student id%d\n", STU_ID(i)); goto error;);
}
pResult->update_qps = bench_ts_end (UPD_COUNT);
xdb_stmt_close (pStmt);


bench_print ("------------ %s DELETE %d ------------\n", ORDER_STR(bRand), STU_COUNT);
pStmt = xdb_stmt_prepare (pConn, "DELETE FROM student WHERE id=?");
XDB_CHECK (pStmt != NULL, goto error);
bench_ts_beg();
for (int i = 0; i < STU_COUNT; ++i) {
pRes = xdb_stmt_bexec (pStmt, STU_ID(i));
XDB_RESCHK (pRes, bench_print ("Can't delete table student id%d\n", STU_ID(i)); goto error;);
}
pResult->delete_qps += bench_ts_end (STU_COUNT);

error:
xdb_stmt_close (pStmt);
}
Loading

0 comments on commit 54fda38

Please sign in to comment.