Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add vertices in sorted order #598

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

add vertices in sorted order #598

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5cbc6ba
add vertices in sorted order
Nov 7, 2024
4afeed4
update gitignore
Nov 7, 2024
051c69f
add logs
Nov 7, 2024
1b1661b
add sorted results
Nov 7, 2024
28f4ac4
add baseline results
Nov 7, 2024
dc42a6a
add sorted results
Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ ipch/
*.vspx
*.sap

# Visual Studio Code Settings
.vscode

# Visual Studio Trace Files
*.e2e

Expand Down
84 changes: 84 additions & 0 deletions prec40M/baseline_r64_k10.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
Reading (with alignment) bin file /nvmessd1/fbv4/queries384d.bin ...Metadata: #pts = 13265, #dims = 384, aligned_dim = 384... allocating aligned memory of 5093760 bytes... done. Copying data to mem_aligned buffer... done.
Opened: /nvmessd1/fbv4/gt100_prec40M, size: 10612008, cache_size: 10612008
Reading truthset file /nvmessd1/fbv4/gt100_prec40M ...
Metadata: #pts = 13265, #dims = 100...
Using AVX2 distance computation DistanceL2Int8.
Resizing took: 1.40021s
From graph header, expected_file_size: 7438757948, _max_observed_degree: 64, _start: 11890341, file_frozen_pts: 0
Loading vamana graph /nvmessd1/fbv4/avarhade/prec40M_memory_index_r64_l100.......done. Index has 40469609 nodes and 1819219872 out-edges, _start is set to 11890341
Num frozen points:0 _nd: 40469609 _start: 11890341 size(_location_to_tag): 0 size(_tag_to_location):0 Max points: 40469609
Index loaded
Using 48 threads to search
Ls QPS Avg dist cmps Mean Latency (mus) 99.9 Latency Recall@10
=================================================================================
10 51557.93 1324.67 913.18 9792.77 43.13
20 33640.81 2021.14 1420.26 3249.95 58.38
30 25210.59 2617.97 1896.59 4469.27 66.24
40 20547.19 3168.06 2326.82 5450.12 71.25
50 17328.33 3687.22 2760.19 6599.82 74.44
60 14980.50 4195.96 3193.32 7739.67 76.98
70 13341.03 4685.40 3588.47 8306.07 78.78
80 11462.47 5171.15 4176.58 11172.33 80.21
90 10723.69 5654.48 4462.36 10865.74 81.53
100 9782.57 6129.37 4892.21 12162.22 82.58
Done searching. Now saving results
Writing bin: /home/rakri/avarhade/Dump/tmp_10_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_10_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_20_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_20_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_30_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_30_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_40_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_40_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_50_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_50_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_idx_uint32.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_dists_float.bin
bin: #pts = 13265, #dims = 10, size = 530608B
Finished writing bin.
21 changes: 21 additions & 0 deletions prec40M/baseline_r64_k100.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Reading (with alignment) bin file /nvmessd1/fbv4/queries384d.bin ...Metadata: #pts = 13265, #dims = 384, aligned_dim = 384... allocating aligned memory of 5093760 bytes... done. Copying data to mem_aligned buffer... done.
Opened: /nvmessd1/fbv4/gt100_prec40M, size: 10612008, cache_size: 10612008
Reading truthset file /nvmessd1/fbv4/gt100_prec40M ...
Metadata: #pts = 13265, #dims = 100...
Using AVX2 distance computation DistanceL2Int8.
Resizing took: 1.40809s
From graph header, expected_file_size: 7438757948, _max_observed_degree: 64, _start: 11890341, file_frozen_pts: 0
Loading vamana graph /nvmessd1/fbv4/avarhade/prec40M_memory_index_r64_l100.......done. Index has 40469609 nodes and 1819219872 out-edges, _start is set to 11890341
Num frozen points:0 _nd: 40469609 _start: 11890341 size(_location_to_tag): 0 size(_tag_to_location):0 Max points: 40469609
Index loaded
Using 48 threads to search
Ls QPS Avg dist cmps Mean Latency (mus) 99.9 Latency Recall@100
=================================================================================
100 10331.70 6129.37 4626.05 11214.14 66.81
Done searching. Now saving results
Writing bin: /home/rakri/avarhade/Dump/tmp_100_idx_uint32.bin
bin: #pts = 13265, #dims = 100, size = 5306008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_dists_float.bin
bin: #pts = 13265, #dims = 100, size = 5306008B
Finished writing bin.
70 changes: 70 additions & 0 deletions prec40M/baseline_r64_k25.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
Reading (with alignment) bin file /nvmessd1/fbv4/queries384d.bin ...Metadata: #pts = 13265, #dims = 384, aligned_dim = 384... allocating aligned memory of 5093760 bytes... done. Copying data to mem_aligned buffer... done.
Opened: /nvmessd1/fbv4/gt100_prec40M, size: 10612008, cache_size: 10612008
Reading truthset file /nvmessd1/fbv4/gt100_prec40M ...
Metadata: #pts = 13265, #dims = 100...
Using AVX2 distance computation DistanceL2Int8.
Resizing took: 1.48129s
From graph header, expected_file_size: 7438757948, _max_observed_degree: 64, _start: 11890341, file_frozen_pts: 0
Loading vamana graph /nvmessd1/fbv4/avarhade/prec40M_memory_index_r64_l100.......done. Index has 40469609 nodes and 1819219872 out-edges, _start is set to 11890341
Num frozen points:0 _nd: 40469609 _start: 11890341 size(_location_to_tag): 0 size(_tag_to_location):0 Max points: 40469609
Index loaded
Using 48 threads to search
Ls QPS Avg dist cmps Mean Latency (mus) 99.9 Latency Recall@25
=================================================================================
30 25253.53 2617.97 1885.01 10497.99 60.18
40 20654.60 3168.06 2316.21 5201.40 65.38
50 17413.17 3687.22 2747.47 6720.12 68.82
60 15010.45 4195.96 3187.87 7331.08 71.60
70 13300.10 4685.40 3599.56 8146.48 73.66
80 11990.02 5171.15 3991.75 8909.45 75.35
90 10918.86 5654.48 4385.05 9846.92 76.87
100 9976.54 6129.37 4799.29 10763.89 78.14
Done searching. Now saving results
Writing bin: /home/rakri/avarhade/Dump/tmp_30_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_30_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_40_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_40_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_50_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_50_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_idx_uint32.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_dists_float.bin
bin: #pts = 13265, #dims = 25, size = 1326508B
Finished writing bin.
56 changes: 56 additions & 0 deletions prec40M/baseline_r64_k50.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
Reading (with alignment) bin file /nvmessd1/fbv4/queries384d.bin ...Metadata: #pts = 13265, #dims = 384, aligned_dim = 384... allocating aligned memory of 5093760 bytes... done. Copying data to mem_aligned buffer... done.
Opened: /nvmessd1/fbv4/gt100_prec40M, size: 10612008, cache_size: 10612008
Reading truthset file /nvmessd1/fbv4/gt100_prec40M ...
Metadata: #pts = 13265, #dims = 100...
Using AVX2 distance computation DistanceL2Int8.
Resizing took: 1.70302s
From graph header, expected_file_size: 7438757948, _max_observed_degree: 64, _start: 11890341, file_frozen_pts: 0
Loading vamana graph /nvmessd1/fbv4/avarhade/prec40M_memory_index_r64_l100.......done. Index has 40469609 nodes and 1819219872 out-edges, _start is set to 11890341
Num frozen points:0 _nd: 40469609 _start: 11890341 size(_location_to_tag): 0 size(_tag_to_location):0 Max points: 40469609
Index loaded
Using 1 threads to search
Ls QPS Avg dist cmps Mean Latency (mus) 99.9 Latency Recall@50
=================================================================================
50 321.67 3687.22 3108.74 9766.01 62.96
60 426.77 4195.96 2343.13 6894.97 65.92
70 459.30 4685.40 2177.17 21952.75 68.19
80 416.92 5171.15 2398.49 22705.72 70.07
90 384.96 5654.48 2597.62 23008.43 71.76
100 353.87 6129.37 2825.84 23359.16 73.17
Done searching. Now saving results
Writing bin: /home/rakri/avarhade/Dump/tmp_50_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_50_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_60_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_70_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_80_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_90_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_idx_uint32.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Writing bin: /home/rakri/avarhade/Dump/tmp_100_dists_float.bin
bin: #pts = 13265, #dims = 50, size = 2653008B
Finished writing bin.
Loading
Loading