Skip to content

v0.3.1

Jump to bottom
QueensGambit edited this page Dec 27, 2018 · 15 revisions

Strength Evaluation v0.2.0

CrazyAra played all common available open-source crazyhouse chess engines.

Hardware Setup:

  • Memory (RAM): 31,4 GiB
  • Processor - AMD® Ryzen 7 1700 eight-core processor × 16
  • Graphics - GeForce GTX 1080 Ti/PCIe/SSE2
  • OS type - 64-bit

The various Alpha-Beta engines achieved a Node per Seconds of ~700,000-2,000,00.

CrazyAra v0.2.0 using MCTS had a Node per Seconds of 300 using both CPU and GPU.

Result Overview

All these matches have been conducted using 5min/40 moves time control.

Hyperparameter Settings for MCTS:

  • threads - 8
  • cpuct - 1
  • playouts_empty_pockets - 256
  • playouts_filled_pockets - 512
  • playouts_update_stats - 256
  • max_search_depth - 40
  • max_search_time_s - 30
  • dirichlet_alpha - 0.2
  • dirichlet_epsilon - 0.1
  • threshold_time_for_raw_net_ms - 30,000
Engine Name Estimated Elo Link
Sjeng 11.2 2300 https://github.com/gcp/sjeng, https://sjeng.org/download.html
SjaakII 1.4.1 2353.87 sudo apt-get install sjaakii, http://www.chess2u.com/t5010-sjaak
Sunsetter 9 2703 https://sourceforge.net/projects/sunsetter/files/latest/download
TJchess 1.3 2732.58 http://computer-chess.org/doku.php?id=computer_chess:wiki:lists:variants_engine_list
Stockfish 2018-10-13 3946.06 https://github.com/ddugovic/Stockfish

The elo estimates are taken from according to this tournament: https://lichess.org/forum/team-crazyhouse-engine-development-and-game-analyses/swiss-crazyhouse-tournament?page=3

Rank Name                                  Elo    +    - games score oppo. draws 
   1 stockfish-x86_64-modern  2018-10-13   502  285  150    34  100%   -96    0% 
   2 CrazyAra-0.3.1                         71  113  105    34   59%    -2    0% 
   3 TJchess_1.3_64bit_linux                -5  104   99    34   54%   -21    3% 
   4 Sunsetter-9                           -47  100   97    34   49%    -4    3% 
   5 sjeng                                -203  102  113    34   25%    63    3% 
   6 sjaakii                              -318  106  133    34   13%    60    3%

Superiority Matrix

                                     st Cr TJ Su sj
stockfish-x86_64-modern  2018-10-13     99 99 99 99
CrazyAra-0.3.1                        0    82 92 99
TJchess_1.3_64bit_linux               0 17    71 99
Sunsetter-9                           0  7 28    97
sjeng                                 0  0  0  2
Clone this wiki locally