Internal/quantify/run statistics

[SSoelvsten]

Closes #644

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.311%. Comparing base (897a723) to head (d14ff49).

Additional details and impacted files

@@              Coverage Diff              @@
##              main      #649       +/-   ##
=============================================
+ Coverage   97.309%   97.311%   +0.002%     
=============================================
  Files           90        90               
  Lines         6467      6471        +4     
=============================================
+ Hits          6293      6297        +4     
  Misses         174       174               

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

🔴 Regression Test (QBF 'hex/hein_08_5x5-11_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.49% (stdev: 0.31%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	19336.00	19431.33
Standard Deviation	52.20	59.68

Number of samples: 3

[github-actions]

🔴 Regression Test (QBF 'breakthrough/3x4_19_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -1.26% (stdev: 0.23%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	24073.00	24375.33
Standard Deviation	45.18	55.52

Number of samples: 3

[github-actions]

🔴 Regression Test (QBF 'ep/8x8_7_e-8-1_p-3-4_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.76% (stdev: 0.74%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	29336.33	29560.00
Standard Deviation	106.09	218.08

Number of samples: 3

[github-actions]

🟡 Regression Test (QBF 'ep_dual/8x8_6_e-8-1_p-2-3_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.60% (stdev: 0.96%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	5043.25	5073.50
Standard Deviation	48.42	40.76

Number of samples: 4

[github-actions]

🟡 Regression Test (QBF 'httt/4x4_9_tippy_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.18% (stdev: 1.10%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	10555.00	10574.25
Standard Deviation	116.57	25.86

Number of samples: 4

[github-actions]

🔴 Regression Test (QBF 'breakthrough_dual/3x6_10_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.90% (stdev: 0.63%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	5255.75	5303.25
Standard Deviation	33.01	27.93

Number of samples: 4

[github-actions]

🔴 Regression Test (QBF 'connect4/6x6_11_connect4_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -0.98% (stdev: 0.58%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	12314.25	12434.50
Standard Deviation	17.04	72.45

Number of samples: 4

[github-actions]

🟢 Regression Test (12-Queens)

'origin/internal/quantify/run_statistics' is a change in performance of 0.23% (stdev: 0.88%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	11514.75	11488.50
Standard Deviation	75.97	101.30

Number of samples: 4

[github-actions]

🟡 Regression Test (Picotrav 'arbiter')

'origin/internal/quantify/run_statistics' is a change in performance of -0.20% (stdev: 0.39%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	64716.67	64844.00
Standard Deviation	249.39	177.27

Number of samples: 3

[github-actions]

🔴 Regression Test (QBF 'domineering/5x5_13_bwnib')

'origin/internal/quantify/run_statistics' is a change in performance of -1.45% (stdev: 0.19%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	20109.75	20402.00
Standard Deviation	38.50	13.64

Number of samples: 4

[github-actions]

🟢 Regression Test (Picotrav 'adder')

'origin/internal/quantify/run_statistics' is a change in performance of 0.09% (stdev: 0.57%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	9476.25	9467.75
Standard Deviation	37.85	53.71

Number of samples: 4

[github-actions]

🟢 Regression Test (Picotrav 'mem_ctrl')

'origin/internal/quantify/run_statistics' is a change in performance of 0.80% (stdev: 0.94%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	109058.33	108186.33
Standard Deviation	1025.73	139.55

Number of samples: 3

[SSoelvsten]

You have to love how much changing the code (even if not changing what is run by the program) can affect the compiler's decisions...

[github-actions]

🟢 Regression Test (14-Queens)

'origin/internal/quantify/run_statistics' is a change in performance of 0.04% (stdev: 0.81%).

...	origin/main	origin/internal/quantify/run_statistics
Mean	267760.00	267656.33
Standard Deviation	1087.40	2165.23

Number of samples: 3