> confidence interval (so nothing spectacular). > are outside of the 95% confidence interval with one just outside the 99% > just ran a small test with 100k games on that and 3 out of 60 engines > the suggested tournament between 60 identical engines (using SF 5).
(if successful) rerunning the test to remove that negative spike. Of course, most people would use a standard shared library compile soĪdapting the test to that would need some code/deployment adoption and Įssentially the Ermine build goes from being faster to being slower thanĪ standard compile in that particular revision. (dynamic) compile actually improves performance from to. Performance drop-off from revision to, whereas doing a standard ( ) to ship the engines as self-contained For running the engines on the cluster I use Ermine Non-functional) at depth 20 and what I found out is that the root issue In parallel, I did some longer bench runs (as the change is Was reproducible (significant elo drop - not quite as large but still 9+ If it vanishes, then we have to look into the assumptions that lie behind the error bars. If the elo spike persists, then it's worth looking into why. > The golden rule when you get an unexpected result is "can it be reproduced?" So you could set up a tournament between the three versions concerned. Am I missing something here or does someone have a better explanation for this? We know that Komodo 8 performs much better at longer time controls, but such a huge ELO difference doesn't seem achievable simply by changing Time controls.
So it is surprising that these tests have shown komodo8 as trailing Stockfish5 by 26 ELO(and SF dev version by 75 ELO). However I find it difficult to understand why Komodo 8 is shown to be 26 ELO weaker than Stockfish 5(Around 70 ELO weaker than latest SF development versions).ĬCRL results have confirmed that Komodo8 is clearly stronger than SF5. Variations can be expected considering the opening book is different from fishtest.
This is more or less consistent with the results obtained from fishetest(last regression test showed 41 ELO gain). One question: Is Komodo 1288.00 64-bit is same as Komodo 8?įrom these results it looks like the Stockfish devlopment versions show 41 - 49 ELO gain over SF5. These are very useful results since very few people have such hardware at their disposal ! Retire hackish support for aCC and HP-UX from Makefileįix out-of-bound array access printing ponder move Retire support for Haiku installation directory from Makefile
0 kommentar(er)
