|
|
Post by Ozymandias on Apr 25, 2024 13:27:40 GMT
In absence of a reliable rating list of "bin" books, I'm making my own. I'll try to update it with whatever top public book I can find. I doubt private books will enter the fray, because authors would most likely expect the games, which I don't plan on releasing. In the list, some public books usually populating Sedat's lists, plus the one I use to test everyone against (mine) and at the bottom, the old one I used for so many years (over half a billion games) which serves as an anchor to provide more realistic Elos.
Conditions:
- TC 36s + 0.6s increment, asmbook (latest asmFish) to depth 13.
- CPU Ryzen 3GHZ, 1 core per engine/book pair
- GUI cutechess-cli 1.2.0
- -tbignore50 enabled, everything else default
- Engine: SF dev for everyone. Killfish for my book.
- No adjudication rules other than 6-men Syzygy (also available to the engines during play).
# PLAYER : RATING POINTS PLAYED (%) CFS(%) W D L D(%)
1 LesChats160424 : 3112.7 5568.5 10000 56 --- 1284 8569 147 86
2 Leschats150324 : 3102.2 5606.0 10000 56 --- 1438 8336 226 83
3 HCanFree_24 : 3094.4 5496.5 10000 55 --- 1402 8189 409 82
4 Sailfish_2 : 3074.7 37240.5 72000 52 --- 5021 64439 2540 89
5 OPTIMUS2403 : 3073.0 5193.0 10000 52 --- 753 8880 367 89
6 kk_10+60_190 : 3072.7 21652.0 35000 62 --- 10546 22212 2242 63
7 Tunafish : 3070.6 4970.5 10000 50 --- 612 8717 671 87
..............................................................................................
31 asmbook : 2460.0 294.5 10000 3 --- 6 577 9417 6
White advantage = 8.25
Draw rate (equal opponents) = 50.00 %
I skipped all the experiments I played from 8 to 30 against Sailfish. As you can see, with more variety of play, and more games, Optimus isn't nearly as good as it seems on Sedat's tests. In fact, the only books which pull ahead are HCanFree and LesChats. I doubt a private book would fare better than any of these two.
|
|
|
|
Post by Ozymandias on Apr 29, 2024 5:45:03 GMT
Two books update. HCan latest almost at the level of previous LesChats, but both of them progress thanks to a rise in draw rate. It's not like they find new ways to win (on the contrary), they just plug holes. And the 4 year old Cerebellum proves this trend by retaining second place with almost a 90% draw!
# PLAYER : RATING POINTS PLAYED (%) CFS(%) W D L D(%)
1 LesChats160424 : 3112.8 5568.5 10000 56 --- 1284 8569 147 86
2 Cerebellum3Merge : 3105.7 5468.0 10000 55 --- 991 8954 55 90
3 Leschats150324 : 3102.3 5606.0 10000 56 --- 1438 8336 226 83
4 HCanFree : 3101.8 5413.5 10000 54 --- 1188 8451 361 85
5 HCanFree_24 : 3094.5 5496.5 10000 55 --- 1402 8189 409 82
6 Sailfish_2 : 3074.8 37240.5 72000 52 --- 5021 64439 2540 89
7 OPTIMUS2403 : 3073.1 5193.0 10000 52 --- 753 8880 367 89
8 kk_10+60_190 : 3072.8 30770.5 55000 56 --- 10962 39617 4421 72
9 Tunafish : 3070.7 4970.5 10000 50 --- 612 8717 671 87
..............................................................................................
33 asmbook : 2460.0 294.5 10000 3 --- 6 577 9417 6
White advantage = 9.96
Draw rate (equal opponents) = 50.00 %
|
|
|
|
Post by Ozymandias on May 3, 2024 9:20:59 GMT
And two more books. The focus of attention right now, because of cloning allegations, Turkey, and an old one from Massimiliano Goi. The old one doesn't perform very well, unlike Cerebellum (which is older) but at least has a lower draw rate. As for the Optimus based book, despite being basically equal, the manual changes made had a positive and measurable impact: more wins AND less draws. I think I'll keep using it, but I'll have to change the name to reflect its origins.
# PLAYER : RATING POINTS PLAYED (%) CFS(%) W D L D(%)
1 LesChats160424 : 3113.0 5568.5 10000 56 --- 1284 8569 147 86
2 Cerebellum3Merge : 3105.8 5468.0 10000 55 --- 991 8954 55 90
3 Leschats150324 : 3102.4 5606.0 10000 56 --- 1438 8336 226 83
4 HCanFree240424 : 3102.0 5413.5 10000 54 --- 1188 8451 361 85
5 HCanFree_24 : 3094.6 5496.5 10000 55 --- 1402 8189 409 82
6 TURKEY2404 : 3085.7 5182.0 10000 52 --- 813 8738 449 87
7 Sailfish_2 : 3074.9 37240.5 72000 52 --- 5021 64439 2540 89
8 OPTIMUS2403 : 3073.2 5193.0 10000 52 --- 753 8880 367 89
9 kk_10+60_190 : 3072.9 40819.0 75000 54 --- 12408 56822 5770 76
10 Tunafish : 3070.8 4970.5 10000 50 --- 612 8717 671 87
..............................................................................................
31 Goi241222 : 3056.7 4769.5 10000 48 --- 536 8467 997 85
..............................................................................................
35 asmbook : 2460.0 294.5 10000 3 --- 6 577 9417 6
White advantage = 11.97
Draw rate (equal opponents) = 50.00 %
|
|
|
|
Post by Ozymandias on May 5, 2024 5:48:06 GMT
Latest updates: a book based on a half a million FICS games (best CvsC ones) and Sedat's manual book. Most noticeable, that a book based on a much worse, much smaller DB performs just the same as the one based on the main one, despite using the same parameters for book building. The manual book, being just 49KB in size, allows SF to play most of the moves, which translates in the highest number of draws, as well as the lowest number of wins (among the top ten); that's why tests between engines starting from 2-moves suites are so boring.
# PLAYER : RATING POINTS PLAYED (%) CFS(%) W D L D(%)
1 LesChats160424 : 3113.0 5568.5 10000 56 --- 1284 8569 147 86
2 Cerebellum3Merge : 3105.8 5468.0 10000 55 --- 991 8954 55 90
3 Leschats150324 : 3102.4 5606.0 10000 56 --- 1438 8336 226 83
4 HCanFree240424 : 3102.0 5413.5 10000 54 --- 1188 8451 361 85
5 HCanFree_24 : 3094.6 5496.5 10000 55 --- 1402 8189 409 82
6 TURKEY2404 : 3085.7 5182.0 10000 52 --- 813 8738 449 87
7 Perfect2023 : 3082.7 5139.5 10000 51 --- 627 9025 348 90
8 Sailfish_2 : 3074.9 37240.5 72000 52 --- 5021 64439 2540 89
9 FICS : 3073.2 5005.0 10000 50 --- 710 8590 700 86
10 OPTIMUS2403 : 3073.2 5193.0 10000 52 --- 753 8880 367 89
11 kk_10+60_190 : 3072.9 50674.5 95000 53 --- 13456 74437 7107 78
12 Tunafish : 3070.8 4970.5 10000 50 --- 612 8717 671 87
..............................................................................................
33 Goi241222 : 3056.7 4769.5 10000 48 --- 536 8467 997 85
..............................................................................................
37 asmbook : 2460.0 294.5 10000 3 --- 6 577 9417 6
White advantage = 11.82
Draw rate (equal opponents) = 50.00 %Almost forgot, with Perfect2023, the SF dev was updated and Hash raised to 64MB (from default). |
|
