"Step 2. Check for aborted search and immediate draw. Enforce node limit here. (This only works with 1 search thread, as of Stockfish 2.3.1.)"

click for larger view

White to play. My Stockfish 7 gives +1.74 for White. However, Black can prevent White from making progress.

Unless their tablebase kicks in computers still have trouble recognising a fortress. How does it evaluate this position.

click for larger view

It entered the stage as the top seed, rated 3221, and won outright with 12/15 (+9,-0,=6).

Stockfish 110616 will be used for Stage 2 of the competition, where at 3222, it is now the 2nd seed, behind Komodo 10 (3228).

http://tcec.chessdom.com/archive.php

There's a video "Komodo and Stockfish analyze" at:

http://en.chessbase.com/post/john-h...

It appears at around 2:02 on the video, Stockfish analyzing for move 26.... "likes" 26...Nc2 27. Rxe8 Rxe8 etc. giving it an evaluation of (.34).

But then, after the video author does an alternative look, the move 26...Nc2 is played in the game being analyzed (at 2:15 or so on the video) but Stockfish analysis now thinks the evaluation is closer to (1.00) which I'm assuming means much better for white.

It appeared that Stockfish had a decent amount of think time for 26...Nc2, so my question is why would Stockfish, or any computer, apparently see a move with roughly equal evaluation and then have that evaluation change so drastically when the move is actually played?

Then on Black's turn on move 26, it will analyze until Black's turn on move 31 and evaluate those positions. After Black actually makes move 26, it is now starting from that position and looks until White's turn on move 32. That's a different set of positions, so they can have different evaluations.

1. The engine is looking N plies ahead and sees the forced loss of its queen. So it begins sacrificing pawns and pieces in order to push the loss of the queen to a depth of N+1 so at each ply evaluates the resulting position less to its disadvantage than after the loss of its queen. Eventually, of course, it runs out of pawns and pieces and has to lose its queen anyway. I believe that this was the original version of the horizon effect and it sure looks to a casual observer as though it short circuited.

2. The engine is looking N plies ahead and is therefore completely oblivious of what happens after ply N+1. The most extreme example I ever say (which, alas, I did not save) was when the engine evaluated the position as slightly in White's favor even though on the next move Black had a mate in one. To address this 2nd definition I coined "Ayler AylerKupp's Corollary to Murphy's Law" (AKC2ML) which can be expressed as "If you use your engine to analyze a position to a search depth=N, your opponent's killer move (the move that will refute your entire analysis) will be found at search depth=N+1, regardless of the value you choose for N."

Here is another example. I had Komodo 9.2 ((the strongest chess engine at the time) do an analysis of the following position:

click for larger view

Komodo produced the following evaluation of its 3rd best line: [+0.16], d=30: 27...Ra7 28.Be1 Qd7 29.Qd3 Rg7 30.Kh1 Kh8 31.Bf1 Ne8 32.Ne2 Qc8 33.Qb3 Bh4 34.Bc3 gxf3 35.gxf3 Bf2 36.a6 Rxa6 37.Rxb7 Rxb7 38.Qxb7 Qxb7 39.Rxb7 Ra8 40.Bh3 Nh4 41.Nxf4 Bd4 42.Ne2 Bxc3 43.Nxc3 Nxf3 44.Bf5 Nf6 45.Nb5 and it reached the following position:

click for larger view

But due to the 2nd version of the horizon effect after Komodo played 45.Nb5 it has no idea of the potential of Black's position. And here Black has a forced mate in 5 after 45...Ra2 (a quiet move so a quiescent search extension was not likely to be used) 46.Rb8+ Kg7 47.Rb7+ Kh6 48.Rh7+ (horizon effect version 1 at work) 48...Nxh7 49.Bxh7 Rxh2#

click for larger view

So given its the horizon effect, it appears that if Stockfish looked out one more move, it's initial 26...Nc2's OK as pretty equal would have turned to 26...Nc2 is bad and should not be considered.

Seems strange though that given the position, Black appears to only have a few decent initial moves. Can a computer compensate with a longer horizon with fewer good initial move options? For instance, with three initial good move options look out 50 moves but with six initial good move options or more stick with the standard 30 or whatever move horizon?

This kind of calculation error, and I'm even assuming it's a rare event, would seem to make any given computer suggested move obviously less credible.

It also might support a feeling I've had that a computers advantage over humans is just about all mechanical via brute force processing.

This might indicate that a match between a top GM on his game and a top computer with a 30 min clock for the GM and a 1 minute clock for the computer (or maybe a 60 vs. 2 even) could be very close since the computers horizon would likely need to be pared significantly.

It would also seem relatively fair as a 30 to 1 advantage on time should be viewed as minor given the computers huge mechanical processing advantage.

Interestingly, the latest Stockfish 7 version seems to prune its search tree less aggressively (and hence does not search as deeply) and the latest Komodo 10, like Hiarcs before it, provides the means for the user to tailor their search tree pruning aggressiveness. So both engines seem to be heading towards each other's approach in an attempt to find the best balance between the two.

I've never heard of computer-human time odds matches but in the past I have suggested that time handicapping might be a better approach to attempt to equalize the playing strength between chess engines and humans. Certainly more flexible than giving pawn, piece, and/or move odds as is currently being done since the time handicap can have a higher resolution. And after all, if the computer's main advantage is its ability to perform more calculations than humans, then it seems reasonable to attempt to equalize their playing abilities by reducing the computer's main advantage.

The issue then, of course, is how much of a time handicap should the human be given? I have also suggested that an approach to determine this might be to first play a series of engine vs. engine matches between the top 5 engines and 5 engines rated 500 points below them, initially giving the lower rated engines twice the time as the stronger engines and adjusting the time handicap up or down until the winning percentage of the lower rated engines is approximately the same as the winning percentage of the higher rated engines. Then that time handicap would be a good place to start, and it would not take any human's time other than managing the matches.

That might imply that the Stockfish program, if it relies on a wider range of initial moves to the detriment of depth, is more at risk in shorter time controls.

If that's the case and if Stockfish was to get challenged to a 30 min vs. 1 min human-computer match, how difficult might it be for the Stockfish team to improve its position assessment and thus more aggressively prune? Is the position based algorithms so part of the programs DNA that major changes are difficult?

I would personally categorize search tree pruning techniques into algorithms and heuristics. Algorithms, of which the best known is alpha-beta pruning, reduce the number of branches of the search tree that need to be searched in a way that it can be proven, or at least shown, that a search of the pruned tree is exactly equivalent to a search of the unpruned tree, at least as far as having the minimax algorithm provide the same result.

Heuristics are educated guesses; strategies that in general seem to reduce the amount of searching needed to determine the best move to be played in the minimax sense, but there is no guarantee that the search of the pruned tree will produce the same result as the search of the unpruned tree. The better heuristics will, of course, result in a near equivalence of the two (otherwise they wouldn't be used), but it is possible that a branch of the search tree containing the best move could be pruned.

I think that search tree pruning is the area where the most progress has been made in the last 10 years or so, and are the main reason why current engines can reach much greater search depths than 10-year old engines. There have certainly been increases in hardware capability by increased clock speeds, multiple cores, and multiple systems but given that the number of possible moves increases exponentially with increased depth these hardware improvements would have provided only a small portion of the increased search depth capability.

Still, Stockfish earns the right to progress in the tournament, and also remains the TCEC all-time leading scorer, with 442.5 points in 739 games played.

Stockfish also gains 2 rating points for its efforts...

I don't know who "people" are, but estimates of how much stronger one engine version is than another engine version is usually determined by the head-to-head results of those 2 engines during development of the newer version. These head-to-head games are typically played at very fast time controls in order to be able to play lots of games in a relatively short period of time to be able to get statistically significant results.

I personally prefer to wait until the latest version of an engine is released and entered into engine vs. engine tournaments such as http://www.computerchess.org.uk/ccr... and http://www.husvankempen.de/nunn/. Then you can see the ratings achieved by the latest version of the engine with the ratings achieved by the previous version of the engine. And I have found that early ratings comparisons achieved during development tend to be optimistic.

The reasons are many, but a possibility is that the newer version of the engine is particularly adept at competing against earlier versions of the engine, but that advantage might not show up in games against other engines. That's why both CCRL and CEGT publish "pure" lists which try to detect such situations. And neither site rates or ranks development versions.

A recent example was Komodo 9.3 released earlier this year. I think that it was a rush job in an attempt to wrest the #1 ranking back from Stockfish 7, which was rated 1 or 2 Elo point higher than Komodo 9.2, even though the difference is not significant. But it was a marketing decision. It was claimed by the Komodo development team to be several Elo rating points better than Komodo 9.3 but both CCRL and CEGT rated and ranked it lower than Komodo 9.2. Since Komodo is a commercial engine, I wasn't about to pay for a newer version that in practice was slightly weaker by 10 Elo points or so that the engine I already have. It wasn't until Komodo 9.42 was released that there was a version of Komodo rated and ranked higher than Stockfish 7.

Further suppose that it wants to know certain things from an engine that are not necessarily standard.

For example, can one ask an engine to report back a FEN?

Also, and more importantly, can the engine be asked to determine if a position is a mate or a stalemate?

Note the latter question seems fairly natural and should be quite easy - but the UCI protocol seems to make this rather tricky.

E.g. one can't rely on the <go> output to reach a given depth, or to even output any INFO about such positions (e.g. Stockfish gives one INFO message, Critter gives none).

The problem for a steering program, is that once give <pgn-extract> a game, how do you know what you've got?

If it gets filtered out, you'll block reading the output pipe.

Stockfish 7's choice is 31. Qf4, with an eval over 1.20 for White. It did not come up with Kh2.