ChatGPT writes Prolog for a chess puzzle

John D. Cook tested ChatGPT on a small chess-placement puzzle and got a working SWI-Prolog program that printed 16 solutions.

The task gave ChatGPT a compact logic problem: place a queen, king, rook, bishop, and knight on a 4-by-4 chessboard so no piece attacks another. Cook ran the generated chess4.pl file with SWI-Prolog and received the expected solution list.

The result says more about Prolog than chess. Prolog asks you to describe facts, constraints, and relations, then lets the engine search. That model fits this puzzle well. ChatGPT generated predicates for board squares, piece placement, attack rules, line-of-sight blocking, and output formatting. A human still needs to inspect the logic, but the language gives the model a narrow surface to hit.

The strongest part of the generated code comes from its separation of rules. square/1 defines the board. place_pieces/3 assigns pieces to open squares. safe/1 checks pairs. attacks/4 holds the chess rules. That structure gives the program room to fail in visible ways: a bad knight rule, a missing blocker check, or a swapped coordinate convention would show up in the printed boards.

Cook’s larger point concerns language stability. Prolog has existed for decades, and its syntax has changed little. Models trained on large code corpora can draw from many examples that still compile. Cook contrasts that with Lean and Mathlib, where library refactors can break generated code even when the theorem idea makes sense.

That distinction gives developers a useful heuristic. LLMs can help more with languages and libraries that have stable APIs, dense public examples, and small syntax surfaces. Prolog checks those boxes. A younger ecosystem can still work, but you spend more time reconciling generated code with current library names, imports, and conventions.

The experiment does not prove that ChatGPT can solve broad planning problems through Prolog. A 4-by-4 chess puzzle has a small search space and clear rules. Yet it shows a good workflow: ask the model for a declarative program, run the solver, inspect the results, and tighten the predicates when the output exposes a flaw.

For constraint problems, that workflow has appeal. You describe the rules in Prolog, let the runtime enumerate candidates, and use the LLM to handle syntax and boilerplate. The human keeps responsibility for the model, the constraints, and the interpretation of the answer.