The First AI - Samuel’s Checkers ML System

[smfadmin (OP)]

AI Timeline

1940–45
WWII drives early computing systems

1948–49
Shannon formalizes game AI ideas

1950
Turing proposes machine intelligence tests and game-based reasoning

1951–52
IBM 701 enables real AI experimentation

1952
Samuel builds first checkers program

1953–54
Refinement and early adaptive experiments

1955
First true learning system via self-play emerges

i=SInCRNWnqBQl2rk3

i=Q4HR0pFy31NukdSF

The extensive Copilot version where I ask a lot of questions:

https://copilot.microsoft.com/shares/MrLHhptAPQ6aM8gdqa6bg

The brief ChatGPT version:

Samuel’s Checkers Machine Learning Discussion

User
Tell me all about 1952–1955 — Samuel’s Checkers (first machine learning)

Assistant
Arthur Samuel’s checkers program from 1952–1955 is one of the foundational moments in modern AI and machine learning. Before neural networks, before deep learning, before even the term “artificial intelligence” was widely used, Samuel built a system that could improve its play through experience. That was revolutionary.
The Context: Computers Were Barely Useful Yet
In the early 1950s, computers were enormous, rare, slow, and mostly used for military calculations, census work, or scientific math.
The machine Samuel used was the IBM 701 — IBM’s first major scientific/commercial computer. It had:
• vacuum tubes
• magnetic drum memory
• tiny RAM by modern standards
• punch cards and paper tape
• no operating system in the modern sense
A phone today is incomprehensibly more powerful.
At the time, most people viewed computers as giant calculators. Samuel wanted to prove they could exhibit something resembling intelligence.

───

Why Checkers?
Samuel deliberately chose checkers (draughts) instead of chess.
Chess was too computationally large for 1950s hardware. Checkers was simpler but still:
• strategic
• symbolic
• turn-based
• mathematically structured
• suitable for “look-ahead” reasoning
It was a controlled environment where intelligence could be studied.
This was heavily influenced by:
• Alan Turing
• Claude Shannon
Both had already proposed that games might be ideal laboratories for machine intelligence.

───

The 1952 Version
Samuel’s first version appeared in 1952.
It could:
• represent a board internally
• generate legal moves
• evaluate positions
• search future move trees
• choose moves algorithmically
This alone was already astonishing for the time.
Most programs back then were basically:
• payroll systems
• arithmetic routines
• table lookups
A program “thinking ahead” in a game felt almost science fiction.

───

The Key Idea: Heuristic Evaluation
The computer could not brute-force all future positions.
So Samuel introduced what became a central AI concept:
Heuristic evaluation
The machine estimated whether a position was “good” or “bad”.
It assigned weighted scores to things like:
• piece count
• kings
• board control
• mobility
• threats
Conceptually something like:
genui{"math_block_widget_always_prefetch_v2":{"content":"V(s)=w_1x_1+w_2x_2+\cdots+w_nx_n"}}
Where:
• V(s) = estimated board value
• x_i = board features
• w_i = learned importance weights
This is essentially an ancestor of modern evaluation functions in AI.

───

The Real Breakthrough: Learning (1955)
Between 1952 and 1955 Samuel transformed the system from a programmed game player into a program that could improve through experience.
By 1955 he had implemented:
• self-play
• memory of past positions
• adaptive scoring
• performance-based refinement
This is why it’s often called the first practical machine learning system.

───

How It Learned
Samuel used two major mechanisms:
1. Rote Learning
The system remembered board positions it had already analyzed and reused them.
2. Self-Play Improvement
Copies of the program played against each other repeatedly. It adjusted evaluation weights slightly and kept beneficial changes.

───

Minimax Search
Samuel’s program also used minimax reasoning:
genui{"math_block_widget_always_prefetch_v2":{"content":"\max_{a}\min_{b} V(s_{ab})"}}

───

Why This Was Extraordinary
It demonstrated that machines could improve behavior through experience rather than explicit programming alone.

───

User
Now tell me about the platform it ran on
Assistant
Samuel’s early checkers program ran on the IBM 701 scientific computer.
The IBM 701
It was room-sized, vacuum-tube based, and designed for military/scientific computation.
It had:
• thousands of vacuum tubes
• CRT-based memory
• punch card input
• extremely small memory (~kilobytes)

───

Memory
About 2048 words of 36-bit memory (~9KB total).

───

Input/Output
• punch cards
• paper printouts
• operator consoles

───

User
What language was the game written in ?
Assistant
Samuel’s checkers program was written in machine code and symbolic assembly language for the IBM 701.
There were no high-level languages like FORTRAN at that stage.
Programmers worked with:
• opcodes
• memory addresses
• symbolic assembler instructions

───

User
How did he debug the system
Assistant
Debugging was done using:
• printed memory dumps
• paper output of game states
• manual inspection of register values
• console lights and switches
• step-by-step execution
• punch card verification
He also debugged by observing gameplay failures directly.

───

User
So he built a machine that could learn by playing many games against himself

Assistant
Yes.
The system improved by self-play and adjusting evaluation weights based on outcomes.
It was not intelligent in a human sense, but it was adaptive.

───

User
What proceeded this technology
Assistant
Several foundations preceded it:
• early electronic computers (ENIAC etc.)
• Turing’s ideas on machine intelligence
• Shannon’s minimax and evaluation framework
• cybernetics and feedback systems
• early game theory and decision theory

───

User
Can you give me a timeline
Assistant
Timeline: 1940–1955
1940–45
WWII drives early computing systems
1948–49
Shannon formalizes game AI ideas
1950
Turing proposes machine intelligence tests and game-based reasoning
1951–52
IBM 701 enables real AI experimentation
1952
Samuel builds first checkers program
1953–54
Refinement and early adaptive experiments
1955
First true learning system via self-play emerges

Meta
Arthur Samuel’s checkers program (1959) was basically the first machine learning system that actually worked. It didn’t have rules hand-coded by Samuel — it figured out how to play better on its own.

Here’s how it learned:

01. Two learning methods running together

Rote learning

• The program remembered board positions it had seen before, along with the move it made and what happened next.
• Basically a lookup table of positions → best move, built up over many games.
• This let it search deeper without recalculating: if it recognized a position, it could skip ahead. Think of it as caching experience.   

Generalization learning — the important one

This is where it actually “learned” strategy:

1. Self-play: The program played games against itself. One version kept its evaluation function fixed as a benchmark, the other version tweaked its weights after each game.

2. Board evaluation function: Samuel gave it 38 features to score a board: piece advantage, king advantage, mobility, center control, etc. Each feature had a weight. The program didn’t know the right weights — they started random.

3. Temporal difference learning: After each move, it compared its evaluation of the current position vs the evaluation of the next position. If they disagreed, it nudged the weights of the earlier position to be closer to the later one.

• Richard Sutton later identified this in 1988 as the “earliest and best-known use of a TD method”. Same core idea behind TD-Gammon, Deep Q-Networks, AlphaGo, etc.

4. Iterative improvement: Features that led to wins got higher weights, features that led to losses got lowered. After ∼8-10 hours of self-play, it played better checkers than Samuel could.   

02. Search + learning combo

• Minimax with alpha-beta pruning: It used lookahead search to explore possible moves, like modern game AIs.

• Evaluation function guided search: Instead of brute force, it scored positions to decide which branches to explore.

• Learning improved the scoring: The more it played, the better its evaluation function got, so its search got smarter.   

03. What it was given vs what it learned

Given by Samuel:

• Rules of checkers
• 38 board features that might matter
• The minimax search algorithm

Learned on its own:

• Which features actually mattered and by how much
• Relative weights, even fixing “wrong signs” Samuel put in initially
• Strategy — it ended up reaching strong amateur level   

Why it matters

Samuel coined “machine learning” in 1952 describing this program. The temporal difference learning he invented is the direct ancestor of modern reinforcement learning.
AlphaGo’s lineage literally runs:
Samuel’s checkers → TD-Gammon → Deep Q-Network → AlphaGo/AlphaZero. 

So TL;DR: It learned by playing itself millions of times, constantly tweaking a scoring function based on whether positions led to wins or losses. No human told it “kings are good” — it figured that out from experience.