Biographies · Information theory and machine intelligence
As a boy in Gaylord, Michigan, Claude Shannon built a telegraph with a friend. Their houses were half a mile apart. They made the parts to send each other Morse code, then got hold of some equipment from the local telephone exchange and put in a phone. Years later, at Bell Laboratories, Shannon found a question common to both of those boyhood inventions. How much information could a line carry if noise kept disturbing the signal? His answer laid the foundations of information theory. Shannon recalls the line between the two houses.
Importance of his work for artificial intelligence
Imagine a mouse reaching the same fork in a maze twice. What would convince you it had learned something on its first visit? In a Bell Labs film, Shannon put a mechanical mouse called Theseus in a maze with movable walls. On its first trip it tried passages until it reached the goal. When he put it back at the start, it followed the route it had found. Then he placed it somewhere new. It resumed exploring until it met a familiar part of the route and could follow it home. Watch Shannon move the mouse to a new starting point.
The memory was under the floor. Telephone relays recorded which way to go, and a magnet moved the mouse above them. Betty Moore Shannon, a mathematician who had worked at Bell Labs, did much of the wiring. Lift the mouse off the board and the experience stays with the maze. What if you put the same mouse on a second board? The first board would still hold the directions. A change in behaviour had a physical cause that Shannon could show his audience. Betty Shannon’s contribution · How Theseus worked.
When two switches mean “and”
Shannon earned degrees in mathematics and electrical engineering at the University of Michigan in 1936. At MIT he worked on Vannevar Bush’s differential analyser, a machine that solved equations. Its control circuits used relays, electrically operated switches. Shannon recognised in their wiring something he had studied in a mathematics class. Put two switches in series and current will flow only when both are closed. Put them in parallel and either one can complete the circuit. Here were the operations AND and OR of George Boole’s algebra, carried out by pieces of metal.
Shannon showed how to work in either direction. An engineer could write down what an existing circuit did, or start with the conditions a new one should satisfy and design its switches on paper. His study appeared in 1938. In 1940 MIT awarded him a master’s degree in electrical engineering and a doctorate in mathematics. After a year at the Institute for Advanced Study in Princeton, he joined Bell Laboratories in 1941. Shannon’s paper on relays · MIT’s biographical account.
What can a noisy line carry?
Suppose a friend has hidden a key in one of eight equally likely lockers. Ask whether it is in the left half. With one answer you are down to four lockers. Two more well-chosen questions settle the matter. Each answer to an even split gives you one bit of information. If the key is nearly always in the first locker, guessing where it is presents a different problem. Shannon used entropy to measure the average uncertainty when the alternatives and their probabilities are known.
In A Mathematical Theory of Communication, published in 1948, Shannon also established a limit for transmission through a noisy channel. Below its capacity, suitable codes operating on sufficiently long groups of symbols can make the probability of a decoding error as small as desired. For an engineer designing a link, it meant that a noisy wire could still carry an exceptionally faithful message. The recipient would decide what that message meant. The mathematics answered how reliably its symbols could arrive. Shannon’s original paper.
A chess player has to stop looking ahead
A rook is there for the taking. Before you reach for it, you see what your opponent would play next. How could a machine exercise that small piece of judgement? In 1950 Shannon described two ways to look ahead in chess. A computer could examine all legal moves to a chosen depth, or devote more of its time to promising continuations. Where the search stopped, it would need a way to judge the resulting position. The balance of material between the two players was one clue. The safety of the king was another. The distinction between choosing what to explore and judging what one finds is still recognisable to anyone who writes a chess program. Shannon’s chess paper in the Computer History Museum.
He also tried prediction with human readers. In 1951 he reported experiments in which a reader guessed the next letter of an English passage and then tried again after a wrong guess. The letters already revealed made some continuations much easier to anticipate. His question was how much uncertainty remained. In 1955 he joined John McCarthy, Marvin Minsky and Nathaniel Rochester in proposing the following summer’s Dartmouth research project on artificial intelligence. By then he had approached machine intelligence with a maze, a chessboard and a page of ordinary prose. The letter-prediction experiment · The four authors’ Dartmouth proposal.
The toy room in Winchester
Robert Gallager, a colleague at MIT, remembered that Shannon generally liked to work alone. Edward Thorp, who collaborated with him, remembered arriving at his study one day to find the calculation 2¹¹ = 2048 on the blackboard. Shannon and Betty had married in 1949. When he returned to MIT in 1956, they settled in Winchester, Massachusetts. The house had a room that they called the toy room. It held chess sets, musical instruments and juggling equipment. Gallager’s recollections · Thorp’s account of working with Shannon · Erico Guizzo on the toy room.
Shannon became professor emeritus in 1978. Some years later, the interviewer Anthony Liversidge saw a machine in the toy room whose soft hands juggled steel balls. He asked Shannon how many balls he could juggle himself. Four, Shannon said. With five he could get them into the air. Catching them was the difficulty. Liversidge’s interview with Shannon.
A chronology
- 1916–1936. Grows up in Gaylord and studies mathematics and electrical engineering in Michigan.
- 1938–1941. Publishes his work on relay circuits, completes two graduate degrees at MIT and joins Bell Laboratories.
- 1948. Publishes A Mathematical Theory of Communication in two instalments.
- 1950–1955. Publishes on chess and letter prediction, demonstrates Theseus and co-authors the Dartmouth proposal.
- 1956–2001. Returns to MIT, becomes professor emeritus in 1978 and dies in Medford at the age of 84.
Further reading
In the Bell Labs film of Theseus, watch where Shannon puts the mouse after it has learned its first route. Then turn to his own paragraph in the Dartmouth proposal. Shannon suggests starting with a simple environment and studying what a machine can learn there before asking it to manage a more complicated one. The maze gives that proposal a scale you can picture. Its walls could be moved by hand.
Liked this post? Follow this blog to get more.