The Work of John von Neumann (1903-1957)
In his all-too-brief career, mathematician John von Neumann, one of the
greatest polymaths of all time, managed to have profound impact on mathematics,
quantum theory, economics, computer science, neurology, and other fields.
Over time, as he moved through different disciplines, his work "moved
in the general direction of the post-war scientific disciplines, which had
a decreased emphasis on motion, force, energy, and power and an increased
emphasis on communication, organization, programming and control."
The theme of self-reference in systems runs through much of von Neumann's
work; early on, von Neumann and other mathematicians tried to eliminate
self-reference and the contradictions it seemed to cause in set theory.
Self-reference was a key element in many of von Neumann's later contributions
as well, from his treatment of the apparent regresses of game theory to
the self-reproduction of organisms.
Born in Budapest in 1903, von Neumann was the oldest son of a successful
Jewish banker. He received an exceptional formal and informal education;
family connections exposed him to many of Hungary's intellectual luminaries
of the period, and the Lutheran Gymnasium (one of Hungary's best secondary
schools) provided him with a university tutor to nurture his mathematical
gifts. He enrolled at both the University of Budapest and the University
of Berlin in 1921. From 1923 to 1925 von Neumann studied chemical engineering
at the Swiss Federal Institute of Technology, earning a degree in 1925,
and the following year he earned his Ph.D. in mathematics from the University
of Budapest. He became the youngest privat dozent (assistant professor)
ever to serve at the University of Berlin, and then spent a year at Hamburg,
also as privat dozent. He also held the prestigious Rockerfeller
Fellowship at the University of Göttingen.
From 1930-33, von Neumann was a Visiting Professor at Princeton University,
and in 1933, when Princeton's new Institute for Advanced Study (a non teaching
institution) opened, he became its youngest Professor (a permanent position).
During World War II, von Neumann advised the U.S. government on the war
effort, including the construction of the nuclear bomb. The massive computational
needs of the atomic bomb project led von Neumann to become involved in the
quest for computers that were up to the task. He remained actively associated
with the government after the war; he began a formal association with the
RAND Corporation (a think-tank set up at the behest of the military) in
1948 and became a member of the Atomic Energy Commission in 1954. In 1955
von Neumann developed cancer--perhaps, some have speculated, as a result
of having been witness to an atomic bomb test ten years earlier. Von Neumann
died in February, 1957, leaving behind a partially finished manuscript for
"The Computer and the Brain," his last major intellectual project.
1. Set Theory and Quantum Mechanics
In the early 1920s, the mathematical community faced a fundamental crisis:
mathematicians Bertrand Russell and Alfred North Whitehead had found that
issues of self-reference presented major paradoxes in mathematics. It was
feared that if contradictions truly had been discovered in the theory of
sets, then the existence of these contradictions threatened the very basis
of mathematics itself. In this context, von Neumann participated in a program
formulated by David Hilbert, a leading German mathematician whom von Neumann
would later call the one "great mathematician" he had ever known.
Along with Wilhelm Ackerman and Paul Bernays, von Neumann was one of a small
number of talented young mathematicians seeking, under Hilbert's direction,
to formalize mathematics and set it on a fully articulated and sound basis.
The program made significant headway in clarifying the axiomatic bases of
several branches of mathematics, but was abruptly brought to a halt by Kurt
Gödel's 1931 proof that "no formal system powerful enough to formulate
arithmetic could be both complete and consistent," demonstrating that
complete axiomatization had inherent limits.
In the late 1920s, von Neumann became inspired by Werner Heisenberg, the
leading German theorist of quantum mechanics. Heisenberg had come to Göttingen
to lecture about his Uncertainty Principle, which states that it is impossible
to measure precisely both the position and the momentum of an elementary
particle (with the product of uncertainties being at least Planck's Constant).
Fascinated, von Neumann began work in quantum theory. This led to his Mathematische
Grundlagen der Quantenmechanik (1932), in which he discussed the much-debated
question of indeterminism in quantum theory. Until then, indeterminism was
thought to be the result of hidden parameters which need only be identified
to restore determinism. Von Neumann "concluded that no introduction
of 'hidden parameters' could keep the basic structure of quantum theory
and restore 'causality.'" He argued that the indeterminism was inherent
in quantum theory because of the interaction of the observer and the observed.
2. Game Theory
The roots of game theory go back at least to the nineteenth-century economists
Augustin Cournot and Francis Edgeworth. The formal theory was introduced
in 1921 by Emil Borel, a French mathematician, who wrote about "la
théorie du jeu" in a note which looked at the phenomenon
of bluffing in poker. (Seven years later, von Neumann would arrive at game
theory via the same route.) In addition to noting the possible applications
of game theory to political and economic issues, Borel listed some basic
questions: "[F]or what games is there a best strategy, and how does
one find such a strategy?"
Von Neumann's 1928 contribution was to show that for two-person zero-sum
games, a best strategy could indeed be found. He then extended the analysis
to more general games, "[defining] three-person games and [suggesting]
how to define a general n-person game." Games involving more
than two players presented very different problems than did two-person games
because of the possibility--and necessity--of alliances. "Conflicts
of interest are less pat. What is good for player A may be bad for player
B but good for player C. In such a situation, A and C might form an alliance.
Such coalitions change a game radically."
Von Neumann saw his later project, The Theory of Games and Economic Behavior,
co-authored with Austrian economist Oskar Morgenstern, as nothing less than
a project to revolutionize economics:
In their book, which appeared in 1944, von Neumann and Morgenstern
stated their belief that economics would develop into a rigorous mathematical
science, just as physics had--except that economics was still in an early
stage of development- perhaps, they suggested, analogous to that of physics
in the sixteenth century. The theory of games was the first serious step
toward a comprehensive mathematical economics, which would stimulate empirical
research to test and possible modify its theories.
Morgenstern, who had fled Nazi persecution, had arrived at Princeton in
1939. He and von Neumann began work on the application of game theory to
economics in that year and continued intermittently until 1943; the book
was completed shortly before von Neumann was called away to England on a
war assignment. Between 1939 and 1943, von Neumann was consulting extensively
with the U.S. government on war work, and therefore much of his work was
done in Washington, D.C. The Theory of Games and Economic Behavior,
which "describes the mathematics of the theory of games in great detail
and applies it to many different economic problems, including exchange of
goods between n parties, monopolies and oligopolies, and free trade,"
was an enormous success (by academic standards)--much to the surprise of
its authors and publisher.
Although von Neumann did not return to his study of economics, he remained
interested in game theory and saw its applicability to a broad range of
public activities--including military strategy. Thus, von Neumann's association
with RAND after the war came about because RAND was convinced of the importance
of game theory for strategic purposes in the Cold War. But despite his work
with RAND, von Neumann spent little time on game theory after the Second
World War, turning his attention instead to the emerging disciplines of
computer science and atomic energy.
Others at RAND (and eventually elsewhere) continued to explore game theory
after von Neumann had moved on, using experiments to determine whether and
to what extent the subject actually describes human decision-making. During
the succeeding decades, researchers discovered various major dilemmas which
appear to have analogues not only in human social experience but also in
the rest of the natural world. The most famous of these situations, the
Prisoner's Dilemma, was first discussed in 1950. The Prisoner's Dilemma
concerns situations where, given a choice between 'defecting' from, or 'cooperating'
with, another player, one will suffer for being the lone cooperator, but
benefit from being the lone defector--which leads to the conclusion that
both players will likely defect. Since mutual cooperation has better all-around
results than does mutual defection in this game, the outcome is unfortunate.
The Prisoner's Dilemma and other game- theoretic constructs were thought
to illuminate the tensions inherent in nuclear proliferation, among many
other areas.
In the mid-1950s, von Neumann left RAND to join the Atomic Energy Commission,
a decision-making position which, as an immigrant, he felt extremely honored
to hold. He did not serve for very long, as he became ill shortly after
assuming the position (although the gravity of his condition was kept secret,
and he remained publicly active for about a year). By the end of 1955, however,
he was confined to a wheelchair and attended AEC meetings thus. This has
been construed as (possible) proof that von Neumann was the model for Stanley
Kubrick's Dr. Strangelove. Author William Poundstone has pondered:
Was the wheelchair-bound von Neumann a model for the title character
of Stanley Kubrick's 1963 film Dr. Strangelove or: How I Learned to Stop
Worrying and Love the Bomb? Strangelove, 'Director of Weapons Research
and Development,' is confined to a wheelchair. He speaks of having commissioned
a defense study from the 'Bland Corporation.' As is often the case with
satire, a number of models have been suggested (especially Werner von Braun
and Edward Teller), and there is no reason to think the character was based
on any specific individual.
3. Computer Architecture
During the mid-1940s, programmable electronic computing instruments were
built for the first time. The first electronic computer, the ENIAC, was
programmable only in a very limited sense, however; it had to be rewired
for each new calculation (typically requiring a half-day at least to prepare
the machine for operation). Although, in principle, a flexible machine with
many potential uses, it was used primarily for calculating ballistic trajectories.
The "von Neumann machine," as it came to be called, changed that.
Von Neumann machine is the name given to a class of computers (including
most computers which exist to this day) which share a family of core components
and a logical structure. First posited in a 1945 memo, this was the plan
for a new kind of computer, the "stored program" computer, which
would be far more flexible than its predecessor. Instead of having program
instructions wired in, the "stored program" computer kept its
specific instructions (programs) in its memories, storing the information
in the same manner as it would store any other information (data). To this
end, the computer would necessarily contain five basic components: a control
unit, memory, a calculating unit (CPU), and input and output components
for interacting with human users. The control unit would delve into memory,
finding an instruction or a piece of data, and deal with what it found accordingly.
The memory itself, of course, would need to be conceived of entirely differently
than it had been in the ENIAC, which had demanded far less flexibility and
lower overall capacity. Ideally, this would be accomplished by having one
memory to which the control unit would have near instantaneous access. Since
this was infeasible from an engineering standpoint, the route taken was
to create a hierarchy of memories, with declining speed of access. As realized
in the EDVAC (ENIAC's successor) this option called for a secondary memory
whose task was to transfer imminently pertinent data into the primary memory
for immediate use, quickly enough not to compromise the efficiency of the
computation itself. In return for a slight decline in speed, a hierarchy
of memories could vastly increase capacity.
The concept of the stored-program computer was a realization of a theoretical
construct, the "universal Turing machine," developed by the English
mathematician Alan Turing in the 1930s. Turing machines were idealized computers,
which could follow any pre-designated decision process that could be expressed
mathematically. The "universal Turing machine" was a Turing machine
which could mimic the processes of any other Turing machine by following
instructions outlining how to do so. The stored-program computer did just
that, albeit to a limited extent due to engineering constraints.
Von Neumann's name has become synonymous with modern computer architecture,
but whether von Neumann deserves exclusive credit for the architecture is
rather in doubt. The famous memo, a write-up of ongoing work by engineers
J. Presper Eckert and John Mauchly as well as von Neumann, was distributed
by Herman Goldstine--the army's liaison to the EDVAC project--bearing only
von Neumann's name. Tension over this memo and its authorship evolved into
a patent fight, with Eckert and Mauchly resigning from their academic positions
to pursue their commercial interest in the computer.
4. Automata Theory
In the late 1940s and early 1950s, von Neumann began to have extensive contacts
with the biomedical community, as biologists and engineers took increasingly
convergent routes to learning about information processing. By the late
1940s, this multidisciplinary field of study had acquired the name "cybernetics."
Von Neumann was an active participant in this movement, which brought together
mathematicians, neurophysiologists, psychiatrists, psychologists, and even
sociologists, to deal with topics such as "Feedback Mechanisms and
Circular Causal Systems in Biology and the Social Sciences." Von Neumann
contributed greatly to this exploration through his study of "automata."
An automaton, in von Neumann's analysis, is "any system that processes
information as part of a self regulating mechanism," such as the human
nervous system or a computer.
Von Neumann's work on automata was concerned with a paradox of self replication:
"[O]ne expects the complexity of a species to increase (or at least
remain constant) through evolution, yet it is hard to understand how an
automaton can produce anything more complicated than itself." At first
blush, it seems that a machine ought to be absolutely limited by its own
level of complexity, and yet reproduction and evolution do occur.
Von Neumann designed a self-replicating automaton that could use information
to create progeny, even progeny of increasing complexity. He concluded that
there is a "completely decisive property of complexity," a "minimum
level . . . below which automata are degenerative (can only produce less
complex automata than themselves) but above which some automata can produce
equally or more complex progeny." Moreover, von Neumann elaborated
on the nature of this threshold, above which "open-ended complication"
or "emergent evolution" could occur. The automaton had to have
the capacity to act on symbolically represented information--specifically,
a symbolic description of itself. "Self-replication would then be possible
if the universal constructor is provided with its own description as well
as a means of copying and transmitting this description to the newly constructed
machine."
The self-reproducing automaton, therefore, must have two components which
are wholly distinct from one another--the machine and its description. A
key insight in von Neumann's analysis of self-reproduction is this "categorical
distinction between a machine and a description of a machine." The
description of the machine is symbol, while the machine is matter--but for
the reproduction to be successful, the description must not only be followed,
but must also be duplicated. The description itself thus performs two distinct
functions: "On the one hand, it has to serve as a program, a kind of
algorithm that can be executed during the construction of the offspring.
On the other hand, it has to serve as passive data, a description that can
be duplicated and given to the offspring." It was several years later
that Watson and Crick would discover DNA, the instructions for living automata.
They discovered that, astonishingly, DNA does indeed perform these two functions.
It encodes the instructions for making the appropriate enzymes and proteins
for a cell, and also unwinds and duplicates itself before a cell divides:
"With admirable economy, evolution has built the dual nature of the
genetic material into the structure of the DNA molecule itself."
Today's artificial life theorists have found von Neumann's insights-- developed
fifty years ago, when the chemical processes of life were relatively poorly
understood--to be invaluable to their efforts.
Von Neumann's progress with automata brought him to the forefront of the
cybernetics movement. By the mid-1950s, he was engaged in active discussions
with numerous biological scientists. Eventually, von Neumann abandoned all
his other research efforts to focus on the computer and the brain. In studying
these systems, von Neumann was not interested in the internal workings of
the individual components. Instead, he preferred to borrow an approach from
a 1943 paper on neural networks by Warren McCulloch and Walter Pitts. He
treated the components as "idealized neurons." Both neurons and
vacuum tubes, in von Neumann's analysis, were "black boxes having 'certain
well-defined, outside, functional characteristics' and 'assumed to react
to certain unambiguously defined stimuli, by certain unambiguously defined
responses.' The electrochemical process within the neuron or the electrical
processes in the vacuum tube that underlie the external behavior of these
elements are not disclosed by the axioms."
Although von Neumann preferred to deal with neurons and vacuum tubes as
black boxes, he did compare the two systems on a systemic level, noting
the similarities and differences in their feedback mechanisms--such as their
systems for dealing with errors:
The natural system, he explained, tried to make errors as inconspicuous
as possible: it detects them as they occur, adjusts to minimize their effect,
and repairs or blocks the faulty components while allowing the rest of the
automaton to continue its function. In contrast, the artificial system attempts
to make errors as conspicuous as possible so that they will be brought to
immediate attention, the machine be shut down, and repair or replacement
be immediately commissioned.
In early 1955, von Neumann accepted an invitation from Yale University to
deliver the Silliman lectures the following spring. He planned to use the
lectures as a forum to expand upon a lecture he had given to the American
Psychiatric Association in May 1955. Tragically, von Neumann's encroaching
cancer prevented him from ever delivering the Silliman lectures. He did
begin to write them out, however. Beginning with a summary of the logic
of computer architecture, von Neumann continued with a groundbreaking comparison
of the computer and the brain:
He calculated that the computer requires greater volume, consumes
more energy, and is 10,000 times less efficient than the brain (in binary
actions per unit either of energy or volume) but that the computer compensates
by its considerable advantage (a factor of approximately 5,000) in speed.
He argued that the brain and the computer are logically organized in rather
different ways: the brain favors more but slower switching components while
the computer favors fewer but faster components. He concluded that the computer
is organized for serial operation and the brain for parallel operation,
and he predicted some of the problems that must be confronted in moving
from one mode of operation to the other (finding efficient algorithms to
make use of parallel facilities, for example, or adding extra storage space
for intermediate results in serial operation). He anticipated that computer
designers could profit by modeling features of the human nervous system
in their designs but sounded a cautionary note on possible difficulties.
He took these simple comparisons about as far as he could without a better
knowledge of the nervous system or improved materials and architectures
for building computing systems.
Bibliography
William Aspray (1990) John von Neumann and the Origins of Modern Computing.
Cambridge, MA: The MIT Press.
Steve J. Heims (1980) John von Neumann and Norbert Wiener. Cambridge,
MA: The MIT Press.
H.H. Pattee "Evolving Self-Reference: Matter, Symbols, and Semantic
Closure," Communication and Cognition--Artificial Intelligence,
Vol 12, Nos. 1-2, Special Issue "Self-Reference in Biological and Cognitive
Systems," pp. 9-27.
William Poundstone (1992) Prisoner's Dilemma. New York: Doubleday.
M. Mitchell Waldrop (1992) Complexity: The Emerging Science at the Edge
of Order and Chaos. New York: Simon & Schuster.
Dictionary of Scientific Biography.
Biographical Dictionary of Scientists.
Notable Twentieth Century Scientists.
By Adam Brandenburger and Elizabeth Stein