Neural Networks: The New Connectionism
Chapter 8
I. Introduction: Since WWII, computers have been increasingly present in society. A child’s starter computer has more processing capacity that the roomful of vacuum tubes that constituted the world’s first true computer. From business record-keeping to military planning…blamed for mistakes in billing to students’ failures to get assignments finished to alleged dehumanization of society. Undoubtedly, they are a crucial factor.
A. What have computers meant to psychology?
1. Contributed storing, organizing, analyzing, and retrieving capacity to research, practice, and teaching in psychology.
a. Teaching – to expand the potential of programmed instruction
b. Research – to generate stimuli and present them to experimental subjects, to record responses, and to analyze data in seconds.
c. Practice – to provide forms of treatment, counseling, and assessment via web or internet methods.
2. Contributed a mirrored approach to theorizing about human learning.
a. A way of testing inferences made through intervening variables, which were previously thought of as mere speculation.
(1) Write a computer program to follow the principles of learning that she things organisms follow
(2) Run the program to see what happens
(3) If similar to outcome of human or animal learning, the hypothesized processes are more plausible; if not, the theorist’s assumptions are probably false
b. A way to carry out logic more precisely and thoroughly, not to change the logic of theory construction.
(1) Provide a more precise way of doing what Hull wanted to do; with computers he might have been able to avoid some of his system’s contradictions.
(2) Role of computer program is much like that of a mathematical analysis.
3. Metaphor for human thought, a way of talking about how we think.
a. Comparing how humans and computers process information and solve problems
b. In jest, psychologists study 3 kinds of organisms: humans, animals, and computers
B. Argument between connectionist and cognitive theories
1. Computer is a connectionist organism
a. Molecular level: Its chips, which do or do not pass currents, are comparable to neurons in the nervous system that do or do not fire.
b. Molar level: Computer stores and retrieves information, reasons logically, breaks problems into sub-problems, behaving in cognitive ways.
c. If a “mere machine” built by humans from basically connectionist parts, can operate so effectively in such cognitive ways, cognitions can scarcely be as unscientifically mentalistic as earlier psychologists feared.
d. Resolution of connectionist-cognitive argument: definition of the human being as “the only movable, general-purpose computer that can be produced by unskilled labor.”
2. Both computers and humans are programmed
a. Computers programmed by humans
b. Humans programmed by genes, lifetime of experience, some deliberate, some not
c. With humans, new learning cannot completely override all earlier learning
(1) Unfortunate in emergency situations if a person forgets recent CPR training
(2) Fortunate in that humans are never under the complete control of any one “programmer.”
d. Differences between computers and humans makes humans in some ways less effective but also contributes to our individuality and autonomy.
3. Both computers and humans have programs and subprograms
a. Computer program often includes a command to call some simpler program to do a particular job needed by the larger program.
b. A person working on a task can draw on previously learned ability to do smaller tasks that contribute to the larger one.
c. Ex. Food Network channel; chef’s recipe calls for chopped onions; carpenter’s program for building a bookcase will call for sawing a board.
d. This can be extended to numerous levels.
II. Symbolic Models
A. The earlier computer-oriented view focusing on how a human, animal, or computer operates to solve problems
1. Does not focus on learning (or how the system comes to operate that way)
2. Involves discovering the rules a specialist uses for making decisions, and then programming the computer to follow those same rules.
3. “Expertness” is built in from the start.
B. Logic Theorist – developed by Newell, Shaw, and Simon (1958)
1. Function: given postulates of geometry, to prove certain theorems.
2. Worked like a good high school geometry student; worked forward from theorems already proved to try to find ways of turning one of them into the theorem to be proved; worked backward from the to-be-proved theorem to other statements which, if true, would permit the proof, and then treated these statements as subgoals to be proved
3. Dealt with cognitive structures rather than with S-R connections
4. Could not be used to solve other kinds of problems
C. General Problem Solver – developed by Ernst and Newell (1969) to expand Logic Theorist to deal with a wider variety of problems
1. A problem may be regarded as any discrepancy between an actual state of affairs and a corresponding desired state of affairs; GPS was intended to reduce that discrepancy.
2. In real life, problems vary from “I am in Chicago and want to be in New York, but a snowstorm has closed O’Hare Airport; How can I get to New York?” to “I need an A in organic chemistry to get into medical school, but at this rate I’ll be lucky to get even a B; How can I raise my grade?”
3. GPS generally was able to solve more artificial problems:
a. Crossing the river
(1) Present state: There are 3 missionaries and 3 cannibals on one side of a river.
(2) Desired state: To have all six of them on the other side
(3) Difficulties: The only way across is by a boat that can carry only 2 people at a time and must always have at least one person to row it (i.e., it cannot cross the river empty). If more cannibals than missionaries are ever left together on one side while the boat is crossing, the cannibals will eat the missionaries. (Getting missionaries across in cannibals’ stomachs is not considered a solution!)
(4) Question: What sequence of crossings with the boat will accomplish the transfer?
(5) Solution: The boat crossed 5 times, each time carrying one missionary and one cannibal. The first and fourth times a missionary brings the boat back; the second and third times a cannibal.
b. Coding
(1) Present state: D O N A L D
i. + G E R A L D
1. R O B E R T
(2) Desired state: An example of correct addition in which one particular numeral corresponds to each of the letters above.
(3) Difficulty: The codebook to tell what numeral each of the above letters represents is lost, and we know only that D = 5.
(4) Question: What code will convert the letters into numerals to give a correct example of addition?
(5) Solution: 5 2 6 4 8 5
i. + 1 9 7 4 8 5
1. 7 2 3 9 7 0
D. Significance of GPS
1. Like humans in several ways
a. Solved some problems, but not all
b. Did better if information was presented systematically
c. Performed better with instructions that provided direction
d. Used processes suggestive of “insight” rather than trial and error
e. Used concepts in solving problems to the extent that axioms can be considered concepts
f. Organized itself to do these things using past discoveries to guide future endeavors
2. GPS is one of several programs that can solve such a diverse realm of problems. Still some think these problems are trivial because humans can solve them; it didn’t reveal anything new about human problem solving; it did only what it was programmed to do. Who cares if a program can do what it’s told?
3. Computers have also been programmed to work on problems that no one has been able to analyze so exhaustively.
E. Chess
1. In 1997, a computer program named Deep Blue played a 6-game match against world chess champion Garry Kasparov and won by 2 games to 1, with 3 draws.
2. At any given point there are a limited number of possible moves (approx. 10 to the 120th power), a finite number of possible countermoves, etc.
3. The computer, like people must rely on heuristics rather than algorithms for situations such as this.
a. algorithm = problem-solving procedure in which all alternatives are systematically considered
b. heuristic approach = makes use of various strategies that eliminate and select from among alternatives without having to consider every one separately.
4. Modern chess programs can mechanically compute millions of moves and countermoves within a few seconds, relying on brute force.
5. Though this victory did not make Deep Blue the official chess champion of the world, it suggested that the definition of “world champion” might need to be reconsidered.
III. People and machines: Computer metaphors
A. Computers and humans need not be the same for the metaphor to work but, instead, be sufficiently similar that some features of one can be used as a sort of pattern for some aspects of the other.
B. Complex arrangement:
1. electronic components, called hardware: chips, disks, drives, switches, etc.
2. neural material, called wetware: neurons, various other cells, amino acids, chemical transmitter substances, etc.
C. Flow of ideas
1. computer: input and output
2. human: stimuli and responses
D. computer programs, or software = human cognitive processes, or information processing
E. A computer that responds like an intelligent human being might function as does a human…or…
F. A computer that does very humanlike things might use very different processes to do them.
G. Important differences between brains and computers
1. Computers are fast, brains are slow (100,000x slower)
2. Brains can multi-task on many levels, computers cannot.
3. Human ability to store information in memory is virtually unlimited; no computer even comes close.
4. Human ability to perceive and to recognize complex, changing patterns cannot be matched by computers.
5. Modern robots cannot compete with humans’ ability in locomoting, recognizing people and objects, discriminating shapes and smells, etc.
6. Computers have ability to retrieve flawlessly from memory and perform arithmetical computations rapidly and accurately, much more so than humans.
7. People aren’t good at what computers do, and computers aren’t good at what people do (Allman, 1989).
IV. Connectionist Models
A. Earlier models: Involved sequences of actions based on rules. Whatever form these rules took and whether they were used by a person or a computer, they were rational, goal-directed, purposeful, and highly cognitive. Would be expected to appeal to Tolman and Piaget and gave cognitivism its great impetus.
B. Problems with earlier computer models (beginning in 1980s)
1. The discovery that human experts do not solve problems simply by following rules
a. Novices slavishly follow rules
b. Experts proceed more intuitively, using their great base of experience to select or eliminate courses of action without detailed analysis; looks more like making learned responses to stimuli than like inferring actions from principles (though perhaps not much like either).
2. Consideration of the nervous system
a. Neurons operate much more slowly than computer chips, and it seemed increasingly unlikely that humans could think as effectively as they do if they operated in the same step-by-step way as computers.
b. However, the neurons in the brain are organized in complex ways, with each connecting with many others, so that it would be possible for the brain to increase its efficiency considerably by having different sets of neurons working on a problem at the same time in various complex ways.
c. Animal nervous systems were found to operate more like a network than like a single central processing unit of a computer.
3. Engineering changes of the computer itself
a. More and more powerful machines built so that the amount of computing power in a given space as increased dramatically.
(1) 1st generation: vacuum tubes
(2) 2nd generation: transistors
(3) 3rd: transistorized chips
(4) 4th: miniaturized chips
(5) 5th: parallel processing – different components of the system are working on different parts of a problem at the same time; this is what the human brain does
b. Thus, whereas for some time an attempt has been made to analyze people as if they were computers, now we might say that an effort is being made to design computers to be more like people
C. New approach
1. Regarded process of learning rather than the end product of problem solving.
2. Rather than the “expertness” being built into the program from the start, the program is designed to learn from experience, keeping those procedures that work and dropping out those that don’t.
3. The system might be very inexpert at the beginning, but since it has perfect memory for its experiences, it would learn fairly rapidly what worked and what didn’t.
4. It would learn by examples rather than be programmed with rules.
5. Much like what Thorndike might have imagined.
D. Parallel distributed processing or network models or connectionism
1. Older connectionism referred to connections between stimuli and responses
2. New connectionism refers to the whole interconnectedness of the network
3. Learning processes more cognitive and complicated than even Hull supposed
4. Combines an emphasis on specific connections and on learning by reinforcement that would have appealed to Thorndike or Hull, with an emphasis on the organized operation of whole systems that would have appealed to a gestalt psychologist.
E. Two approaches
1. Focusing on the real nervous systems of simple organisms, analyzing in detail how the individual neurons work together to produce conditioning; very simple learning processes (ex. Sea snail).
2. Focus on more complex networks
a. Studying humans is to complex; over a trillion neurons in the brain estimated.
b. Also unethical; experimental surgery
3. Alternative: construct physical models to imitate human thinking, and study them
4. An illustration: NETtalk
a. Ex. A 6-year old doesn’t learn to read by learning all the rules first but rather that the “rules” are unconsciously made up in the process of learning how to match spoken words to printed symbols (connectionist model).
b. NETtalk is a machine made up of processing units and is called a neural net model; its units serve as an analogy for active neurons in the brian.
c. The essence of the task for NETtalk is to learn to read text.
d. English is complex since sounds vary greatly.
e. Back-propagation rule: uses information about the correctness or appropriateness of its responses to change itself so that the response might be more correct or more appropriate.
f. NETtalk was presented with 1000 words of text read by a first grader and gave it the back-propagation rule, telling the computer to compare its output with the first grader’s reading and work back through its hidden units, readjusting weights to reduce the difference between what it says and what the child said.
g. The computer did so and eventually taught itself to read.
V. Connectionist models: An appraisal
A. Neural networks respond very much like humans do; their fuzzy logic takes the imprecision of the real world into account. As a result, the notion of truly intelligent machines is becoming a reality (Jang, Sun, and Mizutani, 1997).
B. Given the right series of experiences, the neural network computer might categorize and relate things, reaching something that looks like insight. (very different from conventional computer)
1. may be better models of human cognitive processes than symbol-based models
2. suggest that people don’t always think rationally
3. allow for fuzzier logic and emphasize that many aspects of a situation might be involved in a response or conclusion.
4. A neural network that adjusts its own connections is highly compatible with Hebb’s notion that neurons that repeatedly activate each other become increasingly more likely to do so.
C. Connectionist, or neural network approaches, now dominate study of human cognitive processes; applied to variety of fields, including prediction of weather, performance of financial markets, medical diagnosis, and engineering.
D. Some cautions and criticisms
1. Connectionist models are just metaphors, only describing and suggesting, not explaining.
2. Criticism:
a. Computers don’t simulate human emotions all that well.
b. Computer simulations don’t reveal the insight of which human problem solvers are capable; in Gestalt view, such simulations are inadequate models of human cognitive processes.
c. Computer programs tell us very little about how the human nervous system works
d. Neural network models may generate results that are unpredictable and have never been observed in a laboratory (360 units in a connectionist model is not necessarily equal to 360 neural units in the human brain)
e.
With enough time and sufficient layers of hidden units, a neural net
using a back-propagation model may be able to teach itself to “read” a passage
of
Spanish as if it were really English; if the input does not matter, the model
teaches psychologists little about the processes involved in learning.
f. Problem of interference: When neural network models train themselves to recognize pairs of words in what is termed paired-associate learning, and are then given a second set of pairs, the initial learning impedes subsequent learning far more than is the case with human subjects.
VI. Educational implications
A. Computer-based tutoring systems – might be capable of reasoning about students’ understanding and knowledge and thus used to monitor students’ progress and to act like a teacher by guiding students, asking questions, uncovering misconceptions and errors, and so on.
B. Computer systems that mimic other phenomena have instructional application: train pilots by mimicking the in-flight responses of specific aircraft; others that model the functioning of the circulatory system are used in medical schools
C. Computer-based simulations of various environments, labeled virtual reality (VR) have instructional applications; learner experiences certain events and environments and makes choices within that environment. Palenque (Wilson & Tally, 1990) – virtual reality of the Mayan world.
D. Intelligent tutor systems – computer-based instructional programs where the computer is used as a source of information, much as a human tutor might be, but takes into account the student’s strengths and weaknesses.
E. Teach students programming skills: Logo (Papert, 1980, 1993) – simple tool that allows child to explore plane geometry but is sophisticated enough to allow children to investigate the world of differential equations; teach computer literacy.
F. Moore, Redfield, and Johnson (2001) describe dozes of other applications of AI to education, such as teaching students cooperative learning skills, fostering proficiency in problem-finding and problem-solving, for developing task analysis and study skill, for improving memory, for improving verbal interaction skills, for imitating models, etc.
VII. A Field in Progress
A. Much of this chapter is historical, dealing with theories of whose principles and assumptions have well been chewed because they are of a greater age and have been around longer.
B. Subject of this chapter is of a younger age and has yet to be digested by generations of scholars and thinkers; chapter cannot be concluded, nor the models and theories evaluated.
C. History will judge.