REVIEW
Cognition as coordinated non-cognition
Lawrence W. Barsalou  Cynthia Breazeal 
Linda B. Smith
Received: 19 December 2006 / Revised: 3 March 2007 / Accepted: 6 March 2007 / Published online: 11 April 2007
Ó Marta Olivetti Belardinelli and Springer-Verlag 2007
Abstract We propose that cognition is more than a collection
of independent processes operating in a modular
cognitive system. Instead, we propose that cognition
emerges from dependencies between all of the basic systems
in the brain, including goal management, perception,
action, memory, reward, affect, and learning. Furthermore,
human cognition reflects its social evolution and context, as
well as contributions from a developmental process. After
presenting these themes, we illustrate their application to
the process of anticipation. Specifically, we propose that
anticipations occur extensively across domains (i.e., goal
management, perception, action, reward, affect, and
learning) in coordinated manners. We also propose that
anticipation is central to situated action and to social
interaction, and that many of its key features reflect the
process of development.
Keywords Coordination Á Development Á Embodiment Á
Robotics Á Situated cognition Á Social interaction
In short, the practically cognized present is no knife
edge, but a saddle-back, with a certain breadth of its
own on which we sit perched, and from which we look
in two directions in time William James, 1890,
Chapter 15.
There can be little doubt that cognition is informed by the
past. There can also be little doubt that much of cognition
is about the future. What will happen next? Where must I
put my hand so that I pick up the cup? If I say what I am
thinking, how will my listener respond? Human cognition
is magical in its prescience, in its attempts, often successful,
to foresee the future. How should we understand this
core phenomenon of anticipation (also viewed widely as
prediction)? How should we study anticipation so as to
understand its underlying processes and principles?
One possibility is to situate anticipation in cognitive
computation. This fits the traditional view in which mental
life is divided into discrete steps of ``sense-think-act.''
Cognition, by definition, is about the ``think'' part, the
knowledge and processes that mediate perception and action.
From this perspective, knowledge consists of amodal
propositions that represent experience, and anticipation is a
form of inference that operates on this knowledge to produce
further propositions about what is likely to happen
next.
This approach has a long and venerable history in cognitive
science. It makes considerable sense for many reasons,
and has made major contributions to the field. For
example, this view has been adopted widely in the study of
text comprehension (e.g., Kintsch and van Dijk 1978),
problem solving (e.g., Newell and Simon 1972), and reasoning
(e.g., Rips 1994). In general, this approach assumes
that people have stable models (or schemas) of the logical,
causal, and temporal structure of the world, and that a wide
L. W. Barsalou (&)
Department of Psychology, Emory University,
Atlanta, GA 30322, USA
e-mail: barsalou@emory.edu
URL: http://www.psychology.emory.edu/cognition/barsalou/
index.html
C. Breazeal
Massachusetts Institute of Technology,
Cambridge, USA
L. B. Smith
Indiana University, Bloomington, USA
123
Cogn Process (2007) 8:79­91
DOI 10.1007/s10339-007-0163-1
variety of processes operate on this knowledge. For
example, these models support understanding and predicting
how rain and sun contribute to plant growth (e.g.,
Gentner and Stevens 1983), how levers and pulleys work to
increase the force on objects (e.g., Chandrasekaren and
Josephson 2000; Forbus 1993; Hayes 1985), and how
events in a restaurant unfold over time (e.g., Schank and
Abelson 1977). In these classic accounts, the knowledge
that underlies anticipation is assumed to be fundamentally
different in kind, and theoretically separable, from the real
time processes of perceiving, acting, and emoting.
In this article, we present an alternative account that is
spreading throughout cognitive science and cognitive
neuroscience. Although this account takes many forms that
are evolving rapidly, four important themes underlie it
(among others). These four themes, together, create what
we will call the ``magic of human cognition'' as an
emergent consequence. By the ``magic of human cognition''
we will mean (1) the powerful ability to construct
representations and behaviors that support ambitious goal
achievement and social coordination, (2) the robustness
and adaptability of learning across contextual change, (3)
the emergence of intelligence from complex extended
interactions between social agents and the environment
over a developmental time course. The four themes that we
believe create this ``magic'' are as follows.
First, knowledge has no existence separate from process,
but is instead embedded in, distributed across, and thus
inseparable from real time processes (e.g., Barsalou 1987,
1989, 1993; McClelland et al., 1986; O'Regan and Noe
2001; Rumelhart et al., 1986; Jones and Smith 1993;
Samuelson and Smith 2000; Port and van Gelder 1995;
Spivey 2006). From this perspective, there is not a fixed
and separate representation of anything.
Second, to understand cognition, it is essential to
understand fundamental contributions from what have
traditionally been viewed as non-cognitive systems,
including goal management, perception, action, reward,
affect, social interaction, and development. Rather than
cognition being a modular system that operates independently
of these other systems, these other systems play
central roles in cognition per se, and thus are not really
``non-cognitive.''
Third, to understand a cognitive process such as anticipation,
one must consider the question of how to build it.
On way to address this question is to understand how the
process develops from birth over time in biological agents.
Another approach is to engineer the process in artificial
intelligence, for example, in robots.
Fourth, to understand a cognitive process, such as
anticipation, it is insufficient to understand the process in
isolation. Instead, understanding its coordination with other
processes that are typically co-active during real world
cognition is also essential. Indeed, understanding how a
process coordinates with other processes may be as
important, if not more important, than understanding the
internal structure of the process itself.
In the next three sections, we develop the second, third,
and fourth themes just described in greater detail. Because
the first theme--lack of separation between representation
and processing--has been addressed widely, we do not
address it further. Because the latter three themes have
received less attention, we focus on them here. After
developing these latter three themes, we sketch an account
of anticipation from the perspective of these four principles
in the final section.
Cognition depends intrinsically on non-cognitive
processes
We begin with an analogy (or cautionary tale) that may at
first seem far from the issue of anticipation (Smith and
Thelen 1993). The lesson of this analogy is that one cannot
separate cognition from non-cognitive processes, nor from
the coordination of all these processes in real time. The
analogy concerns quadrapedal locomotion in cats. Classic
theories of motor behavior proposed the construct of a
stable central pattern generator (CPG) to explain the patterned
regularity of a cat's alternating limb actions during
locomotion. Accumulating findings, however, raised
problems for this construct. For example, cats walk with
alternating limbs on a treadmill even when their spinal
cords are separated surgically from their brains (e.g.,
Delcomyn 1980). Based on these findings, some
researchers made the stretch that the CPG is in the spinal
cord. The validity of a CPG has been questioned on many
other grounds as well (Thelen and Smith 1994).
Even if a CPG did exist, it alone could not underlie the
stability apparent in the alternating limb action of cats
walking across real terrains. Cats walk backward, forward,
on grass, on hills, on rubble. They side step objects; they
walk when one limb is in a cast. Although a stable
alternating limb pattern is apparent in all cases, the
adaptability and variability of this pattern is remarkable.
Moreover, alternating limb movements in these different
contexts require fundamentally different patterns of muscle
firing to maintain the same global stability of alternation.
If a CPG exists, then it must constantly make
walking happen in globally similar but appropriately different
ways across diverse contexts. Thus, the overall
system that produces walking cannot be solely the CPG,
but instead could be multiple systems that include a CPG.
Another possibility is that there is no CPG and that the
global pattern of alternating limbs emerges from a variety
of interacting systems.
80 Cogn Process (2007) 8:79­91
123
The importance of ``outside'' systems in walking is
analogous with the nature of higher cognition. Theories
that separate the ``think'' part from the perceiving, acting,
and affective parts fail to recognize the importance of these
other systems in cognitive activity (cf. Van Order et al.
2003). Just as there may not be a CPG module that produces
walking, there may not be a pure cognitive module
that produces cognitive processing. Instead, cognition may
emerge from the interaction of many contributing systems,
analogous to how cat locomotion emerges from multiple
systems. As suggested later, anticipation is also likely to
emerge from many systems in this manner. In the following
subsections, we outline contributions to cognition from
non-cognitive systems, including the systems that implement
perception, action, reward, affect, goal management,
motivation, and social interaction.
The brain's modality-specific systems. Increasingly,
researchers argue that systems in the brain for perception
and action are essential parts of the cognitive system (e.g.,
Barsalou 1999b; Damasio 1989; Glenberg 1997). Rather
than the cognitive system being modular, it utilizes
mechanisms in modality-specific systems for its fundamental
representation and processing activities (both in the
brain and in the body). From this perspective, knowledge is
represented as simulations in modality-specific systems,
not as amodal symbol elsewhere in a self-contained modular
system. Understanding perception and action is
essential for understanding cognition.
Consider recent findings that support this conclusion.
Zwaan et al. (2002) found that when people comprehend a
sentence, they construct a visual simulation to represent its
meaning. Similarly, Glenberg and Kaschak (2002) found
that people construct motor simulations when comprehending
sentences (also see Hauk et al. 2004; Zwaan and
Taylor 2006). Chao and Martin (2000) found that when
people view a static object, they construct a motor simulation
to represent how they would act upon it. Simmons
et al. (2005) found that when people view a food, they
construct taste and reward simulations as they anticipate
eating.
Many findings such as these indicate that perception and
action play central roles in cognition (for reviews of
additional evidence, see Barsalou 2003, in press-b; Barsalou
et al. 2003; Martin 2001, 2007). Many behavioral
experiments show that modality-specific variables affect
high-level cognitive tasks, such as language comprehension.
Many experiments in social cognition show that
bodily states affect diverse forms of social inference. Many
neuroimaging experiments show that the brain's modalityspecific
systems become active as people perform cognitive
tasks. Together, the findings from these diverse areas
indicate increasingly that cognition does not operate
independently of modality-specific systems for perception
and action. Instead, mechanisms in these systems play
central roles in cognition per se.
Furthermore, the coordinated relationships between
perception, action, and cognition must be identified to
characterize cognition adequately. Again, much work
shows increasingly that these systems are exquisitely
linked. For example, judging the weight of an object lifted
by another agent requires simulating the lifting action in
one's own motor and somatosensory systems (Bosbach
et al. 2005). The visual perception of the lifting event
interacts with cognitive simulations of motor and somatosensory
activity, such that the cognition in this judgment is
distributed throughout many systems, not just a ``modular
cognitive system.'' Similarly, the ability of pianists to
identify auditory recordings of their own playing depends
on their ability to simulate the motor actions that underlie it
(Repp and Knoblich 2004). Again, the cognition of these
judgments extends into the auditory and motor systems.
Furthermore, even the simple execution of motor actions
involves complex interactions between systems, as the
brain simulates visual and somatosensory models of actions
to guide and correct them (e.g., Wolpert et al. 1999;
Wolpert et al. 2003). These feedforward models of actions
also play extensive roles in anticipating visual perceptions
(Wilson and Knoblich 2005).
Findings such as these indicate that perception, action,
and cognition interface intricately, something that classic
symbolic theories largely ignore. Grounding higher cognition
in modality-specific representations provides one
solution to this implementing this interface. Because the
knowledge representations that underlie higher cognition
are grounded in perception and action, they link cognition
to the sensory-motor interface.
Perusing the evolution of cognition from the simplest
organisms to humans, it is clear that cognition in the
simplest organisms began with perception coupled directly
to action via hard-wired circuitry. Over the course of
evolution, increasingly sophisticated mechanisms evolved
to mediate perception and action. Nevertheless, perceiving
and acting remained foundational to the system as a whole.
A system is not fully intelligent if it cannot perceive the
world and effect change in it. As a prerequisite, the detailed
structure of perception and action must be linked closely
with cognition. Indeed, increasingly powerful perceptual
and motor abilities may have prodded the sophistication of
cognition forward. Building a cognitive system that fails to
integrate these systems is not likely to implement the
magic of human cognition.
The brain's affective and motivational systems. Affect
regulation and motivation play substantial roles in human
activity. Following Damasio (1994), researchers have
Cogn Process (2007) 8:79­91 81
123
become increasingly convinced that cognition divorced
from affect is not rationale, and that optimal cognitive
performance occurs when affective information is included
in decision making. Over time, affective information
accumulates with events and objects, indicating whether
they are generally associated with positive or negative affect.
On later encountering these events and objects,
average affective information becomes available quickly to
support decision making. When this information is lacking,
decision making suffers. Thus, affective mechanisms play
central roles in cognition, and a full understanding of
cognition is impossible without them.
Another example is the central role of the brain's reward
systems in learning (e.g., Ashby et al. 2005; Granger 2006).
As rewards are experienced for stimuli, cortico-striatal
loops integrate this information. Later, when these stimuli
are perceived again, associated reward circuits become
active and determine actions performed on the stimuli. In
this manner, the reward system fundamentally shapes
cognitive processing. More generally, motivational processes
are central throughout cognition, from initiating
basic response systems for hunger, thirst, etc., to controlling
the pursuit of complex social and personal goals.
Without these systems, theories of cognition are incomplete.
Executive systems are also necessary for maintaining
goals in working memory, and for deciding when to pursue
or drop goals.
Havas et al. (in press) provide still another demonstration
of how higher cognition depends on the affective
system. While people read texts that contained emotional
content, they had their faces configured into bodily states
associated with particular emotions. As a result, the time it
took them to judge the sensibility of a critical sentence
depended on whether the emotion it described was consistent
with their current bodily state. The fact that highlevel
cognitive judgments depended on these bodily states
and the affective states that they engendered demonstrates
the pervasive dependency of cognition on contributing
motor and affective systems (for many related phenomena,
see Barsalou et al. 2003). Although emotion, reward, and
motivation are typically studied as independent processes,
they are tightly integrated with each and with the cognitive
system (e.g., Barrett 2006; Barrett et al. 2007).
Systems for social interaction and communication. Researchers
who study the evolutionary origins of the human
cognition system argue that social pressures shaped human
cognition extensively (e.g., Donald 1993; Tomasello et al.
1993). Because increasingly sophisticated social interaction
enabled major gains in evolutionary fitness, powerful
new mechanisms evolved in the human brain that were
absent in earlier species. Included in this list of mechanisms
are joint attention, perspective taking, mirroring,
imitation, and language. By evolving these mechanisms,
humans were able to represent and coordinate complex
social activity, which in turn yielded major gains in controlling
environmental resources and maximizing reproductive
outcomes. From this perspective, failing to include
such mechanisms in an intelligent system precludes it from
achieving the magic of human cognition.
Contemporary research in social cognition again suggests
a rich interdependence among a diverse collection of
neural systems. Findings across social psychology, social
neuroscience, and electro-physiology show that the visual
and motor systems play central roles in social information
processing (for reviews, see Blakemore and Decety 2001;
Gallese et al. 2004; Iacoboni, in press; Rizzolatti et al.
2002). As the action of an agent is perceived, the motor
system simulates the action in the motor system of the
perceiver. This simulation produces comprehension of the
agent's action and generates visual inferences about what
the agent is likely to do next (Wilson and Knoblich 2005).
This ``mirror neuron'' circuit epitomizes a mixture of
systems from which high-level cognition emerges, in this
case, cognition central to social processing.
Mirror systems are also central to learning (e.g., Breazeal
et al. 2005). As agents observe the actions of other
agents, their mastery of skills is facilitated through the
social direction of attention, imitation, and verbal instruction.
Humans probably learn important things more often
from social interaction than they do from isolated individual
interactions with inanimate stimuli. Furthermore,
these socially acquired skills are intrinsic to coordinated
activity in division-of-labor settings, and also to competitive
activity in conflict situations (e.g., Hutchins 1995).
Because human cognition probably evolved to support
unusually extensive and sophisticated social interaction, it
is likely that mechanisms for processing social information
are densely inter-connected with cognitive mechanisms. To
understand the human cognitive system, it will therefore be
necessary to study cognition in its social contexts, not just
when processing non-social information, such as isolated
words and sentences.
Putting it together: an example. One scientific approach
that makes clear the dependency of cognition on noncognitive
processes is the construction of artificial systems
that attempt to exhibit human-like intelligence. Here we
present an example of how cognition emerges from multiple
domains in an anthropomorphic robot (e.g., Breazeal
et al. 2005). Specifically, this example illustrates how a
mirror-neuron-like system can develop from multimodal
coordination during face-to-face play.
The robot brings the following abilities to the task: (1)
the ability to visually track the facial features of a person,
(2) the ability to ``motor babble'' by exercising its initial
82 Cogn Process (2007) 8:79­91
123
repertoire of facial movements, (3) the ability to sense its
own facial configuration, (4) a coarse mapping of ``organ
relations'' that roughly relates regions of the robot's own
face to regions of the perceived faces of others, and (5)
attraction to contingent interactions, where a contingency
metric determines whether a visually perceived movement
is temporally contingent on the robot's own movement.
The task that couples these systems is an imitation game.
As the human developmental literature shows, it is often
adults who take the initiative to imitate the facial expressions
of infants (e.g., Jones 2006). Hence, as Breazeal
et al.'s robot ``motor babbles'' by exercising its repertoire
of facial expressions, a human participant imitates the robot.
Although the robot cannot see its own face, it can sense its
facial configuration via proprioception (i.e., position sensors).
As the robot moves its face from expression to
expression, it observes visually how the human's face responds.
Meltzoff and Moore (1997) posited that the desire
to match and to be matched by others is innately rewarding
to human infants (also see Meltzoff 1996). This is essential
to the robotic simulation, as there is no behavior, no goals,
no learning without motivation. In this simulation, a contingency
metric allows the robot to determine which regions
of its own face the human has chosen to mimic, causing
these regions to become salient. Through this process, the
robot attentively selects pairings of matched regions between
its own face to and those of the human. Via a standard
statistical learning algorithm, these pairings teach the system
about how perceived visual movements of the human's
face map onto the robot's corresponding motor movements.
The inter-modal representations acquired from the
coordination of perception and action allow the robot to
directly compare its motor movements to observed
expressions in the same motor-based coordinate system.
Once this representation exists, the robot can mimic facial
expressions of the human. The robot can also mimic novel
never-produced facial gestures by searching over a weighted
blend space of its motor repertoire to find (and generate)
an adequate match. Furthermore, this learning increasingly
establishes a mirror system that can be used in other tasks
(e.g., Rizzolatti et al. 2002). The same representations can
be used not only to generate the robot's own actions, but to
recognize the same actions in others. Acquiring this ability
has profound implications for many other forms of social
learning, such as true imitation, social referencing, and
other forms observational learning. It develops through the
coordination and interaction of diverse component systems.
Cognition emerges from coordinated sets of processes
Divide and conquer is the standard strategy for making
progress in empirical research, formal modeling, and AI
engineering. Certainly, divide-and-conquer has obvious
strengths, including the ability to isolate processes, rule out
confounding variables, demonstrate control over phenomena,
and establish causal (as opposed to correlational)
relationships. In engineering and formal modeling, isolating
processes greatly reduces the complexity of systems that
must be built and the problems that must be solved. It also
makes analytic understanding and formalization easier.
It is far from sufficient, however, to understand cognition
well enough so that it can be taken apart. Instead,
understanding cognition may require putting it all back
together again so that it actually works as a whole. The
magic of human cognition may reside not in its specific
components but in their coordination. Furthermore, the
study of processes in isolation may lead to fundamentally
wrong accounts of them, or at least accounts that are different
from what they would be in the context of coordinated
activity. A stand-alone AI implementation of a
process can probably not be plugged effectively into a
larger coordinated system without considerable reprogramming,
if not complete redesign. Thus, studying and
implementing a cognitive process in a complete system that
performs many processes together may teach us more
about this process than studying and implementing it in
isolation.
Just because the cognitive science literature is full of
research on isolated cognitive tasks, it does not follow that
this approach will eventually produce a complete account
of cognition. If we understand how a brain implements
many individual processes, it does not follow that we
understand how they interact in a coordinated manner.
Similarly, just because we can implement many individual
processes does not mean that we understand how to
implement them together.
A related theme is that agents in the real world do not
perform individual tasks in isolation. Instead, they perform
sets of coordinated tasks that produce coherent goal-directed
behavior. For example, organisms do not perform
categorization alone. Instead, they perform categorization
together with perception, inference, action, reward, and
affect.
Coordination during situated action. Situated action
provides one way of exploring coordinated processes (e.g.,
Barsalou 2003, in press; Brooks 1991; Clark 1997; Glenberg
1997; Robbins and Aydede, in press). In situated action,
agents have goals. As they navigate their environment
during goal pursuit, they manage goal priorities, based on
motivational states and opportunities in the environment.
At each moment, they also perceive the environment,
categorize entities and events, and draw inferences that go
beyond the information given. They also perform many
kinds of memory retrieval about possible actions, rewards,
Cogn Process (2007) 8:79­91 83
123
affective states, etc. And ultimately, agents act. At each
point, they are not only learning about how to perform
individual cognitive processes, they are also learning how
to coordinate them.
The core set of coordinated cognitive processes associated
with situated action includes goal management, perception,
categorization, inference, action, reward
assessment, and affect (with learning throughout). We
suspect that the coordinated processes underlying situated
action exist in many species, not just in humans (e.g.,
Barsalou 2005). It is easy to imagine many other species
managing goals, perceiving and categorizing the environment,
generating simple inferences about what will happen
next, performing actions based on previous rewards, and
experiencing affect in response to the outcomes of these
actions.
Thus, understanding the coordinated processes that
underlie situated action could be informative about intelligence
across many species. This particular set of coordinated
processes may constitute the core kernel of
intelligence that evolved into human intelligence. Because
this may be the most basic set of coordinated processes in
the human brain, it might make scientific sense to understand
it first, both empirically and theoretically. Rather than
trying to understand the most advanced human abilities
first, such as logic and mathematics, it might be more
tractable to understand how these advanced abilities built
upon more basic abilities that existed previously, such as
the ability to coordinate situated action.
Coordination during social interaction. Another important
set of coordinated processes are those that support the
unusually sophisticated social abilities of humans. As described
earlier, humans are unusual in establishing joint
attention and in representing other minds. Humans also
have unusually good communication systems that allow
them to coordinate shared mental states and complex social
activities. Humans are also unusually good at learning from
each other via observation, imitation, and verbal instruction.
Understanding the coordination of these social processes
is probably central to understanding human
intelligence.
In addition, many of the basic processes that support
situated action probably contribute to social coordination
as well. It is also probably important to study social processes
together with those for situated action, given that
much social behavior revolves around goal-pursuit in the
environment (e.g., Barsalou 1999a; Barsalou et al. 2003;
Smith and Semin 2004).
Coupled bodies and minds: an example. We illustrate the
coordination of social cognition with perception and action
in a robot that represents other minds. The key ingredients
are correlations that emerge from coupled agents who have
similar bodies and similar cognitive systems, and who are
interacting in a shared situational context. This recipe
produces abilities associated with the magic of human
cognition, such as ``mind reading'' inferences about the
internal states of others.
An agent engaged in a task with another agent who has a
similar body and a similar cognitive system can detect the
following correlations:
* correlations between the appearance of the self and the
appearance of others (e.g., hands to hands, feet to feet),
* correlations between the behavior of the self and the
behavior of others (looking to an object),
* correlations between one's bodily behaviors and one's
internal states (e.g., looking left and remembering what
was on the left; maintaining the memory of a goal and
looking in the direction of the goal),
* correlations between the external states of another and
one's own internal states (seeing where someone
looks, looking there oneself, and thinking about that
location).
Numerous experiments in humans document these correlations.
One large area of research shows that bodily
states and affect are tightly coupled (e.g., Strack et al.
1988; for reviews, see Barsalou et al. 2003; Niedenthal
et al. 2005). When a person adopts a bodily state associated
with an affect (e.g., a posture or facial expression), the
affect is elicited (e.g., slumping produces negative affect).
These body-affect correlations, in turn, support empathy
through mirroring (e.g., Blakemore and Decety 2001;
Gallese et al. 2004; Iacoboni, in press; Rizzolatti et al.
2002). If a person's face mirrors the facial expression of
another agent, the previously established correlation between
the somatosensory experience and the associated
affect produce the affect. As a result, the perceiver adopts
the same affective state as the agent. Other time-locked
multi-modal cues further facilitate learning this mapping,
such as the affective speech that accompanies facial
expressions between caregivers and infants (e.g., Fernald
1989).
Through such couplings and coordinations, an artificial
device, such as a robot, can develop ``human-like intuitions''
about the internal states of others (e.g., Breazeal and
Aryananda 2002; Gray et al. 2005). Consider a robot
developed by Breazeal (2003) that learns to perform
affective appraisal (e.g., Plutchik 1991; Izard 1977). During
face-to-face interactions with a human, heterogeneous
processes in the robot become tightly coupled over time.
When the robot imitates the human's facial expressions,
body-affect pathways in the robot evoke the corresponding
affective state. Affective information in the human's speech
signal reinforces this response. All of these multi-modal
84 Cogn Process (2007) 8:79­91
123
states are time-locked because of the similarity in bodies
and body-affect mappings. As a result of these correlations,
the robot learns to associate its internal affective states with
the facial expressions of humans.
Significantly, this ``existence proof'' from robotics
illustrates how similar correlations in humans between
perceptual, motor, and affective states could produce the
ability to ``mind read.'' From the acquired coordination of
multiple systems emerges one of the most important facets
of human intelligence.
The incremental process of development
Developmental researchers have argued for decades that
studying adult cognition in isolation will never be successful
(e.g., Bjorklund and Pelligrini 2000). To the contrary,
understanding the mature adult system requires
understanding the history of biological growth, social
interaction, and cognitive processing that produced it.
Smith and Gasser (2005) proposed that the developmental
process in humans is successful because the developmental
environment contains several important characteristics.
First, partially redundant sources of sensory-motor information
in the learning environment allow babies to educate
themselves, without teachers, simply by interacting with
the world. Second, an incremental learning process over
the course of development creates capabilities--such as
understanding the cause and effect relations among actions
and rattles--that could not exist genetically at birth. Third,
rich statistical structure in the physical world is central to
this incremental learning process. Not only does this
structure scaffold development of the adult cognitive system,
it remains a continual source of constraint and support
during adult learning and behavior. Fourth, extended
exploration is essential for the development of a mature
cognitive system, where exploration discovers structure in
the physical world, and also produces important cognitive
skills, ranging from simple motor behaviors to creativity.
Fifth, the development of a mature cognitive system depends
on extended experience with other agents who
constitute rich sources of instruction at many levels,
including knowledge, skills, and meta-cognition. Sixth, the
development of a mature cognitive system depends on
extended experience with other agents operating together
in a shared environment. As agents share information
symbolically and non-verbally, important knowledge,
skills, and meta-cognition develop.
From this perspective, the system-level properties that
are the signature of human cognition only emerge from an
extended history of coordinating cognitive and non-cognitive
processes in situated action and social interaction.
Anticipation is one such system-level property likely to
develop from a long history of coordinated development.
Although the development of individual processes is certainly
important, the development of their coordination is
no less important. Furthermore, the development of coordination
may affect the internal structure of individual
processes. If so, then fully understanding an individual
process cannot be achieved by studying it in isolation, or
even in coordination during adulthood. Instead, understanding
the developmental history that coordinated the
process with other related processes during its development
is essential.
Also essential is an understanding of the training regimens
that structure the development of coordination.
Critically, these regimens typically include structure in the
physical world and interactions with social agents. Relatively
simple raining regimens may often arise implicitly to
produce simple coordination initially, followed by more
explicit and complex regimens that produce increasingly
sophisticated coordination. Interestingly, there may be no
other way to establish the kind of flexible, programmable
coordination seen in humans. Although genetically based
coordination may be sufficient in simpler species, multiple,
overlapping, training regimens may be the only way to
achieve human levels of coordination. Indeed, the magic of
human cognition may reflect the results of training regimens
to a considerable extent.
What kind of architecture? If these ideas are anywhere
near correct, a key problem becomes the nature of the
architecture that underlies the development of coordination.
One way to approach this issue is to ask how one
might build an artificial agent whose system-level properties
emerge from a developmental history. Hybrid approaches
offer one potential solution. Specifically,
researchers could build upon the (considerable) success of
the classic sense-think-act approach and implement hybrid
systems that attempt to exhibit coordination between traditional
cognitive and non-cognitive processes. Specifically,
researchers could start with a mature theory of
symbolic cognition and attempt to coordinate modules that
implement goal management, perception, action, affect,
reward, social interaction, and development. Building a
hybrid system may be feasible, at least to some extent.
Success would be of considerable theoretical interest and
significance because it would suggest that the modular
approach is feasible. Furthermore, success might suggest
that the cognitive system per se includes something similar
to the relatively modular cognitive system in classic symbolic
theories. At a minimum, success would indicate that
these theories capture important functionality in the brain.
An alternative strategy, and one that we believe cognitive
science should consider, is to develop new architectures
motivated explicitly by the attempt to integrate
Cogn Process (2007) 8:79­91 85
123
diverse domains of natural intelligence, including goal
management, perception, action, cognition, reward, affect,
social interaction, and development. From studying these
domains together and assessing what it would take to build
computational systems that integrate them, alternative
architectures may emerge that are much more powerful
than previous ones.
The development of coordination: an example. A recent
attempt to engineer social referencing in a robot illustrates
how a sophisticated ability can emerge developmentally
from the coordination of multiple systems. Social referencing
is the ability to use the emotional reaction of another
agent to help form one's own affective appraisal of a
novel situation, which can then be used to guide subsequent
behavior (e.g., Feinman 1982). In human infants, social
referencing arises under conditions of uncertainty and
ambiguity, when one's own intrinsic appraisal processes
are not adequate (e.g., Campos and Stenberg 1981). Social
referencing is an important form of emotional communication
and is a developmental milestone for human infants
in their ability to learn about the environment through social
interaction. It is also a behavior built from many
component processes, including emotional empathy
(bootstrapped from early facial imitation as discussed
above), joint attention, and the ability to form affective
memories (associating positive or negative valence to
stimulus representations in memory).
Thomaz et al. (2005) observed the development of social
referencing in a robot and reached the following conclusions
about the developmental process. The robot's
social referencing ability emerged in real-time from the
dynamic coordination of multiple systems during interactions
with a human agent. When the robot encountered a
novel object, the object appraisal mechanism tagged the
object as novel, which biased the affective system to evoke
a mild state of anxiety. In turn, the robot's face expressed a
heightened state of arousal as it looked upon the novel
object. The robot also looked to the human's face to soothe
itself, reflecting a previously established correlation. The
human then reacted in a naturally instructive way, noticing
the robot's anxious reaction to the unknown object, and
showing the object to be safe. For example, the human
often picked up the object and shared her positive reaction
to it with the robot.
Consider further aspects of how this social referencing
process develops. As the human gazes toward the novel
object and reacts to it affectively, the robot's attentional
focus is drawn to the object as well. By computing relative
looking-time towards various objects in the environment,
the robot establishes the novel object as the referential
focus. As the robot's attentional focus shifts to the human
(while maintaining the novel object as the referential
focus), the robot extracts affective information from the
human's face and voice, using the empathic mechanism
described earlier. The resulting change in the robot's
internal affective state triggers an encoding process that
establishes a memory of the object, tagged with the robot's
affective state. Thus, the novel object is appraised with
socially communicated affective information and committed
to long-term memory.
Over time, the robot becomes increasingly sophisticated
in its ability to perform social referencing. Although this
ability is not present initially, it emerges from the soft
assembly of existing processes following interactions with
adult agents. If this is how social referencing develops, then
viewing it as a pre-existing modularized process is misguided.
Instead, this fundamental cognitive ability results
from extended practice at coordinating simpler processes.
The ``developmental'' achievements of this robot
should be interesting to human developmentalists for several
reasons. First, these achievements provide a measure
of how well we currently understand development. To the
extent that we understand the core developmental principles
that produce human intelligence, we should be able to
apply these principles when engineering artificial intelligence.
Second, robots serve as physical platforms on which
we can model complex coordinated processes across domains
(e.g., empathy, social referencing). Robotic platforms
allow researchers to present real time tasks in
physical and social environments to embodied cognitive
systems that are softly assembling processes repeatedly
across development.
Anticipation as coordinated non-cognition
In this final section, we apply our themes to the cognitive
process of anticipation. We first describe important situations
in which anticipation plays central roles. We then
describe how anticipation relies critically on non-cognitive
processes. Finally, we describe how viewing anticipation as
coordinated processes differs from thinking about it as a
single process.
Choosing situations. We believe that optimal rates of
progress in understanding cognition depend on the judicious
choice of situations for scientific study. As suggested
earlier, we believe that situated action is one particularly
important case. Because situated action occurs ubiquitously
across species and is central for survival, it seems central to
understand. To the extent that many cognitive processes
evolved to support situated action, studying and understanding
this situation should be essential to understanding
these processes. For these reasons, we begin with situated
action. Because social interaction also appears highly
86 Cogn Process (2007) 8:79­91
123
central in human intelligence, we consider social situations
as well.
As described earlier, situated action includes the following
processes: goal management, perception, categorization,
inference, action, reward, affect, and learning.
During situated action, an agent is typically moving
around with a goal in mind, perceiving the environment,
and categorizing what is present. Once categorizations are
made about perceived objects and events, the categories
accessed generate likely inferences about events, actions,
rewards, and affects that could follow. The agent then
selects an action and performs it. Events in the world
follow that produce rewards, which in turn produce affect
and learning. Over time, this approximate cycle iterates
constantly, taking many variations that depend on current
conditions and the agent's expertise (including developmental
level).
In social situations, other agents are also present. They,
too, are typically pursuing goals that may be competitive or
cooperative. They, too, are having cognitive and affective
states. They may further model behaviors that serve as
instruction, and may offer instruction explicitly, either
verbally or by doing.
Situated anticipation. Where does anticipation arise in
these situations? Everywhere, in intricately coordinated
manners. Once a goal is selected, plans and possible
courses that plans could take are anticipated. Relevant
stimuli in the environment are anticipated as well. When
entities and events in the environment are perceived and
then categorized, category knowledge generates anticipations
in the form of categorical inferences, including potential
actions that could help achieve the current goal. As
these actions are entertained, their consequences are
anticipated, including reward and affect. As feedback is
encountered, it specifies whether anticipations were correct
or incorrect, producing extensive learning at multiple levels.
We assume that neural simulations underlie all these
anticipations in the relevant modality-specific systems
(Barsalou, in press).
The coordination among all these anticipations is
extensive. An active goal must coordinate with perception
to identify goal-relevant information in the environment.
The goal must also coordinate with working memory to
maintain goal-relevant information, such as relevant stimuli
in the environment to find. Once a relevant entity is
identified, extensive coordination between memory, perception,
and action must occur to interact effectively with
it. As the consequences of actions become available,
coordination between anticipated rewards and affects must
occur to evaluate what has happened so far, and what to do
next, if anything. Further coordination must occur with
goal management, starting the cycle all over.
Many additional anticipations arise in social situations,
falling under classic topics in social psychology, such as
person perception and causal attribution. On perceiving an
agent act, for example, a social perceiver anticipates further
actions that are likely to follow. Perceiving an embodied
state, such as a facial expression or posture, similarly produces
anticipations about subsequent actions. Inferring the
mental states of agents also leads to extensive anticipations
about what the agents will do next. Situations, too, play
central roles in producing anticipations, given that particular
behaviors occur in particular situations. In general,
tremendous amounts of anticipation occur during social
interaction. Again, we assume that neural simulations in the
relevant modality-specific systems underlie these anticipations
(Barsalou, in press; Decety and Gre`zes 2006).
Anticipation is central to instructional interactions and
to collaborative work. In instructional settings, the teacher
generally anticipates what should happen next in the domain
of study much better than can the student. The teacher's
job is often to teach students sequences of operations
that achieve goals, including mental operations and
assessments, not just physical actions. As students become
increasingly competent, they become increasingly adept at
anticipating what to do next and what should happen as a
result, no longer requiring the teacher's assistance. In
collaborative work, each co-worker must anticipate what
other co-workers will do, and how the collaborative process
will evolve. Extensive coordination between agents in
all these settings is essential for success.
Studying situated anticipation empirically. How should a
rigorous experimental psychologist approach the study of
situated anticipation? Obviously, the situations just described
are so complex that controlling and analyzing them
with classic methods is not feasible. Nevertheless, we believe
that these situations should play a central role in
motivating experimental research.
Typically, experimental paradigms are chosen with little,
if any, interest in their ecological relevance. Instead,
the primary reasons for selecting a paradigm are ease of
implementation in the laboratory, potential for rigorous
control, and tractability in mathematical modeling. For
example, research on anticipation in cognitive psychology
has been dominated by research on lexical and semantic
priming from words. Clearly, much elegant work has resulted
from this approach. We believe, however, that this
work likely to have little impact until it demonstrates its
applicability to real world problems. What potential
implications does our understanding of semantic priming
have for situated action and social coordination? Because
these situations are usually not considered when choosing
research paradigms, these paradigms have little potential
for informing our understanding of them.
Cogn Process (2007) 8:79­91 87
123
Attempting to rigorously understand situated action and
social interaction have tremendous potential to drive
experimental research forward. Consider how framing
experimental work in this manner could generate novel and
potentially valuable research paradigms. Specifically,
consider situated action. Researchers could take each specific
form of anticipation in situated action described earlier
and develop a paradigm for understanding it. Although
these researchers would be isolating mechanisms, they
would be isolating mechanisms that belong to a larger
coordinated system that we know is central to human (and
non-human) activity. As researchers increasingly understand
specific forms of anticipation in situated action, they
could begin to study the coordination of different forms.
Although this would increase the complexity of experimental
paradigms, studying small subsets of a coordinated
system in a controlled manner is feasible.
By framing experimental investigations in this manner,
entire systems of coordinated processes from multiple domains
can be investigated that ultimately yield the magic of
cognition. Anticipation would be studied in goal management,
perception, action, reward, and affect. Not only
would we understand individual anticipation processes, we
would understand a set of individual processes that operate
together in an ecologically important situation.
Along with classic experimental work, qualitative and
descriptive methods would be valuable as well. Before
beginning analytic laboratory work, it is essential to describe
the component processes of the target situation, and
to document the underlying patterns of coordination. Rather
than relying on arm chair assessments of what a target
situation contains, rigorous assessments of its content
should be made, using standard observational and correlational
techniques. Such studies could be viewed as
analogous to the extensive documentation of phenotypes
that preceded more analytic laboratory work in genetics.
Before identifying genetic mechanisms, it was necessary
first to identify the hereditary distributions of phenotypic
patterns to be explained. We believe that a similar state of
affairs exists in cognitive science. We first need to identify
the components of important situations, such as those for
situated action and social interaction, along with their
statistical patterns of distribution. Once we have this
descriptive information, laboratory paradigms can then be
used to isolate important processes, identify their properties,
and establish their coordination with other processes.
Implementing situated anticipation. How should situated
anticipation be implemented in AI systems? What is the
best way to implement all the forms of anticipation that
occur across domains during situated action and social
interaction, and to create effective coordination between
them? An obvious answer is robotics, more specifically,
robotics in social environments under developmental
training regimens.
Building an autonomous agent that captures the magic
of human cognition requires the inclusion of all relevant
domains, including goal management, perception, action,
categorization, inference, affect, learning, and communication.
Furthermore, these autonomous agents need to
operate effectively in situated action and social interaction.
Given the central role of developmental accumulation,
judicious choice of training regimens within these situations
is central as well.
Getting all the processes from these domains to work
increasingly together across a developmental trajectory
requires solving the coordination problem. Solving the
coordination problem may also help specify the individual
processes correctly in the first place. Clearly, there may be
times when implementing a process in a circumscribed toy
domain has its benefits. Ultimately, however, the process
must work effectively together with other processes across
domains in a complete autonomous agent. For these reasons,
we believe that the gold standard for implementing
anticipation should be implementing it in robots who
experience developmental trajectories in social domains.
Anticipation, goals, and higher-level coordination: examples.
Piaget's (1952) book, The Origins of Intelligence,
presents an insightful case of how anticipation emerges in
infants. Piaget placed, for the very first time, a rattle in his
4-month-old infant's hand. As the infant moved the rattle,
it came into sight and made noise. The sight and sound
aroused and agitated the infant, inducing further bodily
motions and causing the rattle to move even more rapidly
in and out of sight, and to make even more noise.
Infants at this age have very little organized control over
their hands and eyes. They cannot yet reach for a rattle. If
given one, they do not necessarily shake it. If the infant
accidentally moves the rattle, however, visual, auditory,
and somatosensory consequences result, capturing the infant's
attention. As these unintentional events repeat, the
infant increasingly gains intentional control over shaking
the rattle. Piaget allowed the infant to play with rattle
repeatedly for several days and observed the emergence of
anticipatory action. At the mere sight of the rattle, the infant
would begin to move its hands in the coordinated
pattern acquired from past experience. The unplanned and
untaught relations between actions and outcomes constituted
a self-organizing system that led to anticipatory
representations of cause and effect. Piaget referred to such
patterns as secondary circular reactions, namely, perception-action
loops that arise from an embodied multimodal
system behaving in the physical world.
Piaget believed that these emergent perception-action
loops are foundational for development because they create
88 Cogn Process (2007) 8:79­91
123
opportunities for learning. In the rattle example, the repeated
activity teaches the infant how to control its body,
what actions bring held objects into view, and how sights,
sounds and actions correspond to one another. Importantly,
this learning happens without an initial goal. Instead, the
goal--the intention to shake the rattle--emerges from
simply placing an embodied organism who has sensorymotor
systems, a motivational system, and a memory in a
particular physical environment.
A more recent example of how a new goal emerges
through anticipation during perception-action cycles is the
experimental procedure known as ``infant conjugate reinforcement''
(Rovee-Collier and Hayne 1987). Infants (as
young as 3 months) are placed on their backs, with their
ankles attached by a ribbon to a mobile suspended overhead.
The mobile, which produces interesting sights and
sounds, provides the infant with many time-locked correlations.
Significantly, infants themselves discover these
relations through their own movement patterns. The faster
and harder infants kick, the more vigorously the mobile
jiggles and sways. Further kicking results as the perception-action
cycle repeats itself. Infants become so highly
engaged that they smile, laugh, and become angry when the
contingency is removed.
The goals and perception-action cycles that emerge from
rattle and mobile shaking illustrate how complex forms of
anticipation emerge from placing an embodied agent in a
physical situation that complements its intelligent capacities.
As the agent learns to anticipate all the relevant
streams of information, they become exquisitely coordinated
with one another, and intelligence emerges. We
suspect that much of human cognition--and especially its
magic--emerges in this manner.
Conclusion
We realize that we have made ambitious requests. We have
requested that researchers integrate non-cognitive domains
with cognition. We have requested that researchers study
the coordination of processes, not just individual processes
in isolation. We have requested that researchers study the
developmental time course of coordinated processes in situated
action and social interaction.
These requests arise from our increasing belief that
cognition is more than a collection of independent processes.
Instead, we believe that cognition, and especially
the magic of human cognition, emerges from deep
dependencies between all the basic systems in the brain,
including goal management, perception, action, memory,
reward, affect, and learning. We also believe that human
cognition greatly reflects its social evolution and context,
as well as major contributions from a developmental
process. Because we believe that human cognition reflects
all these dependencies, we believe that it is necessary to
change how we study it.
The process of anticipation is a paradigm case for our
themes. Anticipations occur across all domains in a highly
coordinated manner. Anticipations are central to situated
action and to social interaction, and they grow in
sophistication as the result of a developmental trajectory.
We believe that our understanding of anticipation will
proceed most rapidly if examined from this perspective.
We also believe that the results of such study will move
cognitive science and cognitive neuroscience forward
significantly.
Acknowledgment DARPA contract BICA FA8650-05-C-7255
awarded to three authors supported their collaboration on this project.
In addition, Lawrence Barsalou was supported by DARPA contract
FA8650-05-C-7256 and National Science Foundation Grant BCS-
0212134. We are grateful to Arthur Glenberg and an anonymous
reviewer for helpful comments on a draft of this article.
References
Ashby FG, Ell SW, Valentin VV, Casale MB (2005) FROST: a
distributed neurocomputational model of working memory
maintenance. J Cogn Neurosci 17:1728­1743
Barrett LF (2006) Emotions as natural kinds? Perspect Psychol Sci
1:28­58
Barrett LF, Mesquita B, Ochsner KN, Gross JJ (2007) The experience
of emotion. Annu Rev Psychol 58:373­413
Barsalou LW (1987) The instability of graded structure: Implications
for the nature of concepts. In: Neisser U (ed) Concepts and
conceptual development: ecological and intellectual factors in
categorization. Cambridge University Press, Cambridge, pp 101­
140
Barsalou LW (1989) Intraconcept similarity and its implications for
interconcept similarity. In: Vosniadou S, Ortony A (eds)
Similarity and analogical reasoning. Cambridge University
Press, Cambridge, pp 76­121
Barsalou LW (1993) Flexibility, structure, and linguistic vagary in
concepts: Manifestations of a compositional system of perceptual
symbols. In: Collins AC, Gathercole SE, Conway MA (eds)
Theories of memory. Lawrence Erlbaum Associates, London, pp
29­101
Barsalou LW (1999a) Language comprehension: archival memory or
preparation for situated action. Discourse Process 28:61­80
Barsalou LW (1999b) Perceptual symbol systems. Behav Brain Sci
22:577­660
Barsalou LW (2003) Situated simulation in the human conceptual
system. Lang Cogn Process 18:513­562
Barsalou LW (2005) Continuity of the conceptual system across
species. Trends Cogn Sci 9:309­311
Barsalou LW (in press). Embodied cognition. Annu Rev Psychol
Barsalou LW (in press-b). Situating concepts. In: Robbins P, Aydede
M (eds) Cambridge handbook of situated cognition. Cambridge
University Press, New York
Barsalou LW, Niedenthal PM, Barbey A, Ruppert J (2003) Social
embodiment. In: Ross B (eds) The psychology of learning and
motivation. vol 43. Academic, San Diego, pp 43­92
Bjorklund DF, Pellegrini AD (2000) Child development and evolutionary
psychology. Child Dev 71:1687­1708
Cogn Process (2007) 8:79­91 89
123
Blakemore SJ, Decety J (2001) From the perception of action to the
understanding of intention. Nat Rev Neurosci 2:561­567
Bosbach S, Cole J, Prinz W, Knoblich G (2005) Inferring another's
expectation from action: the role of peripheral sensation. Nat
Neurosci 8:1295­1297
Breazeal C (2003) Emotion and sociable humanoid robots. In:
Hudlika E (ed) International journal of human computer
interaction 59:119­155
Breazeal C, Aryananda L (2002) Recognizing affective intent in robot
directed speech. Auton Robots 12:85­104
Breazeal C, Buchsbaum D, Gray J, Gatenby D, Blumberg B (2005)
Learning from and about others: towards using imitation to
bootstrap the social understanding of others by robots. Artif Life
11:1­32
Brooks RA (1991) Intelligence without representation. Artif Intell
47:139­159
Chandrasekaran B, Josephson JR (2000) Function in device representation.
Eng Comput 16:162­177
Campos J, Stenberg C (1981) Perception, appraisal, and emotion: the
onset of social referencing. In: Lamb M, Sherrod L (eds) Infant
social cognition. Erlbaum, Hillsdale, pp 273­314
Chao LL, Martin A (2000) Representation of manipulable man-made
objects in the dorsal stream. Neuroimage 12:478­484
Clark A (1997) Being there: putting brain, body, and world together
again. MIT Press, Cambridge
Damasio AR (1989) Time-locked multiregional retroactivation: a
systems-level proposal for the neural substrates of recall and
recognition. Cognition 33:25­62
Damasio AR (1994) Descartes' error. Grosset/Putnam, New York
Decety J, Gre`zes J (2006) The power of simulation: imagining one's
own and other's behavior. Brain Res 1079:4­14
Delcomyn F (1980) Neural basis of rhythmic behavior in animals.
Science 210:492­498
Donald M (1993) Precis of ``Origins of the modern mind: three stages
in the evolution of culture and cognition.'' Behav Brain Sci
16:739­791
Feinman S (1982) Social referencing in infancy. Merrill Palmer Q
28:445­470
Fernald A (1989) Intonation and communicative intent in mother's
speech to infants: is the melody the message? Child Dev
60:1497­1510
Forbus K (1993) Qualitative process theory: twelve years after. Artif
Intell 59:115­123
Gallese V, Keysers C, Rizzolatti G (2004) A unifying view of the
basis of social cognition. Trends Cogn Sci 8:396­403
Gentner D, Stevens AL (eds) (1983) Mental models. Lawrence
Erlbaum Associates, Mahwah
Glenberg AM (1997) What memory is for. Behav Brain Sci 20:1­55
Glenberg AM, Kaschak MP (2002) Grounding language in action.
Psychon Bull Rev 9:558­569
Granger R (2006) Engines of the brain: the computational instruction
set of human cognition. AI Mag 27:15­32
Gray J, Breazeal C, Berlin M, Brooks A, Lieberman J (2005) Action
parsing and goal inference using self as simulator. In: Bowerman
M, Levinson S (eds) Language acquisition and conceptual
development. In: Proceedings of fourteenth IEEE workshop on
robot and human interactive communication (Ro-Man05). Cambridge
University Press, Cambridge, pp 111­128
Hauk O, Johnsrude I, Pulvermu¨ller F (2004) Somatopic representation
of action words in human motor and premotor cortex. Neuron
41:301­307
Havas DA, Glenberg AM, Rinck M (in press) Emotion simulation
during language comprehension. Psychon Bull Rev
Hayes PJ (1985) The second naive physics manifesto. In: Hobbs JR,
Moore RC (eds) Formal theories of the commonsense world.
Ablex, Norwood
Hutchins E (1995) Cognition in the wild. MIT Press, Cambridge
Iacoboni M (in press) Understanding others: Imitation, language,
empathy. In: Hurley S, Chater N (eds) Perspectives on imitation:
from cognitive neuroscience to social science. MIT Press,
Cambridge
Izard C (1977) Human emotions. Plenum, New York
James W (1890) The principles of psychology. Dove, Mineola
Jones S (2006) Infants learn to imitate by being imitated. In: Fifth
proceedings of the international conference of development and
learning
Jones SS, Smith LB (1993) The place of perception in children's
concepts. Cogn Dev 8:113­139
Kintsch W, van Dijk TA (1978) Toward a model of text comprehension
and production. Psychol Rev 85:363­394
Martin A (2001) Functional neuroimaging of semantic memory. In:
Cabeza R, Kingstone A (eds) Handbook of functional neuroimaging
of cognition. MIT Press, Cambridge pp 153­186
Martin A (2007) The representation of object concepts in the brain.
Annu Rev Psychol 58:25­45
McClelland JL, Rumelhart DE, the PDP Research Group (1986)
Parallel distributed processing: explorations in the microstructure
of cognition. vol 2. Psychological and biological models.
MIT Press, Cambridge
Meltzoff A, Moore MK (1997) Explaining facial imitation: a
theoretical model. Early Dev Parent 6:179­192
Meltzoff AN (1996) The human infant as imitative generalist: a 20
year progress report on infant imitation with implications for
comparative psychology. In: Heyes CM, Galef BG (eds) Social
learning in animals: the roots of culture. Academic, New York
Newell A, Simon HA (1972) Human problem solving. Prentice-Hall,
Englewood Cliffs
Niedenthal PM, Barsalou LW, Winkielman P, Krauth-Gruber S, Ric F
(2005) Embodiment in attitudes, social perception, and emotion.
Pers Soc Psychol Rev 9:184­211
O'Regan JK, Noe A (2001) A sensorimotor account of vision and
visual consciousness. Behav Brain Sci 24:939­1011
Piaget J (1952) The origins of intelligence in children. International
Universities Press, New York
Plutchik R (1991) The emotions. University Press of America,
Lanham
Port R, van Gelder T (eds) (1995) Mind as motion: explorations in the
dynamics of cognition. MIT Press, Cambridge
Repp BH, Knoblich G (2004) Perceiving action identity: how pianists
recognize their own performances. Psychol Sci 15:604­609
Rips LJ (1994) The psychology of proof: deduction in human
thinking. MIT Press, Cambridge
Rizzolatti G, Fadiga L, Fogassi L, Gallese V (2002) From mirror
neurons to imitation: facts and speculations. In: Meltzoff AN,
Prinz W (eds) The imitative mind: development, evolution, and
brain bases. Cambridge studies in cognitive perceptual development.
Cambridge University Press, New York, pp 247­266
Robbins P, Aydede M (eds) (in press) Cambridge handbook of
situated cognition. Cambridge University Press, New York
Rovee-Collier C, Hayne H (1987) Reaction of infant memory:
implications for cognitive development. In: Reese H (ed) Parallel
distributed processing: explorations in the microstructure of
cognition, vol 2. MIT Press, Cambridge
Rumelhart DE, McClelland JL, The PDP Research Group (1986)
Parallel distributed processing: explorations in the microstructure
of cognition. vol 1. Foundations. MIT Press, Cambridge
Samuelson LK, Smith LB (2000) Children's attention to rigid and
deformable shape in naming and non-naming tasks. Child dev
71:1555­1570
Schank RC, Abelson RP (1977) Scripts, plans, goals, and understanding:
an inquiry into human knowledge structures. Lawrence
Erlbaum Associates, Hillsdale
90 Cogn Process (2007) 8:79­91
123
Simmons WK, Martin A, Barsalou LW (2005) Pictures of appetizing
foods activate gustatory cortices for taste and reward. Cereb
Cortex 15:1602­1608
Smith LB, Gasser M (2005) The development of embodied cognition:
Six lessons from babies. Artif Life 11:13­30
Smith ER, Semin GR (2004) Socially situated cognition: cognition in
its social context. Adv Exp Soc Psychol 36:53­117
Smith LB, Thelen E (1993) (eds) A dynamical systems approach to
development: applications. MIT Press, Cambridge
Spivey M (2006) The continuity of mind. Oxford University Press,
New York
Strack F, Martin LL, Stepper S (1988) Inhibiting and facilitating
conditions of the human smile: a nonobtrusive test of the facial
feedback hypothesis. J Pers Soc Psychol 54:768­777
Thelen E, Smith LB (1994) A dynamic systems approach to the
development of cognition and actions. MIT Press, Cambridge
Thomaz AL, Berlin M, Breazeal C (2005) An embodied computational
model of social referencing. In: Proceedings of fourteenth
IEEE workshop on robot and human interactive communication
(Ro-Man05)
Tomasello M, Kruger A, Ratner H (1993) Cultural learning. Behav
Brain Sci 16:495­552
Van Orden GC, Holden JG, Turvey MT (2003) Self-organization of
cognitive performance. J Exp Psychol Gen 132:331­350
Wilson M, Knoblich G (2005) The case for motor involvement in
perceiving conspsecifics. Psychol Bull 131:460­473
Wolpert DM, Doya K, Kawato M (2003) A unifying computational
framework for motor control and social interaction. Philos Trans
R Soc Lond Biol Sci 358:593­602
Wolpert DM, Ghahramani Z, Jordan MI (1999) An internal model for
sensorimotor integration. Science 269:1880­1882
Zwaan RA, Stanfield RA, Yaxley RH (2002) Language comprehenders
mentally represent the shapes of objects. Psychol Sci 13:168­171
Zwaan RA, Taylor LJ (2006) Seeing, acting, understanding: motor
resonance in language comprehension. J Exp Psychol Gen
135:1­11
Cogn Process (2007) 8:79­91 91
123