Second
World Congress on Biomimetics,
Nano-Bio & Artificial Muscles
Symposium:
Towards
Artificial Rodents
Special
session chaired by Jean-Arcady Meyer & Agnès Guillot
Organizer
& Chair: Dr.
Jean-Arcady Meyer ( jean-arcady.meyer@lip6.fr)
AnimatLab/LIP6, 8 rue du capitaine Scott, 75015 Paris,
France
Co-Chair: Agnès Guillot
AnimatLab/LIP6, 8 rue du capitaine Scott, 75015 Paris, France
1.
Biological fundations
Neurobiologically
based proposals for structuring navigation strategies for autonomous agents.
Sidney
I Wiener and Angelo Arleo
Laboratoire de Physiologie de la Perception et de
l’Action, CNRS - Collège de France, Paris, France
Neurobiological
studies have revealed brain systems that are likely substrates for spatial
orientation and navigation (that is, planning and executing displacements to a
point not perceptible from the point of origin). Neuroanatomical studies have
demonstrated the architecture and microcircuitry of multiple parallel and
hierarchical pathways. Experimental lesions of the respective specific modules
have shown deficits in particular types of cognitive processing (such as path
integration, or mapping functions) indicating that these structures carry out
complementary functions. For example the hippocampal system is implicated in
employing configurations of environmental cues for self-localization - and is
capable of reconstructing spatial representations in the presence of only a
partial subset of the cues learned previously. In contrast, lesions to the
caudate nucleus impair approaches to visible cues, but spare the capacity to
employ configurational cues. These respective brain structures thus are vital
for enabling particular characteristic strategies employing certain types of
cues and signal processing.
The
signal processing for these processes is revealed in neurophysiological
recordings of neuronal activity in navigating rats. These show the elemental
information, such as coding in 'place cells', selective for the position of the
rat in its environment and 'head direction cells' selective for the cardinal
orientation of the head. In addition to on-line processing, these systems are
also involved in memorization and recall of this information.
On
the basis of these experimental data, an integrative framework is put forward
where these processes are organized hierarchically according to complexity and
during extended experience, processing becomes more rapid and automatic and is
successively transferred to structures specialized for this. Many tasks can thus
solicit alternation between different systems according to the types of cues and
processing at each stage of a multistep route. This provides an efficient system
by reducing computational demands to simpler types of processing whenever
possible.
2.
Sensors
Towards
a Robot Whisker System Modelled on the Rat Mystacial Vibrissae
Ben
Mitchinson* , Tony Prescott*, Kevin Gurney*, and Peter Redgrave*, Chris Melhuish**,
Tony Pipe**, Martin Pearson**, Ian Gilhespy**
* Adaptive
Behaviour Research Group, University of Sheffield, UK
** Intelligent
Autonomous Systems Laboratory, University of the West of England, UK
We
will outline our current programme of research aimed at developing a
multi-whisker sensory system modelled on the mystacial vibrissae of the
laboratory rat. Neurobehavioural studies suggest that the rat whisker system is
used in locomotion, maintenance of equilibrium, object recognition and surface
discrimination. The behavioural cues discernible by the whiskers are likely to
include location, proximity, relative velocity, size, and texture of nearby
surfaces and objects. One striking characteristic of this system is the
"whisking" movement employed during exploration behaviour, in which
the vibrissae are swept back and forth across objects and surfaces in a
synchronised wave at a rate of between 5 and 11 hertz. This whisking movement
has been likened to a human running their fingertips over a surface, and it
appears that the rat is able to exploit both the spatial and temporal
characteristics of the pattern of deflections derived from whisking to glean
information about characteristics such as surface texture.
We
will report the initial results of a three year project to develop an active,
multi-whisker system capable of object detection and surface-discrimination.
We will also report on some pilot neurobehavioural studies of rat whisking in a
simple navigation task.
3.
Control architectures
The
neural basis of spatial orientation in rats: Electrophysiology, computational
modeling, and robotics
A. Arleo *, T. Degris **,
C. Boucheny *, S. I. Wiener *
* Laboratoire
de Physiologie de la Perception et de l’Action, CNRS - Collège de France,
Paris, France
** AnimatLab/Laboratoire d’Informatique de Paris 6,
Université Pierre et Marie Curie, Paris, France
Place
and direction representations are crucial for most models elaborating flexible
spatial behavior. This has led to the hypothesis that hippocampal place (HP)
cells and head direction (HD) cells may constitute vital neural bases for
cognitive processes underlying navigation of rats. Indeed, electrophysiological
single-cell recordings suggest that HP neurons can encode the position of the
rat in allocentric (world centered) coordinates, while HD neurons provide an
allocentric representation of the orientation of the head of the rat projected
onto the azimuthal plane. However, the paucity of experimental evidence at the
intermediate levels (e.g., neural populations, macro-circuits, and system level)
makes it difficult to explicitly relate the single-cell level (e.g.,
electrophysiological properties of HP and HD cells) to the behavioral level and
hence understand how HP and HD cells enable spatial navigation.
Theoretical
modeling and neuromimetic robotics may offer useful tools to systematically
explore hypotheses concerning these relations. Since neuromimetic artifacts are
simpler and more `experimentally transparent' than biological systems, they may
permit new hypothesis to be examined at different hierarchical levels and lead
to a better understanding of the mechanisms supporting cognitive behavior of
animals. Conversely, a biologically-founded approach to model spatial cognition
offers the attractive prospect of developing autonomous systems that emulate the
navigation capabilities of animals and may lead to beneficial application in
improving design of more powerful and adaptive robots.
We
propose an integrative neuroscience and engineering approach involving
electrophysiological experiments, computational modelling, and neuromimetic
robotics for studying spatial orientation. Here, we present a prototypic case
and focus on HD cells, first presenting a series of electrophysiological
experiments addressing the following issue: How do actual thalamic HD cells
update their directional representation on the basis of visual landmarks in the
environment? Secondly, a computational model of HD cells based on a continuous
attractor network of leaky integrate-and-fire spiking neurons will be presented.
The weight distribution determining the intrinsic dynamics of the attractor
(i.e., recurrent inhibitory and excitatory signals) has been established by
means of a genetic algorithm. The model captures several electrophysiological
properties of single HD cells and, most importantly, permits to investigate the
state transition dynamics (e.g., during visual reorientation) at the level of a
large population of formal HD cells. The experimental validation of the model
has been done via a robotic implementation, which has produced an artificial HD
system endowing a robot with allocentric spatial orientation capabilities.
Towards
a robot control architecture modelled on the 'integrative core' of the rat brain
Mark
Humphries, Tony Prescott, Kevin Gurney, and Peter Redgrave
Adaptive
Behaviour Research Group, University of Sheffield, UK
We
will outline our current programme of research aimed at developing a biomimetic
'integrative core' for robust and effective robot control modelled on key
structures in the rat brain. Our general focus is on two specific brain systems
that are hypothesized to play an important role in action selection-the basal
ganglia (BG) and the reticular formation (RF). This paper will specifically
report on the development of a new model of the reticular formation inspired, in
part, by the classic RF model of Kilmer, McCulloch, and Blum, 1969.
We
will report preliminary robotic experiments using this model that aim to show
that this structure can support a form of decentralized action selection.
Finally, we will discuss the possibility that the RF forms a key element of a
layered control architecture within the rat brain, functioning to provide the
neural substrate for action selection during basal ganglia development or
following damage to the basal ganglia.
Using
Rodent Hippocampal Models for Robotic Simultaneous Localisation and Mapping
Gordon
Wyeth and Michael Milford
School of
Information Technology and Electrical Engineering, University of Queensland 4072
Australia
To navigate successfully in a novel environment, a robot needs to be able to perform
Simultaneous Localization And Mapping (SLAM). In this paper we
present a biologically inspired system called RatSLAM that uses visual
input from a colour camera to carry out SLAM on an autonomous robot.
The
heart of the RatSLAM system is based on current integrative models of the
rodent hippocampal complex. The RatSLAM system is shown to effectively
perform parameter self-calibration and SLAM in one-dimension. Tests in two
dimensional environments reveal the inability of the RatSLAM system to maintain
multiple pose hypotheses in the face of ambiguous visual input.
These
results support recent rodent experimentation that suggests that current
models of the hippocampal complex are not the complete solution to SLAM
problem.
Navigation
skills of an artificial mouse
Verena
Hafner
Artificial
Intelligence Laboratory, Department of Information Technology, University of
Zurich, Switzerland
Mice
and rats are probably the best studied vertebrates with respect to their
navigation skills. The discovery of place cells in the rat's hippocampus in the
early 1970s inspired the modelling of such cells - whose firing rate correlates
with the spatial position of the animal - and the study of their importance for
navigation behaviour. This work shows how such a model can be applied and
optimised on a mobile robot which learns to navigate within its environment
through exploration. The place cell model is enhanced with local navigation
skills. The major sensory information is gained from a camera providing the
robot with omnidirectional vision, similar to the large visual field of rats and
mice. We are currently investigating how tactile information from an active
whisker array on the robot can be used as an additional sensory modality for the
place cell model.
Complex
Cell Representations for Robot Localisation
Gordon
Wyeth and David Prasser
School of
Information Technology and Electrical Engineering, University of Queensland 4072
Australia
Various
researchers have explored the hierarchical nature of the mammalian visual cortex
for the purposes of building visual classifiers. In this paper we use a hierarchical
model of the visual cortex to learn the localisation of an autonomous
robot based on visual input. A key factor in the development of a
biologically plausible vision system for localisation is the representations
used in the complex cells of the visual cortex. The invariance properties
of the complex cell representations are critical to successful robot
localisation.
The
effectiveness of two different complex cell models is evaluated, and is
shown to be suitable as the basis for localisation. Tests with a simple
neural network show that reasonable localisation can be achieved with
single-shot learning.
4.
Learning
Actor-Critic
models of reinforcement learning in the basal ganglia: from natural to
artificial rats
M.
Khamassi *,**, B. Girard*,**, A. Guillot*, and A. Berthoz**
* AnimatLab/Laboratoire d’Informatique de Paris 6,
Université Pierre et Marie Curie, Paris, France
** Laboratoire de Physiologie de la Perception et de
l’Action, CNRS - Collège de France, Paris, France
Actor-Critic architectures have been proposed as models
of dopamine-like reinforcement learning mechanisms in the rat’s basal ganglia.
In such models, an Actor network learns to select
actions so as to maximize the weighted sum of future rewards, computed on line
by another network, a Critic. The Critic predicts this sum by comparing its
estimation of the reward with the actual one by means of a Temporal Difference
(TD) learning rule, in which the error between two successive predictions is
used to update the synaptic weights. A strong resemblance between the activation
patterns of dopaminergic neurons that project to the basal ganglia, and the TD
error prediction signal centers these models on the role of dopamine in
the Critic. A
recent review of numerous computational models, built on this principle since
1995, concluded on several issues raised by the inconsistency of the detailed
implementation of Actor and Critic models with known basal ganglia anatomy and
physiology. Here we consider some of the main issues, updated with anatomical
and neurophysiological knowledge.
We also illustrate the consequences of different
hypotheses concerning the Critic by comparing three existing models, connected
with the same Actor, in a simulated robot as it performs the same reward-seeking
task. During this task, the robot freely moves in
a plus maze where a reward is distributed at different sites and referent to
site-specific local stimuli. The robot has to select sequences of locomotor acts that enable it to
reach the goal from any place in the maze. The differences between the simulated
Critics lie in their capacity to build short or long sequences of
stimulus-response associations to attain the reward, and on their capacity to
construct them autonomously – i.e. with or without preprogrammed associations.
The structure of these learned behavioral sequences is compared with the
neurophysiological activation data of brain striatal cells recorded in real rats
performing the same task.
A general discussion evokes the growing number of works
that question the two main assumptions on which all Actor-Critic models are
designed, i.e. the role of dopamine as a reinforcement signal, and the TD rule
as a learning process for the basal ganglia.
The perspective of this work is to
implement learning mechanisms processing in the basal ganglia in the control
architecture of the artificial rat Psikharpax, already inspired by the same
neural structures.
Artificial
Rodents that Can Predict the Future
Nestor
A. Schmajuk
Department of
Psychological and Brain Sciences, Duke University, U.S.A.
We
describe a neural network model that characterizes a remarkable large number of
classical conditioning paradigms. The network
(a)
describes behavior in real time,
(b)
contains simple and configural stimulus representations, and
(c)
includes attentional control of both storage and retrieval.
The
behavioral properties of the model are supported by rigorous computer
simulations, which are consistent with a large body of experimental data. The
network can be applied to the design of predicting mechanisms that can be used
in an artificial rodent.
5.
Evolution
The
Cyber Rodent Project
Eiji Uchibe, Genci Capi,
Stefan Elfwing, Anders Eriksson, Hirofumi Suzuyama, Kenji Doya
ATR Computational Neuroscience Laboratories; CREST,
JST, Japan
Successful
acquisition of behaviors of autonomous robots by learning and evolution often
requires appropriate selection of reward or fitness functions and
meta-parameters of learning and evolution algorithms. Automatic regulation of
these factors is not only necessary for building truly automatic robots, but
also helpful for understanding the mechanisms of motivation and emotion in
animals and humans.
In
order to elucidate the functions of the reward system, we built a robotic
platform, Cyber Rodents, which have the same vital constraints as biological
agents, namely, self-preservation and self-reproduction.
Cyber
Rodents are two-wheel driven rodent-like robots that can recharge itself from
battery packs scattered in the field and can copy their ' genes' (programs or
parameters) through IR communication ports.
We
report our preliminary results of our simulations and experiments, such as
evolution of metaparameters and hierarchical architectures.
6.
Integrative projects
The
Psikharpax project: towards building an artificial rat.
Jean-Arcady Meyer *, Agnès
Guillot *, Patrick Pirim **
* AnimatLab/Laboratoire d’Informatique de Paris 6,
Université Pierre et Marie Curie, Paris, France
** BEV, Boulogne
Billancourt, France
The
Psikharpax project aims at endowing a robot with a sensory-motor equipment and a
neural control architecture that will afford some of the adaptive capacities
that are exhibited by real rats.
Concerning
the sensory-motor equipment, the article first introduces the robot's visual
system and focuses on the implementation of the vestibulo-ocular reflex, on the
management of saccades, and on the mechanisms of landmark detection. Then it
describes the robot's tactile (whiskers) and auditory (electronic cochlea)
systems.
Concerning
the control architecture, the paper specifies how localization and navigation
capacities are implemented thanks to a biomimetic model of hippocampal and
para-hippocampal structures in the rat brain. Likewise, it also reports how
action selection capacities are afforded by another biomimetic model inspired by
the anatomy and physiology of the basal ganglia. Finally, a model calling upon
the cortical columns in the frontal cortex makes planning possible.
Preliminary
results on the implementation of learning mechanisms in these structures are
also presented.
Finally,
the paper describes how Psikharpax will integrate these adaptive mechanisms and
structures in a functional and coherent system that will be able to explore an
unknown environment, to build a cognitive map of it, and to use this map to plan
trajectories to places where the robot's will fulfil its various motivations,
like "eating", "drinking" and avoiding
"predators".
The
integrative biomimetic architecture of Psikharpax : action selection and
navigation in realistic 3D environments
L.
Lachèze*, B. Girard**, A. Guillot*, and J.-A. Meyer*
* AnimatLab/Laboratoire
d’Informatique de Paris 6, Université Pierre et Marie Curie, Paris, France
** Laboratoire
de Physiologie de la Perception et de l’Action, CNRS - Collège de France,
Paris, France
A
biomimetic control architecture inspired by the rat basal ganglia integrates
action selection and navigation through the simulation of dorsal and ventral
circuits. The input module of the ventral part corresponds to the nucleus
accumbens (Nacc), which selects locomotion actions calling upon various
navigation strategies and motivations. The input module of the dorsal part
corresponds to the dorsal striatum, which is in charge of the selection of
non-spatial tasks and of the coordination with Nacc's decisions. This
architecture was previously implemented on a simulated robot that had to survive
in an environment where places for `energy ingestion' and `energy digestion'
could be found. The corresponding results demonstrated that the robot was able
to ensure its survival with the selection of efficient sequences of actions,
using its abilities of map-building and path-planning.
In
this work, the same architecture is tested with another robot and another
environment that are more complex than the previous ones. In particular, the
physical characteristics of the robot are more realistic, as they closely
emulate those of an existing robot. Moreover, the robot has to deal with a
greater set of motivations and actions, and is equipped with more complex visual
and orientation systems. Several experiments are performed in specific
environments that serve to assess specific adaptive abilities. Other experiments
involve a large office-like environment, cluttered with numerous obstacles, in
which the full robot's adaptive capacities are put at work. Results demonstrate
the capacity of the architecture to generate behavioral sequences that ensure
survival. They also support the next step of the research, i.e. the embodiment
of such a biomimetic model into a real robot and a real office.