- Learning control policies for humanoid robot grasping and
manipulation through human interaction (Fagg, McGovern, Lane). When humans
perform tasks involving reaching, grasping, and manipulation,
they bring to bear a tremendous amount of domain knowledge that
incorporates precedence relationships, redundant solutions, and a
representation of goal. We are designing ML techniques that can
use observations of a human performing a complicated task to construct
a general policy for behavior that accomplishes the task in
novel situations (de Granville et al., 2006). This work combines
relational learning
techniques with supervised and reinforcement learning.
- Multi-robot coordination through ML and hybrid control
(Fierro, Hougen, Fagg).
Coordinating teams of autonomous mobile robots is
much more challenging than maneuvering a single robot (Fierro et al., 2006; Fierro and Song, 2005). We
explore and combine results from three research areas
related to multi-agent coordination in order to develop strategies for
maneuvering teams of robots through obstacle-ridden environments while
optimizing their performance on tasks such as escorting (McClintock and Fierro, 2008). Specifically,
basic principals from hybrid system theory (Fierro et al., 2002)
are used to formulate the
formation reconfiguration problem as a switched system (Das et
al., 2002). The
formation-keeping controller is based on the idea of potential-field
control (Clark and Fierro, 2007) to maintain fixed pair-wise
inter-agent distances . ML can
generate optimized system parameters for the team of
robots once the formation switching logic has been defined. Students
will implement the hybrid learning controller on teams of simulated and
real mobile robots.
- Multi-agent search of dynamic targets (Fierro, Hougen, Fagg).
This project considers the task of searching a polygonal environment
for the presence of a dynamic target using a coordinated team of
multiple mobile robots (Clark and
Fierro, 2005; Fierro et al., 2005). The goal is to develop
cooperative search control strategies that enable a team of
robots to detect a target when observations shared with
teammates are not perfect. Several strategies for cooperative
search will be evaluated and compared under the same
performance metric (Ferrari et al., 2007; Perteet et al.,
under review).
- Robot relays (Fierro, Lane). Mobile robots equipped with
wireless networking capabilities can provide network connectivity to
mobile users (Das et al., 2003). In this project, we
investigate motion planning algorithms for team robot relays to
maintain the connectivity of a mobile user to a static base station.
The problem will be posed as a pursuit-evasion problem where the
goal of the robot relays is to always detect the target. The
target is assumed to move in a non-cooperative manner by trying to
break connectivity. The key challenge here is to develop learning
techniques that allow the team of mobile relays to maintain the
target's connectivity (Branca and Fierro, 2006; Fierro et al., 2005).
- Implementing a robotic game (Fierro, McGovern). In this
project, students create a game that allows groups of school
children to interact with a robot in an informative and interesting
demonstration. This will help the children to see that science and
engineering can be fun and will encourage them to consider computer
science as they begin thinking about careers. The platform for this
project is the Evolution Robotics Scorpion robot. This platform was
chosen because it provides designers with a set of advanced robotic
behaviors which can be combined and augmented to develop a
high-level game learning based module (Perteet et al., 2007; Cruz et al., 2007).
-
Maximizing Fitness while Minimizing Trial Time (Hougen).
Evolutionary computation methods such as genetic algorithms and
genetic programming proceed by alternating between generating
candidate solutions and evaluating their fitness. When using these
methods for learning control of physical systems, evaluating fitness
often consists of running the system through tests of its abilities,
known as trials. The accepted wisdom seems to be that one
should run several trials per candidate solution (Moriarty, Schultz &
Grefenstette, 1999).
Unfortunately, this is an expensive undertaking---whether learning
using actual hardware or high-fidelity simulations, trial time will
almost surely be the limiting factor in learning. Fortunately, we
have found reason to question this accepted wisdom. In a surprising
result we have found that, for one benchmark problem at least,
minimizing the number of trials per fitness evaluation resulted in
faster learning (that is, a faster increase in fitness per trial run)
(Hougen, Carmer & Woehrer, 2003). This project will explore the relationship between
trials per evaluation and fitness increase over a range of additional
benchmark problems and testing conditions.
-
Memetic Learning in Real Robots (Hougen). Memetic
learning is a novel
machine-learning method that combines components from group-level
learning methods (including various forms of evolutionary computation)
and individual-level learning methods (such as reinforcement learning).
Memetic learning is based on the way humans learn through imitation of
successful behavior they observe. Imitation should allow systems to
learn more quickly than they would if they were restricted to learning
based solely on their own experiences. This is because they have the
experiences of others to guide their own learning. This
is particularly important in learning from data in real physical
systems, such as robots, because experiences are so expensive to
collect in these systems. In simulations of control problems memetic
learning has proven quite effective at learning with fewer experiences
than similar evolutionary computation methods (Hougen, Carmer &
Woehrer, 2003; Eskridge & Hougen, 2003). However, memetic
learning has yet
to be implemented on real robots. This project will implement memetic
learning on real robots.
-
Collection and Representation of Multi-Agents for Visual Analysis
(Weaver, McGovern, Fagg, Hougen, Fierro) Collecting and representing data is often a significant
practical challenge in the implementation of useful and usable visual
analysis tools, regardless of the system being studies. Rendering and
interaction depend on coherent, structured representation of the
spatial, temporal, and relational characteristics of both objects and
the multi-dimensional space in which they reside. Moreover, analysis
in a particular domain often benefits from the precomputation of
particular spatio-temporal-relational aggregates, such as the average
centroid speed and the convex 2-D region explored by different teams
of robots over the course of an experiment. The difficulty is in
knowing which data dimensions to collect and which aggregates to
precompute in advance, in order to support particular kinds of
analysis. In this project, students will work with faculty and other
students as potential users of visual analysis tools for studying
movement of one or more simulated or real agents. They will elicit
users wants and needs, develop an appropriate data representation with
algorithms for preprocessing and aggregate calculation, and implement
software for collecting data into a persistent storage
format. Possible systems of study can come from many other REU
projects, including robots or robot teams, machine learning agents,
evolving storm features, and interactive art sensor networks.