REU Site on Integrated Machine Learning Systems:


Andrew H. Fagg

Rafael Fierro Dean F. Hougen Amy McGovern Terran Lane Chris Weaver


  • Learning control policies for humanoid robot grasping and manipulation through human interaction (Fagg, McGovern, Lane). When humans perform tasks involving reaching, grasping, and manipulation, they bring to bear a tremendous amount of domain knowledge that incorporates precedence relationships, redundant solutions, and a representation of goal. We are designing ML techniques that can use observations of a human performing a complicated task to construct a general policy for behavior that accomplishes the task in novel situations (de Granville et al., 2006). This work combines relational learning techniques with supervised and reinforcement learning.

  • Multi-robot coordination through ML and hybrid control (Fierro, Hougen, Fagg). Coordinating teams of autonomous mobile robots is much more challenging than maneuvering a single robot (Fierro et al., 2006; Fierro and Song, 2005). We explore and combine results from three research areas related to multi-agent coordination in order to develop strategies for maneuvering teams of robots through obstacle-ridden environments while optimizing their performance on tasks such as escorting (McClintock and Fierro, 2008). Specifically, basic principals from hybrid system theory (Fierro et al., 2002) are used to formulate the formation reconfiguration problem as a switched system (Das et al., 2002). The formation-keeping controller is based on the idea of potential-field control (Clark and Fierro, 2007) to maintain fixed pair-wise inter-agent distances . ML can generate optimized system parameters for the team of robots once the formation switching logic has been defined. Students will implement the hybrid learning controller on teams of simulated and real mobile robots.

  • Multi-agent search of dynamic targets (Fierro, Hougen, Fagg). This project considers the task of searching a polygonal environment for the presence of a dynamic target using a coordinated team of multiple mobile robots (Clark and Fierro, 2005; Fierro et al., 2005). The goal is to develop cooperative search control strategies that enable a team of robots to detect a target when observations shared with teammates are not perfect. Several strategies for cooperative search will be evaluated and compared under the same performance metric (Ferrari et al., 2007; Perteet et al., under review).

  • Robot relays (Fierro, Lane). Mobile robots equipped with wireless networking capabilities can provide network connectivity to mobile users (Das et al., 2003). In this project, we investigate motion planning algorithms for team robot relays to maintain the connectivity of a mobile user to a static base station. The problem will be posed as a pursuit-evasion problem where the goal of the robot relays is to always detect the target. The target is assumed to move in a non-cooperative manner by trying to break connectivity. The key challenge here is to develop learning techniques that allow the team of mobile relays to maintain the target's connectivity (Branca and Fierro, 2006; Fierro et al., 2005).

  • Implementing a robotic game (Fierro, McGovern). In this project, students create a game that allows groups of school children to interact with a robot in an informative and interesting demonstration. This will help the children to see that science and engineering can be fun and will encourage them to consider computer science as they begin thinking about careers. The platform for this project is the Evolution Robotics Scorpion robot. This platform was chosen because it provides designers with a set of advanced robotic behaviors which can be combined and augmented to develop a high-level game learning based module (Perteet et al., 2007; Cruz et al., 2007).

  • Maximizing Fitness while Minimizing Trial Time (Hougen). Evolutionary computation methods such as genetic algorithms and genetic programming proceed by alternating between generating candidate solutions and evaluating their fitness. When using these methods for learning control of physical systems, evaluating fitness often consists of running the system through tests of its abilities, known as trials. The accepted wisdom seems to be that one should run several trials per candidate solution (Moriarty, Schultz & Grefenstette, 1999). Unfortunately, this is an expensive undertaking---whether learning using actual hardware or high-fidelity simulations, trial time will almost surely be the limiting factor in learning. Fortunately, we have found reason to question this accepted wisdom. In a surprising result we have found that, for one benchmark problem at least, minimizing the number of trials per fitness evaluation resulted in faster learning (that is, a faster increase in fitness per trial run) (Hougen, Carmer & Woehrer, 2003). This project will explore the relationship between trials per evaluation and fitness increase over a range of additional benchmark problems and testing conditions.

  • Memetic Learning in Real Robots (Hougen). Memetic learning is a novel machine-learning method that combines components from group-level learning methods (including various forms of evolutionary computation) and individual-level learning methods (such as reinforcement learning). Memetic learning is based on the way humans learn through imitation of successful behavior they observe. Imitation should allow systems to learn more quickly than they would if they were restricted to learning based solely on their own experiences. This is because they have the experiences of others to guide their own learning. This is particularly important in learning from data in real physical systems, such as robots, because experiences are so expensive to collect in these systems. In simulations of control problems memetic learning has proven quite effective at learning with fewer experiences than similar evolutionary computation methods (Hougen, Carmer & Woehrer, 2003; Eskridge & Hougen, 2003). However, memetic learning has yet to be implemented on real robots. This project will implement memetic learning on real robots.

  • Collection and Representation of Multi-Agents for Visual Analysis (Weaver, McGovern, Fagg, Hougen, Fierro) Collecting and representing data is often a significant practical challenge in the implementation of useful and usable visual analysis tools, regardless of the system being studies. Rendering and interaction depend on coherent, structured representation of the spatial, temporal, and relational characteristics of both objects and the multi-dimensional space in which they reside. Moreover, analysis in a particular domain often benefits from the precomputation of particular spatio-temporal-relational aggregates, such as the average centroid speed and the convex 2-D region explored by different teams of robots over the course of an experiment. The difficulty is in knowing which data dimensions to collect and which aggregates to precompute in advance, in order to support particular kinds of analysis. In this project, students will work with faculty and other students as potential users of visual analysis tools for studying movement of one or more simulated or real agents. They will elicit users wants and needs, develop an appropriate data representation with algorithms for preprocessing and aggregate calculation, and implement software for collecting data into a persistent storage format. Possible systems of study can come from many other REU projects, including robots or robot teams, machine learning agents, evolving storm features, and interactive art sensor networks.

REU Home
Back to Projects

fagg [[at]]

Last modified: Wed Jan 28 16:30:55 2009