William James [1890] wrote, “The baby, assailed by eyes, ears, nose, skin, and entrails at once, feels it all as one great blooming, buzzing confusion.”

Future robots will face the same problem. They will include mobile robots and intelligent vehicles, self-monitoring spacecraft, whole-building environmental monitoring systems, and arrays of sensors spread over large regions of earth, sea, or space. They will be equipped with increasingly rich sensory systems. MEMS technology will provide complex distributed sensor arrays with irregular, variable structure more like biological systems than like engineered systems. Long-lived robots will need to adapt to deteriorating (or improving) sensor systems. Intelligent systems such as these must learn the properties of their sensors and effectors, and adapt to changes, through their own experiences in the environment.

We are working to develop a learning agent with a domain-independent set of mathematical tools that can start with an uninterpreted sensorimotor interface to its environment, and can learn a hierarchy of representations for its experience and its world. The agent will take an incremental “bootstrap learning” approach that combines multiple machine learning algorithms to converge on the desired state of knowledge. The success criterion is for the agent's model of its sensorimotor system and its environment to support reliable planning, allowing the agent to formulate and achieve high-level goals.

Our Spatial Semantic Hierarchy (SSH) [Kuipers, AIJ, 2000] provides the target for this learning process. We assume that the structure of the environment itself, as perceived by the robot's sensors, will define locally distinctive states that can be reached by hill-climbing control laws from anywhere within their local neighborhoods, and qualitatively uniform segments of the environment within which trajectory-following control laws can take the robot reliably from one distinctive state to the neighborhood of another. The robot travels in its environment by selecting control laws to move reliably from one distinctive state to another. These elements naturally abstract to a causal graph of states and actions, and to a topological graph of places and paths. Both graphs may be annotated with local measurements, and local metrical maps may be constructed for the neighborhoods of distinctive states and places. The multi-layer map structure in the SSH eliminates the problem of cumulative estimated position error. Hill-climbing control laws eliminate small amounts of accumulating uncertainty, making it possible for the causal and topological maps to represent position in terms of graph nodes and arcs.

Figure 1 shows the multiple representations of the Spatial Semantic Hierarchy, and their dependencies.

Our approach exploits the continuity of interaction with the physical world, rather than assuming a discretizing abstraction from the beginning. We exploit the properties of dynamical systems such as control laws to define qualitative abstractions matched to the structure of the environment.

The problem for our learning agent can then be restated as the problem of learning the sensory features and motor commands necessary for hill-climbing and trajectory-following control laws, starting with uninterpreted sensors and effectors in an unknown environment.

In previous work [Pierce & Kuipers, AIJ, 1997], we solved a version of this problem for a simulated mobile robot with a variety of sensors including a ring of sonar range-sensors. The agent learned the structure of its sensory system, including the sonar ring; then it learned the effects of actions and identified a small set of primitive actions; finally it identified local state variables and defined homing, open-loop and closed-loop control laws. These control laws were sufficient to define distinctive states, and thus to bootstrap to the Spatial Semantic Hierarchy.

Figure 2 shows the various intermediate representations developed during the bootstrap learning process.

Figure 3 (a,b,c) are stages of behavior during bootstrap learning:

     (a) random wandering
     (b) open-loop control laws
     (c) closed-loop control laws

Bootstrap learning requires choreographing several machine learning algorithms. For example, to learn the structure of the sensor ring, we applied a similarity measure based on correlation to assess the similarities among the sensors, then used multi-dimensional scaling to project the sensors into a high-dimensional space with positions consistent with their similarities. Principal component analysis (PCA) made it possible to select two dimensions that captured most of the variance, so the sensor positions were projected into those two dimensions. This step is the critical ontological change that introduces spatial position for sonar sensors. With this shift, we can estimate spatial as well as temporal changes, and motion becomes a meaningful concept. With a characterization of motion, it becomes possible to collect motion data from actions and apply PCA again to identify a small set of primitive actions.

Our goal in our current work is to extend this approach, adding new learning methods, scaling up to physical robots, with real sensors including vision, operating in real environments.