The real world is composed of sets of objects that have multidimensional properties and relations. Whether an agent is planning the next course of action in a task or making predictions about the future state of some object, useful task-oriented concepts are often encoded in terms of the complex interactions between the multi-dimensional attributes of subsets of these objects and of the relationships that exist between them. The Spatiotemporal Multidimensional Relational Framework (SMRF) is a data mining technique that extends the successful Spatiotemporal Relational Probability Tree (SRPT) models. From a set of labelled, multi-object examples of some target concept, the SMRF learning algorithm infers both the set of objects that participate in the concept, as well as the key object and relational attributes that characterize the concept.

In contrast to other relational model approaches, SMRF trees do not require that categorical relations between objects to be defined a priori. Instead, the SMRF learning algorithm infers these categories from the continuous attributes of the objects and relations in the training data. In addition, our approach explicitly acknowledges the multi-dimensional nature of attributes, such as position, orientation and color in the creation of these categories.

SMRF has been successfully applied to a number of problem domains involving simulated real-world dynamics, such as robot soccer and blocks world tasks [1], in addition to problems involving synthetic two- and three-dimensional object configurations [2,3].

The following figures show SVM decision values using SMRF forest evaluations to predict (a) whether a balance scale will tip left or right with a new block added at a given location and (b) for whether a pass from the player in yellow will succeed to a receiver at a given location.

  1. Palmer, T. J., Bodenhamer, M. and Fagg, A. H. (2012) Learning to Predict Action Outcomes in Continuous, Relational Environments, Proceedings of the International Conference on Development and Learning (ICDL)
  2. Bodenhamer, M., Palmer, T. J., Sutherland D. and Fagg, A. H. (2012), Grounding Conceptual Knowledge with Spatio-Temporal Multi-Dimensional Relational Framework Trees Artificial Intelligence and Robotics Technical Report #1138, University of Oklahoma
  3. Bodenhamer, M., Bleckley, S., Fennelly, D., Fagg, A. H., and McGovern, A. (2009) Spatio-Temporal Multi-Dimensional Relational Framework Trees,, In the Proceedings of the International Workshop on Spatial and Spatiotemporal Data Mining, IEEE Conference on Data Mining, Miami, FL, Electronically Published