Remembrance of Things Past

Leslie Pack Kaelbling
Brown University

Abstract

Behaving effectively in partially observable environments requires remembering something about the past. Partially observable Markov decision processes (POMDPs) provide a formal model for describing and evaluating control problems that require memory. An agent in an unknown POMDP has two main strategies: learn a model of the POMDP, then solve for a good policy or learn a policy directly.

I will begin by describing an algorithm for learning POMDP models for robot navigation, which, coupled with previous work on controlling POMDPs, yields a behavior learning system. Then, I'll talk about some very recent work on direct approaches to learning policies for POMDPs without first learning a model. I'll conclude with a description of a new project on learning models for visual navigation in humans and robots.

Biographical Information

Leslie Pack Kaelbling is Associate Professor of Computer Science at Brown University. She previously held positions at the Artificial Intelligence Center of SRI International and at Teleos Research. She received an A. B. in Philosophy in 1983 and a Ph. D. in Computer Science in 1990, both from Stanford University. Prof. Kaelbling has done substantial research on programming paradigms and languages for embedded systems, on mobile robot design and implementation, and on reinforcement learning algorithms. Her current research directions include integrating learning modules into systems programmed by humans, algorithms for learning and navigating using hierarchical domain representations, and methods for learning perceptual strategies. She is an NSF Presidential Faculty Fellow, a member of the AAAI Executive Council, a member of the IJCAI Advisory Committee, and the 1997 recipient of the IJCAI Computers and Thought Award.
Eyal Amir
Last modified: Wed Mar 3 16:39:42 PST 1999