Building Graphical Interactions with Constrained Optimization
Michael Gleicher
Advanced Technology Group
Apple Computer
(Work also performed in the Department of Computer Science at
Carnegie Mellon University)
Direct manipulation has become the prefered interface for controlling graphical objects. Despite their success, the ad hoc manner with which such interfaces have been designed and implemented places restrictions on interaction techniques that can be provided to users. In this talk, I will present a new approach to graphical manipulation that provides a systematic method for providing flexible, combinable interactive controls. The Differential Approach to graphical interaction uses constrained optimization to couple user controls to graphical objects. By permitting a variety of controls to be freely combined, the Differential Approach provides a new set of abstractions that enable new types of interaction techniques and new ways of modularizing applications.
The talk begins with an example graphical application that motivates the problems to be addressed. I will describe how the problem of graphical manipulation can be viewed as a constrained optimization problem, and how this view leads to the Differential Approach. I will then outline the major components of realizing the approach: mathematical techniques for the optimization problems, implementation techniques to address practical issues, and system architecture to encapsulate the abstractions. Demonstrations of how the abstractions are used will include interfaces for manipulating 2D and 3D objects, simulating mechanisms, controlling views, and adjusting lighting. Finally, I will discuss a graphics toolkit built with the approach, and some prototype applications.