current page



Lecture on Oct 7, 2009. (Slides)


  • Required

    • Postmortem of an example, Bertin. (pdf)

    • Visual information seeking: Tight coupling of dynamic query filters with starfield displays, Ahlberg & Shneiderman. (html)

    • Visual exploration of time-series data, Hochheiser & Shneiderman. (pdf)

  • Optional

    • Generalized Selection via Interactive Query Relaxation. Heer, Agrawala, Willett. CHI 2008. (pdf)

    • The cognitive coprocessor architecture for interactive user interfaces. George Robertson, Stuart K. Card, and Jock D. Mackinlay, Proc. UIST '89, pp 10-18. (pdf)

    • Exploration of the Brain's White Matter Pathways with Dynamic Queries. Akers, Sherbondy, Mackenzie, Dougherty, Wandell. Visualization 2004. (html)

  • Demos

  • Videos


nmarrocc wrote:

Now that were talking about interfaces for visualizations we have to study the trade off between expressiveness vs usability. If you think of the language as an interface you have sql as the most expressive kind of interface, but least usable. On the other end of the spectrum you have these dynamic query widgets, usable but limited expressiveness. I would love to see some kind of graphic depicting the collation between expressiveness and ease of use plotted against how specific a task is. It seems to me that the more specialized your graph/problem is the easier it is to find a perfect amount of expressiveness and usability for the task at hand.

vagrant wrote:

I have to say that, after going through the proposal for the TimeSearcher application, I find the tool a little intimidating to use. I referred back to the study on visual information seeking and realized that at least on the surface, the TimeSearcher does in fact practice many of the principles advocated--perhaps my skepticism stems from my lack of experience with data visualization software in general.

At the same time, that is not to say that I don't find the features described exciting--nearly every science fiction vision of the future features some demonstration of dazzling data manipulation. As the readings have stated, data visualization is about application, not art, but first impressions matter; as Bertin's writing suggests, how a graphics-based tool impresses a first-time user can be the difference between its adoption and its abandonment. So I guess my real want is not to do with the interactive capabilities in a tool such as TimeSearcher, but its interface. That said, perhaps if I used the TimeSearcher application, I'd find it intuitive after little effort.

The demos associated with the readings and demoed in class are great counterexamples to the issues I've described. They are clean, simple in concept, and immediate in engagement. Perhaps there is an argument to be made for how broad or specific a utility a graphical visualization tool features. With clear-cut uses and limited scope for data input, it becomes easier to tune the interface of a representation toward those uses. With a tool meant to accommodate less specific data, such as TimeSearcher, there seems to exist less options in the way of dressing up the interface.

cabryant wrote:

Bertin's “postmortem of an example” is a compelling analysis of the process of decision making and the role of visualization therein. That being said, it makes strong claims with regard to which steps are achievable via automation, claims which merit revisitation in a data-rich, digital age.

Bertin's initial posit is of the first stage: defining a problem by posing fundamental questions cannot be automated. This claim becomes ever more dubious with advances in data mining and information extraction. One can imagine programmed entities combing through newly generated exabytes of data and applying many of the predefined, basic questions that characterize the initial stages of investigation within a given domain. It is also conceivable that, given the plethora of data available in this networked age, that programs could perform sufficiently sophisticated data abstraction to identify reasonable questions that have been explicitly or even implicitly published.

Of the following three stages, Bertin acknowledges (or does not disallow the potential for) the effectiveness of automation. In this class alone we have seen automated examples of defining the data table and adopting a processing language. The third, processing the data, seems to be the more complex, although Bertin claims that this stage is visual rather than semantic, and may be automated. If this is indeed the case, then it is conceivable that the generation of new questions and/or algorithms for describing data relationships may be automatically derived from the results of this step (as was done in a non-automated fashion by the PRIM-9 team after discovering visual patterns in multi-dimensional space).

Bertin argues that the final stages of interpretation, identification of extrinsic relationships, decision making, and/or communication cannot be automated. Of these, interpretation appears most difficult to automate. The identification of extrinsic relationships would seem to fall into the realm of what is achievable through data mining, and suggestions for decision making could conceivably be derived given concrete questions defined in stage one. Communication may be facilitated by supporting data views at different levels of abstraction, something we have seen in many of the examples of interactive visualization software.

Finally, in the spirit of Bertin, I think much may be gained by performing a postmortem of FilmFinder. One crucial step, and one which is likely directly responsible for the effectiveness of the final product, is the diligent consideration of the semantics of the target domain (accomplished by conducting informal interviews with video store clerks and film aficionados). In terms of what could have been improved, user-selection of the axes may be a warranted feature (e.g. number of academy awards / nominations, film budget, etc. versus popularity rating). Regardless, I find the product to be fairly impressive given the expectations of the age, as exhibited by the statement that existing tools “employ novel approaches such as hypertext links[!]”

jieun5 wrote:

I share some of the same attitude with @vagrant on the program TimeSearcher. I actually downloaded and tried out the software as a potential tool for homework 2, and ended up finding it inappropriate for my (specific) desired purpose.

I wanted to visualize pitch-rhythm trajectory of a piece of music with a rather strict form, such as a Bach fugue, in which short musical motifs repeat in different voices in transposed-imitations. Since music is a time series data, I thought that this would potentially be very interesting. However, I soon found barriers that were too difficult to overcome; some of this had to specifically with the design of TimeSearcher, and others were problems specific to music.

As for the former kind of barrier, the fact that I had to re-format my data specifically for the software was extremely inconvenient; TimeSearcher cannot understand input data unless it is formatted specific to its own file format guidelines. Another barrier was that it is difficult to represent events happening at "random" non-uniform time periods. TimeSeracher2 tries to overcome this problem and allow for a continuous input values for time by providing flexible time sampling intervals, but there seems to be limitations to this-- especially for things like music, where even tens of milliseconds difference in timing can have a huge impact on expression.

As for the latter type of barrier, music is difficult to visualize using TimeSearcher because it is difficult to represent silences. For instance, if we wanted to outline the pitch-rhythm contour for each of 4 voice-parts of a piece of music, where the vertical scale represents pitch and horizontal scale represents time, pitched notes should be denoted by line segments while rests are best represented by blank space. But querying things such as slopes and angles on these four series of broken line segments could easily result in an error because of discontinuities. Also, it makes little sense to interpolate between two "data points" across time when dealing with musical contours, as one can with stock-prices. There are many other difficulties, such as inability to represent dynamics (though this could be achieved by varying line thickness/ width), articulation styles (stacatto vs. legato vs. slurred), or multiple-notes per voice-parts (i.e. double stops on a violin, or playing chords on a piano). In this way, I found many assumptions that the software makes that are simply inappropriate for certain time-series data (such as music), though at a first glance this was not obvious.

malee wrote:

~ @nmarrocc: I agree that there is a tension between expressiveness and usability, and ideally there would be one interaction that is both. However, if that is too hard, it's also possible to support multiple interactions of varying expressiveness/usability. : ) For one viz system I worked on, we allowed power users to specify what they wanted with SQL while more casual users employed typical mouse interactions to explore the data.

On a related note, I am a big fan of dynamic query filters and direct manipulation, which seem to be relatively common. I think that visualization systems that support dynamic query filters would be much improved if they also supported the ability to take snapshots of the system as opposed to confining the user to always work in the same XbyY rectangle on a screen. Much like how having different worksheets in Tableau facilitated my exploration in creating a visualization, being able to take snapshots of a viz system would help users explore the data, compare/contrast, combine etc.

fxchen wrote:

This talk gave me a lot to think about with regards to many user apps. For example, I can't help but think Google Maps can make a lot of improvements with better query relaxation. It'd be incredibly powerful to be able to fine-tune my query within a map interface, including the type of filtering system available in Tableu.

Over the summer, I worked on a vis system for HPC clusters that fused data from cpus, physical data, and other metrics together to present a entirely new way to explore data for system administrators / engineers. I plan on taking concepts from some of these readings to my group.

bowenli wrote:

Similar to @cabryant, I am kind of reluctant to believe Bertin's claims about automation. He claims that the simplification step of the visualization can be entrusted to the machine. He makes this step sound trivial, even though I feel like it is prettymuch where all the work and insight went into the visualization. Transforming the visualization from unusable to usable seems to be the magic step that requires iteration and human feedback.

Filmfinder: I think the tightly coupled sliders they talk about is impressive for a system. I'm a bit worried that the automation may be a bit confusing for new users who don't expect it. For example, if the range of actors was suddenly limited it may not be clear why or how to get back. This seems to happen a lot with people who are new to computers and then they form false connections between actions.

@malee: I think dynamic query filters and multitouch would be awesome.

zdevito wrote:

I think it is important to make a difference between interactive visualizations such as zipdecode, NameVoyager, or the LA Homicide Plot, and tools that create interactive visualizations such as TimeSearcher or Tableau. In the former the interactive controls are tailored to the particular data being explained, while in the latter the controls are some reasonable default but they don't necessarily make sense of the actual. Hence interactive visualizations feel intuitive but are inherently less expressible: they can only show what their designer wanted to show, while visualization tools are more expressive at the cost of initial usability.

Furthermore, for interactive visualization, one must anticipate that someone using your visualization will want to look at some relationship that you have not provided directly. While the majority of people will use only a small piece of the functionality, any particular individual may want to look at one particular detail in depth. This makes it important to provided a more expressive interface to the data for any particular visualization. This could mean providing way more semantic encodings than what any one person would use at once or in the extreme this could mean providing the raw data as a downloadable option along side with the visualization. In either case, interactivity allows people to ask more detailed questions of the data, making the task of designing an accurate visualization more difficult.

dmac wrote:

The LA County Homicide data shown in class was very impressive, and I think my favorite part about it was how smoothly it transitioned between views. You could actually track a data point as it moved from one context to a completely different one. This seems to be where the interplay of interaction and animation is strongest. A visualization can be animated but not interactive, or interactive but not animated. When it has both, it can either be beneficial or detrimental. I can easily imagine a situation where too much animation in an interactive visualization would detract from it. However, when done correctly, I think animating an interactive visualization can greatly improve the user's understanding. One risk of an interactive visualization is confusing users by having too many views or options, but if you animate in such a way as to lower that barrier it can greatly enhance the viewing of the visualization.

vad wrote:

The infovis wiki lists all the interaction techniques that were covered in class and has cool demos for some of them: There are a great number of applications for these techniques; I am surprised that I wasn’t more familiar with them.

wchoi25 wrote:

I was most intrigued by the "output-is-input" idea, of using tight coupling to allow progressive refinement and make visualization state obvious. I think that as more and more complex visualization systems get developed that let people explore huge data sets in various ways, there emerges an important meta-question: how can one visualize the state of the visualization? Ideally, the visualization itself does the job. But the position of the sliders next to it, dynamic scale reference as zoom level changes, radio-buttons, and check-boxes are all extremely important communicator of state. They are useful not just for their inputting functions but for their communicative output. It is exciting to think about creative new ways in which the state of an interactive visualization system might be exposed, shared, and restored across users & machines.

alai24 wrote:

In Bertin's example, it almost seems as if by extrinsic information, he means intuition that involves some 'big picture' examination. It is the visualization's job to facilitate this process.

Dynamic queries are a fantastic way to discover new relationships between elements in a dataset and I agree with wchoi25 regarding the importance of the actual inputs and how the state is presented. I haven't thought about that much, but it is actually a pretty integral part of an interactive visualization.

jqle09 wrote:

@malee - I think the idea of taking snapshots of a vis system would be especially great for multiple user systems. Designs could be explored and iterated on in multiple directions and would allow for much greater interactivity with the visualization and with other users. A lot more data can certainly be explored as well.

I also agree with wchoi25 w.r.t. interactive visualization systems that can be exposed, shared, and restored across user machines. Individually being able to interact with data gets supplemented with interacting with other people interacting with data. I don't remember if Jeff talked about this but his project (read about it in the book Beautiful Data), which allowed for visual states to be saved and annotated so that other users could find and also comment, is a great example of interactive interactive visual systems.

I didn't notice this from the demo shown in class but the name voyager lets drill down into a name to find more information like derivation, and even lets you vote on things like 'Does this name sound': smart, sexy and so on. I certainly added some nice comments for my name. I think being able to provide different levels of interaction and community around vis systems is important, and offers us the ability to ask more questions.

by the way I was also mad impressed by the demos shown in class. interactive visualizations are too cool.

aallison wrote:

I think the idea of snapshots is critical to an interactive visualization system, as others have argued above. The idea of snapshots made me think of how many people pass around locations of ridiculous things in Google Streets. While GS is simply a collection of photos, it is certainly analogous to a large data repository that requires exploration to find interesting information.

I think of an interactive viz system as a meta-visualization, hiding a wealth of snapshot visualizations waiting to be discovered by the users of the system. I certainly agree, especially after using Tableau, that direct manipulation is very important for an interface to be easily approachable and explorable.

The paper on query relaxation made me think about how I do selections in a complex medium such as a photoshop image. In am image consisting of many many pixels, PS provides numerous tools to help the user select exactly what they want to select, with options for greater refinement. Could tools such as those used in PS apply in an analogous way to data visualizations? Could I "feather" my selection-query in a set of data? Or "smart-select" a region by smearing my cursor in the general area that I am interested in?

rnarayan wrote:

Systems such as FilmFinder (and its newer and fancier incarnations on, iTunes, amazon and netflix) make recommendations by comparing user sampled items to a knowledgebase that connects data through a similarity measure. Key elements missing in such recommendation listings however are the reasons why the recommendation was made. A more recent approach uses a visualization where similar items and clusters of items are related to each other in a spatial map where trends can be more readily identified. From this, a personalized heat map could also be derived.

Plz see (their prototype implementation is a bit weak in responsiveness since it seems to utilize a server feed of images instead of client-side caching or AJAX)

They also describe a generalized method for reduction of high dimensionality datasets into a 2D spatial representation, while preserving the structural, clustering and neighborhood info, so that it can be employed to interact with relational datasets in different domains. From the interaction standpoint, it seems to be a significant improvement over dynamic query techniques using sliders and other UI widgets as in the many examples in the two Schiederman papers.

Yet, it seems we still have a long way to go in being able to handle the typical dataset that is large and ever-growing. A drawback that becomes quickly apparent when using a spatial metaphor is display techniques for rollup of data. Clustering nodes by frequencies, etc. as in Spotfire (the commercial product from Ahlberg and Schneiderman's efforts, now acquired by Tibco) seems to be the predominant technique, yet it masks the cognition afforded by visual examination.

Another aspect that behooves careful re-consideration is lumping and dumping all pattern recognition to the human visual processor. For instance, the TimeSearcher project interaction of selecting a time capsule for tracking evolution of two other dimensions seems constrained by a geometric window of capture. Firstly, a more generalized query relaxation as presented in class can help in the selection process. Secondly, the onus of pattern-recognition can be shifted from merely visual to an integral approach using data-mining algorithms. A good need for such capability is for instance, a cradle-to-grave analysis of a product, where the same selected dimensions are not the only variables that need be tracked at each step. Other application domains that cry out for such sophistication include supply-chain management, drug discovery, etc.

nornaun wrote:

This is a dumb comment. Still, I really admire the assistance's effort in making all the chart for his manager in Bertin reading. Had he lived in this era, he could just use some software. This is a good example of how technology makes it easier to process data. However, I thought that, for me, the physical chart makes more sense and fun to play with while the digital presentation can be more expressive in some respects. The digital presentation is also limited to specific hardware like monitor's size and what not. It would be great if we can add some interaction in physical world to breach this barrier and enhance experience and perception in visualizing data too.

tessaro wrote:

Ben Fry's zipdecode represents an elegant and confirmatory example of how data visualization can make quick work of a previously obtuse or even vexing organizational system. Two ideas that make this successful. One the simplicity of the encoding, the use of the postal centers alone to effectively 'draw' the outlines and population densities of the US. The second is the deft use of scaling with the zoom function as a way of coupling the organizational logic of the zipcode system with the encoding strategy of the map itself. Zooming in to the map with each successive digit reveals the visualization's encoding method and the systems hidden rational in parallel - unifying macro and micro readings with elegance and concision. This kind of compression achieves a kind of completeness that turns something of modest curiosity into an exemplar of understanding, in effect answering its simple query so clearly that it became how I think about how zipcodes work in my minds eye. A unexpected result indeed.

anuraag wrote:

It seems like the most successful interactive operation performed by the example interactive visualizations (zipdecode, NameVoyager, LA Homicide, etc.) is in filtering (zipdecode and NameVoyager also perform view adjustment operations to show better views of the filtered data). The great value of visually filtering in an interactive visualization is the immediate feedback: because viewing the filtered results visually supports much faster cognition than having to comprehend the raw data, there is a tighter feedback loop of trying a filter, evaluating whether it produced the desired result, and then backtracking to modify it or moving on to the next desired operation.

It seems here that one of the values of animation is in enhancing our understanding of how the filtering operation is happening. If all we see is a single-step change from the visualization of the unfiltered data to the filtered data, it is harder to understand how the filter was applied, and so if the result is not what we expected or intended, then it is more difficult to determine what modifications to the filter are needed to obtain the desired result. One of the values of animation seems to be to illustrate the "inner workings" of the interactive tool - the tool creator's way of showing us, the viewer, how he or she interprets our request and applies it.

codeb87 wrote:

After reading Hochheiser and Shneiderman's article on TimeSearcher, I am quite impressed with their creation. I think the innovation of TimeSearcher is that it uses the enhanced understanding afforded to us by a visualization (ie visual maximums, minimums, trends, comparisons, etc) channels that directly to us as we interact with and filter the data. I think it is beautiful because the query itself takes place directly on top of the data, and it is for this reason that the queries made with TimeSearcher are so well informed and relevant. The overhead of the feedback/refinement process is next to zero. Not only is the temporal overhead very small, our understanding of how each query is relevant to us (ie assessing the results of each refinement), is presented with the same clarity and benefit of the entire data set (ie we still see the entire data set, and can compare the result data with the entire set, and simply move the mouse to refine).

Though TimeSearcher does a wonderful job of implementing the data selection process interactively, it makes me curious about what sort of interactivity would be useful in the data analysis process. This could be anything from tranforming the data/axes, to intuitive ways to zoom, compare change over time, draw trend lines, or quantitatively compare two or more different data series. Basically I think it would be exciting to extend the interactivity into the analysis of the data, although this might become very specific depending on what types of analysis are useful/relevant.

cwcw wrote:

From an educator's standpoint, I have to wonder about the implications of visualizations that allow the viewer to manipulate them directly. For example, Tableau integrates tools like "highlight" and "sort," that allow users to show the same data in different ways. Similarly, we've seen visualizations in class that incorporate sliders or toggles and animation to help tell the story. I'm inclined to believe that, where such features are appropriate to the visualization, they increase effectiveness. A person learns better by DOING--that is, if the user/viewer is allowed to manipulate and (actively) interact with a set of data, he/she is more likely to understand it and remember it, than if he/she is passively receiving the information as an audience member.

mikelind wrote:

to cwcw - I agree that doing can be an incredibly useful tool to get people actively engaging with a concept which will often lead to a greater internalization of the idea. I think there are some interesting problems around using interaction in this way, such as how much latitude should be given to the user by the interaction creator to play with certain settings if there is a clear idea that is meant to be expressed. I think reaching the point where a user is able to discover enough about a visualization on their own to help them internalize it as well as get across the original goal of the visualization is a difficult task, but if done effectively can probably be the most rewarding type of visualization creation.

saip wrote:

I believe that applications where one can visualize the data, and in addition, visually interact with the data, probably modifying the data in the process, can be exciting. For example, is a online music site that one can use to visualize songs as collections, grouped in relation to one another. One can see this visually and with sufficient permissions, visually change the one's own collections to suit preferences.

Another interesting point in class was the use of non-conventional interaction techniques for effective interaction with visualization. For example, the demo shown in class of the tool for brain pathway visualization using gesture based interactions was interesting. This provided a means of selecting brain fibers using gestures instead of combination of using more complicated non-natural selection procedures. This is particularly useful in some scenarious where the data (nerves in this case) are in large numbers.

Leave a comment