One of FXPAL’s papers at the ACM Multimedia conference this year describes FACT, an interactive paper system for fine-grained interaction with documents. The FACT system consists of a small camera-projector unit, a laptop, and ordinary paper documents. The system works as follows: a user makes pen gestures on a paper document in the view a of a camera-projector unit. FACT processes these gestures to select fine-grained content and to apply various digital functions. For example, the user can choose individual words, symbols, figures, and arbitrary regions for keyword search, copy and paste, web search, and remote sharing. FACT thus enables a computer-like user experience on paper. This paper interaction can be integrated with laptop interaction for cross-media manipulations on multiple documents and views. FACT can be used in the application areas such as document manipulation, map navigation and remote collaboration.

I just read an interesting post by David Karger about PIM, end-user programming, data publishing, and lots of other interesting HCI ideas. The premise is that purpose-built applications for PIM impose strict schemas on their users, making it difficult to adapt, repurpose, or integrate the data with other applications. The alternative is something like Evernote, that lumps everything into one bucket, access to which is mediated largely by search. The tradeoff, then, is between a relatively undifferentiated interface backed by search on one hand, and a large number of siloed applications with dedicated interfaces.

David describes several systems (interfaces) his students built that leverage the Haystack framework for storing arbitrary data, and suggests that it’s possible to structure these data management tasks as authoring problems rather than as programming, thereby making flexible, extensible, customized interfaces more widely accessible.

Continue Reading

Bill van Melle, Thea Turner, and Eleanor Rieffel contributed to this post

FXPAL’s work on the MyUnity Awareness Platform has received considerable attention from the popular press and the Internet blogosphere in recent weeks, following a nice write-up in MIT’s Technology Review. That article, despite its misleading headline, correctly relays the core motivation for the work: to improve communication among workers in an increasingly fragmented workplace. However, some writers who picked up on that article focused instead on the sensational aspects of having technology monitor people’s behaviors and activities while they are working. They incorrectly described some of the platform’s technical details, overstated what the platform does and what it is able to do with the data it collects, and failed to mention the numerous options we offer users to control their privacy. We thought we should clear up some of these misconceptions and clarify the technical details.

Continue Reading

Tweetdeck recently published an interesting summary of their testing efforts for a Twitter client for Android. The short post enumerates the set of hardware and operating system versions they had to contend with in testing their software. I counted about 250 different devices and over 100 versions of the OS in the Tweetdeck charts in a population of 36K beta-testers, many of whom, admittedly, are early adopters who are more likely to use wacky devices and odd versions of the OS. But still, that’s a lot of potential wierdness.  This proliferation of versions and configurations can be seen as a sign of the vitality of the platform, but it is also suggestive of some potential problems.

Continue Reading

A writer for the TC blog, Erick Schonfeld, recently posted a description of an encounter he had with a Stanford student at a drug store trying to recruit users to experiment with a paper prototype. The prototype and study were being carried out as a requirement for an HCI course the student is taking. The TC writer, in short, found the whole experience ridiculous, especially with respect to all of the whiz-bang, interactive demos he is used to seeing. While, as many point out, paper prototyping is a standard technique in HCI, that does not mean that it always works or is always appropriate. In fact, in my experience with early stage prototypes I was overall underwhelmed with paper prototyping. But I realized over time that experience did not reflect a problem with the prototyping tool per se, but rather a lack of understanding of the context of the user.

Continue Reading

The September-October issue of ACM interactions includes an essay by Don Norman and Jacob Nielsen in which they critique various aspects of gestural interactions as implemented on the new crop of touch-enabled devices: the iPhone, iPad, the Android family, and the like. The gist of their concern is that the design of these interfaces, while incorporating useful and pleasurable interactions based on touch (swipe, pinch, etc.) also introduce a range of usability problems well known to the HCI community.

Continue Reading

Ken Hinckley, Koji Yatami, and several other people from MSR published an interesting analysis of how to combine pen and touch input on table-top displays. The work draws inspiration from observations of how people manipulate paper to derive design guidelines for bi-manual and bi-modal interaction. The paper contributes a thorough description of integrated touch and pen-based interaction and offers a thorough analysis of design principles that underlie these kinds of interactions.

Continue Reading

Fox@fury is an interesting blog I have started following. Kevin Fox displays a keen design eye for offering small usability improvements that can potentially yield important usability benefits.

In a recent post, for example, he offered a simple suggestion for how to alter the appearance of Chrome tabs to improve the user’s ability to determine which of many open tabs is active. In another, he analyzed the Android app market to show some of the fundamental flaws in its design.

Continue Reading

Michael Bernstein and a cast of thousands published an interesting paper at UIST 2010, which was honored with the Best Student Paper award. The paper describes and evaluates Soylent, a tool that uses Mechanical Turk to generate corrections and suggestions to improve writing. (The name Soylent is not a substitute for dairy in the weeks leading up to Easter; rather, it is derived from the film Soylent Green.)

This work is interesting in a number of ways: it automates the distribution and collection of Mechanical Turk tasks and then integrates the results into an interactive system, it recognizes the limitations of fully-automated approaches, and it suggests a design pattern that can be applied in other contexts .

The main contribution of this paper is the idea of embedding paid crowd workers in an interactive user interface to support complex cognition and manipulation tasks on demand. These crowd workers do tasks that computers cannot reliably do automatically and the user cannot easily script.

Continue Reading

Yesterday Volker Roth came back for a visit and to give us a preview of the talk he will give next week at UIST 2010 on his work with Philipp Schmidt and Benjamin Güldenring on The IR Ring: Authenticating users’ touches on a multi-touch display. The work supports multiple users interacting with the same screen at the same time with different access and control permissions. For example, you may want to show me a document on a multi-touch display, but that does not mean you want me to be able to delete that document. Similarly, I may want to show you a particular e-mail I received, without giving you the ability to access my other e-mail messages, or to send one in my name. Roth et al. implemented hardware and software add-ons for a multi-touch display that restrict certain actions to the user wearing the IR ring emitting the appropriate signal. Users wearing different rings have different access and control privileges. In this way, only you can delete your document, and only I can access my other e-mail messages.

Roth and his coauthors frame their work as preventing “pranksters and miscreants” from carrying out “their schemes of fraud and malice.” To me, the work is most compelling as a means to avoid mistakes and to frustrate human curiosity. Continue Reading