[vimeo 10723525]

StruckAxiom recently had the good fortune to build a multitouch installation piece for a PepsiCo internal conference. The timeline was extremely tight, but we knew that if we could harness the new multitouch capabilities of Flash Player 10.1, we could create a simple but engaging experience. We had a lot to learn to use this new feature, and I'd like to share some of those lessons here.

(For more details about the installation, please check out the StruckAxiom blog post on the subject.)

Plan your gestures

Our installation allowed a select few interactions: 'panning' around or spinning the virtual space, rotating the view of the space, zooming in and out, and selecting an item.

We considered existing multitouch paradigms, since there's little use in reinventing the wheel. Example: even among Apple products there are two alternatives for panning or scrolling—two fingers on the trackpad on a MacBook, or a single finger on iPhone. We opted for single-finger panning, as we wanted to save two-finger gestures for zooming and rotating.

Mind your hardware

Since our timeline was incredibly short, we didn't have time to build our own multitouch input, so we had to rely on commercially available solutions. We settled on technology supplied by NextWindow:

The NextWindow displays integrate smoothly with Windows 7—as of this writing, Mac OSX doesn't support multitouch displays. A note: optical multitouch technology only allows for two simultaneous touches. Unfortunately, occasionally the cameras that track touches get a bit flummoxed and begin to behave unpredictably.

Multitouch.inputMode

To get Flash Player to dispatch multitouch events, you have to set Multitouch.inputMode. You've got two options (aside from MultitouchInputeMode.NONE), and it's either one or the other:

  • MultitouchInputMode.TOUCH_POINT: tracks individual touch point events - when touches begin, move, and end. However, no native gestures.
  • MultitouchInputMode.GESTURE: dispatches events for native gestures. All gestures are two-fingered. Only one gesture can be detected at a time—Flash Player determines whether the gesture should be interpreted as a pan, rotate, or zoom gesture. No individual touch point events are fired.

Writing custom gestures

We chose to use MultitouchInputMode.TOUCH_POINT and write custom gestures for two reasons—we wanted the ability to have a single-finger pan, and we also wanted to be able to scale and rotate at the same time—the native gestures can seem a little horsey because they can only perform one at a time.

To create custom gestures, we wrote a TouchManager class that kept track of touch events. We built our TouchManager to hold our two possible touch points in an array, tracking when a touch began (TouchEvent.TOUCH_BEGIN), moved (TouchEvent.TOUCH_MOVE), or ended (TouchEvent.TOUCH_END). Whatever class was relying on multitouch input would check the TouchManager on each frame, to see how many touch points are active, and then get the change in the touches over the last frame.

TouchEvent.touchPointID

TouchEvents come fully loaded with a touchPointID property, so that you can tell which touch point triggered the event—wasn't that thoughtful? I imagined that perhaps the first touch point would have a touchPointID of 0, the second would be 1, etc. Seems to be totally hardware dependent, however - using the Dell SX2210T monitor, the first point had a touchPointID of 2, and the second was 3. However, the NextWindow overlay started with a touchPointID of 2, and then proceeded to 4, perhaps because of an irrational fear of odd numbers. So, a note: don't count on the first touchPointID being 0, and don't rely on touchPointIDs being consecutive.

Using Flash Player 10.1

Developing for Flash Player 10.1, which is still in Beta at the time of this writing, was really fairly smooth. We only ran into one major problem: we were initially developing our application as an AIR app. However, while interacting with the piece, the TouchEvents seemed to get backed up, like a clogged pipe—suddenly we'd stop receiving TouchEvents, even though the user was interacting, and then all of the missed TouchEvents would dispatch in a flood to catch back up. We tried changing it over to play in a browser, and that problem disappeared—apparently it was just a difference in the Flash Player and AIR Beta versions.

The problem with AIR has been reported to Adobe, so I trust that they're getting that fixed as 10.1 approaches release.

Last words

The multitouch capabilities in Flash Player 10.1 are a great addition to the platform. Despite the ubiquity of mobile touch devices (iPhone, Blackberry, Android, et al), seeing multitouch on a larger scale is a huge crowd-pleaser—our piece was incredibly simple in terms of concept and execution, yet comparisons to Minority Report were made more than once. Not sure that multitouch monitors are really going to catch on—the process of using a traditional, mouse-centric OS with touch interactions is pretty clunky and painful. Maybe the capabilities will come in handy as 10.1 comes to mobile devices, or maybe it will be restricted to Flash-driven installations. Either way, Adobe did an excellent job with Multitouch in Flash—pretty easy to implement, pretty solid—and I look forward to see what the Flash community does with it.