Photography by Jeff Singer
As a Ph.D. student in applied mathematics at the University of North Carolina in Chapel Hill, Holz set his mind to developing a device that would fill in where the mouse fell short. And he asked Buckwald — who, as an undergraduate at Georgetown University in Washington, D.C., built and sold a startup to the Yellow Pages — to help bring the technology to market.

It wasn’t easy.

“You’d think that tracking motion should be relatively simple if you can see it with a camera,” Buckwald says. “But it turns out it’s very difficult. Fingers look very small on camera, and people move them very quickly.”

To set itself apart from what already existed, the device would have to track those movements faithfully in all directions — not just up and down and left and right ­— without the latency, or time-lapse, of other motion sensors. What’s more, the sensor needed to be small (the Leap Motion Controller is about the size of a pack of gum), affordable (it retails for $80) and not suck up too much of a PC’s brainpower (it uses a very low percentage of a computer’s CPU).

How Buckwald and Holz pulled this off is a matter of some secrecy.

“But I can tell you that the cameras we use are available over the counter,” Buckwald says. “The advances here aren’t really in the hardware. We’re talking about some very complicated mathematics.”

Leap Motion’s device has emerged as one of the most talked-about new high-tech gadgets on the market. But Buckwald expects more than buzz; he’d like to see his motion sensor “change the world.” For that, though — for the Leap Motion Controller to become more than just a nifty way to trash a document or slice digital fruit — the company will need help, according to Michael Copeland, a senior editor at Wired magazine.

“For this kind of 3-D interface to be something other than a technological detour, you need to be able to do something completely new, and for that you need an army of smart, creative developers,” Copeland says. “They are the people who will make this new, fun and worth our time. Without their ideas, motion control isn’t the future of computing. It’s a gimmick.”

Buckwald agrees, which is why Leap ­Motion has distributed its sensor at no cost to more than 10,000 developers around the world and challenged them to dream up apps for it that in July will be made available through the company’s app store, ­Airspace. Already they’ve had progress, and on a recent afternoon at Leap Motion’s offices in San Francisco, Buckwald takes a break to show some off. Sitting at a keyboardless computer, he dabbles at a virtual potter’s wheel, digging his fingers into digital clay to craft an elegant 3-D vase. That project completed,­ Buckwald calls up another application. Through it, his computer screen fills with schools of frenzied digital fish. Buckwald holds up his hands, and the tip of each finger appears on the screen, represented by what looks like an underwater spotlight. The fish seem startled. Buckwald jabs a finger toward them, and they scatter. When he moves his hand away, the fish regroup.

His movements are swift, constant and natural, and so is the sensor’s reaction to them.

Buckwald smiles. Not exactly Tom Cruise in Minority Report, but very much the stuff of science fiction.

“It’s impossible to predict everything that will come of this,” he says. “But that’s what’s most exciting — the absolute certainty that things will be created that we haven’t even dreamed of yet.”

Josh Sens is an Oakland-based freelancer who has also written for Men’s Journal and The New York Times.