Stanford 'Frankencamera' platform available on Nokia N900 ahead of unveiling at graphics conference

Jul 22, 2010 BY DAVID ORENSTEIN
The programmable Frankencamera took this photo by synchronizing the light from a strobe and a flash.

Stanford's open-source digital photography software platform, "Frankencamera," which allows users to create novel camera capabilities, is available as a free download for Nokia N900 "mobile computers" starting today. Next week at the SIGGRAPH conference in Los Angeles, the Frankencamera engineering team will describe the platform and several sample apps created with it.

"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy, who leads the project. "We are releasing code so that people can create new imaging applications on their Nokia N900s."

In addition, the researchers have been awarded a $1 million grant from the National Science Foundation, shared with colleagues at MIT, to begin making professional-style, single-lens reflex (SLR) cameras, equipped with the software platform, for free distribution to computational photography professors around the country. Non-academics could buy the camera at cost. Levoy said he expects those cameras will be available within a year.

Program and shoot

Computational photography refers to the ways computers can extend the capabilities of digital imaging by combining multiple photographs taken with different camera settings to create an image that could not be taken in a single shot, or with an ordinary camera.

This video is not supported by your browser at this time.
Eddie Tavala holds the Frankencamera, programmed to individual needs of the photographer. It is part of an attempt by a team of Stanford researchers to build an open-source camera architecture. The group has just released a version of their software for Nokia smartphones.

Some of these new ways of combining images can be done in Photoshop or another such program, but until now they could not be done inside the camera, Levoy said. That's because commercial cameras are closed to development by all but their manufacturers. Frankencamera, on the other hand, brings computational photography directly to the camera, by making the camera a programmable platform.

Frankencamera began in 2006 when Levoy and Kari Pulli, a Nokia Fellow who heads a research team at Nokia Research Center (NRC) Palo Alto, and a former research associate in Levoy's lab, reasoned that computational photography shouldn't be relegated to klunky research equipment in academic labs, as it has been for years. Instead it should be developed for use in the field on portable, consumer-friendly cameras.

"We thought it was time to make the research more nimble and get the results into smaller form-factors," said Pulli. "At NRC, we believe in an open innovation model that enables consumers and university researchers to use our research algorithms, and add their own, to create even more interesting capabilities."

At SIGGRAPH the researchers will describe how the Frankencamera platform exposes all the photographic and computational hardware on the camera - light sensors, flashes, focus, shutters and image processors - to a programmer's control through a software interface. The interface is part of a software "stack" that brings together familiar programming elements: a Linux operating system and the ubiquitous C++ programming language.

"The N900 is a camera phone, but it runs a version of Linux almost as complete as that installed on personal computers," Pulli said.

Putting the "app" in aperture

To help demonstrate and inspire development with Frankencamera, the team will discuss six apps they’ve created on the platform. Using a prototype of their SLR version, for example, Levoy’s group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards’ trajectories and also caught them still in mid-air with remarkable clarity.

Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.

An app called "low-light imaging" shoots both exposure speeds in rapid succession and then automatically combines them, resulting in a photo that is both bright and sharp.

Another app offers a different take on the low-light problem. Dubbed "lucky imaging," the app takes advantage of a motion sensor attached to the Nokia N900 to prevent shaky hands from adding blur to a long exposure. In this mode, the camera shoots constantly but only stores images taken when the camera is at its most still. The result is the sharpest possible image.

Levoy said he hopes the entire computational photography community will step forward with other ideas and that they will share them with each other online. The apps will be found through the Frankencamera project website.

For Levoy, a key question is whether commercial manufacturers - both of dedicated cameras and camera smart phones - will also open up their platforms. Since the Frankencamera effort was first publicized last fall, Levoy said, he has had a steady stream of meetings, especially with smart phone makers.

"What we're hoping is that if making your camera programmable adds value to the camera that this could shift the entire camera industry," Levoy said. "We'd like to democratize cameras the way that the Maker Faire is democratizing crafts so that anyone can play the game."

In addition to Nokia Research Center, funding comes from Adobe Systems, Kodak, Hewlett-Packard and the Walt Disney Company. In addition to authors at Stanford and NRC, other authors on the SIGGRAPH paper are affiliated with Ulm University in Germany, the University of California-Santa Barbara and Disney Research in Zurich, Switzerland.

Explore further: Thanksgiving travel woes? There's an app for that

More information: graphics.stanford.edu/projects/camera-2.0/

add to favorites email to friend print save as pdf

Related Stories

Carl Zeiss lens enables highest quality in camera phones

Sep 09, 2004

The lens is supplied by Carl Zeiss, the camera module and electronics by Dialog Semiconductor: with this collaboration, the two companies aim to take part right from the beginning in shaping the growing market for high quality ...

Recommended for you

Thanksgiving travel woes? There's an app for that

Nov 26, 2014

Traveling by plane, train or automobile can be a headache. Mixing in Thanksgiving can make it a throbbing migraine. Technology provides some pain relief in the form of apps to let you know which roads are ...

Singapore moves to regulate taxi booking apps

Nov 21, 2014

Singapore on Friday announced new rules for mobile taxi booking apps, including US-based Uber, in the latest move by governments around the world to regulate the increasingly popular services.

Protecting personal data in the cloud

Nov 20, 2014

IBM today announced it has patented the design for a data privacy engine that can more efficiently and affordably help businesses protect personal data as it is transferred between countries, including across private clouds.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.