Stanford 'Frankencamera' platform available on Nokia N900 ahead of unveiling at graphics conference

Jul 22, 2010 BY DAVID ORENSTEIN
The programmable Frankencamera took this photo by synchronizing the light from a strobe and a flash.

Stanford's open-source digital photography software platform, "Frankencamera," which allows users to create novel camera capabilities, is available as a free download for Nokia N900 "mobile computers" starting today. Next week at the SIGGRAPH conference in Los Angeles, the Frankencamera engineering team will describe the platform and several sample apps created with it.

"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy, who leads the project. "We are releasing code so that people can create new imaging applications on their Nokia N900s."

In addition, the researchers have been awarded a $1 million grant from the National Science Foundation, shared with colleagues at MIT, to begin making professional-style, single-lens reflex (SLR) cameras, equipped with the software platform, for free distribution to computational photography professors around the country. Non-academics could buy the camera at cost. Levoy said he expects those cameras will be available within a year.

Program and shoot

Computational photography refers to the ways computers can extend the capabilities of digital imaging by combining multiple photographs taken with different camera settings to create an image that could not be taken in a single shot, or with an ordinary camera.

This video is not supported by your browser at this time.
Eddie Tavala holds the Frankencamera, programmed to individual needs of the photographer. It is part of an attempt by a team of Stanford researchers to build an open-source camera architecture. The group has just released a version of their software for Nokia smartphones.

Some of these new ways of combining images can be done in Photoshop or another such program, but until now they could not be done inside the camera, Levoy said. That's because commercial cameras are closed to development by all but their manufacturers. Frankencamera, on the other hand, brings computational photography directly to the camera, by making the camera a programmable platform.

Frankencamera began in 2006 when Levoy and Kari Pulli, a Nokia Fellow who heads a research team at Nokia Research Center (NRC) Palo Alto, and a former research associate in Levoy's lab, reasoned that computational photography shouldn't be relegated to klunky research equipment in academic labs, as it has been for years. Instead it should be developed for use in the field on portable, consumer-friendly cameras.

"We thought it was time to make the research more nimble and get the results into smaller form-factors," said Pulli. "At NRC, we believe in an open innovation model that enables consumers and university researchers to use our research algorithms, and add their own, to create even more interesting capabilities."

At SIGGRAPH the researchers will describe how the Frankencamera platform exposes all the photographic and computational hardware on the camera - light sensors, flashes, focus, shutters and image processors - to a programmer's control through a software interface. The interface is part of a software "stack" that brings together familiar programming elements: a Linux operating system and the ubiquitous C++ programming language.

"The N900 is a camera phone, but it runs a version of Linux almost as complete as that installed on personal computers," Pulli said.

Putting the "app" in aperture

To help demonstrate and inspire development with Frankencamera, the team will discuss six apps they’ve created on the platform. Using a prototype of their SLR version, for example, Levoy’s group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards’ trajectories and also caught them still in mid-air with remarkable clarity.

Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.

An app called "low-light imaging" shoots both exposure speeds in rapid succession and then automatically combines them, resulting in a photo that is both bright and sharp.

Another app offers a different take on the low-light problem. Dubbed "lucky imaging," the app takes advantage of a motion sensor attached to the Nokia N900 to prevent shaky hands from adding blur to a long exposure. In this mode, the camera shoots constantly but only stores images taken when the camera is at its most still. The result is the sharpest possible image.

Levoy said he hopes the entire computational photography community will step forward with other ideas and that they will share them with each other online. The apps will be found through the Frankencamera project website.

For Levoy, a key question is whether commercial manufacturers - both of dedicated cameras and camera smart phones - will also open up their platforms. Since the Frankencamera effort was first publicized last fall, Levoy said, he has had a steady stream of meetings, especially with smart phone makers.

"What we're hoping is that if making your camera programmable adds value to the camera that this could shift the entire camera industry," Levoy said. "We'd like to democratize cameras the way that the Maker Faire is democratizing crafts so that anyone can play the game."

In addition to Nokia Research Center, funding comes from Adobe Systems, Kodak, Hewlett-Packard and the Walt Disney Company. In addition to authors at Stanford and NRC, other authors on the SIGGRAPH paper are affiliated with Ulm University in Germany, the University of California-Santa Barbara and Disney Research in Zurich, Switzerland.

Explore further: Review: 'Hearthstone' card game is the real deal

More information: graphics.stanford.edu/projects/camera-2.0/

add to favorites email to friend print save as pdf

Related Stories

Carl Zeiss lens enables highest quality in camera phones

Sep 09, 2004

The lens is supplied by Carl Zeiss, the camera module and electronics by Dialog Semiconductor: with this collaboration, the two companies aim to take part right from the beginning in shaping the growing market for high quality ...

Recommended for you

Review: 'Hearthstone' card game is the real deal

4 hours ago

Video game publishers don't take many risks with their most popular franchises. You know exactly what you are going to get from a new "Call of Duty" or "Madden NFL" game—it will probably be pretty good, ...

Microsoft expands ad-free Bing search for schools

Apr 23, 2014

Microsoft is expanding a program that gives schools the ability to prevent ads from appearing in search results when they use its Bing search engine. The program, launched in a pilot program earlier this year, is now available ...

Growing app industry has developers racing to keep up

Apr 20, 2014

Smartphone application developers say they are challenged by the glut of apps as well as the need to update their software to keep up with evolving phone technology, making creative pricing strategies essential to finding ...

Android gains in US, basic phones almost extinct

Apr 18, 2014

The Google Android platform grabbed the majority of mobile phones in the US market in early 2014, as consumers all but abandoned non-smartphone handsets, a survey showed Friday.

Hackathon team's GoogolPlex gives Siri extra powers

Apr 17, 2014

(Phys.org) —Four freshmen at the University of Pennsylvania have taken Apple's personal assistant Siri to behave as a graduate-level executive assistant which, when asked, is capable of adjusting the temperature ...

User comments : 0

More news stories

Facebook buys fitness app Moves

Facebook has bought the fitness app Moves, which helps users monitor daily physical activity and their calorie counts on a smartphone.

Cell resiliency surprises scientists

New research shows that cells are more resilient in taking care of their DNA than scientists originally thought. Even when missing critical components, cells can adapt and make copies of their DNA in an alternative ...