The city, 2.0. (w/ Video)

Augmented reality tours of real monuments, instant visualisation of virtual objects on living cityscapes and user-generated, digital story telling around famous landmarks are just some of the new applications made possible by breakthrough European research into mixed reality. This is the city, 2.0.

You are walking through modern-day Cologne on a beautiful morning radiant with the promise of summer. Abruptly, you are approached by a short, elf-like character.

Instantly, the cityscape morphs strangely and you now stand, in the same street, before a tent that obviously dates from Medieval times. You have entered a time warp. To escape the various time periods you must complete a series of challenges posted by the elves.

This was the augmented experience of around 60 volunteers at the TimeWarp field trial held in real-world Cologne in January 2010. The trial was a test and a demonstrator of one of the applications developed by European researchers at the IPCity project.

It had a surprising impact. The media were fascinated, the participants enthralled and the passers-by mystified: 'Why were these cyborgs bursting with technology and gesturing oddly at empty space?'

The IPCity project coordinator, Rod McCall, explains: "The users quickly got used to the augmented reality technology that they carried, and they would act in odd ways. Passers-by were often confused about what exactly was going on and even intervened, but it was a convincing and fun experience for the participants."

Real, virtual experiences

It is a very unusual concept; it was a real experience, the users did interact with elves that were invisible to the rest of the world. It happened in a world that mixed virtual people and objects with modern streetscapes in a new cross-reality. For the users, it was a new fusion between real and virtual worlds.

TimeWarp is a mobile augmented reality game originally played in the Old Town of Cologne by two players. They take on the role of agents working for the so-called ChronoGuard, investigating breaches in the time-space continuum. Their task is to locate little gnome-like robots called ‘Heinzelmännchen‘, inspired by the real Heinzelmännchen legend from Cologne.

To complete the task, players use their ultra-mobile PCs (UMPCs) to travel into different time periods, evoked by a mix of music, sound-effects and virtual 3D objects and characters superimposed onto reality via VR glasses.

In this way, players visit Roman, Medieval and future versions of the surrounding city where they are confronted with different tasks and dilemmas. In addition, the UMPC has a motion sensor which allows players to interact with the game more intuitively.

While the hardware elements are fairly standard, it is their combination with subtle effects that created an engaging and enjoyable experience for users.

"It also brings them into contact with the history and culture of the city, so it has an educational and participative element, too," notes McCall.

Presence, interaction, experience

That was a crucial element of all IPCity's work. The applications revolved around presence, interaction and experiences with the city, though many of the techniques and technologies could be adapted to other scenarios.

The City Tales application, for example, sought to place a user-generated digital layer over the physical fabric of the city, a bit like a Facebook of the physical world, where users could post personal stories about the city based on a physical element rooted at a specific point in the landscape.

Storytelling is a particularly interesting example of IPCity's work, because it demonstrates the range of tasks the project had to execute. For example, GPS is not nearly accurate enough to map virtual contents accurately into the cityscape, therefore alternative approaches had to be developed.

For example, IPCity developed a localisation technology driven by computer vision. It automatically recognises real-world elements based on captured images from the user’s location. For example, users can leave virtual graffiti on real-world locations - without incurring the wrath of city authorities.

Other work included authoring tools, so companies or city authorities can easily set up services and create appropriate content for it. The authoring tools work with all the applications developed by the project.

Furthermore, device integration and software interfaces had to be developed for each application, and fundamental to all applications was the theoretical and analytical work that defined the rules for creating engaging mixed reality experiences.

Through the looking glass

In some respects the work of IPCity represents a radical shift in direction for digital technologies. For much of the history of the internet, people entered deeper and deeper into the virtual world, through social networking, gaming and virtual reality.

But in IPCity, ‘virtuality’ reverses direction, bringing internet concepts into the real world, with objects in the streetscape unlocking virtual characters, messages and information. Instead of people entering the digital landscape of the internet, digital objects are beginning to colonise the real world. So, prepare to be V-Ex’d!

This is the second of a two-part special feature on IPCity appearing on ICT Results. Part 1. www.physorg.com/news198306374.html