Immersive Game System Allows Physical Interaction Between Players

December 22, 2009 By Lisa Zyga, feature
In the virtual reality game, the player’s avatar mirrors the player’s actions. Credit: Tedjokusumo, et al. ©2009 IEEE.

( -- With a new immersive multiplayer game system, researchers are further blurring the line between gaming and the real world. Using a mouse and keyboard sounds kind of quaint compared to the system developed and tested by Jefry Tedjokusumo, Steven ZhiYing Zhou, and Stefan Winkler of the National University of Singapore (Winkler is currently with Symmetricom in San Jose, California).

The designers have created a few varieties of a first-person shooter (FPS) game in which players use a gun or sword to battle other players. In a game called “Death Match,” the goal is to kill as much as possible until one player’s health level reaches zero, which ends the game. The traditional way to do this is with a keyboard and , but in the new study, the researchers have developed game systems in both a and an augmented reality environment (where gaming imagery is superimposed on a player’s view of the real world).

“Unlike traditional FPS games where the user usually sits and controls the game using a joystick/keyboard/mouse, our immersive game system forces the user to move his or her entire body to aim, run, jump, crouch etc,” Tedjokusumo told “Our immersive system can also be used to train for shooting accuracy, and the result will be very close to the real world performance.”

In the system, players wear head-mounted displays and play the game in a rectangular room without obstacles. The head-mounted display provides each player with an image of the scene in front of them, which is the equivalent to the view on a computer or TV screen. Each player also carries a wand, which can be set up to be used as either a gun or a sword. Both the head-mounted display and the wand are tracked by a tracking system inside the room, and players receive feedback in real time. As players move through the room, they can find and pick up virtual game items, such as bullets, armor, weapons and health. Besides walking, players can also jump to dodge enemy bullets, and the player’s avatar mirrors the player’s actions.

The augmented reality system is similar to the virtual reality system, except that a player’s head-mounted display has a camera mounted in front to capture the player’s real-world view, and the game can be played in any open environment. The researchers built a game engine to calibrate the real-world data with the gaming environment (which includes the player’s weapon, other virtual game items, and sound effects). In this system, players move around in the real world using a virtual gun or sword to shoot or slash other players.

For both games, the researchers developed a novel method for presenting the player’s view and the weapon’s view separately. With this feature, players can turn their heads to look in various directions while aiming their weapon in a different direction, even if it’s out of the player’s view. To enable the players to aim, a small window called Gun Radar appears at the bottom of the screen, showing the weapon’s point of view. With two different points of view, can shoot in any direction, even behind their backs.

The researchers invited volunteers to test both new gaming systems, and measured the results through subjective and objective methods. In the subjective user study, volunteers reported that they preferred the virtual reality game mode over both the augmented reality and keyboard/mouse modes. However, the objective user study showed that the and mouse mode allowed for more accurate aiming of bull’s-eye targets in the virtual world. Overall, volunteers preferred the virtual reality mode since it exactly mimicked their own physical movements, including full-body movements, neck movements, and weapon aiming.

“We are currently promoting the mixed/augmented reality system to the mass market,” Tedjokusumo said. Zhou’ company, MXR Corporation (, is in charge of this project. On the other hand, the virtual reality system was made for specific research purposes since it uses an expensive, sophisticated tracking system. To learn more about this game, visit the Interactive Multimedia Lab (

Explore further: Increased success a 'virtual' certainty for rugby players (w/ Video)

More information: “Immersive Multiplayer Games With Tangible and Physical Interaction.” Jefry Tedjokusumo, Steven ZhiYing Zhou, and Stefan Winkler. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. To be published.


Related Stories

Pendulum Finds Virtual Soulmate

June 14, 2007

What's nerdier than creating an online avatar that fights dragons and raids strongholds? Creating a virtual pendulum that you can sync up to your real-life pendulum. Leave it to physicists to do just that, resulting in a ...

Gesture recognition

December 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Augmented Reality: Science Fiction or Reality? (w/ Video)

July 7, 2009

( -- Computer graphics have come a long way since the birth of Atari Games over 30 years ago. Today, computer graphics seem very real and some day researchers will pull graphics out of your television or computer ...

Recommended for you

1 in 3 Michigan workers tested opened fake 'phishing' email

March 16, 2018

Michigan auditors who conducted a fake "phishing" attack on 5,000 randomly selected state employees said Friday that nearly one-third opened the email, a quarter clicked on the link and almost one-fifth entered their user ...

World's biggest battery in Australia to trump Musk's

March 16, 2018

British billionaire businessman Sanjeev Gupta will built the world's biggest battery in South Australia, officials said Friday, overtaking US star entrepreneur Elon Musk's project in the same state last year.

Origami-inspired self-locking foldable robotic arm

March 15, 2018

A research team of Seoul National University led by Professor Kyu-Jin Cho has developed an origami-inspired robotic arm that is foldable, self-assembling and also highly-rigid. (The researchers include Suk-Jun Kim, Dae-Young ...


Adjust slider to filter visible comments by rank

Display comments: newest first

1.3 / 5 (13) Dec 22, 2009
At the rate were going soon we will just have people going around killing people in hopes of respawns. I don't think games like this are productive to our society unless we want to continue being a warlike nation. I understand their are positive ramification such as towards the technological applications and well as fighting obesity but I don't think programs like this would help anything but bring a new level of realism to humans. I understand some people man has become passive compared to older generations. This is true but we should be going toward peace unless we still want to use war as a mechanism for lowering our worlds population. If this is the case... we have missiles and bombs to do that job quicker than shotguns and swords.
4.4 / 5 (7) Dec 22, 2009
First person digital experiences could be used to market and sell real estate, holidays, cars, appliances, meetings etc. Some of the first pictures taken depicted nudity... that did not stop everyone from using the camera to portray the beauty of this world we live in.

Any negative perspective on the ramifications regarding use of this tech, while somewhat relevant, will ultimately be far outweighed by the massive social benefits of this tech.
5 / 5 (1) Dec 22, 2009
Open to some awesome pranks, place objects in the room not modeled in the map. >:]
4.7 / 5 (3) Dec 23, 2009
Themo, aggressive and warlike behaviour is part of the human condition - it's part of our evolutionary drive - surely immersive game environments is a fantastic way to channel that natural drive in a way that doesn't endanger anybody?

I'm all for peace but not sure a 'hippie' view that we should all just start being nice is ever going to work for the mass of people who are never going to have the space or resources to make them happy. Immersive gaming may even reduce our population in the long run.
4.7 / 5 (3) Dec 23, 2009
The main problem with such virtual reality is limited space in real world, games would require a 2D treadmill (or a giant trackball) on which to run.
1 / 5 (2) Dec 24, 2009
Finally! We got a real motivation for kids to do physical exercise! Brilliant! :-D
3 / 5 (1) Dec 24, 2009
The main problem with such virtual reality is limited space in real world, games would require a 2D treadmill (or a giant trackball) on which to run.

Or, we could just use the technology being developed that reads brain signals and convert them into computer movements. Just like they did with monkeys to make them control a virtual arm.
1 / 5 (1) Dec 24, 2009
I meant, apes, not monkeys.
not rated yet Jan 02, 2010
The augmented reality part of it is exciting. Imagine future tech of this, being able to wear your headset and play a game out in the real world. All real life objects would be converted in real time to the digital representations. Players and non players could easily be identified, so players wouldn't disturb those not playing. It would basically use the real world as the game world, the entire planet could be an MMO game. Amazing possibilities.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.