Physical keyboards make virtual reality typing easier

May 8, 2017 by Allison Mills, Michigan Technological University
To find out how to type better with virtual reality technology, computer scientists used a light-up virtual display, autocorrect algorithms and a physical keyboard. Credit: Kiran Udayakumar

What's better than a holographic keyboard? A real one, apparently. New research from computer scientists at Michigan Technological University delves into the different ways to type in a virtual reality (VR) space. They're presenting their work at ACM Conference on Human Factors in Computer Systems (CHI 2017), noting that while cranking out text is an integral part of our digital lives, it's a field of research that has a surprising lack of emphasis in VR development.

"Lots of people are buying head-mounted displays, but it's mostly for video games," says Scott Kuhl, an associate professor of computer science at Michigan Tech. "We're trying to figure out how we can use a head-mounted display for office work like writing and editing a document or sending a message to someone."

Though these mundane tasks aren't ideal yet in VR, by wearing a headset that looks an oversized sleeping mask, VR can make an enchanted forest walk feel real—or at least surreal—and can transport a viewer halfway around the world. The key is balancing what a person can see through the head-mounted display and what they can touch physically. For typing, people need both visual and tactile feedback.

Head-Mounted Display

James Walker, a lecturer in computer science, led the research as part of his dissertation, working with Kuhl. He says the challenge stems from the fact that people need to see what they're typing—a bit difficult with an over-eye headset on—so he developed a light-up virtual keyboard synced with a physical keyboard. This virtual keyboard lets a VR typist see in the head-mounted display what keys they typed on the physical keyboard.

Other VR typing systems rely on either mid-air virtual keyboards or overlaying of real-world video into the VR display. But both approaches require extra equipment such as tracking cameras that can be error-prone and intrusive. People's texting performance also declines with only virtual keyboards.

"Typing in midair is very fatiguing," Walker says. "Our solution is noteworthy because it enables people to continue using their physical peripherals, which gives the best performance, and it doesn't need any extra hardware or require superimposing a video feed into the virtual environment."

Credit: Michigan Technological University
VelociTap: Autocorrect Algorithm

To assess the effectiveness of a physical keyboard, Walker conducted an experiment in which participants typed on a keyboard they couldn't see. In one part, participants wore a head-mounted display—in this case, an Oculus Rift. In the other part, participants used a desktop monitor with their view of the keyboard occluded by a cover. In each part, Walker tested participants' performance with and without his virtual keyboard.

To start, a large number of people reported their typing prowess to be expert or at least proficient. Being human, participants made mistakes—lots of them—especially those without the virtual keyboard lighting up keys. That makes this experiment a perfect set-up not just for testing VR text hardware, but also to examine how autocorrect fits in.

"People underappreciate the redundancy in natural language," says Keith Vertanen, an assistant professor of computer science who assisted Walker and Kuhl with a language model to correct participants' typing. "Our recognition touch screen program, VelociTap, is extremely accurate as it's been trained on billions of words."

Still, Vertanen says he and the rest of the team were pleasantly surprised by how well the autocorrect algorithm shifted from its touch screen origins to predicting intended letters on a physical keyboard. In addition, the team observed that error rates declined as people continued typing with the VelociTap feedback and the model corrected about two-thirds of the text errors.

Full Texting

The team's research has focused on the input of short, informal text messages. Next steps include supporting the input of numbers, punctuation and capitals, and to allow the editing and styling of text.

"The long-term goal is facilitating arbitrary input while wearing a head-mounted display," Walker says.

The team knows how to catalyze that integration: use what's at hand. Physical keyboards can help push VR into the text-filled world of work and play.

Explore further: Acer's Iconia laptop skips keys in favor of double touch screens

More information:

Related Stories

Google rolls its own keyboard app for Android 4.0 and up

June 6, 2013

( —Google Maps, Google Drive, Google This, Google That….But there is always room for one more new arrival from Google, and now it is in the form of an app called Google Keyboard. Available at the Google Play ...

Speech recognition faster at texting

August 25, 2016

Smartphone speech recognition software gets a bad rap. Most users find the nascent technology to be frustratingly slow, and there are entire blogs dedicated to documenting examples of its biggest – and sometimes hilarious ...

Lenovo applies for a patent for a grip tablet keyboard

May 29, 2012

( -- With the introduction of the tablet computer, users of such devices have been forced to make some tradeoffs regarding keyboards. Virtually all tablets make use of image display and finger touching or tapping ...

Recommended for you

Meteorite source in asteroid belt not a single debris field

February 17, 2019

A new study published online in Meteoritics and Planetary Science finds that our most common meteorites, those known as L chondrites, come from at least two different debris fields in the asteroid belt. The belt contains ...

Diagnosing 'art acne' in Georgia O'Keeffe's paintings

February 17, 2019

Even Georgia O'Keeffe noticed the pin-sized blisters bubbling on the surface of her paintings. For decades, conservationists and scholars assumed these tiny protrusions were grains of sand, kicked up from the New Mexico desert ...

Archaeologists discover Incan tomb in Peru

February 16, 2019

Peruvian archaeologists discovered an Incan tomb in the north of the country where an elite member of the pre-Columbian empire was buried, one of the investigators announced Friday.

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet May 09, 2017
We need an online database of 3D keyboard models. Then put lighthouse trackers on the physical keyboards and sync them up with the virtual counterpart. There have been a variety of attempts to implement text input systems with motion controllers but it's hard to imagine something working better than a physical keyboard.
not rated yet May 09, 2017
I'm sure this is OK for general typing tasks. However, almost no tasks are of that type. In real world applications you always have many abbreviations and task specific vocabulary (where task/context can shift from email to email).

In coding it gets even worse. There you create a lot of artificial words/variable names on the spot and are using no end of special characters. I once went for a week of typing classes hoping to improve my coding speed. In the end it didn't help at all. While I became faster at typing words the many special characters used didn't allow for a flow to develop. In coding autocorrect features can be downright annoying. (AutoCOMPLETE is the bees knees, though)

I still think the overlay of a camera feed is the best approach (where 'best' still doesn't feel optimal as it breaks the immersion when moving one's head)
not rated yet May 09, 2017
Several of us are working on this problem. Some use the Twiddler chording input device to some satisfaction. We're working on bringing full workstation capability to a neck worn device that will be held like a sandwich. It can be seen at SoLaTiDo (dot com). If anyone would care to have some input on the approach, I would be happy to hear it.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.