Medical monitor with eyes and ears

February 1, 2016, Fraunhofer-Gesellschaft
The "proxemic monitor" is connected to a PC, which in turn is connected to the database in which all the vital data of the patients in intensive care converge. Credit: Fraunhofer HHI

In intensive care Units (ICU), every second counts. In emergencies, doctors and nurses need to make the right decisions quickly. Fraunhofer researchers have developed a smart "proxemic monitor" which has optimized the processes in the most sensitive area of a hospital and clearly shows the data of the connected medical devices while avoiding false alarms. The screen can be controlled from a distance, without contact, by means of gestures and voice commands, thereby reducing the risk of transmitting pathogens.

Everything had just been quiet in the of the . Suddenly, though, there is excitement: the alarms on several in different rooms are sounding. Monitors are flashing and beeping. A drop in , – the computer monitors show exactly what has to be done and where. Doctors and nurses rush to the rooms of the patients. Medical equipment there provides detailed information on the status of the . "It's not easy to keep track of everything here during hectic situations", says Paul Chojecki, scientist from the "Vision & Imaging Technology" Department at the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, in Berlin.

Controlled by contact-free gestures

Together with his team as part of the joint project "control room", the scientist has developed an intelligent monitor. It quickly shows doctors and nurses the most important information about the vital signs of their patients. The screen has interfaces to the medical equipment in the room, as well as with the information systems in the hospital, and can be controlled by contact-free gestures and . The graphics of its display adapt to the distance from which it is viewed. "From the door, the doctor sees the data in a correspondingly large size. As he gets closer, the screen displays detailed information", Chojecki explains. A 3D camera provides the necessary data. The user interface is programmed based upon the web, so it is also suitable for mobile devices, such as tablets. The scientists are presenting a demonstration of the "proxemic monitor" at CeBIT 2016 (Hall 6, Stand B36). A practical test in cooperation with Uniklinik RWTH Aachen is planned for later this year. 

The system evaluates the data of the medical devices on the basis of the smart alarm design of the project partner, the Medical Engineering Department of Aachen University Hospital. This prevents false alarms. "Intensive care doctors have told us that this is a big problem. Current devices stick stubbornly to determined limits, without including all of the relevant factors that are necessary in order to comprehensively assess the risk situation. In addition, the acoustic signals produce a very loud noise, which is not beneficial to the health of either the staff or the patient", Chojecki says.

Fewer pathogens transmitted

Another advantage of the gesture control is that the doctor or health care provider does not have to touch the devices directly. "The transfer of pathogens in hospitals, particularly in intensive care units, is still a problem. The obligatory hand hygiene is sometimes forgotten, and viruses as well as bacteria are carried from room to room", Chojecki describes. The smart screen can be programmed at the bedside, as well as used in the control room (with a different configuration). Chojecki adds: "That's an important distinction, since it is not legally permitted for all of the instrument functions in the patients' rooms to be accessed from the control room. It has to be ensured that the doctor actually examines the patient before changing the treatment".

Three different cameras and a microphone scan the area in front of the monitor. Using the video data, the built-in software of the HHI analyzes whether there are people in the room, how far away they are from the screen, and what movements they are making. Depending on the distance, the display and functionality of the monitor changes. "Our monitor distinguishes between near, medium, and further distance. The cameras cover a maximum distance of four meters", Chojecki explains. From the medium distance, the cursor can be controlled with arm movements, and commands or short reports can be input by voice. With pre-programmed gestures, for example, a video call can be started, in order to have discussions with other physicians within or outside of the hospital. "We have given the monitor eyes and ears so as to allow for multi-modal interaction between the user and the system. Our software records distances and movements of the user in a contactless manner, interprets them, and converts them into commands for operating systems or machines", Chojecki explains.

3 questions for Paul Chojecki

Scientist, Vision & Imaging Technology Department, Fraunhofer HHI, Berlin

1. What are the next steps in this project?

We want to display more information, such as context-sensitive workflows, on the proxemic monitor so as to aid in the decision-making process. Another goal to reach by 2017: Integration of camera-based detection of users, as well as of their roles and authorizations. As a result, information and control elements can be provided more easily to individuals who are authorized and denied to those who are not.

2. What are the current trends regarding the topic of "Vision & Imaging Technology"?

Very evidently applications for augmented and virtual reality (VR): with professional 360° recording systems, the real world can be represented ever more accurately in VR glasses. With contact-free interaction via gestures, the user can move more easily in virtual worlds. Moreover, it is possible to reconstruct people in 3D and involve them in interactive TV formats or VR worlds.

3. What other topics are you working on at the moment?

We are describing and standardizing gestures. There are more and more systems and solutions with contact-free interaction. Concerning this area, it is important that more attention be paid in the future to ensuring that standards are adhered to. As a result, users will not have to re-learn the operating concept every time they want to control a system by gestures. For this purpose, we are currently developing a gesture description system and cooperating with other partners on the standard "DIN SPEC 91333 – Contact-Free Gesture Control for Human-System Interaction."

Explore further: iPoint 3D - Using fingers as a remote control

Related Stories

iPoint 3D - Using fingers as a remote control

February 19, 2009

( -- The 'iPoint 3D' allows people to communicate with a 3-D display through simple gestures - without touching it and without 3-D glasses or a data glove. What until now has only been seen in science fiction ...

The helping hand in the operation room

November 2, 2015

Surgeons have their hands full in the operating room: Among other things, clamps have to be set and tools have to be held. Until now, surgical assistants have been responsible for these tasks. In the future, a metal hand ...

Gesture-driven computers

February 29, 2008

It isn’t always easy to communicate with a computer. Two Fraunhofer Institutes will be presenting new possibilities of man-machine interaction at CeBIT in Hanover (Germany) on March 4 through 9. They will demonstrate how ...

Star chef points the way

February 10, 2010

Master chef Johann Lafer is a virtuoso in the kitchen -- and with modern technology too. At his cookery school the TV celebrity adopts a high-tech approach to make things easier in the kitchen with the touchless iPoint-Presenter.

Window shopping, virtually

January 5, 2011

Researchers want to make shopping trips a special experience in future by enabling passers-by to operate window displays with hand and facial gestures. Four cameras record the 3-D positions of people’s hands, face and ...

Recommended for you

Apple closing iPhone security gap used by law enforcement

June 14, 2018

Apple is closing a security gap that allowed outsiders to pry personal information from locked iPhones without a password, a change that will thwart law enforcement agencies that have been exploiting the vulnerability to ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.