An Army scientist recently won a best paper award at the Association for Computing Machinery's 26th Conference on User Modeling, Adaptation and Personalization for discovering that most people cannot distinguish between liking a user interface and making good choices.

Dr. James Schaffer, U.S. Army Research Laboratory scientist stationed at ARL West, and his collaborators at the University of California, Santa Barbara, Drs. John O'Donovan and Tobias Höllerer, received the best paper award at the conference held in July at Nanyang Technological University in Singapore.

So, does technology really enhance our decision-making ability?

The paper, "Separating User Experience from Choice Satisfaction," addresses this question and furthers the theory that underpins the evaluation of recommender systems, which are designed to help users make good choices.

Simply put, recommender systems are artificially intelligent algorithms that use big data to suggest additional products to consumers based off of things such as past purchases, demographic information or search history, for example. Think of the "people you may know" feature that exists on many of today's .

In recommender systems, it has been assumed that users form very complex mental models of user interfaces.

This is reflected in current measurements, which elicit subjective responses on a wide range of system features.

However, ARL's new results contradict this assumption and even demonstrate that a person's subjective satisfaction with their decisions are all strongly influenced by their cognitive state and traits.

"User experience and choice satisfaction can easily be conflated when good system design creates positive feelings about an experience, artificially leading participants to think good decisions have been made," Schaffer said. "This can lead to false positive situations, where researchers may assume good decisions are being made due to a 's appearance or ease of use."

The authors drew from historic work on the cognition of happiness to generate a measurement strategy that can better account for this conflation.

The Army continues to push for increased modernization of its forces, with notable efforts including the Android Tactical Assault Kit and allocating funding towards researching new AI and machine learning methods to assist command and control personnel.

Recommender systems and other forms of AI are expected to play a key role in battlefield decision making, but academic and corporate approaches to designing such systems often fail when transitioned onto the battlefield due to the increased cost of failure.

"The current state of the art in recommender systems likely would have led the U.S. Army's modernization in the wrong direction, and the results from the paper are a warning against any type of subjective evaluation being done at, for instance, military exercises," Schaffer said.

Schaffer's research helps form the basis for evaluation strategies that can help the Army distinguish between technology that boosts performance and technology that simply has a wow factor.

In fact, this research indicates we should see the opposite: frustration on the part of the decision makers likely means something is being accomplished.

Schaffer expressed his satisfaction with the award and believes it shows the promise of the Open Campus initiative.

"This research idea only matured due to chatting with an ARL scientist from the Human Research and Engineering Directorate, Dr. Benjamin Files, which I think really highlights the benefit of ARL's Open Campus initiative," Schaffer said. "This paper was also only possible due to a collaboration with UCSB, which shows good things happen when ARL collaborates with outside universities. I probably would not have had the discussion that highlighted the feasibility of this idea in a different environment."

Provided by The Army Research Laboratory