'Artificial empathy' could help stores improve recommendations for customers

March 18, 2016, University of Cambridge
'Artificial empathy' could help stores improve recommendations for customers
5th Avenue, 2013. Credit: Ingfbruno

Visual data will revolutionise the way companies talk to their customers, according to researchers at the Cambridge Judge Business School.

Imagine trying on a piece of clothing, only to find that when you've put it on, a computer is able to read your facial expressions and body language so that it can make recommendations for other items of clothing you might like. If Shasha Lu has anything to do with it, this will become a reality.

Visual data has become a main source of corporate and public (unstructured) big data and the focus of Lu's work is how to harness the power of these images and videos to mine valuable information by combining image/video analytics with current marketing models. "Ultimately," she explains, "it's about gaining better customer insights to solve business problems and improve business practice."

As a new domain in marketing research, Lu explains that it's all about combining computer science techniques with existing marketing models to develop managerial tools for business. "It was during my PhD that my interest in artificial empathy – automatically inferring people's internal states based on the information they emit – really took off," she says. "My supervisor at the time, Professor Min Ding, was really interested in this area and I didn't take much convincing to share his enthusiasm because I'd always been so fascinated by artificial intelligence.

"My father worked in business when I was little, so I was influenced by that. Plus, I was a great fan of science fiction, especially the innovative stuff like time machines. The idea of getting a machine to come to useful conclusions from someone's expressions and behaviour was therefore massively exciting to me and we now have the technology and computing power to actually make this possible."

Moving to Cambridge Judge Business School to further her interest in leveraging image and was a huge move, she admits. "I came from Shanghai, which is a very big city, similar to London in that it never sleeps. Cambridge, on the other hand, provides a much quieter, small-town life, but I love it."

Among Lu's main interests outside work is fashion, which has made her clothing project – entitled A Video-Based Automated Recommender System for Garments – even more motivating.

"When people, especially females, go out shopping, we tend to try a garment on before making the final purchase. When we do so in front of the mirror, the sales person often observes two key pieces of information. The first is whether the customer likes it or not, which is inferred by her emotional response from her facial expression. The second is which particular part of the garment she likes or dislikes, which is observed through behavioural response. If she's touching her collar or scratching the lower hem with an annoying expression on her face, for example, we may infer she doesn't like that bit of the garment.

"Our project uses a camera mounted on top of the mirror – placed outside the changing space – to capture the evaluation process and analyse and behavioural responses. Then we combine it with data showing what other customers' reactions are to the same garment in order to make recommendations to the current customer in real time. In addition, the store can use other pieces of information, such as height and body shape, to make better recommendations or provide better service."

Customising movie trailers is another project that Lu is involved in. "This is still ongoing, so we don't have concrete findings yet, but as it stands all move trailers target the same group of people; if there's, say, a trailer with an action element that's watched by someone that doesn't go in for action films, they may well be put off seeing it, whereas in fact the movie might have plenty of non-action that makes the film attractive to this viewer," she explains.

"We know that it's possible to analyse the content of movies to extract all kinds of information – lighting, colour, motion and shot, for instance – all of which relates to people's preferences, and we are working on using this to customise movie trailers according to people's preferences, thereby improving the effectiveness of the movie trailer and getting people to watch movies they might otherwise dismiss."

Even the nature of online dating may be influenced by Lu's work in the future. "We know that certain information about someone's face in an online profile picture has an impact on the person looking at them. That could be anything from the distance between the eyebrows and nose or to the distance between the nose and mouth. We propose to analyse people's preferences in order to provide suggestions of candidates that match the types of faces the person is most likely to go for."

Currently, says Lu, there's a relatively small group of academics involved in using image and video data in the business world in this way, albeit on a global basis, with growing interest at Harvard Business School, where work is being done, for instance, on inferring people's preferences for television commercials. "But it's a growing area," she says.

Marketing, points out Lu, can feel theoretical and distant, whereas the work she's involved in is very accessible. "Not only is this area of work innovative and full of potential, but it's about the everyday aspects of living, and it's this combination that makes it a really exciting area to be involved in."

Explore further: Filmmakers just know how to help you read on-screen emotions

Related Stories

Filmmakers just know how to help you read on-screen emotions

January 7, 2016

Movie makers implicitly know how important it is to be able to read someone's expression, and that distance influences how easy this is done. This explains why close-ups of an actor appear on screen for shorter periods than ...

Asymmetry of an emotion

March 3, 2016

Though probably not at the top of the concerns of people affected by facial hemiparalysis, it's nonetheless a problem that should not be overlooked. Facial hemiparalysis not only makes it impossible to produce expressions, ...

Why smiles (and frowns) are contagious

February 11, 2016

Smile! It makes everyone in the room feel better because they, consciously or unconsciously, are smiling with you. Growing evidence shows that an instinct for facial mimicry allows us to empathize with and even experience ...

Technology to analyze customer behavior in stores

March 8, 2016

Proximus, a start-up in the Vivero de Empresas del Parque Científico de la Universidad Carlos III de Madrid (UC3M Science Park Business Incubator), has developed technology for creating a map of shopper behavior in supermarkets ...

Recommended for you

Coffee-based colloids for direct solar absorption

March 22, 2019

Solar energy is one of the most promising resources to help reduce fossil fuel consumption and mitigate greenhouse gas emissions to power a sustainable future. Devices presently in use to convert solar energy into thermal ...

EPA adviser is promoting harmful ideas, scientists say

March 22, 2019

The Trump administration's reliance on industry-funded environmental specialists is again coming under fire, this time by researchers who say that Louis Anthony "Tony" Cox Jr., who leads a key Environmental Protection Agency ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.