Using artificial intelligence to gain insights into personality

When it comes to hiring, it can be a challenge for an employer to find the perfect person for the job. As a result, organizations often utilize personality scales as an aid in determining whether a candidate is the right fit.

While there are several widely used personality tests on the market, organizations may be looking for traits or skills that are not measured by scales already in existence. Creating a new scale—which takes the work of experts such as personality, organizational, social, or clinical psychologists—can be time-consuming and costly.

With this in mind, Ivan Hernandez, an assistant professor in the Virginia Tech Department of Psychology, wanted to find a way to make the creation of personality scales easier and more accessible.

"As psychologists, there are so many different aspects of personality that we'd be interested in measuring," said Hernandez. "But the hard part is, how do you do it? How do you come up with the right questions to know if a person is a good friend, to know if a person would be a diligent worker, to know if a person is emotionally intelligent?"

While those questions are typically devised by subject matter experts, Hernandez has suggested an alternative source—artificial intelligence.

Working with research consultant Weiwen Nie of Hogan Assessment Systems Inc. Hernandez has created a framework for using various natural language processing models to assist researchers with developing valid psychological scales.

In the traditional method of creating personality scales, subject matter experts are called on to create a pool of items that may fit a particular personality trait—for example, "I like going to parties" as a measure of extraversion. That pool forms the basis for the scale creation, which is administered and tested before being deployed.

In Hernandez's framework, a transformer-based language model generates the artificial intelligence-based item pool (AI-IP), consisting of a million new items—far more than any group of experts could create. Additional language processing models narrow the pool to the items most relevant to the desired construct, such as extraversion.

Essentially, this multimodel framework enables researchers to create longer, cohesive scales from a small set of relevant items.

The best part? Scales created using the AI-IP perform just as well as scales that are created through the human process of validation and calibration.

"When we give these items to people and we show them real items that were not made by a computer, people just can't tell the difference," said Hernandez. "This song and dance that we do as humans of making personalized scales through committee can really be resolved by leveraging the internalized knowledge of an artificial intelligence model."

The framework designed by Hernandez and Nie can help organizations cut down on the time and expense involved in creating personality scales. By relying on artificial intelligence to create the pool of items, the inherent subjectivity, inconsistency, and bias of humans is also bypassed.

More importantly, the framework, which is openly available, achieves Hernandez's goals of improving accessibility to personality measurements. Now anyone—whether it's a lawyer who wants to gauge the trustworthiness of a jury pool or a college student wondering about their new roommate's cleanliness—can create a personality scale.

"This framework was intended for organizational use, but I truly think it's something that can help everyday people," said Hernandez. "People are interested in personality—but I think they just perhaps don't know of more validated ways to go about exploring their interest than the measures most commonly seen on social media or in pop culture."

An article describing the framework and how it was created is being published in a 2023 Personnel Psychology special issue focused on artificial intelligence and machine learning applications in personnel selection and staffing.

Those interested in experimenting with the AI-IP generation tool can access the application online.