Study finds gender bias in open-source programming

May 1, 2017 by Matt Shipman
Credit: North Carolina State University

A study comparing acceptance rates of contributions from men and women in an open-source software community finds that, overall, women's contributions tend to be accepted more often than men's - but when a woman's gender is identifiable, they are rejected more often.

"There are a number of questions and concerns related to gender bias in computer programming, but this project was focused on one specific research question: To what extent does gender bias exist when pull requests are judged on GitHub?" says Emerson Murphy-Hill, corresponding author of a paper on the study and an associate professor of computer science at North Carolina State University.

GitHub is an online programming community that fosters collaboration on projects. When people identify ways to improve code on a given project, they submit a "pull request." Those pull requests are then approved or denied by "insiders," the programmers who are responsible for overseeing the project.

For this study, researchers looked at more than 3 million pull requests from approximately 330,000 GitHub users, of whom about 21,000 were .

The researchers found that 78.7 percent of women's pull requests were accepted, compared to 74.6 percent for men.

However, when looking at pull requests by people who were not insiders on the relevant project, the results got more complicated.

Programmers who could easily be identified as women based on their names or profile pictures had lower pull request acceptance rates (58 percent) than users who could be identified as men (61 percent). But woman programmers who had gender neutral profiles had higher acceptance rates (70 percent) than any other group, including men with gender neutral profiles (65 percent).

"Our results indicate that gender bias does exist in open-source programming," Murphy-Hill says. "The study also tells us that, in general, women on GitHub are strong programmers. We don't think that's because affects one's programming skills, but likely stems from strong self-selection among women who submit pull requests on the site.

"We also want to note that this paper builds on a previous, un-peer-reviewed version of the paper, which garnered a lot of input that improved the research," Murphy-Hill says.

The paper, "Gender Differences and Bias in Open Source: Pull Request Acceptance of Women Versus Men," is published in the open-access journal PeerJ Computer Science. The paper was co-authored by Josh Terrell, a former undergraduate at Cal Poly; Andrew Kofink, a former undergraduate at NC State; Justin Middleton, a Ph.D. student at NC State; Clarissa Rainear, an undergraduate at NC State; Chris Parnin, an assistant professor of computer science at NC State; and Jon Stallings, an assistant professor of statistics at NC State. The work was done with support from the National Science Foundation under grant number 1252995.

Explore further: Women accepted as better coders as long as no gender link

More information: Josh Terrell et al, Gender differences and bias in open source: pull request acceptance of women versus men, PeerJ Computer Science (2017). DOI: 10.7717/peerj-cs.111

Related Stories

Women accepted as better coders as long as no gender link

February 13, 2016

Wow, gender bias has appearances of residing in computer programming—off the charts, so to speak, but accorded serious attention in a new study. The findings come from scientists at California Polytechnic State University ...

Study shows software developers' skills improve over time

April 29, 2013

There is a perception in some tech circles that older programmers aren't able to keep pace with rapidly changing technology, and that they are discriminated against in the software field. But a new study from North Carolina ...

Recommended for you

When words, structured data are placed on single canvas

October 22, 2017

If "ugh" is your favorite word to describe entering, amending and correcting data on the rows and columns on spreadsheets you are not alone. Coda, a new name in the document business, feels it's time for a change. This is ...

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.

15 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

tblakely1357
2.7 / 5 (7) May 01, 2017
Any research that has 'political' implications is automatically suspect.
Bart_A
2.8 / 5 (11) May 01, 2017
Bogus study. There are so few women programmers out there to begin with. Stop trying to find bias at every nook and cranny. It gets very old.

PTTG
2 / 5 (4) May 01, 2017
ITT: Men who can't program insult women who can.
s86u
2.6 / 5 (5) May 01, 2017
very unfortunate for those of you who are of the same gender/race as those the government deems not worthy of preferential treatment. You would think you should be able to request some of the benefits of the wealthier people of your race/gender for whom the government is adjusting its statistical justice scales.
vacuumforce
3.7 / 5 (3) May 02, 2017
"Study finds gender bias in population."
Dingbone
May 02, 2017
This comment has been removed by a moderator.
Lex Talonis
2.3 / 5 (3) May 02, 2017
Maybe once the women out themselves, all their less than desirable attributes come out with them.

As my lesbian buddy once noted, "Men don't have a monopoly on being arseholes."
mdblack98
3.7 / 5 (3) May 02, 2017
Gee...go ahead and read the paper. The first article by Rachel Nabors they provide of a female example explicitly says "All the rejections were polite. All of them were justified". There was no perception of gender bias by her at all.
Correlation is not causation.
Without identify the types of pull requests one HUGE assumption is being made that all pull requests are equal. Nothing could be further from the truth.
rrwillsj
1.3 / 5 (3) May 02, 2017
LT, remember when that utterly ridiculous book "Men are from Mars, Women are from Venus" was so popular? Every damn party or get-together, some smug jerk (of whatever sex) would repeat that rubbish.

Both my wife and I found it irritating as hell, so she came up with the perfect response.

"No, not Mars or Venus."
"Women are from Luna."
"And Men are from Uranus."
"Women may be raving assholes several days a month."
"But Men are assholes everyday!"
Ensign_nemo
3 / 5 (4) May 02, 2017
The differences here were -4.1% for all female pull requests, -3% for identifiable female pull requests, and +5% for non-identifiable female pull requests.

Those are pretty small differences.

I suspect that if you compared two subsets of the data with some obviously irrelevant characteristic such as men with beards and men without beards, you'd find the same statistical fluctuations.

If we were studying particle physics this large an effect would be relevant, but people are more complex than particles. A 3, 4, or 5% variation in behavior could be mostly random chance.

Opinion polls taken with samples of 1000 people usually have margins of error of about the same magnitude as the "bias" found in this study.
anonymous8675309
3 / 5 (4) May 02, 2017
Read the article but not the paper. Any reference to the gender of those rejecting the pull requests? Do females more often reject female's pull requests?

Echo mdblack and Ensign.

Bogus, worthless study put out there to imply exactly what some stupid ideology wants implied. The authors either have an agenda or (more likely) are pathetic puppets possessed by ideas of which they aren't even aware.

Here's some real homework: Read up on Camille Paglia and Jordan B. Peterson. Maybe I'm just possessed by their ideas but at least I have an idea of who's pulling my strings.
PTTG
1.8 / 5 (5) May 02, 2017
The differences here were -4.1% for all female pull requests, -3% for identifiable female pull requests, and +5% for non-identifiable female pull requests.

Those are pretty small differences.

I suspect that if you compared two subsets of the data with some obviously irrelevant characteristic such as men with beards and men without beards, you'd find the same statistical fluctuations.

If we were studying particle physics this large an effect would be relevant, but people are more complex than particles. A 3, 4, or 5% variation in behavior could be mostly random chance.

Opinion polls taken with samples of 1000 people usually have margins of error of about the same magnitude as the "bias" found in this study.


Do the study then, and tell us.
MarsBars
3 / 5 (6) May 03, 2017
Stop trying to find bias at every nook and cranny.

Who needs to search in nooks and crannies when it's out there in the open for all to see?
Dingbone
May 03, 2017
This comment has been removed by a moderator.
rrrander
3 / 5 (4) May 03, 2017
Just STFU and do your jobs, you losers with persecution complexes. Everyone today is thoooooo thensitive!!

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.