Google tries to reduce pushback bias in reviewing developer software code

close up programmer man hand typing on keyboard on computer desktop for input coding language to software for fix bug and malfunction of system in operating room, technology concept

Image: Getty Images/iStockphoto

Google is trying to make the software development code review process more equitable after discovering that women, Black+, Latinx+, and Asian+ developers are more likely to experience code change pushback than white, male engineers. It also found that older developers were more likely to get pushback than younger developers.

Google revealed details about code review pushback in its Study “The Pushback Effects of Race, Ethnicity, Gender, and Age in Code Review”published in the computer magazine Communications of the ACM.

The study looked at the day-to-day experiences of traditionally underrepresented engineers in tech.

TO SEE: Software skills will get you far, but you don’t have to be a programmer to get big in technology

The study found that “excess pushback” costs Google more than 1,000 additional engineer hours per day, or about 4% of the estimated time engineers spend responding to reviewer comments. The costs were borne by non-white and non-male engineers, it found.

“Code review is essentially a decision-making process, where reviewers have to decide if and when a code change is acceptable; thus, code review is subject to human bias,” noted Google researchers Emerson Murphy-Hill, Ciera Jaspan, Carolyn Egelman and Lan Cheng. .

They found that women at Google were 21% more likely to receive pushback than men during code review. Also, Black+ developers had 54% higher odds than White+ developers; Latinx+ developers had 15% higher odds than White+ developers; Asian+ developers had 42% higher odds than White+ developers; and older developers were more likely to get pushback than younger developers.

Before the study, the authors mistakenly thought Asian developers would face less opposition because of stereotypes, but the study showed otherwise. “We hypothesize that those who identify as Asian will be viewed more positively than those who identify as White because Asians are stereotyped as having higher role congruence in tech fields,” they noted.

For context, the researchers explained that at Google, code changes must be reviewed by at least one other engineer. Most reviewers are on the same team as the author. Authors can choose or be assigned their reviewers through the code review tool, which Google calls Critique.

“The code review tool gives authors and reviewers the opportunity to learn about each other, including their full names and photos (more in the supplemental material),” they explained.

To address these code review issues, Google has researched the effectiveness of anonymous code reviews, which it hopes will narrow the pushback gaps faced by developers from different demographics.

Last year, it tested the idea by asking 300 developers to do their code reviews without the author’s name at the top of the report. It did this using a browser extension that removed the author’s name. A potential problem with anonymous code reviews is when the reviewer needs to contact the author for complex discussions.

TO SEE: Upgrade your job: 5 ways to get that career boost

All Google code is in one big repository. When a technician wants to make a change to a code, they create a “changelist”, which is similar to pull requests on GitHub that must be vetted and approved.

The results of the extension experiment showed that review times and review quality appeared consistent with and without anonymous review. They also found that for certain types of reviews, it was more difficult for reviewers to guess the author of the code.

“By continuously experimenting with anonymous code review, we hope to narrow the gaps in the pushback that developers from different demographics are facing. And through this work, we want to inspire other companies to take a closer look at their own code reviews and think about adopting anonymous author code review as part of their process,” Murphy-Hill said.

Leave a Comment

Your email address will not be published. Required fields are marked *