Technology Economist Susan Athey Adds DOJ Role to Her Multidimensional Career

After winning awards for technical progress in economic theory and industrial organization, he was chief economist at Microsoft, conducted original research combining machine learning with econometric modeling, pioneered the new field of technical economics, helped develop establishing and promoting Stanford HAIand starting the Golub Capital Social Impact Lab at Stanford, Susan Athey will now try a new hat as chief economist in the antitrust division of the United States Department of Justice (DOJ).

The broad nature of her work and career has garnered her respect in multiple academic fields. “Susan is a force of nature. She goes from machine learning to business strategy to technology policy to social impact, producing deep ideas at every turn,” says Jonathan Levinthe Philip H. Knight Professor and Dean at Stanford Graduate School of Businesswhere Athey is the endowed professor of economics of technology.

While Athey will continue her GSB tenure part-time as she steps into her new government role, the shift in focus provides an opportunity to reflect on the significant impact she has had throughout her career and while at Stanford.

A role model for HAI

Few researchers are a better example of the multidisciplinary mindset fostered by Stanford HAI then Athey. Her tendency to immerse herself in different fields dates back to her college days at Duke University, where she graduated in 1991 with a triple major in economics, mathematics, and computer science.

She then became a professor of economics and business at MIT, Harvard University, and from 2013 at Stanford, but even in economics Athey’s interests were diverse: in 2007 she won the prestigious John Bates Clark Medal for her contributions to multiple subfields, including industrial organization, microeconomic theory, and econometrics.

But it was during a leave of absence from academia to serve as chief economist at Microsoft from 2008 to 2013 that Athey made a surprising connection between her passion for economics and the tools of AI and machine learning.

They already knew that digitization and technical platforms were going to play an important role in the economy, and that search engines were about to play an inordinate role. She also knew that the research community was just beginning to address questions about designing digital markets and what healthy competition looked like in those markets, and she was excited to help develop that research.

But once she started working at Microsoft, Athey also discovered something she didn’t expect: the potential for machine learning to address economic problems. The creators of the Bing search engine conducted experiments in ways economists could only dream of. They simultaneously ran thousands of randomized A/B tests and asked large numbers of “what if” questions to better understand which search results should come out on top and how to run auctions to set ad prices on a search page. By comparison, she says, economists would typically conduct one experiment per year.

“Microsoft used an artificial intelligence system composed of hundreds of algorithms all working together to create a search results page,” she says. “That was something new.”

Until then, in the field of economics, data mining and machine learning were pejorative terms for a less sophisticated form of statistics. “They were seen as a mechanical exercise to separate cats from dogs,” she says. But at Microsoft, Athey saw an opportunity to combine the computational advances of predictive machine learning with statistical theory so that researchers could better understand causal effects not only in business applications like the search engine, but also in the social sciences and economics. It was a revelation that launched her in a new direction of research and helped define her as one of the early tech economists.

Machine learning and causal effects

From her experience at Microsoft, Athey realized that the insights of predictive algorithms could be harnessed in new ways by combining them with recent advances in econometrics and statistics. For example, machine learning algorithms can be adapted to answer cause-and-effect questions in economics, such as what will happen if we change the minimum wage? Expand immigration policy? Increase prices? Want to merge two companies? “Predictive machine learning can’t just solve these questions, but it can help,” she says.

For example, Athey has used machine learning to look at the impact on consumers of personalized pricing, a form of price discrimination in which consumers are charged different prices based on their willingness to pay. Traditional economic methods would provide aggregated solutions to that problem, she says. They might study one product category at a time, taking into account the demand for, say, different brands of yogurt or towels. By applying machine learning methods to historical consumer purchase data, Athey’s research group can estimate personalized consumer preferences for multiple products simultaneously.

In turn, by building these predictive models of consumer choice, researchers can ask even bigger questions about things like what happens to prices when you apply a tariff, or if generic drugs come to market. “As input for answering these questions, we want to understand how consumers make choices,” Athey says. And machine learning provides that input in a way that allows researchers to do this work more efficiently, on a larger scale, and in a more personalized way. “Assuming everyone is the same, you get different answers than assuming people have different preferences,” she says.

A pioneer in the field of technical economics

Athey’s position of chief economist at Microsoft ended in 2013, but her tenure there defined her as one of the first people to be considered a “tech economist.” It is a field that she has since helped establish as an independent discipline by convening early conferences in the field and guiding countless students along that career path.

“Now tech economists hold an annual conference that attracts about 800 participants,” she says. “And we have a specialized labor market, because being a technical economist is a separate profession that people can practice.”

Athey also has written about what it means to be a tech economist. “It’s partly a career, but it’s also a combination of different fields of study,” she says. Tech economists are studying the impact of digitalization on the economy, considering market design, privacy, data security, fairness, competition policy and more, she says. “They also help create and analyze business models and competitive strategy, and they connect the models with data to guide decisions.”

Promoting AI for Good

At Microsoft, Athey not only witnessed an unexpected deep dive into machine learning and AI, but also witnessed the challenges these technologies presented: ethical and legal issues, First Amendment issues, fairness and bias, privacy and copyright, and the prevalence of unforeseen consequences as people manipulated or played the system in response to market shifts or new rules.

Because of these observations, Athey developed a desire to influence the ways machine learning and AI would evolve in the world. When she returned to academia full-time, her first steps in that direction included helping plan the launch of HAI and then becoming one of HAI’s founding associate directors. “Stanford HAI was really made to address these issues,” she says. “We want to make AI beneficial to humans and we want to avoid all these unintended consequences.”

Athey also wanted to translate the successful applications of AI from the for-profit sector to the social impact sector. This urge led her to Golub Capital Social Impact Lab at Stanford. “We are bringing the technical toolkit to social impact applications,” she says. For example, the Social Impact Lab has conducted case studies of digital education technology to improve student learning; developed and implemented approaches to targeting educational messages to maximize farmer engagement; developed and evaluated digital tablet applications that guide nurses in guiding patients; and developed methods to prioritize candidates for clinical trials of COVID-19 drugs.

Connecting the dots to the DOJ

Applying machine learning to interesting social problems in the Golub Capital Social Impact Lab is a bottom-up approach to driving change, Athey says. In contrast, in her new job as chief economist in the DOJ’s antitrust division, she will do her best to tackle the problems of the digital economy from above. “Government laws and policies affect everything from how competition works to what mergers go through, to what investments people make,” Athey says.

By moving to the DOJ, Athey hopes to continue many of HAI’s efforts to help governments adapt to an era of rapidly changing technology, particularly around the use of data in industry and government. “Because technology like artificial intelligence moves so fast, it’s hard for governments to keep up,” she says. “We need to figure out how all branches of government will be prepared to take us through another era.”

It’s a great time for Athey to try her hand at government work, Levin says. “At a time when technology is on the rise and fostering competition is essential, I can’t think of anyone I’d rather have at the DOJ than Susan.”

Stanford HAI’s mission is to advance AI research, education, policy and practice to improve the human condition. Learn more

Leave a Comment

Your email address will not be published. Required fields are marked *