Neither DALL-E 2 nor Imagen is currently available to the public. Yet they share a problem with many others who already are: They can also produce disturbing results that reflect the gender and cultural biases of the data they’re trained on — data that includes millions of images pulled from the Internet.
The bias in these AI systems poses a serious problem, experts told CNN Business. The technology can perpetuate hurtful prejudices and stereotypes. They are concerned that the open nature of these systems — making them adept at generating all kinds of images from words — and their ability to automate image creation means they can automate biases at scale. They can also be used for nefarious purposes, such as spreading misinformation.
“Until that damage can be prevented, we’re not really talking about systems that can be used in the open, in the real world,” said Arthur Holland Michel, a senior fellow at the Carnegie Council for Ethics in International Affairs, who conducts research. towards AI and surveillance technologies.
Documenting Bias
Lama Ahmad, policy research program manager at OpenAI, said researchers are still learning how to measure bias in AI, and that OpenAI can use what it learns to adjust its AI over time. Ahmad led OpenAI’s efforts to collaborate with a group of outside experts earlier this year to better understand issues within DALL-E 2 and provide feedback so it can be improved.
Google has turned down a request for an interview from CNN Business. In their research paper introducing Imagen, the Google Brain team members wrote that Imagen appears to code for “various social biases and stereotypes, including a general preference for generating images of people with lighter skin tones and a tendency to use images of different occupations.” to align with Western gender stereotypes.”
The contrast between the images these systems create and the thorny ethical issues is stark to Julie Carpenter, a research scientist and fellow in the Ethics and Emerging Sciences Group at California Polytechnic State University, San Luis Obispo.
“One of the things we have to do is we have to understand” AI is very cool and can do some things really well. And we should partner with it,” Carpenter said. “But it’s an imperfect thing. It has its limitations. We have to adjust our expectations. It’s not what we see in the movies.”
Holland Michel is also concerned that no amount of security can prevent such systems from being used maliciously, noting that deepfakes — an advanced application of AI to create videos that claim to show someone doing or saying something they aren’t really have done or said – were initially used to create fake pornography.
“It follows that a system orders of magnitude more powerful than those early systems could be orders of magnitude more dangerous,” he said.
Hint of bias
Because Imagen and DALL-E absorb 2 words and spit out images, they had to be trained with both types of data: pairs of images and related text captions. Google Research and OpenAI filtered out harmful images such as pornography from their datasets before training their AI models, but given the large size of their datasets, it is unlikely that such efforts will capture all such content, and the AI systems will not be able to to produce harmful results. In their Imagen paper, Google researchers pointed out that, despite filtering out some of the data, they also used a massive dataset known to contain pornography, racist utterances, and “harmful social stereotypes.”
Filtering can also lead to other problems: For example, women are more represented than men in sexual content, so filtering out sexual content also reduces the number of women in the dataset, Ahmad said.
And really filtering these datasets for bad content is impossible, Carpenter said, because people are involved in decisions about labeling and removing content — and different people have different cultural beliefs.
“AI doesn’t understand that,” she said.
Some researchers are thinking about how it might be possible to reduce biases in these kinds of AI systems, but still use them to create impressive images. One possibility is to use less, rather than more, data.
Alex Dimakis, a professor at the University of Texas at Austin, said one method consists of starting with a small amount of data — say, a photo of a cat — and cropping it, rotating it, making it a mirror image, and so on turned on, to effectively create many different images from one image. (A graduate student who advises Dimakis contributed to the Imagen research, but Dimakis himself was not involved in developing the system, he said.)
“This solves some of the problems, but it doesn’t solve other problems,” Dimakis said. The trick in itself won’t make a dataset more diverse, but its smaller scale allows people working with it to pay more attention to the images they record.
royal raccoons
For now, OpenAI and Google Research are trying to keep the focus on cute photos and away from images that could be distracting or show people off.
There are no realistic-looking images of people in the vivid preview images on Imagen’s online project page of DALL-E 2, and OpenAI says on its page that it has “used “advanced techniques to avoid photorealistic generations of faces of real individuals, including those of public figures.” This protection can prevent users from getting image results for, say, a prompt that tries to show a specific politician performing some sort of illegal activity.
“Especially researchers, I think it’s really important to give them access,” Ahmad said. This is partly because OpenAI wants their help in studying areas such as disinformation and bias.
Still, as Google Research noted in its Imagen paper, “Even when we focus generations away from people, our preliminary analysis indicates that Imagen encodes a range of social and cultural biases in generating images of activities, events, and objects.”
A hint of this bias can be seen clearly in one of the images Google posted on its Imagen webpage, created from a prompt that reads, “A wall in a royal castle. There are two paintings on the wall. One on the left is a detailed oil painting of the royal raccoon king, on the right is a detailed oil painting of the royal raccoon queen.”
The picture is just that, with paintings of two crowned raccoons – one wearing what appears to be a yellow dress, the other in a blue and gold jacket – in ornate gold frames. But as Holland Michel pointed out, the raccoons wear western-style royal outfits, though the prompt didn’t specify anything about what they should look like except look “royal.”
Even such “subtle” displays of bias are dangerous, Holland Michel said.
“By not being blatant, they’re really hard to catch,” he said.