Early stage meta open source AI translation tool that works in 200 languages

Social media conglomerate Meta has created a single AI model that can be translated into 200 different languages, including many that are not supported by current commercial tools. The company is open sourcing the project in hopes that others will build on its work.

The AI ​​model is part of an ambitious R&D project by Meta to create a so-called “universal speech translator”, which the company believes is important for growth across its many platforms – from Facebook and Instagram to developing domains like VR and AR. Machine translation not only allows Meta to better understand its users (thus improving the ad systems that generate 97 percent of its revenue), but could also be the foundation of a great app for future projects like his augmented reality glasses

Machine translation experts told The edge that Meta’s latest research was ambitious and thorough, but noted that the quality of some of the model’s translations would likely be well below that of better-supported languages ​​such as Italian or German.

“The main contribution here is data,” Professor Alexander Fraser, an expert in computational linguistics at LMU Munich in Germany, told me. The edge† “What matters are 100 new languages [that can be translated by Meta’s model]†

Meta’s achievements stem, somewhat paradoxically, from both its scope and focus of his research. While most machine translation models can only handle a handful of languages, Meta’s model is all-encompassing: it’s a single system that can translate in over 40,000 different directions between 200 different languages. But Meta is also interested in including “low resource languages” in the model – languages ​​with less than 1 million publicly available translated sentence pairs. These include many African and Indian languages ​​that are not usually supported by commercial machine translation tools.

Meta AI researcher Angela Fan, who worked on the project, shared: The edge that the team was inspired by the lack of attention for languages ​​with fewer resources in this area. “Translation doesn’t even work for the languages ​​we speak, so that’s why we started this project,” Fan says. “We have a motivation for inclusion of something like – ‘what does it take to produce translation technology that works for everyone’?”

Fan says the model, described in a research paper here, is already being tested to support a project that will help Wikipedia editors translate articles into other languages. The techniques developed in creating the model will soon also be integrated into Meta’s translation tools.

How do you rate a translation?

Translation is a difficult task at the best of times, and machine translation can be notoriously unreliable. When applied widely across Meta’s platforms, even a small number of mistakes can produce disastrous results, such as when Facebook a message from a Palestinian man mistranslated from ‘good morning’ to ‘hurt them’, leading to his arrest by the Israeli police.

To evaluate the quality of the output of the new model, Meta created a test dataset consisting of 3001 sentence pairs for each language covered by the model, each translated from English to a target language by someone who is both a professional translator and a native speaker. is.

The researchers ran these sentences through their model and compared the machine translation to the human reference sentences using a benchmark common in machine translation known as BLEU (which stands for BilEnglish Etaxation youteaching).

BLEU allows researchers to assign numerical scores that measure overlap between sentence pairs, and Meta says the model yields a 44 percent improvement in BLEU scores for supported languages ​​(compared to previous state-of-the-art work). But as is often the case in AI research, assessing progress against benchmarks requires context.

Although BLEU scores allow researchers to determine the family member advancement of various machine translation models, they do not provide absolute measure of software’s ability to produce human-quality translations.

Remember: Meta’s dataset consists of 3001 sentences and each one has only been translated by one person. This provides a basis for assessing translation quality, but the total expressive power of an entire language cannot be captured by such a small piece of real language. This problem is in no way limited to Meta – it is something that affects all machine translation work and is especially acute when assessing languages ​​with few resources – but it shows the magnitude of the challenges facing the field.

Christian Federmann, a principal research manager working on machine translation at Microsoft, said the project as a whole was “commendable” in its desire to expand the scope of machine translation software to less-covered languages, but noted that BLEU scores by themselves give only a limited measure of output quality.

“Translation is a creative, generative process that can result in many different translations that are all equally good (or bad),” Federmann said. The edge† “It is impossible to give general levels of ‘BLEU score goodness’, as these depend on the test set used, the reference quality, but also inherent characteristics of the language pair under investigation.”

Fan said that the BLEU scores were also supplemented with human evaluation, and that this feedback was very positive and garnered some surprising responses as well.

“A really interesting phenomenon is that people who speak languages ​​that have few resources often have a lower bar for translation quality because they don’t have any other resource,” said Fan, who is a speaker of a low-resource language herself. Shanghainese† “They’re super generous, so we should really go back and say ‘hey, no, you have to be very precise, and if you see a mistake, call him.'”

The Power Imbalances of Business AI

Working on AI translation is often presented as an unequivocal good, but creating this software poses particular difficulties for speakers of low-resource languages. For some communities, Big Tech’s focus is: just not welcome: they don’t want the tools necessary to preserve their language in the hands of anyone but their own. For others, the problems are less existential, but more focused on quality and influence.

Meta’s engineers explored some of these questions by interviewing 44 speakers of low-resource languages. These interviewees highlighted a number of positive and negative effects of opening their language to machine translation.

For example, a positive point is that speakers with such tools have access to more media and information. They can be used to translate rich resources such as English Wikipedia and educational texts. However, if low-resource speakers consume more media generated by speakers of better-supported languages, this may reduce incentives to create such material in their native language.

Balancing these issues is a challenge, and the issues we encountered even within this recent project show why. For example, Meta’s researchers note that of the 44 low-resource speakers they interviewed to research these questions, the majority of these interviewees “were immigrants living in the US and Europe, and about a third of them identified as technical workers.” – meaning their perspectives are likely to be different from their home community and biased from the start.

Professor Fraser of LMU Munich said the research was nevertheless certainly conducted “in a way that increasingly involves native speakers” and that such efforts were “commendable”.

“Overall, I’m glad Meta did this. More of this from companies like Google, Meta and Microsoft, all of which have a lot of work in machine translation with few resources, is great for the world,” said Fraser. “And of course some of the thinking behind why and how to do this also comes from academia, as well as the training of most of the researchers mentioned.”

Fan said Meta tried to avoid many of these social challenges by broadening the expertise they consulted on the project. “I think when AI develops, it’s often very technical — like, ‘Okay, where are my computer science PhDs? Let’s get together and build it, just because we can.’ But actually we were working with linguists, sociologists and ethicists for this,” she said. “And I think this kind of interdisciplinary approach focuses on the human issue. Like, who wants this technology built? How do they want it to be built? How are they going to use it?”

Just as important, Fan says, is the decision to open source as many elements of the project as possible — from the model to the evaluation data set and training code — that should help redress the power imbalance inherent in a company working on such an initiative. Meta also offers subsidies to researchers who wish to contribute to such translation projects, but are unable to fund their own projects.

“I think that’s very, very important because it’s not like one company will be able to solve the machine translation problem holistically,” Fan says. “It’s everyone — worldwide — and that’s why we’re really interested in supporting this kind of community effort.”

Leave a Comment

Your email address will not be published.