Quality prediction, to decide which machine translations to look at?#192

We have too much content and not enough bandwidth to look at every the machine translation, in all languages.

If Weglot integrated a machine translation quality prediction API like integrated machine translation APIs, we could use it to work on only those few machine translations that are below 10% quality.

So we’d get “hybrid” translation - part human, part machine. https://machinetranslate.org/hybrid-translation

It looks like Bablic will be adding quality prediction (“quality estimation”) for this exact purpose: https://resources.unbabel.com/blog/elevating-web-localization-and-translation-capabilities-with-bablic

Full disclosure:

I’m one of the co-founders of ModelFront, the main provider of machine translation quality prediction, but we use Weglot for modelfront.com and we actually need this to have any hope of translating our content in decent quality in 3 languages, let alone 10.

Thanks Adam, interesting indeed. We also think that the translations that you want to review in “priority” are not necessarly only the ones with low score but also that are the “most viewed” across the website. For example, it’s more important to edit a translations that is in the header of all pages even if it’s 70% than a 10% that is on one page.
We are working on a way to show you the “most viewed” translations.
For what you suggest, it’s also interesting so I will check with the machine transalation providers if we can get that “score” information within the API response.

Exactly - priority for human editing is a function of value first, and then quality.

Getting a number for the value requires not AI, but good ol’ business logic from Weglot and/or the Weglot user - e.g. titles have more value than some sentence buried deep in an article, best-selling items have more value than long-tail items, certain languages are more important for certain companies than others…

(It’s frankly insane that most translation workflows totally ignore differences in value within a content type. In the complex incumbent TMSes, it’s not easy, let alone standard, to use a different workflow for different types of segments in each document, like titles and descriptions.)

Then quality prediction like ModelFront can help with getting a number for the quality.

Together, Weglot and ModelFront can make quality translation e.g 20x to 100x more efficient. (If 10-25 in 100 segments are actually valuable, and 1-5 in those 10-25 actually require human review and editing.)

As far as the machine translation providers, those integrated in Weglot - Google Translate, DeepL, Microsoft Translator do not provide any type of score in the API.

(Google does provide a machine translation quality prediction score, but not in the Google Translate API, only as a feature in Google Cloud Translation Hub, its TMS.)

It’s unlikely that they’d ever provide what Weglot users need - an independent score, that could be used for translations from multiple APIs and even on the human-edited translations.

Here’s a relatively current list of machine translation quality prediction providers.