Do you want to translate English content into Simplified Chinese? You may be tempted to use one of the many free translation software solutions, such as Bing Translator, Google Translate, AI-powered Deepl or Baidu Fanyi. But are they really reliable and accurate?
To answer that question, we tested all these services plus some of the newest generative AI tools: chatGPT with GPT-3.5 (standard model), GPT-4 (most advanced model) and Baidu ERNIE bot.
To test their performance, we gave them the same task: translating an English text into Simplified Chinese. We also got a native Chinese speaker to do the same task. In this article, we want to reveal the results of our test and show you how each translation option fared. Get ready for an insightful comparison!
For our test, if the translation of English text to Chinese is just effective as Transcreation to Chinese, we tested different Translation SaaS offers against a human native Chinese speaker.
The Chinese Translation Tools we tested
- Google Translate
- Bing Translator
- Deepl.com
- Baidu Fanyi (Baidu Translate)
- Open AI chatGPT (3.5) prompted to translate
- Open AI chatGPT (4) prompted to translate
- Baidu ERNIE bot prompted to translate
- Human Native Chinese speaker
Google Translate, Bing Translator, Deeepl.com, and Baidu Fanyi are online tools, with a pretty straightforward user interface: you input your original text into a field on the left side, choose your target language, and receive the translated version on the right side.
For chatGPT we wanted a similar approach, so we prompted “Please translate the following text from English to Mandarin Chinese for Mainland China in Simplified Chinese: …”.
For ERNIE bot being very similar to chatGPT from the idea, we did the same: Only we translated the prompt first from English to Chinese. Although ERNIE bot is trained as a bilingual language model and understands both Chinese and English, we assumed that being a Chinese model, we maybe get better results if the prompt itself would be Chinese: Of course, the text that we wanted to be translated to Chinese had to be provided in English, otherwise, it wouldn’t have made sense.
OK, I do not want to let you wait and read too much (if you are interested in actually reading the translations, the comments, and detailed scores, head over to the translation vs. transcreation article). Here is the scoring board – here is the list sorted from best to worst (score):
It came as no surprise that the human Chinese native speaker wrote the best Chinese translation text, avoiding pitfalls with too-literal English translations.
However, I was surprised to see my favourite translation tool Deepl.com on the bottom of the list.
It wasn’t unexpected to find Baidu in second place – though not its Fanyi service, but rather the AI-driven ERNIE bot. This large language model is presently one of the biggest public models trained on a huge amount of Chinese data points.
Google Translate, beloved by many around the world, came in just behind OpenAI’s ChatGPT with its GPT-3.5 model – which did surprise me, as I’d have expected the GPT-4 model to do better than 3.5.
In short, the best Chinese translations come from a human native speaker, with AI models like Baidu’s ERNIE bot and Open AI’s GPT-3.5 close on their heels. Google Translate is not far behind either, and the last positions are taken by Bing Translator and Deepl.com.
Disclaimer: I will personally stick to my favorite tool Deepl.com for quick translations to Chinese. It is fast, convenient and my colleagues seem to understand quite well, which message I want to get across. But for serious content like for a big Enterprise website, I would not rely on translation at all, but choose transcreation instead.