AWEJ for Translation & Literary Studies, Volume 7, Number 3. August 2023 Pp.19- 34

Comparing the Performance of Google Translate and SYSTRAN on Arabic Lexical Ambiguity

Department of English, College of Arts and Sciences, Prince Sattam Bin AbdulAziz University, Wadi ad-Dawasir, Saudi Arabia


Machine translation systems face many challenges when dealing with Arabic lexical ambiguity, which affects the quality and accuracy of their translations. This study investigates how two popular MT systems, Google Translate and SYSTRAN, handle three problematic linguistic features of the Arabic language: homonyms, heteronyms, and polysemes. A test suite was designed to include sentences that contain these features in different contexts and domains. The translations produced by the two MT systems were evaluated by four independent evaluators for intelligibility and accuracy using a four-point scale. Results showed that both MT systems struggled with the three chosen linguistic features, with average scores below 40%. Google Translate outperformed SYSTRAN in almost every sentence in the test suite. Both systems scored better in intelligibility than in accuracy. Heteronyms proved to be the most challenging for both MT systems due to the unique design of Arabic discretization, which is not yet recognized by translation systems. This study contributes to the field of machine translation by providing a comprehensive analysis of Arabic lexical ambiguity and its impact on MT quality, as well as suggesting possible ways to improve MT systems for Arabic-English translation.

Cite as:

Aldawsari, H. A. H. (2023). Comparing the Performance of Google Translate and SYSTRAN on Arabic Lexical Ambiguity.  Arab World English Journal for Translation & Literary Studies 7 (3):   DOI: http://dx.doi.org/10.24093/awejtls/vol7no3.2


Abbad, H., & Xiong, S. (2020). Multi-components System for Automatic Arabic Discretization. Lecture Notes in Computer Science, 12035,341–355. https://doi.org/10.1007/978-3-030-45439-5_23

Ali, M. K. (2020). Quality and Machine Translation: An Evaluation of Online Machine Translation of English into Arabic Texts. Open Journal of Modern Linguistics, 10(05), 524–548. https://doi.org/10.4236/ojml.2020.105030

Almahasees, Z. (2021). Analysing English-Arabic Machine Translation: Google Translate, Microsoft Translator and Sakhr.Routledge. https://doi.org/10.4324/9781003191018

Almeman, K.,& Lee, M. (2014). Morpheme-based language models for improving the speech recognition of Arabic dialects. In the 5th International Conference on Arabic Language Processing (CITALA’14) (pp. 49-56). http://www.citala.org/papers/paper_19.pdf

Alotaibi, M. (2019). Evaluating machine translation systems for Arabic language. Journal of King Saud University – Computer and Information Sciences, 31(4), 46-55.https://doi.org/10.1016/j.jksuci.2018.07.001

Alqudsi, A., Omar, N., & Shaker, K. (2014). Arabic machine translation: A survey. Artificial Intelligence Review, 42(4), 59–72. https://doi.org/10.1007

AlShaikhli, M. (2022). Problems of Machine Translation Systems in Arabic. Journal of Language Teaching and Research, 13(4), 755–762. https://doi.org/10.17507/jltr.1304.08

Alsohybe, N., Dahan, N., & Ba-Alwi, F. (2017). Machine-translation history and evolution: Survey for Arabic-English translations. Current Journal of Applied Science and Technology, 23(4), 1-19. https://doi.org/10.9734/cjast/2017/36124.

Arnold, D., Balkan, L., Meijer, S., Humphreys, R. L., & Sadler, L. (1994). Machine Translation: An Introductory Guide.Manchester: NCC Blackwell.

Aman, A., Wu, L., & Lu, C. (2020). Towards a hybrid machine translation system for low-resource languages: The case of Uyghur-English. Machine Translation, 34(1), 3-28. https://doi.org/10.3390/info12030098

Azmi, S. (2013). Homonymy in Jordanian colloquial Arabic: A semantic investigation. ELLS, 3(3), 69-76. https://doi.org/10.5539/ells.v3n3p69

Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to Align and Translate.International Conference on Learning Representations. https://arxiv.org/pdf/1409.0473

Bergeron, D. (1990). Heteronyms. English Today, 6(04), 39-44. https://doi:10.1017/S0266078400005150

Chitu, A. (2007). Google Switches to its Own Translation System| Google Operating System. Retrieved from: http://googlesystem.blogspot.co.uk/2007/10/google-translate-switches-to-googles.html

Creswell, J.W. and Creswell, J.D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE Publications, Inc.https://doi.org/10.1002/nha3.20258

Crystal, D. (2008). A dictionary of linguistics and phonetics (6th ed.).Oxford, UK: Blackwell. http://dx.doi.org/10.1002/9781444302776

Dictionary.com (2023). The definition of heteronym.Retrieved from: http://dictionary.reference.com/browse/heteronym.

Hutchins, W. J. (2001). Early Years in Machine Translation. In Amsterdam studies in the theory and history of linguistic science. John Benjamins Publishing Company. https://doi.org/10.1075/sihols.97

Hutchins, W., & Somers, H. (1992). An introduction to machine translation.London: Academic Press.

Moorkens, J., Castilho, S., & Gaspari, F. (2019). Translation Quality Assessment: From Principles to Practice. Springer.https://doi.org/10.1007/978-3-319-91241-7

Panman, O. (1982). Homonymy and polysemy. Lingua, 58(1-2), 105-136.

Poibeau, T. (2017). Machine Translation. In The MIT Press eBooks. The MIT Press. https://doi.org/10.7551/mitpress/11043.001.0001

Rivera-Trigueros, I. (2021). Machine Translation Systems and quality assessment: A systematic review. Language Resources and Evaluation, 56(2), 593–619. https://doi:10.1007/s10579-021-09537-5

Saldanha, G., & O’Brien, S. (2013). Research methodologies in translation studies. Abingdon, Oxon: Routledge. https://doi.org/10.4324/9781315760100

Sennrich, R., Haddow, B., & Birch, A. (2016). Neural machine translation of rare words with Subword units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1715–1725. https://doi:10.18653/v1/p16-1162

Sommerlad, J. (2021). Google Translate: How does the multilingual interpreter actually work? The Independent. Available at https://www.independent.co.uk/tech/how-does-google-translate-work-b1821775.html

Soudi, A., Farghaly, A., Neumann, G., & Zbib, R. (2012). Challenges for Arabic Machine Translation. John Benjamins Publishing. https://doi.org/10.1075/nlp.9

Systransoft.com (2023). SYSTRAN: 50 Years of MT Innovation | SYSTRAN Translation Technologies. Retrieved from: https://www.systransoft.com/systran/translation-technology/systran-50-years-of-mt-innovation/

Turovsky, B. (2016). Found in translation: More accurate, fluent sentences in Google Translate. Google.Retrieved from: https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/

Yule, G. (2010). The study of language (4th ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511757754


Hamad Abdullah H Aldawsari is an Assistant Professor of Translation Studies at the College of Arts and Sciences, Prince Sattam bin Abdulaziz University (PSAU), Wadi Addawasir, Saudi Arabia. He completed his MA and PhD at the University of Birmingham, UK. His research interests include Translator Studies & Linguistics. ORCID ID https://orcid.org/0009-0001-2041-9760