ArticlesMarch 2024

Assessing Arabic Proverb Translation Accuracy: A Comparative Study of ChatGPT and Google Translate with Pedagogical Applications

By Gisele El Khoury, St. Lawrence University

Gisele El Khoury

Introduction

Due to their cultural specificity, metaphorical language, contextual reliance, regional differences, and idiomatic usage, several Arabic proverbs are extremely challenging to translate. Proverb translation is regarded as difficult since it calls for linguistic and cultural expertise. According to House (2009, p.11), “Translating is not only a linguistic act; it is also a cultural one.” Some proverbs are challenging to translate directly into other languages because they frequently rely on metaphors and cultural allusions ingrained in a particular culture. Furthermore, a proverb’s meaning might change significantly depending on the context in which it is employed, making literal translations useless. Regional dialectal differences and the lack of direct translations in other languages contribute to this difficulty as well. 

AI tools can facilitate engaging activities with proverbs in the classroom, helping in translation, explanation, and contextualization. These tools enable educators to engage in a new way to let students explore proverbs and offer translations, usage contexts, and explanations. They also assist in creating interactive exercises, enhancing comprehension and cultural understanding. Given the difficulty of translating proverbs, however, we as educators need to evaluate the accuracy of these tools, which is why I undertook this investigation.

As part of this study project, I evaluated ChatGPT translation accuracy for thirty Arabic proverbs and colloquial expressions. I created a carefully selected collection of thirty popular Arabic proverbs and slang terms to ensure a variety of cultural and situational circumstances. Some of these language idioms are difficult for machine translation algorithms to interpret correctly because of their cultural value and complex meanings. They contain statements with strong cultural, ethical, and historical resonance in Arabic-speaking communities.

This article is structured into two parts. Part one consists of two phases: in phase one, I assessed ChatGPT’s ability to translate the selected proverbs and convey their intended meanings accurately. The second phase involved a comprehensive comparison between the translations generated by Google Translate and ChatGPT, evaluating how effectively and consistently these popular machine translation systems conveyed the meanings of the proverbs.

In part two, I delve into the pedagogical applications of proverbs and ChatGPT in the classroom. This examination uncovers the valuable opportunities for language learning and cultural understanding that arise from integrating proverbs and language models like ChatGPT.

Part 1: ChatGPT vs. Google Translate

Phase 1: Translating the proverbs through ChatGPT

In my analysis of thirty proverbs and colloquial phrases from the Levant, I referred to the source “Popular Proverbs by Nasser Isleem” to ensure the accuracy of their meanings. In his book, Nasser Isleem provides a unique perspective on the culture’s treasured linguistic legacy through his vast collection of more than a thousand proverbs from Palestine and the Arabic-speaking world. We used ChatGPT 3.5 and prompted it: “Translate the following Arabic proverb into English taking into consideration its cultural context.”

Below is a chart that summarizes the data, followed by an explanation of the outcomes and a detailed table with the proverbs, their translation using ChatGPT, the correct meaning, and if ChatGPT rendered them correctly.

Picture 1- Graph showing the percentage of correct proverbs - Correct - 37%; Incorrect - 53%; Can be one of the meanings - 10%
Picture 1 – Graph showing the percentage of correct proverbs
  1. “Correct” (11): For 11 of the 30 proverbs, the meanings that ChatGPT provided were accurate and aligned with the intended interpretations. 
  1. “Incorrect” (16): A substantial portion of the proverbs, precisely 16 out of 30, had meanings that were inaccurately presented or did not correspond to their intended interpretations.
  1. “Can be one of the meanings” (3): In three instances, the proverbs’ meanings could be considered one of the possible interpretations, as certain proverbs may have multiple layers of meaning or contextual variations.

This table summarizes the proverbs and the findings. Click to jump below the table.

Arabic Proverb English translation on ChatGPT Meaning as explained by ChatGPT Correct or incorrect
إسأل عن الجار قبل الدار Ask about the neighbor before the house. This proverb emphasizes the importance of getting to know your neighbors and their character before making decisions or judgments about the place or house you intend to live in. It underscores the significance of good relationships with neighbors for a harmonious living environment. Correct
إضرُب عصفورين بحجر Kill two birds with one stone. This idiom describes a situation where a single action or effort accomplishes two different objectives or solves two problems simultaneously, making it more efficient and effective. Correct
إللي بيدري بيدري و إللي ما بيدري بيقوك كف عدس He who understands understands, and he who does not understand, you can explain until you’re blue in the face. This proverb conveys that some people are receptive to understanding and learning. In contrast, others may be resistant or difficult to teach, no matter how much effort is put into explaining something to them. Incorrect

He who knows knows, and he who does not know says: “A handful of lentils.”

Meaning: Said when someone knows a secret and cannot reveal it, lest he or she cause a scandal or conflict between two parties. It can also be said to those who think that someone is angry for some insignificant reason when, in fact, the reason is quite serious.

 

حَظّه بيفلَق الحجر His luck can split a rock. This phrase is often used to describe someone fortunate, to the extent that their good fortune seems almost supernatural or extraordinary. It emphasizes that some individuals have exceptionally good luck in various aspects of their lives. Correct
تَيَّس عَبدك يا رب Guide your servant, O Lord It is a prayer or supplication expressing a desire for guidance and assistance from a higher power, in this case, from God or the Lord. Incorrect

He became so stubborn, oh Lord!
(English proverb: Stubborn as a mule)

Meaning: This applies to someone who becomes stubborn or disagreeable. The word تَيس means “Billy Goat” and the verb  تَيَّس means he became extremely stubborn.

Commonly, students will criticize failures using this word, for example: “ تَيَّس in math or  تَيَّس in Arabic!”

حامل السّلم بالعَرض Bearer of the ladder in the parade This phrase is often used to describe someone doing something unnecessary or irrelevant to the situation, similar to the English idiom “carrying coals to Newcastle.” It implies that the person’s actions or contributions are unnecessary and may be counterproductive. This could be one of the meanings.

The other meaning is a person who never agrees with group decisions and opinions. He enforces his will even if others object.

لله يرحمك يا با إللي ما علّمتني القراية

 

May God have mercy on you, oh Father, who didn’t teach me how to read This phrase is an expression of respect and remembrance for a person’s father who, for some reason, didn’t provide them with an education, particularly the ability to read. It carries a sense of longing and acknowledgment of the importance of education and the role parents play in it. Correct
حَماتك بِتحِبك

 

Your mother-in-law loves you. This phrase is often used humorously or sarcastically to suggest that a person’s mother-in-law has affection for them. Still, it can also be used genuinely to express a positive or warm relationship with one’s mother-in-law. Incorrect

This is said to someone who comes to visit another person where he will be asked to sit and partake in the food while the host says  حَماتك بِتحِبك

 

زَيّ مِسمار جُحا

 

Like a nail in Juha’s head Juha is a popular character in Arabic folklore known for his humorous and often foolish actions.  Incorrect

Correct translation: Like Juha’s nail

Meaning: This proverb describes someone who employs different excuses to visit someone’s house as often as possible.

خَيرها بِغيرها Better than it with something else This phrase is often used to convey that replacing or letting go of something or someone is better, as better alternatives are available. It implies that one should seek or accept a superior option or solution rather than sticking with what is currently in place. Incorrect

Correct translation: Better luck next time.

Meaning: This proverb encourages people to keep trying where they previously failed.

 

وافَقَ شَنٌّ طَبَقة

 

A tray has agreed with its flatbread. This phrase describes a situation where two things are a perfect match or go together seamlessly, like a tray and the flatbread that fits on it. It’s used to express that two things or people are perfectly suited for each other or in harmony. Correct
دايرَة على حَلّ شَعرها

 

Trying to solve her hair This phrase describes a person who is overly preoccupied with their appearance or is making excessive efforts to fix their hair or grooming. It can also be used more broadly to refer to someone overly concerned with their appearance and trying to make themselves look perfect. Incorrect

Correct translation: She does what she desires.

Meaning: This proverb describes an immodest woman who does not acknowledge religion or social customs and becomes the subject of gossip.

الدنيا قايمِة قاعدِة

 

The world is built on opposites. This phrase reflects the idea that life is full of contrasts and contradictions. It implies that there are ups and downs, good and bad, and various opposing elements in the world, and these opposing forces and elements make the world balanced and complete. It’s a philosophical expression that acknowledges the duality and complexity of life. Incorrect

Correct translation: It is up and down (complete chaos)

Meaning: A proverb describing a state of confusion and turmoil.

دِق عَ الخشب

 

Knock on wood This phrase expresses a superstitious belief that one should knock on wood to ward off bad luck or prevent a positive situation from turning negative. It’s often used when someone is talking about something they hope will continue to go well, and they want to avoid jinxing it. The act of knocking on wood is a common superstition in many cultures. Correct
راحت السَّكرة و إجِت الفَكرة The drunkenness has gone, and the thought has come This phrase conveys that after intoxication or carelessness, a person becomes sober or serious and starts thinking more clearly or sensibly. It’s often described as a shift from indulgence or distraction to a more focused and thoughtful state.  
رَفِع عَتَب

 

Raise an objection or Express disapproval. It is used when someone raises concerns or objections, indicating dissatisfaction or disagreement with a particular situation, decision, or action. It is a way of expressing criticism or dissent Incorrect

.. so as not to be blamed.

Meaning: This may describe someone who executes a task not because they want to do it or feel responsible for doing it but because they want others to speak well of their work or at least not criticize them for doing a bad job.

رِجلُه بِالقَبِر

 

His foot is in the grave. This expression indicates that someone is very close to death or is in a critical condition, typically due to illness or old age. It’s a metaphorical way of saying that the person’s life is hanging by a thread, and they may not have much time left. Correct
زَيّ الأطرَش في الزَّفِة

 

Like a blind person at a wedding This phrase often describes someone oblivious or unaware of their surroundings, context, or events. It implies that the person is in a situation where they cannot fully appreciate or understand what is happening, just as a blind person at a wedding cannot see or comprehend the festivities. Correct
سَبع البورومبو

 

—– This phrase does not have a specific meaning in Arabic, and it appears to be a sequence of words that do not form a coherent phrase or expression. Incorrect

Correct translation: Boromboh lion.

Meaning: Boromboh is like calling someone Don Quixote. It describes a coward who pretends to be more than he or she is.

سَنّ سنانُه

 

He has sharpened his teeth. This expression is used metaphorically to indicate that someone is well-prepared, ready, or skilled in dealing with a particular situation or challenge. It suggests that the person has acquired the necessary knowledge or experience to handle a task effectively.

Top of Form

Bottom of Form

 

Correct
زَيّ الزِفِت

 

Like asphalt This phrase describes something very sticky or difficult to deal with, much like asphalt, a thick and sticky substance. It implies that the situation or thing in question is problematic or challenging. Not 100% correct

Translation: Like tar

Meaning: This is used to like the English expression “I don’t feel well.”

شِحّاد و مشارَط

 

stingy and nitpicky This phrase describes someone who is not only tight-fisted with their money but also excessively meticulous or overly concerned with small details, especially in financial matters. It implies that the person is not generous and is overly critical or picky about spending or sharing resources. Incorrect

Correct translation: A beggar and he bargains.
English proverb: Beggars cannot be choosers.

Meaning: This proverb refers to a needy person complaining about what he received for free.

شو جاب لَجاب!

 

What did he bring for the crow? This phrase is often used to express disapproval or skepticism when someone has gone to a great effort or trouble, but their actions seem unnecessary or fruitless. It’s a way of questioning the value or purpose of someone’s actions when the result does not seem to justify the effort. Incorrect

Correct translation: There is no comparison between what he brought and the other one.

English proverb: There is no comparison.

Meaning: Said to show the vast difference between two people or things where one is clearly better.

شو مَحَلُّ من الإعراب

 

What is the point of the grammatical analysis? This phrase questions the relevance or significance of discussing grammatical or linguistic details when they may not be necessary or when the main point or message is clear without such analysis. It’s a way of emphasizing a conversation or argument’s practical or essential aspects rather than getting lost in the minutiae of language analysis. Incorrect

Correct translation: What is he in terms of “the case ending markers in Arabic”?

(English expression: and who is he supposed to be?)

Meaning: Said when a person is mentioned in front of someone who does not know that person. It normally expresses contempt.

صَحّ النّوم

 

Good sleep or Sound sleep. It’s an expression used to wish someone a restful and refreshing sleep or to acknowledge that they had a good night’s sleep. Incorrect

Correct translation: Good sleep!

Meaning: Said to someone who tries to achieve something when it is clearly too late to do so.

طَق حَنَك

 

Mind your own business” or “Keep your nose out of it.” It’s often used when someone is nosy or intrusive, and you want to tell them to stay out of your affairs or not meddle in things that don’t concern them. Incorrect

Correct translation: Cracking Jaws

Meaning: This saying refers to idle people’s worthless or senseless talk.

جَوابُه تحت باطُه

 

His answer is under his doorstep. This expression describes a person who avoids giving a direct or clear answer to a question, usually by being evasive or not addressing the issue directly. It suggests that the person is not forthright or honest in their response and is trying to divert attention or responsibility. Incorrect

Correct translation: His answer is under his armpit.

Meaning: This describes a clever, witty person who quickly gives correct and suitable answers.

خالِف تُعرف

 

Differ, and you will be known. This phrase conveys that expressing your opinions, taking a different stance, or being unique will stand out and be recognized for your individuality or the distinctiveness of your views. It emphasizes the value of being true to yourself and not conforming to the expectations of others. Can be one of the meanings.

Correct translation: Be contrary, be known.

Meaning: This proverb describes someone who seeks fame and likes to show off by disagreeing with others.

عِذر أقبح مِن ذَنب

 

An excuse is uglier than a sin. This phrase implies that making excuses for wrongdoing or mistakes is worse than acknowledging the wrongdoing itself. It emphasizes taking responsibility for one’s actions rather than trying to justify or excuse them. Correct
خَلّيه يبَلِّط البحر

 

Let him pave the sea. This phrase expresses that someone is trying to accomplish something impossible or extremely difficult. It’s akin to saying, “Let him attempt the impossible” or “Let him try the most challenging task.” It is often used to convey skepticism or doubt about the feasibility of a person’s ambitions or efforts. Incorrect

Correct translation: Let him lay riles on the sea floor.

Meaning: This proverb refers to a person who pays no attention to the warnings, threats, or negative reactions of others.

It also said for a person who keeps threatening others of doing something bad to them, never less the “others” don’t care about his threats.

End of table

Phase 2: Comparison between ChatGPT and Google Translate

The second phase involved comparing ChatGPT and Google Translate. I used Google Translate to translate the same list of proverbs that had been initially translated with ChatGPT. Here is the chart summarizing the findings, followed by an explanation of the results, and a table comparing both translations: 

Picture 2 - Graph reflecting the comparison between ChatGPT and Google Translate - Same meaning - 37%; Both incorrect - 53%; ChatGPT is better - 10%; Google Translate is better - 0%
Picture 2 – Graph reflecting the comparison between ChatGPT and Google Translate
  1. “Same” (11): In 11 out of the 30 proverbs tested, ChatGPT and Google Translate produced identical translations. This consistency suggests that these two machine translation tools are equally effective in capturing the intended meaning and providing accurate translations for certain proverbs.
  1. “Both incorrect” (16): A significant portion of the proverbs, precisely 16 out of 30, posed challenges for both ChatGPT and Google Translate. Both tools delivered translations that deviated from the intended meaning in these cases, highlighting the complexities of translating proverbs accurately.
  1. “ChatGPT is better” (3): In three instances, ChatGPT outperformed Google Translate by providing a more accurate translation of the proverbs.
  1. “Google Translate is better” (0): Interestingly, Google Translate did not outperform ChatGPT in any of the tested proverbs.

Click to jump below the table

Arabic Proverb English translation with ChatGPT English translation with Google Translate Which tool did it better
إسأل عن الجار قبل الدار Ask about the neighbor before the house. Ask about the neighbor before the house Same
إضرُب عصفورين بحجر Kill two birds with one stone. Kill two birds with one stone Same
إللي بيدري بيدري و إللي ما بيدري بيقوك كف عدس He who understands understands, and he who does not understand, you can explain until you’re blue in the face. Whoever knows, knows, and whoever doesn’t know, give you a palm of lentils Both are incorrect
حَظّه بيفلَق الحجر His luck can split a rock. His luck breaks the stone Same
تَيَّس عَبدك يا رب Guide your servant, O Lord Your servant’s goat, O Lord Both are incorrect
حامل السّلم بالعَرض Bearer of the ladder in the parade Ladder holder crosswise Same
لله يرحمك يا با إللي ما علّمتني القراية

 

May God have mercy on you, oh Father, who didn’t teach me how to read May God have mercy on you, my father, who did not teach me to read Same
حَماتك بِتحِبك

 

Your mother-in-law loves you. Your mother-in-law loves you Same
زَيّ مِسمار جُحا

 

Like a nail in Juha’s head Juha’s nail costume Both are incorrect
خَيرها بِغيرها Better than it with something else Make her better than others Both are incorrect
وافَقَ شَنٌّ طَبَقة

 

A tray has agreed with its flatbread. Shin Tabaqa agreed Both are incorrect
دايرَة على حَلّ شَعرها

 

Trying to solve her hair She decided to undo her hair Both are incorrect
الدنيا قايمِة قاعدِة

 

The world is built on opposites. The world is standing and standing Both are incorrect
دِق عَ الخشب

 

Knock on wood Knock on wood Same
راحت السَّكرة و إجِت الفَكرة The drunkenness has gone, and the thought has come The drunkenness went away and the idea came Same
رَفِع عَتَب

 

Raise an objection or Express disapproval. Raising a threshold Both are incorrect
رِجلُه بِالقَبِر

 

His foot is in the grave. His foot is in the grave Same
زَيّ الأطرَش في الزَّفِة

 

Like a blind person at a wedding Al-Atrash’s costume in the wedding ceremony ChatGPT is better
سَبع البورومبو

 

—– Seven burombos Both are incorrect
سَنّ سنانُه

 

He has sharpened his teeth. His teeth ChatGPT is better
زَيّ الزِفِت Like asphalt Asphalt costume ChatGPT is better
شِحّاد و مشارَط

 

stingy and nitpicky Beggar and scalpel Both are incorrect
شو جاب لَجاب!

 

What did he bring for the crow? What happened to him? Both are incorrect
شو مَحَلُّ من الإعراب

 

What is the point of the grammatical analysis? What is the subject of parsing? Both are incorrect
صَحّ النّوم

 

Good sleep or Sound sleep. Sleep well Both are incorrect
طَق حَنَك

 

Mind your own business or Keep your nose out of it. Taq Hanak Both are incorrect
جَوابُه تحت باطُه

 

His answer is under his doorstep. His answer is under his cover Both are incorrect
خالِف تُعرف

 

Differ, and you will be known. What is different is known Same
عِذر أقبح مِن ذَنب

 

An excuse is uglier than a sin. An excuse worse than a sin Same
خَلّيه يبَلِّط البحر

 

Let him pave the sea. Let him smooth the sea Both are incorrect

End of table

Detailed Findings

After evaluating the Arabic proverbs and their translations and then comparing the two tools (ChatGPT and Google Translate), I ended up with this comparative analysis:

  1. Consistency of Translation:
  • For many of the proverbs tested, ChatGPT and Google Translate produced identical translations, indicating consistency between these machine translation tools. Proverbs such as “إسأل عن الجار قبل الدار”(Ask about the neighbor before the house)  and “حَظّه بيفلَق الحجر” (his luck can split a rock) were translated accurately and uniformly by both programs.
  1. Accuracy of Translation:
  • While both ChatGPT and Google Translate demonstrated competence in some instances, there were notable variations in translation accuracy.
  • In several cases, both tools delivered translations that deviated significantly from the original meaning of the proverbs, as evidenced by “إللي بيدري بيدري و إللي ما بيدري بيقوك كف عدس,” which were inaccurately translated by both tools. In my opinion, since this proverb has a story behind it (mentioned below in #4 of this section), unless you know what the story’s background is, you will not fully comprehend the intricacies and depth embedded within the proverb itself.
  • On the other hand, ChatGPT outperformed Google Translate in a few instances, such as the translation of ” سَنّ سنانُه .” This proverb is often used when someone has prepared themselves and is ready for a certain act (it’s like sharpening your teeth to prepare yourself to eat).
  1. Interpretation of Nuances:
  • Proverbs frequently contain colloquial phrases and cultural quirks that necessitate close reading to provide an appropriate translation. ChatGPT and Google Translate found it difficult to convey these subtleties during our evaluation reliably.
  • The proverb “تَيَّس عَبدك يا رب” (Stubborn as a mule) serves as an example of the challenges both tools faced in interpreting the intended meaning accurately. ChatGPT translated it as “Guide your servant, O Lord,” and Google Translate rendered it as “Your servant’s goat, O Lord.” This proverb applies to someone who suddenly becomes stubborn or disagreeable. The word “tais” means “billy goat,” and the verb “tai-yass” means he became extremely stubborn. It is common that students will criticize failures using the word “tais”. People will say, “So-and-so is tais in math or tais in Arabic.”
  1. Cultural Loss:
  • Our study’s conclusions draw attention to how difficult it is to translate proverbs because of their strong cultural and historical roots. While ChatGPT and Google Translate offer valuable assistance in basic translation tasks, they may not consistently preserve the subtleties and cultural significance of proverbs. Human oversight and editing are often necessary to ensure accurate and contextually appropriate translations.
  • The proverb ” إللي بيدري بيدري و إللي ما بيدري بيقوك كف عدس” was translated in both programs as “He who understands, understands, and he who does not understand, you can explain until you’re blue in the face” (ChatGPT), and as “Whoever knows, knows, and whoever doesn’t know, give you a palm of lentils” (Google Translate). Both translations have nothing to do with the correct meaning of the proverb!

According to Isleem (p. 41), this is the story behind the proverb:  

During the lentil harvest season, a farmer had his crop stacked in the fields. He and his workers would spend nights guarding the piles to protect the lentils from birds and potential theft. One night, a worker noticed another worker advancing toward the farmer’s wife, proposing they sleep together on the lentil piles. Concerned, the observer informed the farmer, who became furious and chased away the seducer.

As the seducer fled, he passed through neighboring farmers’ fields, where people stopped him and asked why he was running. He explained, “I have this bushel of lentils, and the man chasing me won’t let me go.” Although the explanation sounded odd, the seducer continued to run.

Upon the farm owner’s arrival, the onlookers questioned, “Why are you pursuing that man over just a handful of lentils?” In response, he uttered, “Whoever understands, understands, and whoever doesn’t, simply says: ‘A handful of lentils.'”

I tried multiple prompts (in English), hoping that I could elicit the correct translations, as shown below:

1- Translate the following Arabic proverb into English

2- Translate the following Arabic proverb into English, taking into consideration its cultural background

3- Translate the following Arabic proverb into English, taking into consideration its cultural background about a story related to a person who couldn’t tell the real reason because it would be humiliating or embarrassing (to him), even though the apparent reason is making him look irrational. He’d rather look irrational than be humiliated.

Despite all these different prompts, the AI consistently offered the same incorrect translation and interpretation. This highlights a limitation in the model’s ability to grasp the nuances and subtleties of the provided content and revise its conclusions.

Picture 3 - Screenshot from ChatGPT - The Arabic proverb translates to "He who understands understands, and he who does not understand, you can explain until you're blue in the face" in English. It emphasizes that some people grasp things easily, while others may require extensive explanation, but they may never truly comprehend.
Picture 3 – Screenshot from ChatGPT

However, when I provided the proverb in Arabic and prompted it (in English) to provide an Arabic explanation of the proverb, the results were much closer to the correct meaning. 

These findings highlight how differently machine translation algorithms can perform when dealing with proverbs. They also emphasize the importance of considering these utterances’ cultural quirks and context. The findings show that even though ChatGPT and Google Translate provide helpful translation support, human oversight, and experience are frequently required to guarantee the most accurate and culturally appropriate translations of proverbs. 

Part 2: Pedagogical Applications of Proverbs and ChatGPT in the Classroom

In this section, I will discuss why, despite the inaccuracies observed in proverb translation by ChatGPT, it is still worthwhile to consider including this tool as a part of students’ learning about proverbs in the target language, as well as how to deal with those inaccuracies.

The use of translation of proverbs with ChatGPT in an Arabic classroom from a pedagogical point has both benefits and considerations.

Benefits

  1. Language Exposure: ChatGPT can give students easy and fast access to various proverbs from different Arabic-speaking regions. This exposure helps students become more familiar with the richness and diversity of the Arabic language.  
  2. Vocabulary Expansion: Translating proverbs with ChatGPT exposes students to new vocabulary and idiomatic expressions. It allows them to explore the meanings and usage of words in real-life contexts.
  3. Comparative Analysis: The translations produced by ChatGPT can be contrasted with translations from other students, teachers, or other resources. Students can assess translations critically through this comparative study, which helps them better comprehend linguistic nuances, as well as obtaining a more accurate view of the capabilities of the tools that they are using. The previous sections have demonstrated that ChatGPT struggles to translate cultural complexity, which brings us to the next benefit: “Cultural Understanding.”
  4. Cultural Understanding: Arabic proverbs have a rich cultural history. Proverb translations with ChatGPT can shed light on the cultural allusions, historical settings, and moral principles connected to these sayings. This improves students’ awareness of culture. Suppose we take as an example the proverb زيّ مسمار جحا (Like Juha’s nail.). Working with this proverb opens the door to discussing who Juha is, a popular figure in Arab folklore known for his wit and humor. The story behind this proverb is that Juha sold his house to a man but retained ownership of one single nail on a wall. Initially, the new owner didn’t think much of it, but Juha began visiting the house frequently, insisting on seeing his nail. His visits became increasingly bothersome, occurring at all hours and even interrupting the man’s private moments. Fed up with Juha’s constant presence, the frustrated owner finally relinquished the entire house back to Juha, exclaiming, “Take your nail and take your house! I don’t want it!”Understanding the context of Juha adds depth to the proverb’s meaning and allows students to appreciate the layers of cultural significance embedded in such sayings.  
  5. Critical Thinking: Talking about the accuracy of ChatGPT’s translations and possible mistakes pushes students to grow as critical thinkers and problem solvers. They gain the ability to challenge translations, use a variety of resources as they learn about the target language, and offer more precise substitutes.
  6. Developing Digital Literacy Skills: Technology in language instruction helps students become more adaptable and comfortable with digital tools, preparing them for the increasingly digital world of learning and communication. In the current digital age, students depend increasingly on technology for their educational needs. However, without digital literacy skills, students may struggle to effectively navigate and utilize these tools (including ChatGPT). Thus, I recommend the following steps: First, we must teach them how to “effectively” interact with it (i.e., by outlining appropriate prompts or commands). To ensure ChatGPT understands your request, structure prompts as complete sentences, provide context (for example, if asking for a proverb, provide the cultural context or specify the region), use keywords related to the desired output (for instance, when seeking a translation, include the languages involved), mention the audience, request examples, and review responses for accuracy, relevance, and coherence. You can also offer corrective feedback to improve future interactions. Secondly, students need to evaluate the information by cross-referencing with credible sources, which prompts responsible information consumption. By doing so, they can identify any biases or inaccuracies that may be present. This critical thinking skill is important in today’s digital age, where misinformation and fake news can easily spread. 

Considerations

  1. Accuracy: As mentioned throughout this study, while ChatGPT can provide useful translations, it is not infallible. Students should know that ChatGPT’s translations may contain errors or lack context, and teachers should emphasize the importance of cross-referencing and verifying translations.
  2. Linguistic Variations: Dialects and regions can have differences in Arabic proverbs. It is possible (and likely) that ChatGPT doesn’t always correctly reflect these regional differences. 
  3. Supplementary Learning: ChatGPT should be supplementary to traditional teaching methods. It should not replace teacher-led discussions, explanations, and guidance on cultural and historical aspects of proverbs.
  4. Proper prompting:  Precision in prompting ChatGPT leads to more contextual translations and a deeper understanding of cultural nuances. Students can follow this step-by-step process when prompting ChatGPT for proverbs:
    a) Literal translation: Start by asking ChatGPT for a literal translation of the proverb from Arabic to English. A simple prompt can be: Translate this Arabic proverb into English. This step will allow students to understand the words and their meanings with the proverb.
    b) Example sentences: Having the proverb in context allows students to understand its usage and nuances. For example: Can you provide example sentences using the proverb ‘[insert Arabic proverb]’? Including sentences where this proverb is used would help better understand its context and application.
    c) Authentic translation: In this last step, students can ask for an authentic translation of the proverb’s meaning in English. This step permits them to get a clear understanding of the intended message behind the proverb. A good prompt would be: Could you provide an accurate English translation or an explanation that reflects the authentic meaning of the Arabic proverb ‘[insert Arabic proverb]’? It’s crucial to capture this proverb’s essence or cultural context for a better understanding.

Example of Activities 

1. Look for equivalents in English (or other languages): One of the activities I enjoy when using proverbs is asking students to identify the equivalents of Arabic proverbs in other cultures or languages. While some proverbs may have identical wording between Arabic and English, others might have variations while conveying a similar meaning. This exercise enhances language skills, fosters cross-cultural awareness, and underscores the idea that wisdom is universal, transcending linguistic boundaries. It’s a valuable tool for students to appreciate the diversity of human expression and the shared wisdom of different cultures.

Below is a table showing Arabic proverbs and their English equivalents:

English Equivalent In Arabic
He had the devil’s own luck حَظّه بيفلَق الحجر
Stubborn as a mule تَيَّس عَبدك يا رب
To go against the grain  حامل السّلم بالعَرض
Birds of a feather flock together  وافَقَ شَنٌّ طَبَقة
Knock on wood دِق عَ الخشب
Baggers can’t be choosers  شِحّاد ومشارَط

2. Original proverbs: ChatGPT can generate entirely new proverbs that have never been 

seen or used before. To test this activity, I asked ChatGPT to ‘Create a new Arabic proverb about the role of parents in their children’s lives, emphasizing guidance, love, and support for their growth and well-being.’ Subsequently, I prompted it to produce proverbs reflecting Egyptian culture. Here are the proverbs ChatGPT provided:

Picture 4 - Screenshot from ChatGPT (example from Egypt) -Certainly! Here's a new proverb in Arabic inspired by the culture of Egypt, underlining parents' involvement in their children's lives: Translation: "Parents are like the sun on one's face." This proverb reflects the Egyptian cultural appreciation for the sun and how it represents the warmth, guidance, and constant presence of parents in their children's lives.
Picture 4 – Screenshot from ChatGPT (example from Egypt)

Then I instructed it to utilize words related to the Gulf region, and here’s what it provided:

Picture 5 - Screenshot from ChatGPT (example from the Gulf region) - Certainly! Here's a new proverb in Arabic using words from the Gulf region, highlighting parents' involvement in their children's lives: Translation: "Parents are like pearls in the heart of the sea." This proverb draws inspiration from the Gulf region's association with pearls and emphasizes the precious and valuable role of parents in their children's lives.
Picture 5 – Screenshot from ChatGPT (example from the Gulf region)

And if ChatGPT is prompted to use words from the Maghreb, you get the following: 

Picture 6 - Screenshot from ChatGPT (example from the Maghreb) Of course! Here's a new proverb in Arabic using words from the Maghreb region, emphasizing parents' involvement in their children's lives: Translation: "Parents are like the blossom in the midst of the olive tree." This proverb draws inspiration from the Maghreb region's rich olive culture and symbolizes the nurturing and vital presence of parents in their children's lives.
Picture 6 – Screenshot from ChatGPT (example from the Maghreb)

When you compare the three proverbs, you notice that the choice of words was tailored to reflect the region. In the Gulf, the sea is often viewed as a source of wealth, and pearls hold great value. Therefore, using a regional metaphor, the proverb emphasizes parents’ preciousness in their children’s lives. An olive tree, a significant cultural and economic symbol, was used for the Maghrib. The proverb reflects the nurturing and central role of parents within the context of the olive tree, emphasizing their importance in their children’s lives. In Egypt, the sun holds great significance, both culturally and historically, as it symbolizes warmth and guidance. 

When engaging in activities involving AI-generated proverbs, it is important to note that the proverbs generated by AI, like ChatGPT, are not authentic in the traditional sense. This exercise is designed to prompt critical thinking and cultural assessment. Therefore, while exploring these AI-created proverbs, students can compare and contrast them with authentic proverbs. They can also assess their resemblance to real cultural sayings and their ability to encapsulate the essence of a specific region’s culture. This comparison helps analyze nuances in language and culture, allowing students to discern what elements contribute to a proverb’s authenticity and cultural significance.

3. Proverb riddle:  This is a wordplay in which a scenario, narrative, or description is presented in a way that alludes to a well-known proverb or saying. ChatGPT can provide riddles upon your prompt; you have to specify the proverb or saying you want it to create a riddle for (or you can leave it up to ChatGPT to choose any proverb). Students must identify the proverb based on the information mentioned in the riddle.

For example, I requested it to provide a riddle about this proverb القرد بعين إمو غزال.
ChatGPT responded with this:

I’m a creature that swings from tree to tree,
In the forest, I’m as nimble as can be.
Though my mother’s eye sees a different scene,
In her view, I’m graceful, almost like a queen.
What am I?

In Arabic:

أنا مخلوق يتأرجح من شجرة إلى شجرة،
في الغابة، أنا حريص جدًا.
على الرغم من أن عين أمي ترى مشهدًا مختلفًا،
من وجهة نظرها، أنا نعمة ، تقريبًا مثل ملكة.
ما أنا؟”

4. Meme Generator: Finding memes that utilize proverbs or even creating memes that refer to proverbs with a meme generator can be a creative way to engage with language and cultural expressions. Using a Meme Generator (for example, imgflip) and asking students to input a traditional proverb or a modified version with an appropriate image that emphasizes its meaning is an innovative approach to blending humor with timeless sayings.
This method enables students to visually reinterpret and express old wisdom in a contemporary setting, increasing engagement and understanding of the proverb’s meanings. 

5. Prompting comparison: As mentioned earlier, when I provided the proverb in Arabic and prompted it in English to provide an Arabic explanation of the proverbs, the results were much closer to the correct meaning. The AI could better interpret the proverb and provide a meaningful explanation in Arabic than in English. This suggests that perhaps the language model has a stronger understanding of the nuances and cultural context of the proverb when it is prompted in its original language. 

Students can be asked to prompt ChatGPT with the same proverb in both English and a second language and observe any differences or subtleties in the translations provided. By comparing how ChatGPT handles the same proverb in different languages, students can gain insights into the complexities of translation and the significance of context in conveying meanings accurately across languages. 

Conclusion

In summary, integrating ChatGPT into the classroom for teaching Arabic proverbs has several benefits, including vocabulary enrichment, cultural awareness, and critical thinking development. However, it should be used alongside traditional teaching methods to ensure accuracy and context preservation. This study revealed that ChatGPT and Google Translate struggle with translating proverbs due to their complex cultural implications and idiomatic expressions. Human oversight is crucial for contextually accurate translations. This work underscores the importance of context, cultural awareness, and language proficiency in achieving precise proverb translations and suggests the potential for future machine translation advancements.

Classroom activities involving Arabic proverbs can include providing English equivalents, exploring equivalent sayings in students’ languages or cultures, and comparing meaning nuances across languages. Students can also be encouraged to create riddles based on Arabic proverbs, and ChatGPT can assist by generating riddles on their prompts. Additionally, students can prompt ChatGPT to generate original proverbs on specific topics, enhancing language skills and critical thinking while promoting cultural exchange. These activities enrich the learning experience and contribute to a deeper understanding of proverbs.

References

Isleem, N. (2010). Popular Proverbs: An Entrance to Palestinian Culture (English and Arabic Edition) Bilingual Edition. Passia Palestine Academy for Science and Information.

Hanki, J. (1998). Arabic Proverbs.

House, J. (2014). Translation: A Multidisciplinary Approach. New York: Palgrave Macmillan.

One thought on “Assessing Arabic Proverb Translation Accuracy: A Comparative Study of ChatGPT and Google Translate with Pedagogical Applications

  • Very interesting read and comparison! Thank you for providing this invaluable contribution to our understanding of AI and using it in the classroom.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *