IALLT Interview Project, Episode 13, with Mireia Toda Cosi, Md Mahbubul Amin, and Fred Poole: Testing the Limits of AI Solutions for LCTL Instruction
Mireia Toda Cosi, University of Maryland
Md Mahbubul Amin, Michigan State University
Fred Poole, Michigan State University
DOI: https://www.doi.org/10.69732/OKEB8961
In this interview, Jeff moderates a panel discussion among educators who are either instructors or native speakers of less commonly taught languages (LCTLs). Mireia Toda Cosi focuses on Catalan, Mahbubul Amin on Bangla, and Fred Poole on Chinese (Mandarin). Each was tasked in advance with using an AI solution using their target language and were given questions and prompts in common to test the limits, affordances, and reliability aspects of the resulting responses. The panel concludes with a brief discussion of takeaways, and noted that this exercise represented a snapshot in time, which perhaps they will repeat in the future for comparison. The questions and prompts for each AI solution in the target languages were:
- What are the 4 skills that comprise the basis for second language education? If it doesn’t know, add in that they come from ACTFL and see if that triggers the correct response.
- How can I use (insert name of app being tested) to promote each of these skills?
- Create a short article of 300 words in (target language) at the intermediate low (or use another scale like the European scale or the US govt) about the cultural traditions surrounding the family unit in (a country that uses the TL).
- Now change the level to advanced high
- Now change the level to novice low
- Create a set of 5 comprehension questions for learners in (TL) based on the top article in today’s issue of (major newspaper in TL)
- Now create a vocabulary-based matching? / multiple choice? / translation? quiz of 10 questions based on that article. Show me the questions and the correct answers. All instructions and the content of the quiz should be in (TL).
- What languages do you support and for which ones do you have the 5 largest language models to rely upon?
- Do you hallucinate? How can I be sure I can trust the statements you make and the examples you give?
- Do you interact with other AI solutions such as ChatPGT, Claude, or others? How do you learn from them?
- Finally, each of you should create a prompt with the intention of testing the limitations or veracity of the AI you’re using to see if you can deceive it, catch it in a hallucination, or illustrate a shortcoming that a language educator would not have. The idea is to see to what extent we as language professionals cannot rely on the AI alone and must vet and review what it presents to us.