Lost in translation

In my upper-intermediate class I suggested that the students use an online translation app to check up on their own writing. The response baffled me: Surely the level of quality of these apps was so low, they couldn‘t be helpful at all. Everybody seemed to know that the results of machine translations were only funny gibberish.

I countered by explaining about artificial intelligence and neuronal networks. Still, the skepticism was immense. So, I told the students, we would have to check for ourselves and run a few tests. Next lesson I suggested the following experiment as a learning project for the next two full lessons. The students used their iPads as a main tool for the project.

The experiment

Conduct an experiment to test the quality of online translation apps. Take at least 3 of those translators and have them translate the following text into English.

Analyse and compare the quality of the results. Rate the overall quality and set up a ranking to show the relative differences.

In order to conduct a thorough analysis you’ll have to find suitable criteria to measure the overall quality and check for idiomatic language. Use a monoloingual dictionary for reference.

Try and verify your findings by running a second test using the second text.

Document your findings and upload them to this padlet. Choose a suitable way of documenting. Remember to back up any claims you make with conclusive data.

The outcome

The results as such left a lot to be desired. The students had huge problems with the complexity of the task. This was mostly due to

⁃ time constraint (two lessons were simply not enough!),

⁃ lack of familiarity with the tools of project management,

⁃ their English proficiency level to measure the quality of a translation,

⁃ lack of familiarity with scientific methods.

Next time, I would try a more structured approach and allow for a more generous time frame.

The learning

Still, I consider the experiment a success as it provided valuable learning affordances. The project sparked two intense and interesting discussions in class: The first one was all about how science works and how often we are prone to jump to conclusions based on very poor data.

The second discussion came from discovering the staggering level of quality that online translation has reached. None of the students had expected this beforehand. Now, for weeks, we had to talk about what this meant for their schoolwork and language learning in general.