SCIC – universities 2015: Lessons learned

Every year in March the European Commission’s interpreting directorate (nowadays DG Interpretation, but for must of us still DG SCIC) gather representatives from the universities they collaborate with. This year was my second time, but with some 200 participants and a programme filled to the brim it is still a quite overwhelming experience. The webcast is online and you can watch it here.

The European institutions contribute actively to the training of conference interpreters. As university we can apply for funding for our interpreting programmes, we get support through virtual classes with trainers from the institutions, and DG SCIC (Interpretation) sends trainers to the universities for Pedagogical Assistance. There is also a multitude of virtual support (Speech repositorySCICtrain, ORCIT, podcasts) and with non-institutional spin-offs such as SpeechpoolA word in Your Ear and Interpreter training resources).

All of the above gives good support for the universities, and once a year we get together to exchanges views. The conference has one big challenge though, with so many participants it’s difficult to create enough interaction to make room that exchange of views. And we’re conference interpreters after all, so we love to have or say, but due to time constraint it’s usually not many discussions. This year was different though, the organizer made a great effort to create interaction through panels with trainers, one focusing on the training support and the other one on how to bridge the famous gap between training and free-lance test. The conference was also preceded by a survey to the universities on the support from DG SCIC (Interpretation). All this together with twitter-feed and e-mail questions provided ample opportunities for exchange. So, what did I learn. First, a few things I really should now by now, as experienced conference goer…

1) Bring a tissue. By now, I have understood that I will cry when the Leopoldo Costa laureate gives the acceptance speech. Maybe it’s a sign of becoming an old lady, or simply proof of the talented laureates. In any case @Goldsmith_Josh ‘s speech was eloquent, witty and touching.

2) Be early. Despite my infinite struggles I often arrive last minute. It is rarely a problem as I’m not late, but arriving last minute means that the room is full and it’s really hard to find the people you’d like to see. Even more difficult if you don’t know their faces…

3) Be prepared. I flatter myself for having good memory of names and faces, but when I, for the second consecutive time, don’t recognize a colleague I both like and would like to see, there are NO excuses!

And then on to the teaching practice. My take home messages:

4) Booth practice. I will start working in the booth with my students in training. We can tell our students what to do in the booth a million times, but unless we actually work with them they will not experience what it’s really like. On a side note, I have to say that I love my teaching team at TÖI, because when I came back and told them about my experiences and that I would like to go into the booth, they answered that they had already started trying that out!

5) Ambassadors’ scheme. At London Met they have started an ambassadors’ scheme of alumni students who can help new students through exercises and as renumeration the ambassadors will be given extra classes and practice time – perfect if you prepare for a test, for instance. You can read more about it here.

6) Bridging the gap. The “bridging-the-gap-between-the-training-programme-and-the-free-lance-test” discussion is an eternal debate. But I’m inspired (by the ambassadors’ scheme among other things) to continue to support students after the exam and until a freelance test. Contrary to some of my peers, I’m not sure we can bring students up to EU-working speed in one year. But, I do believe we can provide enough tools to, as Roderick Jones put it, make the students hit the ground trotting. And if we get our former students to trot-on we will soon get them into a nice regular canter. Danielle D’Hayer of London Met has a very good contribution to the debate here.

7) Participation in tests. One of the forms of pedagogical assistance universities can get is participation in final exams, but Roderick Jones suggested trainers could get invited to sit on freelance tests. Wonderful! That way we would get first hand experience on demands, and we could also coach unsuccessful students.

8) Mentors. Yes we need them! Not sure how a perfect mentor programme would work, but quite convinced we need to develop them and keep improving them. As @ApatrideDude put it on twitter “I have a mentor-shaped hole in my life”.

9) Lifelong learning. One of the suggestions from the survey was a training for trainers scheme for experienced trainers. Sounds like music to my ears! The SCIC-universitites conference is a little bit like that, but only one representative per university can go, wouldn’t it be a wonderful boost for our teaching staff to be able to participate in such courses.

Finally, my two stars and a wish. Two things I really liked and one thing I would like to improve:

Star: The panel idea was absolutely excellent. Star: I very much appreciated the survey. Wish: A morning of smaller workshops on different topics related to exchange of practice. For instance, applying for and reporting of funding, teaching methodologies, study visits, virtual classes, using SCICtrain and other on-line resources, and so forth.

Thank you so much for this conference! Already looking forward to next year.

How to be a teacher’s pet – what all my interpreting students need to know

Äpple Stefan Svensson Flickr

Äpple Stefan Svensson Flickr


On Monday, our spring term starts and I will teach public service interpreting. Here are some tips for my students to dwell on over the week-end and which go beyond be on time, be polite and give your teacher an apple. Some tips are general for all students, other more specific for interpreting.

1. Come prepared

For a lecture it means you will have read the text. It does not mean you have to know it by heart or fully understand it, but the discussions will be so much more interesting if you have actually read the text. Then I don’t have to tell you what it says, I can start by explaining and discussing it. I prefer when my lectures are interactive, but that depends on you too, if you don’t have a clue of what I’m talking about, there will be less interaction. For an interpreting exercise class it means having studied the topic and the terminology. The reading list is not optional, it’s not just for fun, or if you have time. I have found and screened those books and articles because they will help you build on your knowledge base to become a better interpreter. If we run an exercise on thyroid cancer you will have to know how the thyroid gland works.

2. Let me know if you’re not coming

This depends on group size of course, but typically interpreting classes are small groups and many of our activities will have to be adapted on whether you’re there or not. On top of that you’re studying to become an interpreter, interpreters just don’t show up, you’re either there or you have provided someone to replace you. So this is part of your professionalism. For class, the least I ask is that you inform me if you’re not coming. Please note that almost all our classes are compulsory, so you will need to hand in additional tasks in case you cannot come.

3. Of course you can ask questions!

All the time, I’m here for that. I will answer your questions both face to face and over e-mail or telephone. But please check our web platform first. I have put all our documents there, and most of your questions will probably be answered there. Do make the effort and search there first, please. I will answer your e-mail on office hours, every working day, but please bear with me if it takes a few days before I answer, I may have to look for the information myself or prioritize things. If it takes more than three days you are welcome to remind me, after all e-mails get lost. If it’s urgent, tell me right away. And, do say thank you. It will work just as well for you today, as it did when your mother taught you to do it when you were two or three.

4. Check your terminology

We will work together on terminology, but you need to create your own word lists and do your own terminological research. Google translate is not enough, neither are the bilingual dictionaries you find online. Many of them are quite good, mind you, but they just aren’t enough. What you are, typically, looking for is highly specialized and context bound terminology, quite the opposite of what you find in general dictionaries. When you build your word lists you will need to check the meaning for the term in several different contexts, you will also need to check whether a generally accepted translation exist for that particular term or not. Furthermore, you may have to questions your source, which could be, for instance, official translations, did those translators use the right or appropriate term? In your word list you will have to state the term, its meaning, its different translations and your sources for that.

5. Respect practice time

On top of time in class, I have booked rooms where you can practice, I will also provide ample training material. You will be requested to keep a log for your practice. I did not do that just for fun, or because I have a particular liking for keeping you at university. Practice is the absolute core of becoming an interpreter. Learning to interpret is about automatizing processes and those processes will only get automatized if you practice. I will teach you how to assess yourself. I will also try my best to give you all the possibilities to get that practice, take them!

6. Record yourself

Record yourself in class and record yourself during your practice sessions. Listen to the recordings too. Listening critically to yourself when you work or practice will help you improve immensely. Nowadays, your phone or computer will probably have a recording device so you won’t even have to invest in new devices.

This is my humble advice to you as students. If you follow them you will not just make me happy, you will make fine interpreters. As usual, I invite my fellow teachers reading the blog to add their opinion. What’s your advice to your students?

Let me introduce myself – the interpreter’s introduction

Vector handshake

Vector handshake (Photo credit: Wikipedia)


When you arrive at a meeting where you will interpret, you will have to introduce yourself. Well, maybe not if you’re part of the staff at an international institution, then you’ll just slip into your booth and do your job. But in all other contexts you will have to tell somebody who you are and what you’re doing there. So how do you go about it?


When I arrive at a more conference-like meeting I will just see the person responsible for the interpreters and a short: “I’m Elisabet Tiselius, Swedish booth”, will do. The only thing they’re interested in is that we are there and ready to start working. If there’s a particular tricky terminology or concept you may go and see your delegate and ask for clarification or explanation, but otherwise you sit tight and wait for the meeting to start.


In public service meetings though, it’s very important that you give a clear introduction in both your languages. You cannot assume that the participants have worked with interpreters before, and even if they have worked with an interpreter before, you cannot assume they are familiar with the ethical guidelines for interpreters. Shenyun Wu gives very good advice on his blog Their Words, Your Voice.


On our first interpreting exercise class the students are asked to create a written cheat sheet for their interpreting presentation. I ask them to put the following there:


1) The interpreter’s impartiality, 2) everything expressed in the room will be interpreted, 3) the participants are responsible for the conversation and must address each other not the interpreter, 4) the interpreter uses the first person, 5) the interpreter observes absolute confidentiality, and 6) the interpreter will ask questions if anything is unclear.


The cheat sheet should be written in both their languages and they must start using it immediately at exercises so that it become their second nature. It’s important to keep it short and simple, just as Shenyun Wu writes, many case officers will not see the importance of taking an extra minute to clear those things out. Yet, it is important for the interpreter to stress these points, too often it is obvious that the client or the case officer/doctor/lawyer is not aware of the interpreter’s role or responsibilities.


And what about interrupting? Shenyun Wu includes a note on interruption in the introduction. I think it may be a good idea. To interrupt smoothly is an art and something budding interpreters need to practice a lot as well. I will come back to that in another post.


My students notice, when they do their compulsory interpreting observations that there are still many both professional interpreter users and professional interpreters who do not know of, or use, the interpreter’s introduction. This is unfortunate as it strengthens and clarifies the role of the interpreter. So, what do you do? Do you introduce yourself? How do you do it? And do you find it helpful?


Happy New Year!

FIRE WORKS in curacao

(Photo credit: Jessica Bee)

We were out walking today and talked about what was the best thing about 2013. For my part it was easy to answer – finishing and defending my PhD! Yes on November 26 I finally defended my PhD, proof here. But 2013 has been a particular eventful year for me. Most of which I have touched upon in my previous post so I won’t dwell anymore on that, suffice to say that I really wish I had had some more time for blogging (no new posts and 500+ unread posts in my feedly…) and twittering these past months. When I met with Michelle (@InterpDiaries), in April I think it was, we both agreed that it isn’t ideas that lack when it comes to blogging, if only there were some more hours in a day.

During autumn I’ve been busy teaching two introductory courses on interpreting, one in Bergen (TOLKHF) and one in Stockholm (ToÖ I). At TÖI (the Institute for Interpreting and Translation Studies), I’ve also been busy launching our Facebook page and our Twitter account (@TOI_SU). On the course developing side I’ve been busy revamping the course Interpreting II, and I’m very excited to see how it turns out as it will start on January 20th. Traditionally we have always given Interpreting II only with Swedish and one other language. This time it will be Swedish and four other languages which means a completely new approach to both teaching and learning interpreting. I have integrated Lionel’s (@Lioneltokyo) approach to start teaching consecutive, when you have students interpret from notes before they actually start to learn note taking. Next term for Interpreting III we will also take on simultaneous interpreting for public service interpreters. I’m still thinking about this since the techniques in public service settings are not the same as in conference settings. There is also very little course literature on simultaneous interpreting for PSI so we’ll see where that takes me. If you have any suggestions, please comment!

As my PhD project drew to an end this year I have also thought about what to do next of course and I have one or two threads I would like to pursue. My most recent research interest has been young children and interpreting or child language brokering as it has been coined (professor Harris writes a lot about it on his blog). I applied for money for a series of workshops on children and interpreting in the Nordic Countries, we did not get it this time but I will apply again, and hope to be successful eventually. I will also try to get project money for a project on just mapping the practice and the ethics around it in the Nordic Countries.  I would, of course, also like to continue to follow and interview my fellow interpreters. I hope they will let me continue to record them and investigate them. I have such a wonderful material that I would like to build on. And finally, I hope to start up a project with my friend Emilia Iglesias Fernández of Granada. We have been in touch again and I hope it will lead to something. All of this will of course not come to an end (or maybe not even to a start) during next year, but I’m positive some things will.

This autumn I have also been the extremely proud guest blogger at Rainy London’s blog. I get tired just reading about my own week, but that was a very fun culmination of an extremely busy spring. An I was very flattered to be mentioned by Jonathan Downie (@jonathandownie) in Ligua Grecas blog.

When I write this I sit in front of my bedroom window at our house in the country side, it is pitch black outside and in the distance I can see the lights from our neighbours’. My projects now are like the lights out there gleaming in the distance. I hope I will find my way there and catch up with them. Any new years resolutions? No, not really, they only give me bad conscience as I’m really not good at following them, but if any – being better to stay in touch.

I wish you all the best for the New Year and I hope that you have plenty of projects that you will be able to carry through too. Thank you all my friends, (both in and outside Internet) for staying in touch and being so supportive this year. I hope we will continue our discussions in 2014.

Good tidings

Summer 2013

Regular readers of the blog will know that summer is a less active period for me when it comes to blogging. Other activities, also known as family life,tend to take over. With three darlings out of school, and two extra darlings who join us for the summer, and an additional four ponies, three dogs, a cat, in-laws, and lots of friends around the house, academic life, writing, or just time in front of the computer is very limited. This year the usual summer silence has dragged out well into the new term. Rest assured that the silence is not final though :-)

New home, new job and…

There are however lots of changes in my life and hence the reason for the silence. I am happy to share these news with you now, as things are clearing up (and darlings are back in school). First of all we have relocated back to Sweden this summer. After another summer in boxes, I’m now back home ready to start flying (not really looking forward to Swedish autumn, but there we are).

Second big news; I have taken up a part-time position as lecturer at the Institute for Interpreting and Translation Studies at Stockholm University. This means that I have the privilege of teaching interpreting both in Norway and Sweden. My course TOLKHF continues as usual at the University of Bergen, and this term I will teach part of the introductory course to Interpreting and Translation. I have a few lectures and a few seminars to plan so I hope to share some ideas with you and also hopefully get some feedback. I’m slowly getting used to leaving freelance life after almost 20 years of freelancing. I will not completely stop interpreting – it’s too fun, and I need to stay in touch with the business both for my students and for my research.

And finally, my thesis is submitted! I handed it in just before summer and just before our move. Now I have firm indications that I will defend end of November. I’m extremely happy to be done, but naturally, when something as important is no longer part of your life it’s a little bit empty too. In case you’re in Bergen end of November, let me know and I’ll give you all the details.

7th EST Congress

Apart from these more important changes in life I have also participated in two great events. This week I attended the 7th congress of the European Society for Translation Studies where I held a presentation on the results of my thesis, I got good feed back and it was a really nice conference. I met @jonathanddownie in person, very nice experience, an amazing amount of energy, and an interesting thesis in the making. Jonathan co-organized a panel on religious interpreting a very under-researched area that professor Harris has written about on several occasions on his blog Unprofessional translation. My friend Adelina Hild was there too and gave some impressions from her ongoing research on religious interpreters.

I also had a chance to chat with Pekka Kuijamäki who is behind the project of translation and interpreting in WWII Finland (blog here). Very interesting project that I think professor Harris has blogged about too. I was also happy to meet Céderic again. First time we met was this spring in Mons at the conference “Le Nord en Français” and he gave me the inspiration to a previous post. This time I got to listen to a presentation of Céderic’s research project on how intonation affects the understanding in interpreting, very interesting project and an extremely ambitious data collection.

The EST congress is an exhausting exercise with more than 400 participants (record for Germersheim, the organizing university) and over 20 parallel panels. The best thing about EST though is that ”everyone” is there, so you get to socialize with most people in Translation Studies i Europe and beyond, very nice setup. Congratulations to the organizing committee at the FTSK at Germersheim too, who managed this huge event.


This spring I submitted a proposal for InterpretAmerica and was accepted to their Interpret-Ed sessions, a ten-minute-talk that was be web-streamed and recorded for publication on their home page. Splendid opportunity to talk about one of my findings that I still cannot get to grips with. And absolutely daunting!

InterpretAmerica was a great opportunity to meet and network with colleagues on the other side of the Atlantic (and a few from this side too). Brandon Arthur is just as nice in real life as he seems on Street leverage, and imagine I had to cross the Atlantic to meet Ian Andersen from SCIC interpreters :-). It was also great to meet Stephanie Jo Kent again, and to get to know Cris Silva, and finally meeting Andrew Clifford @GlendonTransl8 in person, and many, many others.  And yes, I got to meet and talk to the Babelverse guys Joseph and Mayel, we had a good discussion, I’m not totally convinced yet, but I’ll admit they want to listen to the profession, their FAQ page is new, and they seem to have listened to a lot of the questions that came up in Reston and are looking to answer them.

The whole event was extremely well-organized (great work Katharine and Barry!) and with a nice open atmosphere. Apart from the keynotes and panels (the program is available here) there were also thematic workshops. Although the workshop on social media was extremely tempting, I decided to go to the workshop on vicarious trauma, separate post will follow on that. At the speaker dinner I got to sit next to the charming Saima Wahab, author of “In my father’s country”, she was supposed to be the second keynote, but could not make it last-minute. I was very happy I got to talk to her at dinner, I’m reading her book now.

If you want to listen to my topic that I cannot really come to grip with, you can watch it here:

I can also recommend that you listen to the other Interpret-Ed talks. It was very interesting to hear both Cris, Victor, Michelle and Stephanie. Victor’s approach to managing medical interpreting is absolutely amazing.

Now I’m looking forward to an intensive autumn with students to teach, courses to plan, articles to write and of course a thesis to defend. And some interpreting, what about you?


Interpret America, here I come!

Plaza at Lake Anne in Reston Virginia

Plaza at Lake Anne in Reston Virginia (Photo credit: Wikipedia)

I’m extremely excited! My proposal for Interpret America was accepted. I’ve been wanting and aiming to attend InterpretAmerica since it started in 2010, but other things have gotten in the way. I admit I don’t work on the American continent, but their program is always very interesting and this year is no exception.

But there are two reasons in particular that makes this extra special. First I’m one of five speakers in their new Interpret-ED format, so I had to submit a video proposal, and the talk will be recorded and broadcasted – cool! Second, I get to talk about things I found in my research about interpreting and practice. I will not spill the beans already, but I’m very much looking forward to hear your reactions on my findings. I was very surprised myself and have spent a lot of time thinking about it.

I’m also very much looking forward to meet other colleagues in Reston, both new faces and old friends. There are also a few tweeps I hope to meet in person. And, as I have called for a discussion between inventors of new technologies and interpreters I’ll be in the front row for the plenary on new technologies (don’t worry I neither bark nor bite).

This year’s program follows up on previous years discussions of creating a professional identity and how to form the profession. There’s also a key-not on creating presence in social media and a whole panel on social media with Nataly Kelly (our own Interprenaut and of course Found in Translation, CSA and now Smartling), Brandon Arthur (from Street Leverage)  and Ian Andersen (who is behind the European Commission’s interpreting unit’s popular Facebook page  among other things). I’ll be in the front row there too :-)

And then, there’s the book talk and book signing – Saima Wahab, Pashto interpreter, will talk about and sign her book “In my Father’s country”, and Nataly Kelly will sign her book “Found in translation”. I’ve already read Nataly’s and Jost’s book (maybe I should bring it and get it signed or will that seem too eager?), but I’m very much looking forward to read Saima’s.

So, on the 14 and 15 of June I’ll be spending 48 intensive hours in Reston, Virginia. Come join me there or be sure to watch the video afterwards and tell me if you agree or not.

How to assess interpreting

Use new evidence of learning to replace old

Use new evidence of learning to replace old (Photo credit: dkuropatwa)

This is not my first and surely not my last post on assessment. If you’re looking for the other posts just type “assessment” in the search box to the right. Last Friday (March 15, 2013) I gave a talk on process and expertise research in the Nordic countries at the conference “Le Nord en français” at the University of Mons (one of my alma maters, actually). I also presented the results of my PhD project. All this in 20 minutes, so you can imagine I didn’t have the time to be very thorough.

One of the questions that came up was how I actually went about doing my assessment, and why I choose this particular methodology and not others. As I didn’t really get round to go through my assessments thoroughly, I thought I’d try to do it here. Thanks for the discussion Cédéric, if you stop by and read the post, don’t hesitate to comment or ask more questions.

When I set out to investigate interpreters at different levels of experience I understood quite early that I had to evaluate or assess their product one way or the other. I did not want to assess them based only on my own judgment. I preferred to have “independent/objective” judges, as I was afraid I would be biased both as an interpreter myself and as a colleague to several of my informants. So, fairly early on I decided to use groups of assessors rather than asses myself.

1. Choosing an instrument

Next, I had to choose the instrument for assessment. A popular method for assessing interpreting both in research and otherwise is to use a componential approach. Components typically cover fluency, correctness (terminology, grammar, syntax), sense consistency (with original), logical cohesion, intonation, accent, style and more (or less). Assessors evaluate each component in order to get a complete evaluation of the interpreting. There were several reasons why I did not want to use this componential approach. First, different researchers had pointed out potential problems when using this type of assessment. Heike Lamberger-Felber found in her PhD that it was very difficult to get consistent results from a componential assessment. But, while the rating of the different components varied a lot, the assessors’ rankings of the different interpreters were almost in agreement. Angela Collados-Aís and her ECIS research team have published several reports on assessment, pointing out that although the assessors in their different studies all agree on the level of importance of different components (e.g. fidelity to the original is the most important), other components (e.g. native accent) affect how the most important ones are rated. So a foreign accent would give a lower score for fidelity, although the interpretings word wise were identical. Another important aspect for me was that I wanted to use people without personal experience as an interpreter to be assessors. The reason behind it was that the Swedish interpreting community is so little that it would be almost inevitable for interpreter-assessors to recognize interpreter-informants.

2. Carroll’s scales

So, I started looking at other types of assessment and soon found a type of Lickert-scale used by Linda Anderson already in the late 1970’s. She used two scales created by John Carroll in 1966 to assess machine translation. John Carroll LINK specialized in language testing and he was a big critic of the discreet point theory. The discrete point theory claims that from certain features in a language learner’s production you can predict the learner’s proficiency in that language (rings a bell? if not – reread the paragraph above). When Carroll developed his instrument for translation he said that a translation can be perfectly true to the original but incomprehensible or perfectly comprehensible but completely untrue to the original. Therefore he developed two scales one for intelligibility (comprehensible or not) and the other for informativeness (different from the original or not). The translations were assessed using both scales. Linda Anderson then applied them as they were to her data collected from conference interpreters. She did not dwell much on using the scales, but seemed to fear that they were too blunt.

The scales had not really been used since then, but I found them appealing and wanted to test. One issue was that the scales had served as basis for creating the scales for the US court interpreter accreditation test (FCICE) and this test had been very criticized for its accuracy (or lack thereof). Andrew Clifford has investigated those tests and argues that there may not be any significant difference between the different test constructs. I do not argue against Clifford’s conclusions, on the contrary, but I think the problem lies in how the court accreditation test was developed and is used, rather than a problem with the original scales.

More than one researcher (but far from all) have sniggered at me for using scales that old, which clearly did not create a spin-off in the interpreting research world. If they weren’t used again it must be because they weren’t good, right? But since I’m a stubborn risk-taker I decided to go ahead. What more fun than to dance with the devil? (Yes, I am being ironic in case you wonder…)

3. Tiselius’ adaptation (sounds grand talking about myself in third person right?!)

The scales had to be adapted of course. They were created for translation and I was going to use them for interpreting. Furthermore, there were nine scale steps, some of them difficult to discern from one another. I wanted clear differences between the scale steps, and no middle step, no number five where everything generally OK could be put. Therefore I changed the definitions from written to spoken language and from English to Swedish. I also reduced the steps from nine to six, merging a few that were very similar.

Now only using the scales remained …  When it came to using the scales I had to decide whether to use sound files or transcripts. After all, interpreting is the spoken word, and should it be assessed on the basis of written words? And if I wanted to use non-interpreters as assessors then I would have to justify that. Presumably, interpreters, especially those who have jury training, would be better than non-interpreters at evaluating interpreting.

4. Interpreters or non-interpreters?

I had both interpreters and non-interpreters rate the first batch of interpretings (on transcripts as I did not want the interpreters to recognize their peers). It turned out that in raw figures the interpreters were slightly more severe, but the scores from the two groups correlated and the difference was not significant. These results indicated that I could use either interpreters or non-interpreters.

5. Sound-files or transcripts?

I designed a study where the intelligibility part of the interpretings was assessed by non-interpreters from both sound-files and transcripts. One group assessed transcripts (with normalized orthography and punctuation) and the other sound files. The sound files got slightly worse scores than the transcripts, but again the difference was not significant and all the scores correlated. So from this respect I could use either sound-files or transcripts.

I ended up going for transcripts. This decision mostly came from the insight that Collados Aís provided on how deceitful the voice is when it comes to assessment of product. Pitch, intonation, accent, security and so forth affects the impression of the quality of the product. Clearly, this aspect is important for the assessment of the interpreting, but with the aim in this study to assess only the skill to transfer an entire message in one language into another it seemed wise to exclude it, too many confounding variables.

6. The assessment

The assessment units ended up looking like this:


First the raters saw only the interpretation and they rated that according to the scale from completely unintelligible to completely intelligible, from 1 (lowest) to 6 (highest). They also had a sheet with the full explanation of each step of the scale next to them when rating. If you’re curious I left a copy of the sheet in English here.


Then the raters unfolded the sheet of paper and the European parliament’s official translation showed up at the bottom. Then they rated the informativeness of the interpreting, i.e. the difference between the original and the interpretation. This time from no difference compared to the original to completely different compared to the original. Now the scale is inverted so 1 is the best score and 6 the worst. You may wonder why the scale is inverted this time; I decided to stick with Carroll’s original proposal where a low score is equal to little difference. The zero on the scale means that the interpreters added information not present in the original. This typically happens when something implicit is explicitated or when an additional information or hedge is given.

7. Did it work?

The results I got in my cross-sectional material were very promising, clear differences where I would expect them, i.e. between non-interpreter subjects and interpreter subjects, and between novice interpreters and experienced interpreters. The inter-rater variability, that is the variability of the scores between the different raters, was also low. So far, I’m not sure about the results for my longitudinal material. I did not see differences where I expected them. This may be due to a failing instrument (i.e. my scales) or less difference of the interpreting products than what I expected. To be continued…

Now, there are a few more things to try out with my scales. Obviously, an interpreter trainer would not start transcribing their students’ interpretings and divide them into assessment files before assessing or grading them. But, presumably, the scales could work in a live evaluation as well. I have not yet had an opportunity to test them, but I’m looking forward to that, and I will of course keep you posted.


Anderson, L. 1979. Simultaneous Interpretation: Contextual and Translation Aspects. Unpublished Master’s Thesis. Department of Psychology, ConcordiaUniversity, Montreal, Canada

Carroll, John, B. 1966. “An Experiment in Evaluating the Quality of Translations.” Mechanical Translations and Computational Linguistics 9 (3-4): 55-66.

Collados Aís, Á., Iglesias Fernández, E. P. M. E. M., & Stévaux, E. 2011. Qualitätsparameter beim Simultandolmetschen: Interdisziplinäre Perspektiven. Tübingen: Narr Verlag.

Clifford, Andrew. 2005. “Putting the Exam to the Test: Psychometric Validation and Interpreter Certification.” Interpreting 7 (1): 97-131.

Lamberger-Felber, H. 1997. Zur Subjektivität der Evaluierung von Ausgangstexten beim Simultandolmetschen. In N. Grbic & M. Wolf (Eds.), Text – Kultur – Kommunikation. Translation als Forshungsaufgabe (pp. 231–248). Tübingen: Stauffenburg Verlag.

Tiselius, E. 2009. “Revisiting Carroll’s Scales.” In Testing and Assessment in Translation and Interpreting Studies. C. Angelelli and H. Jacobson (eds.). 95-121. ATA Monograph Series. Amsterdam: Benjamins.