Research on interpreting quality: interdisciplinary perspectives

This is a short summary of a round table discussion held at the Second International Conference on Quality in Interpreting, in Almuñecar, Spain 2010. The summary is my own perception of the presentation and any mistakes in the summary are of course due to my misunderstanding.

The Round table participants were

Teresa Bajo Molina (Experimental psychology), Universidad de Granada, España
Jorge Bolaños Carmona (Statistics), Universidad de Granada, España
Emilio Delgado López-Cózar (Research assessment), Universidad de Granada, España
Daniel Gile, Université Paris 3 – Sorbonne Nouvelle, France
Heike Lamberger-Felber (Coord.), Karl-Franzens-Universität Graz, Österreich

In this session interpreting was provided so discussions were allowed to flow seamlessly between English and Spanish.

Heike Lamberger-Felber moderated this discussion and started by letting the participants outside Interpreting Studies introduce themselves.

Teresa Bajo Molina (BM) is a cognitive psychologist. Her first contact with Interpreting Studies was when she supervised Presentacion Padilla Benítez’s PhD. She is interested in interpreting because it gives a window in to researching how you’re pushing the cognitive system to the limit.

Jorge Bolaños Carmona (BC) statistician is above all interested in helping to create statistical rigour.

Emilio Delgado López-Cózar (DLC) works on the assessment of research but also on methodology for research.

BM: Stresses that the most important lesson from cognitive psychology is the fact that there is a limit for the cognitive resources and that when it comes to interpreting that picture is very clear. Challenges are similar in Cognitive Psychology and Interpreting Studies since you are studying something that is not immediately perceivable. In order to push Interpreting Studies further there is a need solid hypotheses.

Moderator: What about training in research methodology? Generally in Interpreting Studies there is not much such training around.

BM: In cognitive psychology we spend a lot of time on that, about one term of PhD training is devoted to research methodology.

Moderator: And what about strengths and weaknesses in Interpreting Studies?

BM: I try to stay up to date on publications in Interpreting Studies and I see a lot of good things but also weaknesses. Good questions are being asked, but it is also a very heterogeneous field, very fragmented and I have a problem finding a strong base. Many studies are exploratory and control is missing. I have the feeling that you give up on controlling variables with the ambition to make the experiment closer to the actual object you study. But if you give up control then you cannot draw valid conclusions.

Moderator: How reliable are statistical tools if they are used by untrained researchers in a simplistic design?

BC: Seen from a statistical point of view we can make enormous mistakes in that context, and it’s even worse if there is a poor design. A poor design can destroy years of work. If you want to collect qualitative data you need to consult with experts. You can of course also ask for help from a statistician if you already have collected a quantitative material, but typically they would want something in return of course, such as publication in important publications, effect in academia and so forth.

Here I (blog author) have to add a very personal comment – the fact that Interpreting and Translation journals usually have low ranking scores (for reasons not related to their academic quality but to the fact that there is a small discipline, often not represented in ranking bodies) is an ENORMOUS disadvantage for our discipline.

But back to BC: More cooperation between interpreting resarchers and statisticians is however good and necessary. A word of warning; when asking IT-experts for help to design a web based study, do check that they have sufficient knowledge in statistics. Finally it is important to first know WHAT questions you attempt to answer and THEN look for the proper method and statistical design.

Moderator: We often have a problem with the great variability in our data, many things influence our object of study. How does that affect our aim to do inferential tests?

BC: There are two important variables – those we want to study and those we don’t want to study. The ones we don’t want to study must be eliminated. This is also true for instance for sociological studies where many variables interact and interfere. The conditions of the analysis have to be absolutely right.

Moderator: Is there any use of making inferential statistics on small data? This is another reality in interpreting studies that we often deal with small data.

BC: Of course we can use small data, but only with the right number of repetitions. Often in interpreting you have small data, but you count, for instance, the total occurrence of something e.g. errors. Say you analyse the work of 5 interpreters, you will get a large number of date from the repetition of each individual. So it is rather a question of a small SAMPLE that can still generate a large amount of DATA.

If it’s clear that something is different, why use statistics? Because small variation could affect your data in ways that you did not necessarily expect.

Moderator: Research assessment has also become a field of study and Emilio Delgado López-Cózar will present that.

DLC: In order to evaluate research we use quantitative methods such as bibliometrics, rankings, impact indexes. All these measures are important for people working with research policy. Quantitative outcome of research is measured through publications. Qualitative data consists of for instance peer-review. Research assessment can be done ex ante e.g. when you apply for funding for research or ex post when you assess the results of a project.

Moderator: And what about key terms that often obsess people, such as citation index and impact factor.

DLC: Citations in humanities are delicate. You have to be cautious because of the size of different fields. If there are not many researchers in a filed you have no critical mass and then the number of citations is lower.

Moderator: How do you assess field that you do not know well?

DLC: Bibliometrics is our mirror and we need to look in that mirror. When assessing a field you look at PhDs, publications, journals, research projects, conferences and so forth. You also make a content analysis; the reflection of actual problems, evaluation of tools, methodology, technique and so forth.

Again I as blog author would like to stress the comment I did above on journal ranking in translation studies.

Moderator: I would like to hear from Daniel Gile what you would expect from multidisciplinary in Interpreting research.

DG: About multidisciplinary approach in interpreting studies, we have seen the advantages and challenges of multidisciplinary in this round table. If we want to study the nature of interpreting, cognitive psychology, linguistics and communication are important areas. But it is important not to loose the characteristic features of interpreting out of sight. You cannot, for instance, control away variables to the extent that you loose your sample. Our weakness in Interpreting Studies is that we look up to methods developed in another field and swallow them without a critical eye. What we need is help to understand the CONCEPTS of issues in cognitive disciplines in order to be able to adapt those concepts to Interpreting Studies.

Question from the audience (Franz Pöchhacker): What about other research designs such as surveys and web-based surveys?

General answer from the panel: Surveys as method is a good and easy accessible method, but you have to understand their limitation, such as controlling who is answering and that each respondent only answers once and so forth. The important issue for any research design is that the design is rigorous. If the design and method is good then also exploratory studies are fine.

Moderator: And what would it take for you as researches from other studies to enter into a specific project in interpreting studies?

BM: I am already involved in interpreting projects but you have to understand that it involves making sacrifices.

BC: I would look for a project with the possibility to create something that others doesn’t have already. However it is difficult for researchers in the beginning of their career, who are building their CV to engage in fields that do not give immediate bibliometric feed back.

On this note the panel ended its discussions. Again I think it very well sums up our problem. I would also like to point out that the panel did not touch upon mixed method design and had no representative from sociological research (or other areas working with qualitative methods). Neither was neuroscience or phonetics/phonology mentionned. Although I realise the time was very limited, I would have appreciated that aspect as well when it comes to multidisciplinary.

I also found both professor Bajo Molina’s and professor Gile’s comments on the field very interesting. Professor BM talked about a fragmented field where it’s difficult to find a strong base. Professor Gile talked about not uncritically swallowing methods from other fields. Although the trend today is towards multidisciplinary I believe that Interpreting Studies should work more on the centripedal power rather than the centrifugal power. Interpreting Studies are multidisciplinary per se and will gain more on working on that common solid ground than on stressing differences.


Any comments?

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s