Tag Archives for language interpretation

How to assess interpreting

Use new evidence of learning to replace old

Use new evidence of learning to replace old (Photo credit: dkuropatwa)

This is not my first and surely not my last post on assessment. If you’re looking for the other posts just type “assessment” in the search box to the right. Last Friday (March 15, 2013) I gave a talk on process and expertise research in the Nordic countries at the conference “Le Nord en français” at the University of Mons (one of my alma maters, actually). I also presented the results of my PhD project. All this in 20 minutes, so you can imagine I didn’t have the time to be very thorough.

One of the questions that came up was how I actually went about doing my assessment, and why I choose this particular methodology and not others. As I didn’t really get round to go through my assessments thoroughly, I thought I’d try to do it here. Thanks for the discussion Cédéric, if you stop by and read the post, don’t hesitate to comment or ask more questions.

When I set out to investigate interpreters at different levels of experience I understood quite early that I had to evaluate or assess their product one way or the other. I did not want to assess them based only on my own judgment. I preferred to have “independent/objective” judges, as I was afraid I would be biased both as an interpreter myself and as a colleague to several of my informants. So, fairly early on I decided to use groups of assessors rather than asses myself.

1. Choosing an instrument

Next, I had to choose the instrument for assessment. A popular method for assessing interpreting both in research and otherwise is to use a componential approach. Components typically cover fluency, correctness (terminology, grammar, syntax), sense consistency (with original), logical cohesion, intonation, accent, style and more (or less). Assessors evaluate each component in order to get a complete evaluation of the interpreting. There were several reasons why I did not want to use this componential approach. First, different researchers had pointed out potential problems when using this type of assessment. Heike Lamberger-Felber found in her PhD that it was very difficult to get consistent results from a componential assessment. But, while the rating of the different components varied a lot, the assessors’ rankings of the different interpreters were almost in agreement. Angela Collados-Aís and her ECIS research team have published several reports on assessment, pointing out that although the assessors in their different studies all agree on the level of importance of different components (e.g. fidelity to the original is the most important), other components (e.g. native accent) affect how the most important ones are rated. So a foreign accent would give a lower score for fidelity, although the interpretings word wise were identical. Another important aspect for me was that I wanted to use people without personal experience as an interpreter to be assessors. The reason behind it was that the Swedish interpreting community is so little that it would be almost inevitable for interpreter-assessors to recognize interpreter-informants.

2. Carroll’s scales

So, I started looking at other types of assessment and soon found a type of Lickert-scale used by Linda Anderson already in the late 1970’s. She used two scales created by John Carroll in 1966 to assess machine translation. John Carroll LINK specialized in language testing and he was a big critic of the discreet point theory. The discrete point theory claims that from certain features in a language learner’s production you can predict the learner’s proficiency in that language (rings a bell? if not – reread the paragraph above). When Carroll developed his instrument for translation he said that a translation can be perfectly true to the original but incomprehensible or perfectly comprehensible but completely untrue to the original. Therefore he developed two scales one for intelligibility (comprehensible or not) and the other for informativeness (different from the original or not). The translations were assessed using both scales. Linda Anderson then applied them as they were to her data collected from conference interpreters. She did not dwell much on using the scales, but seemed to fear that they were too blunt.

The scales had not really been used since then, but I found them appealing and wanted to test. One issue was that the scales had served as basis for creating the scales for the US court interpreter accreditation test (FCICE) and this test had been very criticized for its accuracy (or lack thereof). Andrew Clifford has investigated those tests and argues that there may not be any significant difference between the different test constructs. I do not argue against Clifford’s conclusions, on the contrary, but I think the problem lies in how the court accreditation test was developed and is used, rather than a problem with the original scales.

More than one researcher (but far from all) have sniggered at me for using scales that old, which clearly did not create a spin-off in the interpreting research world. If they weren’t used again it must be because they weren’t good, right? But since I’m a stubborn risk-taker I decided to go ahead. What more fun than to dance with the devil? (Yes, I am being ironic in case you wonder…)

3. Tiselius’ adaptation (sounds grand talking about myself in third person right?!)

The scales had to be adapted of course. They were created for translation and I was going to use them for interpreting. Furthermore, there were nine scale steps, some of them difficult to discern from one another. I wanted clear differences between the scale steps, and no middle step, no number five where everything generally OK could be put. Therefore I changed the definitions from written to spoken language and from English to Swedish. I also reduced the steps from nine to six, merging a few that were very similar.

Now only using the scales remained …  When it came to using the scales I had to decide whether to use sound files or transcripts. After all, interpreting is the spoken word, and should it be assessed on the basis of written words? And if I wanted to use non-interpreters as assessors then I would have to justify that. Presumably, interpreters, especially those who have jury training, would be better than non-interpreters at evaluating interpreting.

4. Interpreters or non-interpreters?

I had both interpreters and non-interpreters rate the first batch of interpretings (on transcripts as I did not want the interpreters to recognize their peers). It turned out that in raw figures the interpreters were slightly more severe, but the scores from the two groups correlated and the difference was not significant. These results indicated that I could use either interpreters or non-interpreters.

5. Sound-files or transcripts?

I designed a study where the intelligibility part of the interpretings was assessed by non-interpreters from both sound-files and transcripts. One group assessed transcripts (with normalized orthography and punctuation) and the other sound files. The sound files got slightly worse scores than the transcripts, but again the difference was not significant and all the scores correlated. So from this respect I could use either sound-files or transcripts.

I ended up going for transcripts. This decision mostly came from the insight that Collados Aís provided on how deceitful the voice is when it comes to assessment of product. Pitch, intonation, accent, security and so forth affects the impression of the quality of the product. Clearly, this aspect is important for the assessment of the interpreting, but with the aim in this study to assess only the skill to transfer an entire message in one language into another it seemed wise to exclude it, too many confounding variables.

6. The assessment

The assessment units ended up looking like this:

Intelligibility

First the raters saw only the interpretation and they rated that according to the scale from completely unintelligible to completely intelligible, from 1 (lowest) to 6 (highest). They also had a sheet with the full explanation of each step of the scale next to them when rating. If you’re curious I left a copy of the sheet in English here.

Informativeness

Then the raters unfolded the sheet of paper and the European parliament’s official translation showed up at the bottom. Then they rated the informativeness of the interpreting, i.e. the difference between the original and the interpretation. This time from no difference compared to the original to completely different compared to the original. Now the scale is inverted so 1 is the best score and 6 the worst. You may wonder why the scale is inverted this time; I decided to stick with Carroll’s original proposal where a low score is equal to little difference. The zero on the scale means that the interpreters added information not present in the original. This typically happens when something implicit is explicitated or when an additional information or hedge is given.

7. Did it work?

The results I got in my cross-sectional material were very promising, clear differences where I would expect them, i.e. between non-interpreter subjects and interpreter subjects, and between novice interpreters and experienced interpreters. The inter-rater variability, that is the variability of the scores between the different raters, was also low. So far, I’m not sure about the results for my longitudinal material. I did not see differences where I expected them. This may be due to a failing instrument (i.e. my scales) or less difference of the interpreting products than what I expected. To be continued…

Now, there are a few more things to try out with my scales. Obviously, an interpreter trainer would not start transcribing their students’ interpretings and divide them into assessment files before assessing or grading them. But, presumably, the scales could work in a live evaluation as well. I have not yet had an opportunity to test them, but I’m looking forward to that, and I will of course keep you posted.

References

Anderson, L. 1979. Simultaneous Interpretation: Contextual and Translation Aspects. Unpublished Master’s Thesis. Department of Psychology, ConcordiaUniversity, Montreal, Canada

Carroll, John, B. 1966. “An Experiment in Evaluating the Quality of Translations.” Mechanical Translations and Computational Linguistics 9 (3-4): 55-66.

Collados Aís, Á., Iglesias Fernández, E. P. M. E. M., & Stévaux, E. 2011. Qualitätsparameter beim Simultandolmetschen: Interdisziplinäre Perspektiven. Tübingen: Narr Verlag.

Clifford, Andrew. 2005. “Putting the Exam to the Test: Psychometric Validation and Interpreter Certification.” Interpreting 7 (1): 97-131.

Lamberger-Felber, H. 1997. Zur Subjektivität der Evaluierung von Ausgangstexten beim Simultandolmetschen. In N. Grbic & M. Wolf (Eds.), Text – Kultur – Kommunikation. Translation als Forshungsaufgabe (pp. 231–248). Tübingen: Stauffenburg Verlag.

Tiselius, E. 2009. “Revisiting Carroll’s Scales.” In Testing and Assessment in Translation and Interpreting Studies. C. Angelelli and H. Jacobson (eds.). 95-121. ATA Monograph Series. Amsterdam: Benjamins.

Advertisement

My questions for the hangout with Babelverse

Television in Question Marks.

Television in Question Marks. (Photo credit: Wikipedia)

Hello again,

I hope you have had a good week. As I said in my last post I’m happy you plan a Google hangout, and I eagerly wait for it to happen. Meanwhile, I have posted these questions in your form. I hope there will be time to answer them. I can understand if you don’t have the time in the hangout (I will not pout about that), but please feel free to answer them in another context.

  1. Have you been in touch with professional organizations for interpreters? If you have, could you let us know which ones? I do not want to cross-examine them :-), but I wouldn’t mind seeking their advice.
  2. If I were to take a conference interpreting assignment for you, how far in advance could I expect to be confirmed for the assignment? I realize it depends on when you get the assignment, but let’s assume that you get an assignment on April 15 for May 15. Would you immediately give the available interpreters with the right language combination a firm option for May 15? That way they would block that day for you and begin to prepare, but on the other hand you would have to pay them if the assignment got cancelled. If you wait with the confirmation (the assignment may be called off), you risk not having interpreters available, but you would not have to pay the interpreters for an interpreting not done.
  3. For conferences: How do you assign booth mates? Ideally in a meeting with many languages you would want to have as many languages as possible covered directly. With your tech solution that would be easy peasy as theoretically you could have as many interpreters as you wish assigned to one booth. But then again, how would you remunerate them in that case? Stand by time, mike time or both? Also, how would the interpreter working in one booth know who else is working there and with which languages?
  4. Would it be possible to post a video on what both your booth and the work would look like from the interpreters’ side?
  5. Would it be possible for a few of the professional interpreters who have worked for you to either write a blog post about it or make themselves available for questions, (no I will not tear them to pieces)?
  6. What happens if you’re in the middle of a conference interpreting and there is a technical interruption? I guess it could be 1) on the customer’s side, 2) on my side (either computer or internet connection) or 3) somewhere in between. Would you have interpreters on stand-by? A techie on site at the customer’s? And how would that affect my assignment both time and money wise?
  7. And finally, how do you plan to screen you interpreters? Based on credentials? Customer satisfaction? Peer-evaluation? Combination or something else?

Thanks a lot!

Babel precarity – more questions

Electronic red megaphone on stand.

Electronic red megaphone on stand. (Photo credit: Wikipedia)

Hi Babel guys!

You said in the e-mails you sent after my last post that we should talk so that I don’t continue to mis-inform my fellow interpreters. I agree, we should talk, but I think we should discuss it openly, not in private mails or over Skype. As I see it, I’m not mis-informing my colleagues. I’m in doubt, and I say so, if you don’t agree then it’s your job to prove me wrong.

You see, I am, just as many interpreters rather suspicious. We are suspicious because we have had bad experiences. We’re used to agencies who do not deliver what they promise; or deliver something completely different from their promise. People who wants to earn money and where interpreters are commodities. Interpreters often end up being some sort of hostage because agencies calculate that we will not let the client down once we’re there. And this does not only go for conference interpreting – for PSI interpreting it’s usually even worse. Reluctance over the tech bit is only the top of the iceberg. Over the years, there are more times than I would like to remember where I have been completely duped when it comes to working conditions. So let’s keep the discussion in the public space. Feel free to answer through blog posts, comments or other public means.

You said you want me to sing up as an interpreter for you, but you see I’m not ready to do that before I fully understand what conditions you are offering and how it works, and I’m sorry, but your homepage does not provide that yet. I’m also afraid that I will end up in a hostage situation. There are agencies who innocently ask you to sign up or provide your CV. Once you do that, they will give you working conditions, or a pay you cannot work for. But then you’re enrolled, so they will use your name and CV in different bids, in order to prove that they use professional, high-profile interpreters, but in fact they don’t, they give the job to other, less expensive interpreters. You’re just there as the lady in the window in the red-light district. This happens everywhere in the industry both in PSI and conference interpreting.

I guess I should not be surprised that my last post received quite a bit of attention. I see you have already written a new blog post treating some of the topics I brought up, and although I doubt that I’m such an important power, I suppose the timing of InterpretAmerica’s recent blog post may have something to do with this as well.

I find it surprising though that so far there has been very little open debate and discussions about Babelverse. Having doubts about a particular solution or player does not mean being tech or development hostile. This is a possible paradigm shift, or disruption, as Kathy Allen over at InterpretAmerica calls it. Then it should be justified to air questions and opinions publicly. Yes, I saw that there will be a Google hangout and that the topic will be discussed in a panel at InterpretAmerica (I must have magic timing). It’s very good that it’s happening now, but before this, as far as I know, the only serious attempt to debate it has been an #IntJC some 5 months ago, and quite frankly, it did not dissolve my doubts. I was also wondering about the participants in the Google hangout – are any of your panelists critical of your idea?

Yes, Josef and Mayel, I know you have attempted to have a Skype conversation with me, the first time I aired some doubts. I did not follow-up on that in the end, because I felt that these are questions I’m sure I’m not the only one to ask, and the discussion needed a greater audience, just as you did with #IntJC. No hard feelings, but the sort of secrecy around the set up does not make me less suspicious.

In your mail to me after the last post you say that I’m incorrect in assuming that interpreters are paid per minute. I’d be more than happy to correct that, after all, what I want you to do, is to develop your platform so that it does not create precarity. I have a follow-up question though, you say that “Professional interpreters working on conference or event jobs are highly respected on our platform and are not paid per minute”. Great, but how do you pay these interpreters, and, more importantly, how do you pay the other interpreters? I have read on your homepage several times that pay are counted on the basis on many factors, and in you latest blog post you say that your rates are lower than for instance EU or any larger institutions, but you want interpreters to receive a fair income. Fair enough, I’ll wait for the examples, you say you will provide. Just curious, what is a fair income? And without wanting to sound like a whiner, just on the information sharing platform, EU is actually not a very good payer when it comes to freelance per day remuneration – they play with the fact that they (usually) give many days and that they pay taxes and pension funds.

About the State of the Union, since that is also something you took up in your mail. You said it was purely experimental, and that you product should not be judged on that. I agree, and I did not judge your product on that, I merely stated that there is a huge difference between the type of interpreting in a State of the Union-type situation and an ad hoc relief situation. And that if you sell them as equal products (which honestly one was easily led to believe reading what you wrote at that time) you have me worried.

As I read from your blog you realize that high level interpreting have different requirement and strive to create an appropriate working environment for the SoU- type of interpretings, I’m curious to know; Have you been in touch with any professional organizations such as AIIC, IAPTI, ATA when you developed the working environment? Can you show any examples of how the technical solution works? I see that Nataly Kelly mentions you, and that you will come to InterpretAmerica, but have you actually discussed working conditions, pay and working environment with them? You say that you co-operate with professional interpreters (those who are not as tech hostile and sour as I am, I suppose), that is very good, I’m happy about that. Do you have any references? You are not new to marketing, and you know that direct referral is one of the best things to recruit people. How about adding some references to high-profile interpreters, with their credentials, that would be happy to tell the rest of us more about Babelverse and possibly calm my worry?

If we look at other industries we see that outsourcing or relocation to cheaper countries are a reality for many professions and that it has not necessarily been good for either the professionals or the quality of the product. Patentranslator has a recent post about it. It goes without saying that it is a real fear in our business too. This is not about being tech hostile (although there are tech solutions around that will make you hostile like the one described here) or reluctant to change, as said earlier, I love technology that makes my job easier (and hey, I’m a Swede, we’re the people most open to change in the world according to recent research). But this is about being able to trust new players to not deteriorate working conditions or selling interpreters as commodities, we want to keep our jobs and get a fair pay. I’m sorry guys, but you still have some work to do in order to convince me. There’s also the issue of how you screen your interpreters, but this post is already being too long.

And just to set one last thing straight, in case I sound as a spoilt, luxury interpreter on my high horses who wants my booth and my first class flights all over the world, and who will whine if I’m not given the same food as the delegates – there’s nothing of that in the world I live in. I’m a freelance,  I work both as PSI and conference interpreter, I’m a proud AIIC member and a certified PS-interpreter.  I work for private market, in court, at hospitals as well as for institutions. My home market, and my language combination, is a tiny one. Fighting for decent working conditions for ALL interpreters on my market is a Sisyphean labour, but no one will do it for me, so I’ll take the risk of sounding like an old, sour granny. You need to prove yourself in order to earn trust.

Your turn.

Update: Your should also read the Interpreter Diaries’ open letter to Babelverse, and Dolmetschblog’s take on the issue. Both Michelle and Alexander have been (contrary to me) in direct contact with them. Babelverse’s blogpost that I refer to above is here. There is also an Interpreting.info thread on Babelverse here. Do read the comments in this post, since I asked for a debate it’s fair that everyone is heard or read. And a special credit to Lionel – The Liaison interpreter – who started debating this long before I had even started to think about what it would mean to me.

Language enhancement

C-3PO

When you start an interpreting course one of the first things you that may strike you is how the language you thought you knew just fades away. Interpreting is an extremely complex exercise and your language skills have to be extremely solid. Whether we grew up bilingually or learnt languages later, most of us who are (or were) accepted into an interpreting program probably has the equivalent to a C2 level (mother tongue or near-native level according to the Council of Europe). But let’s face it, when we embark on our first consecutive – it feels like we just learnt our first words in that language.

So, although you are a skilled linguist, you will have to work on enhancing your language skills, and probably also the elusive concept of ‘culture générale’. But how do you do it? Since we’re not C3PO we cannot just add another hard drive or software, we just have to do it the good old way. And you probably already know it, but here’s a repetition.

First of all, listen, read, eat and sleep your language. You may have to do this both with your foreign language and your mother tongue. Unfortunately, there is now way around it – you need to listen to radio, read newspapers, listen to the news, both in your mother tongue and in your foreign language and with all the technical aid today this is not too hard. Log on to iTunes and see which pods suit you. I like NPR (the American National Public Radio), BBC, TV5 Monde, RFI (Radio France Internationale) just to mention a few. Many newspapers also have their own pod casts. And if you subscribe to different news apps you will get short flashes in you mobile.

When I brain stormed with my students someone also said “set your mobile, Facebook or web browser to your foreign language”. Translation is a good exercise too, when you translate shorter, idiomatic texts you get a feeling for expressions, idioms, prepositions and so forth. Attention to prepositions cannot be stressed enough, prepositions are probably one of the most difficult areas of language and preposition use has an unfortunate tendency to break down in stressful situations like interpreting. If you’re unsure about language in use, corpora is a good thing, in many multilingual text corpora, current texts are collected in order to compare language in use. Another way of mastering language in use as professor Harris pointed out in the comments is to learn poems or song lyrics by heart. As dull as it may seem it is a wonderful way of learning expressions and idiomatic language use.

Finally, and unfortunately, there is probably no way round vocabulary swotting. Flash cards is a good strategy here and one of my students mentioned Anki. I have not tried it – in my time we used cardboard and felt pen, but time changes :-). For my part I also joined an amateur theater group in English in order to immerse myself as much as I could without leaving Sweden. There are many other opportunities like that via Internet now, and thanks to different local groups you may also find opportunities to meet people IRL.

What’s your best language enhancement strategy? And do spare me of the pillow method, I’m far from sure it’s the best method.

Update: Just to be very clear – an interpreting course will enhance your language skills, but it is NOT a language course. All the basic language learning, including living and working abroad, will have to be done before the course. Otherwise there is little chance you will survive until your last exam.

Questions for fellow interpreters and teachers

visual note-taking conference call notes

visual note-taking conference call notes (Photo credit: Austin Kleon)

Dear friends,

This is just a short note and a quick question. I’m still suffering from holiday 🙂 and I suppose you are too. But I have to start the final planning of my course this autumn (public service interpreting). And here’s a question for you.

As we have discussed both at #IntJC and other fora I would like to test a much more modular style this term. Apart from the obvious modules: memory, note-taking, consecutive and dialogue. I would also like to work with interpreter’s introduction, appearance, body language, figures, names, registers and collegiality. Do you have any other modules you would like to suggest?

Please help me brain storm!

 

Update August 20, 2012: Thank you so much for all your feed back. I start my course today, and I’m very excited to see how everything will work out.

Interpreting related things I think you should do this week

Dog Looking at and Listening to a Phonograph, ...

The Interpreter Diaries has a very nice post on things to do this summer. There are a lot of stuff I would really like to do there. I’m desperate that I will miss the third InterpretAmerica summit, but I will of course follow the proceedings via Twitter (hash tag #IASummit). I was really planning to go to Germany for the workshop and aiic-meetings there, but due to family increase (our dog just had 5 puppies), I’m not sure I will make it. I will continue trying to go there though. And who would not like to go to Glendon to add a C-language (speaking of which, don’t forget the #IntJC on Saturday June 23 on adding a C-language).

However, there are a few things you could do right now in front of you key board. No need to book tickets, no need to stay in a hotel, just click away.

The first one is Holly Behl’s just for fun survey on translator’s and interpreter’s favourite pets. Go ahead and take it, I’m so curious to know any trends. When it comes to interpreters, I have many colleagues who have cats. One of my friends has two horses, two dogs and two cats. I have one dog (with five puppies for the moment…) and a cat, I have had a horse, but due to time constraints (why does a day only have 24 hours!) I limit myself to just riding. So please let us know what pets you have!

The second one is important! Go and vote for InterpretAmerica to have a small business grant. That way they can continue doing the good things they are doing for interpreting. They need to get 250 votes to be considered for a grant. InterpretAmerica is a great initiative to bring interpreters of all sorts together and lobby for the future. I wish we had the same in Europe. I know I shouldn’t just say it but start it…

If you haven’t signed up for Interprenaut’s newsletter yet it’s time to do it. Very interesting spots on interpreting. Interprenaut cover history, books, social issues and she always has a section with hot topics in interpreting. And speaking of Interprenaut, if you want to leave the booth or the keyboard, get outside and help her produce a promotional video for interpreting. All you need to do is film yourself, if possible near a typical sight of your country, and saying in your language: “I’m an interpreter”. Details here. Deadline is June 30th.

These are my suggestions for key board activities as well as away from key board activities. Did I forget anything?

Update: Oh, I forgot the important survey from Lifeinlincs “How could research help you?” Answer here.

Slightly off topic: What was my life and interpreting jobs like five years ago?

Yesterday the patent translator published a post based on a letter his son sent him five years ago, but that was planned not to reach him until yesterday. The letter inspired him to look back five years. His post inspired me to do the same.

Five years ago we lived in Paris, and I was seriously starting to consider doing a PhD in interpreting. Interpreting jobs had picked up after the blow in 2004 when the Swedish conference interpreting market went absolutely dead. In 2004, The Swedish government decided they should spearhead their English only plan for the European Union. They only used the absolute minimum requirements for interpretation and as a result the market collapsed. Many of my colleagues decided to change careers. In 2007, the marked had picked up, and the fact that I lived in Paris also improved one of my unique selling points (proximity) as they liked to call it.

I did not have any teenagers at home, my oldest was 10! We still had an au pair girl living with us, which sometimes is challenging, but mostly really nice both for children, parents and career. I had the great benefit to ride once a week in central Paris, sometimes very tough (old French pedagogy) and sometimes mesmerizing. Other than taking interpreting jobs I also taught French (yes! me! a foreigner! in France!, but I have an FLE teacher degree mind you) and had I blogged in Swedish about Paris and bilingualism mainly. This blog started a little over a year later.

I thoroughly liked Paris and could have stayed for much longer, but we were homeward bound in the summer and I had to decide whether I should apply for a PhD or not. I knew that a PhD would require a lot of work and not necessarily give any more job afterwards. But I also freshly remembered those years after 2004 when I was happy if I got two days interpreting jobs per month. For me – 10 days is a good average – 10 days of assignment means another 10 days of preparation, and considering you also usually travel to and from your assignment and need a few days for admin and stuff, it means that you will fill up you month both financially and work-wise. Two days, however, all but bankruptcy, and the few days I would get in court or for other assignments (I wrote earlier about the depressing situation forinterpreting jobs in Sweden. Thanks to the best husband in the world and also thanks to the wonderful parental benefits in Sweden (I had days saved up in my benefit scheme), I could stay in business.

I celebrated my birthday that year (an even one) on the night train towards the Pyrenees on our first skiing holiday in France (and a few months later with a grand party at the Swedish club). And in the autumn I enrolled in a PhD program on bilingualism with a focus on simultaneous interprting. When I look back 2007 was a good year, but what can a year in Paris be other than… perfect.