Hello Kerstin, can you give a quick introduction to yourself and your projects ?
Sure. My name is Kerstin Fischer, I am Professor (MSO) for Language and Technology Interaction at the SDU. One of my best known projects is the human-robot interaction lab in Sønderborg. For a comprehensive overview about all my projects, just check my research profile.
What is your academic background?
Originally, I started out as a linguist, with a specialization in cognitive linguistics, pragmatics and language use, working mostly on phenomena of spoken interaction. However, already while I was working on my PhD thesis, I got into contact with human-computer interaction, and I continued working on human-computer dialog as a postdoc in the Verbmobil project in Hamburg. There I met a roboticist who wanted me to evaluate his system, and this is how I started working on human-robot interaction.
Linguistics and human-robot interaction are two different fields of research. How did you continue from that point?
Nowadays, I still do research in linguistics, but most of my work is on human-robot interaction with the goal to create better systems in the long run – that is, robots that are useful, easy to use and that people like to interact with. More recently, I’ve been using robots to study speech and interaction, thus bridging the gap between my two areas of expertise.
The HRI-lab at the SDU Sønderborg offers a wide range of robots.
You are the director of the HRI-lab in Sønderborg. What is this lab about?
Initially, my work in human-robot interaction took place either in cooperation with roboticists who had a robot and wanted to know how it performed or how they should implement the one or other interface. But when I came to Sonderborg, I also needed to provide my students the opportunity to carry out experiments, and I had accumulated so many cool questions I wanted to address that it was clear that I had to build up a lab.
Setting up a new lab from scratch is a serious effort and costly as well. How did you do that and how did you get it funded?
I was lucky that at around this time, I got a really great student, Lars Christian Jensen, who was very interested in robots and in human-robot interaction, and together, we wrote proposals to get funding for some small robots we could program ourselves or have our students operate them. By now, we have a nice little robot zoo with robots for all kinds of purposes.
Next level: Lars Christian Jensen received his doctor's degree in the meantime. To be more precise, it is a PhD in human-robot interaction.
So what kind of research do you do at the HRI-Lab Sønderborg?
In some respects, we do general human-robot interaction research. For instance, we have carried out studies on the legibility of robot approach – that’s a common topic in the field. However, in many other respects, we are quite different from everyone else, for instance, by using robots to study linguistic features, such as charismatic speech, intonation contours or hesitation markers, small words like "um" and "uh".
Sounds interesting, can you give an example for that?
We have one project with the somewhat bulky title "Improving second language pedagogy at the prosody - pragmatics interface using human-robot interaction." In this project, we study the effects of particular linguistic features by having robots produce them. The good thing about robots is that they do exactly what they are programmed to do and can be manipulated in ways humans cannot – which makes them the ideal confederates for our studies. We can even trace the interactional effects of the linguistic feature under consideration – which is pretty impossible using other methods.
Have you recently found something unexpected?
Yeah, many cool things! For instance, we carried out one study on intercultural differences between Danes and Germans, where we investigated different ways of breaking bad news. We had observed that bad news in Denmark are often presented without signs of empathy, when in contrast one would expect such signs in Germany. We first made participants choose between four yummy types of chocolate and had a robot then tell them that the chocolate they had chosen is not available.
Selina Eisenberger used Keepon robots for intonation swap studies.
Why would you choose a robot over a human in this case?
We used a robot because a robot, unlike people, can provide exactly the same information in exactly the same way to as many participants as you wish only with the one difference whose effect is under consideration, here signs of empathy and emotion. What we found was quite surprising, namely that emotional expression is connected to formality in Danish, such that it is only used in formal situations, whereas in informal situations it is regarded to be neither necessary nor appropriate.
The communication between Danes and Germans is notoriously rich in misunderstandings. Pronouncation and intonation are so different. Did you research on that as well?
Of course. In another study, we looked at the effects of using one’s native language intonation when asking questions in a second language. For example, a native speaker of Danish who is learning German may use Danish speech melody when asking a question in German; we wanted to know what impression this makes on native speakers. We had two robots take turns in asking the demographic questions of a questionnaire, where one robot used the original speech melody whereas the other used the speech melody from the other language.
What were the findings of this study?
We found that Danes perceive the robot with German intonation as formal, whereas Germans find the robot with Danish intonation to be dominant. Thus, learning about the speech melody of the target language will be useful, and this is what we are also working on: creating teaching materials for hard to grasp linguistic information like intonation.
Formal Germans, dominant Danes: Poster on teaching intonation.
Better teaching material is required, I did not learn about that during language classes. What other things did you find out about spoken language?
We investigate in human-robot interaction experiments how persuasive certain statements are. For example, in an experiment that lasted almost ten minutes, the robot said only once that most participants drank half a liter after the experiment – or not. This single utterance made people drink almost three times as much after the experiments than if the robot did not say this sentence. So some utterances are really powerful.
So you cover quite a range of topics. How do these correlate with your academic background and how can you use them in day-to-day work?
My own background is in discourse, pragmatics and cognitive linguistics, and thus from these areas stem the phenomena I’m looking at. However, the method we’ve developed in the HRI-lab is suited to analyze very many different phenomena. Thus it is great to collaborate with people who bring in their expertise on other phenomena. Collaborations are a vital part of modern research, combining the skills of different research areas can be very fruitful.
Oliver Niebuhr uses the resources of the HRI-lab for his phonetic research.
Can you give me an example for such a collaboration?
For instance, I collaborate a lot with Oliver Niebuhr. He is a phonetician, and many of the studies we are currently running are the result of our joint work. But we’ve also put together a set of tools for our students and people from all over the world to experiment with linguistic features using our robots and the HRI-lab. Just contact us.
Thank you very much for the interview, Kerstin.
You are welcome.
This interview had been conducted by Sascha Steinhoff in Kolding, 19th of July, 2019.