Have you talked to your robot today? A new AI project will teach humans and robots to communicate through words, gestures and other intuitive cues.
SDU Robotics is participating in a new research project to strengthen communication between humans and robots in manufacturing companies. In particular, the robot must be trained to understand how we feel during work, just as a human co-worker would.
"The idea is to transform robots into colleagues. A robot that understands what you say and is aware of your well-being."
That's what Professor Nobert Krüger from SDU Robotics says. He represents SDU in the Fluently project, a size of nearly 19 million Euros. Fluently will facilitate communication and interaction between the operator and the robot. The project aims to make the title 'operator' redundant because the goal is that we no longer operate a robot but collaborate with it, at the end robot and operator are supposed to be a team.
In the long term, we exchange experiences with the robot as if it were a real colleague.
Redundant daily programming
And it is needed. The cooperation between employees and robots is more crucial than ever because they increasingly share the workspace in Danish production halls. Denmark is one of the world's leading nations in robotics. According to a calculation from Dansk Metal, we are among the ten countries with the most industrial robots per employee.
This development contributes to increasing the productivity of companies and can, at the same time, improve the working environment by leaving more monotonous and tedious work to our robot colleagues. But it also means the employees get a new role in the workplace.
The arrival of the robots has meant that everyone has had to acquire the necessary skills to handle the robots, including programming.
"But imagine if we could get rid of programming the robots and could instead simply communicate with them through speech or gestures and hand signs? That is the goal of Fluently. In short, we want to develop a kind of electronic interpreter that automatically translates speech and gestures into robot instructions. This way, you avoid slow instructions via an IPad. It will create far better collaboration between the employee and the robot and, at the same time, increase productivity," says Nobert Krüger.
The project will develop a 'Fluently Smart Interface unit'. It is a small device equipped with artificial intelligence and software. 'The Fluently Smart Interface device may not enable you to speak fluently robotically, but it will be able to understand what you're saying, and you'll be able to communicate seamlessly.
The robot may not be Freud, but it can be an attentive colleague.
Another central aspect of the project is that thanks to the 'Fluently Smart Interface' unit, the robot can also assess the operator's physiological and psychological state. As it is today, an operator learns to operate a robot.
Still, in the Fluently project, the operator and robot will go to a joint training camp, where they get to know each other after a week of training in a laboratory.
In the training camp, the operator will be attached to a series of biosensors, which read, e.g. temperature, heart rate and sweat production. Along the way, the robot learns to connect the collected data with the operator's voice so that in the future, it can read the mental state based on the voice alone. Because even if a robot reduces ergonomically unfavorable tasks for employees, an operator might still need to feed the robot with items.
The 'Fluently Smart Interface' device lets the robot sense if the operator is tired and slow down, which means that the operator might soon come into a state where he or she feels worn out, and the car factory thus reduces sickness and the downtime that production would otherwise experience by reducing the stress imposed to his human workers.
"In this way the operator gets to know the robot while at the same time the robot learns about operator's preferences and working styles. At the same time, the robot recognizes the operator's voice and can hear if he or she is a little stressed. That way, it knows that now is probably not the time to pick up the pace, but on the contrary, slow down a bit so that the operator can follow along. Or the robot provides information that the human – because of lack of attention – is at risk at overlook. Like a good colleague would do," says Nobert Krüger.
The professor emphasizes that Fluently is not only about increasing productivity; it's about creating a workplace where people and machines harmoniously work side by side as a team.
The project is supported by the European Union's Horizon Europe program and runs from 2022 to 2025.
The project is supported by the European Union's Horizon 2020 programme and runs from 2023 to 2026. Read more at the projects webpage here.