The work in WP5 is focused on developing, implementing and assessing learning based methods to support the application work packages with methods that can encode various types of sensory data and provide both low and high-level decision making in real-time.
The read thread in the conducted scientific work is transfer – considering key scientific problems such as transfer learning, learning transfer, representation learning in the context of transfer learning, skill transfer and adaptation, multimodal learning, to name some. To this end, we have initiated the collaborative work on a survey paper on transfer learning – to understand better how the term is used and what methodologies are used to model the related problems in the areas of robotics, machine learning, computer vision and artificial intelligence at large. To support the intake from the scientific community with specific focus on robotics, a workshop on the topic of transfer learning in robotics is organized at IEEE ICRA 2023. You can find more information here.
One direction of the ongoing work includes transfer learning of a peg-in-a-hole insertion task for manipulators including Panda, UR10, and a Do-It-Yourself manipulator. The focus is on learning transfer between robots. There is also work on exploring a complementary approach for transfer through the design of a multiple objective cost function for a whole-body model predictive control system. It is applied to a humanoid robot TALOS performing a deburring task and it is planned to test it on a Tiago robot. There is also work on the automatic generation of datasets of grasping movements. They have developed a method to generate large repertoires of diverse grasping trajectories (about 10⁴ trajectories in 3-4 hours). The method does not make strong assumptions about the end effector and has been tested in simulation on two-fingers effectors as well as on dexterous hands (Allegro hand). The approach able to produce, in a simulated environment, thousands of grasping trajectories that cover the whole operational space, and in particular the most of the object surface compatible with a grasp of the given end-effector. The focus now is on sim2real transferability and on the exploitation of the dataset to generate grasping controllers.