This page presents our research on robot learning and interaction. The descriptions of past research themes are available here.

### Robot Learning & Interaction

Our work focuses on human-centric robotic applications in which the robots can learn new skills by interacting with the end-users. From a machine learning perspective, the challenge is to acquire skills from only few demonstrations and interactions, with strong generalization demands. It requires: 1) the development of intuitive active learning interfaces to acquire meaningful demonstrations; 2) the development of models that can exploit the structure and geometry of the acquired data in an efficient way; 3) the development of adaptive control techniques that can exploit the learned task variations and coordination patterns.

The developed models must serve several purposes (recognition, prediction, online synthesis), and be compatible with different learning strategies (imitation, emulation, incremental refinement or exploration). For the reproduction of skills, these models need to be enriched with force and impedance information to enable human-robot collaboration and to generate safe and natural movements.

These models and algorithms can be applied to a wide range of robotic applications, with robots that are either close to us (assistive robots in I-DRESS), parts of us (prosthetic hands in TACT-HAND), or far away from us (manipulation skills in deep water with DexROV).

**Techniques:**

- A distinctive view of optimal control
- Probabilistic models of movements and skills
- Geometry-aware statistical learning and regression

**Applications:**

- Assistive teleoperation of a bimanual underwater robot
- Personalized assistance in dressing
- Tensor-structured data for the control of prosthetic hands
- Intuitive user interfaces for the generation of natural movements

### Techniques

#### A DISTINCTIVE VIEW OF OPTIMAL CONTROL

Model predictive control (MPC) and linear quadratic tracking (LQT) are widespread techniques in robot control, but the core formulation of these problems and their associated algorithms can be extended to a wider range of problems, often underexploited in robot learning. In particular, the most simple form of MPC (unconstrained and linear, with a double integrator system) already has advantage for motion synthesis and planning problems, where it can be combined elegantly with probabilistic representations of movements.

This method allows the retrieval of smooth and natural trajectories in a fast and simple manner, by considering variation and coordination constraints. Instead of learning the trajectory directly, the approach allows the learning of the underlying controller to drive the robot. Namely, it learns to reject perturbations only in the directions that would affect task performance (minimal intervention control). This can typically be exploited with torque-controlled robots to regulate the tracking gain and compliance required to reproduce a task in an adaptive manner.

Interestingly, when combined with mixture models, this basic form of MPC also has links with Bézier curves and the use of dynamic features in generative models such as hidden Markov models (trajectory-HMM).

**References:**

Calinon, S. (2016). **A Tutorial on Task-Parameterized Movement Learning and Retrieval**. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2016). **Stochastic learning and control in multiple coordinate systems**. Intl Workshop on Human-Friendly Robotics (HFR). info pdf

#### PROBABILISTIC MODELS OF MOVEMENTS AND SKILLS

In many robotics applications, demonstrations or experiences are sparse. In such situation, it is important to get as much information as possible from each demonstration. We explore approaches encoding demonstrations from the perspective of multiple coordinate systems. This is achieved by providing a list of observers that could potentially be relevant for the movement or the task to transfer. The collected datapoints are organized in a tensor form (multidimensional arrays). A statistical learning approach is then used to determine the variability and coordination patterns in the movement by considering the different coordinate systems simultaneously, which then allows the orchestration of the different coordinate systems to reproduce the movement in a new situation (e.g., adaptation of movements to new positions of objects).

This approach provides better generalization capability than the conventional approach of associating task parameters (what describes the situation) with movement parameters (what describes the skill), which would consist of mapping the two as a joint distribution, and then use regression to retrieve a new motion for a given new situation. The proposed task-parameterized models instead exploit the structure of the task parameters, which can in many robotics problems be expressed in the form of coordinate systems or local projections. It was shown that such approach can provide extrapolation capability that could not be achieved by treating the problem as standard regression.

**References:**

Calinon, S. (2016). **A Tutorial on Task-Parameterized Movement Learning and Retrieval**. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2015). **Robot learning with task-parameterized generative models**. Intl Symp. on Robotics Research (ISRR). info pdf

#### GEOMETRY-AWARE STATISTICAL LEARNING AND REGRESSION

Data in robot learning and control problems have structures and symmetries that are often underexploited. We explore the use of Riemannian manifolds to provide geometry-aware algorithms that can be exploited in several sensing and actuation problems.

Examples of Riemannian manifolds in robotics with well known geometries include stiffness and damping gains, inertia, manipulability ellipsoids (symmetric positive definite matrices), orientations (unit quaternions), period movements (phase variable on circle manifold), or rotary joints (e.g., a 2-link planar robot forms a torus manifold).

We explore the use of differential geometry techniques to treat data of various forms in a uniform probabilistic manner (including data in standard Euclidean spaces). In particular, we exploit the potential that several Riemannian manifolds can be combined together to treat problems such as learning/controlling the pose of an end-effector (position and orientation), or learning/controlling both periodic and discrete movements.

It can also be exploited to revisit common optimization problems in robotics formulated in standard Euclidean spaces. This is typically achieved by removing the constraint of staying on a given manifold, and instead considering an unconstrained problems in spaces that inherently take into account this constraint.

**References:**

Zeestraten, M.J.A., Havoutis, I., SilvĂ©rio, J., Calinon, S. and Caldwell, D.G. (2017). **An Approach for Imitation Learning on Riemannian Manifolds**. IEEE Robotics and Automation Letters (RA-L), 2:3, 1240-1247. info pdf

Jaquier, N. and Calinon, S. (2017). **Gaussian Mixture Regression on Symmetric Positive Definite Matrices Manifolds: Application to Wrist Motion Estimation with sEMG**. In Proc. of the IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS). info pdf

### Applications

#### ASSISTIVE TELEOPERATION OF A BIMANUAL UNDERWATER ROBOT

We develop approaches recasting teleoperation as a collaborative human-robot teamwork. The combination of model predictive control and task-parameterized probabilistic models is explored as a way to cope with teleoperation with long transmission delays (up to a second), and/or the simultaneous teleoperation of many degrees of freedom.

In this approach, the model parameters are first transmitted to both the teleoperator side and the robot side. The task parameters on the robot side can then update the model of the skill at fast pace by local sensing, without requiring the transmission of this change to the teleoperator.

As an example, if the robot has observed that the task of drilling requires the drill to be perpendicular when it approaches a surface, the robot will then automatically orient the drill when it approaches another surface, letting the teleoperator concentrate on the position to drill by delegating the orientation tracking aspect to the robot. The robot will automatically react to perturbations transparently to the user (e.g., when reorienting the surface). The aim is to reduce the cognitive load of the teleoperator for repetitive or well structured tasks.

**Project website:** http://www.dexrov.eu/

**References:**

Havoutis, I. and Calinon, S. (2017). **Supervisory teleoperation with online learning and optimal control**. In Proc. of the IEEE Intl Conf. on Robotics and Automation (ICRA). info pdf

Tanwani, A.K. and Calinon, S. (2016). **Learning Robot Manipulation Tasks with Task-Parameterized Semi-Tied Hidden Semi-Markov Model**. IEEE Robotics and Automation Letters (RA-L), 1:1, 235-242. info pdf

#### PERSONALIZED ASSISTANCE IN DRESSING

We study how to transfer assistive skills to robots, by letting the robot acquire not only movements, but also force and compliance behaviors. We explore extensions of movement primitives frequently employed in robotics to a wider repertoire of skills composed of reactive, time-dependent and time-independent behaviors based on force, impedance, position and orientation information.

Assisting a person to dress is a challenging case study, in which the robot needs to adapt to different morphologies, pathologies or stages of recovery, with varied requirements for movement generation and physical interaction. Since this assistance is person-dependent, with preferences and requirements that can evolve with time, it cannot be pre-programmed. Dressing assistance is typically provided by healthcare workers, which is not always convenient. From the worker perspective, the activity takes time and is not particularly gratifying. From the patient perspective, such assistance is often viewed negatively because it reduces the sense of independence.

In this context, learning from demonstration could provide a solution to transfer dressing assistance skills to the robot. We explore if this could be achieved by means of kinesthetic teaching, where demonstrations are used to let the robot acquire person-specific requirements and preferences.

**Project website:** https://www.i-dress-project.eu/

**Reference:**

Pignat, E. and Calinon, S. (2017). **Learning adaptive dressing assistance from human demonstration**. Robotics and Autonomous Systems, 93, 61-75. info pdf

#### TENSOR-STRUCTURED DATA FOR THE CONTROL OF PROSTHETIC HANDS

We apply Riemannian geometry approaches and tensor methods to the control of prosthetic hands by exploiting several sensor modalities. A tactile bracelet is providing sensory data organized as a cylindric grid, which are augmented with noisy surface electromyography (sEMG) signals typically processed as spatial covariance features. Instead of flattening or vectorizing the above data, we aim at treating the different sources statistically as a joint distribution, by keeping the original geometry and structure of the data.

**Project website:** http://www.idiap.ch/project/tact-hand/

**Reference:**

Jaquier, N., Castellini, C. and Calinon, S. (2017). **Improving Hand and Wrist Activity Detection Using Tactile Sensors and Tensor Regression Methods on Riemannian Manifolds**. Myoelectric Controls Symposium (MEC). info pdf

#### INTUITIVE USER INTERFACES FOR THE GENERATION OF NATURAL MOVEMENTS

Many computer aided design applications require the generation of continuous traces, where the most common interface is to edit the control points of some form of interpolating spline. In this research, conducted in collaboration with Daniel Berio and Frederic Fol Leymarie at Goldsmiths University of London, we study how tools from computational motor control can be used to extend such methods to encapsulate movement dynamics, precision and coordination.

Our approach relies on a probabilistic formulation of model predictive control, by providing users with a simple interactive interface to generate multiple movements and traces at once by defining a distribution of trajectories rather then a single one. A dynamical system is then used to generate natural looking curves stochastically.

The method is applied to the generation of traces and patterns that are similar to the ones that can be seen in art forms such as calligraphy and graffiti.

**Reference:**

Berio, D., Calinon, S. and Fol Leymarie, F. (2017). **Generating Calligraphic Trajectories with Model Predictive Control**. In Proc. of the 43rd Conf. on Graphics Interface. info pdf