This page presents our research on robot learning and interaction. The descriptions of past research themes are available here.


Robot Learning & Interaction

My work focuses on human-centric robotic applications in which the robots can learn new skills by interacting with the end-users. From a machine learning perspective, the challenge is to acquire skills from only few demonstrations and interactions, with strong generalization demands. It requires: 1) the development of intuitive active learning interfaces to acquire meaningful demonstrations; 2) the development of models that can exploit the structure and geometry of the acquired data in an efficient way; 3) the development of adaptive control techniques that can exploit the learned task variations and coordination patterns.

The developed models must serve several purposes (recognition, prediction, online synthesis), and be compatible with different learning strategies (imitation, emulation, incremental refinement or exploration). For the reproduction of skills, these models need to be enriched with force and impedance information to enable human-robot collaboration and to generate safe and natural movements.

These models and algorithms can be applied to a wide range of robotic applications, with robots that are either close to us (assistive robots in I-DRESS), parts of us (prosthetic hands in TACT-HAND), or far away from us (manipulation skills in deep water with DexROV).

Research themes: Applications:

Research themes

A distinctive view of optimal control

Model predictive control (MPC) and linear quadratic tracking (LQT) are widespread techniques in robot control, but the core formulation of these problems and their associated algorithms can be extended to a wider range of problems, often underexploited in robot learning. In particular, the most simple form of MPC (unconstrained and linear, with a double integrator system) already has advantage for motion synthesis and planning problems, where it can be combined elegantly with probabilistic representations of movements.

This method allows the retrieval of smooth and natural trajectories in a fast and simple manner, by considering variation and coordination constraints. Instead of learning the trajectory directly, the approach allows the learning of the underlying controller to drive the robot. Namely, it learns to reject perturbations only in the directions that would affect task performance (minimal intervention control). This can typically be exploited with torque-controlled robots to regulate the tracking gain and compliance required to reproduce a task in an adaptive manner.

Interestingly, when combined with mixture models, this basic form of MPC also has links with Bézier curves and the use of dynamic features in generative models such as hidden Markov models (trajectory-HMM).

References:

Calinon, S. (2016). A Tutorial on Task-Parameterized Movement Learning and Retrieval. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2016). Stochastic learning and control in multiple coordinate systems. Intl Workshop on Human-Friendly Robotics (HFR). info pdf

MPC combined with a statistical representation of the task to achieve. An MPC problem is typically composed of a tracking cost and a control cost (top part of the image), which is minimized to find a set of control commands or a set of tracking gains. The weighting terms and the target references in the cost function can be learned from demonstrations, with a compact generative model of the task (bottom part of the image). Solving the MPC problem with a double integrator system (center of the image) results in an analytical solution corresponding to a controller smoothly tracking the stepwise reference generated by the model, anticipating the step changes and modulating the tracking gains in accordance to the extracted precision and coordination patterns.

Probabilistic models of movements and skills

Statistical learning in multiple coordinate systems, and retrieval in a new situation.


Task-parameterized Gaussian mixture model (TP-GMM) combined with model predictive control (MPC).

In many robotics applications, demonstrations or experiences are sparse. In such situation, it is important to get as much information as possible from each demonstration. We explore approaches encoding demonstrations from the perspective of multiple coordinate systems. This is achieved by providing a list of observers that could potentially be relevant for the movement or the task to transfer. The collected datapoints are organized in a tensor form (multidimensional arrays). A statistical learning approach is then used to determine the variability and coordination patterns in the movement by considering the different coordinate systems simultaneously, which then allows the orchestration of the different coordinate systems to reproduce the movement in a new situation (e.g., adaptation of movements to new positions of objects).

This approach provides better generalization capability than the conventional approach of associating task parameters (what describes the situation) with movement parameters (what describes the skill), which would consist of mapping the two as a joint distribution, and then use regression to retrieve a new motion for a given new situation. The proposed task-parameterized models instead exploit the structure of the task parameters, which can in many robotics problems be expressed in the form of coordinate systems or local projections. It was shown that such approach can provide extrapolation capability that could not be achieved by treating the problem as standard regression.

References:

Calinon, S. (2016). A Tutorial on Task-Parameterized Movement Learning and Retrieval. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2015). Robot learning with task-parameterized generative models. Intl Symp. on Robotics Research (ISRR). info pdf

The proposed task-parameterized approach can generalize the five demonstrations (in semi-transparent color) to new situations, by providing a trajectory distribution adapted to the situation (new position and orientation of the two objects).
    
The conventional method would be to treat the problem as regression. It would consist of jointly learning the movement parameters and the task parameters, corresponding to the position and orientation of the objects (left), and then retrieve a movement based on new task parameters used as inputs of a regression problem. Such approach is generic since the task parameters can take any form, and it can usually interpolate well (top-left). However, such approach fails at extrapolating to situations that are further away (bottom-left). It is for this reason that we prefer to exploit a task-parameterized approach exploiting candidate coordinate systems, providing extrapolation capability in problems that can be structured in the form of coordinate systems, which is most often the case in robotics.

Geometry-aware statistical learning and regression

Data in robot learning and control problems have structures and symmetries that are often underexploited. We explore the use of Riemannian manifolds to provide geometry-aware algorithms that can be exploited in several sensing and actuation problems.

Examples of Riemannian manifolds in robotics with well known geometries include stiffness and damping gains, inertia, manipulability ellipsoids (symmetric positive definite matrices), orientations (unit quaternions), period movements (phase variable on circle manifold), or rotary joints (e.g., a 2-link planar robot forms a torus manifold).

We explore the use of differential geometry techniques to treat data of various forms in a uniform probabilistic manner (including data in standard Euclidean spaces). In particular, we exploit the potential that several Riemannian manifolds can be combined together to treat problems such as learning/controlling the pose of an end-effector (position and orientation), or learning/controlling both periodic and discrete movements.

It can also be exploited to revisit common optimization problems in robotics formulated in standard Euclidean spaces. This is typically achieved by removing the constraint of staying on a given manifold, and instead considering an unconstrained problems in spaces that inherently take into account this constraint.

Reference:

Zeestraten, M.J.A., Havoutis, I., Silvério, J., Calinon, S. and Caldwell, D.G. (2017). An Approach for Imitation Learning on Riemannian Manifolds. IEEE Robotics and Automation Letters (RA-L), 2:3, 1240-1247. info pdf

Minimization of geodesic distances as a gradient descent in the tangent space of the current solution estimate on the Riemannian manifold.


Gaussian mixture model (GMM) on a Riemannian manifold, where the centers of the Gaussians are defined on the manifold, and the covariances are defined in the tangent spaces of the corresponding centers.

Applications

Assistive teleoperation of a bimanual underwater robot

Assistive teleoperation by exploiting probabilistic models for both classification and synthesis purposes.


Minimal intervention controller able to exploit the task variations extracted from the set of demonstrations.

We develop approaches recasting teleoperation as a collaborative human-robot teamwork. The combination of model predictive control and task-parameterized probabilistic models is explored as a way to cope with teleoperation with long transmission delays (up to a second), and/or the simultaneous teleoperation of many degrees of freedom.

In this approach, the model parameters are first transmitted to both the teleoperator side and the robot side. The task parameters on the robot side can then update the model of the skill at fast pace by local sensing, without requiring the transmission of this change to the teleoperator.

As an example, if the robot has observed that the task of drilling requires the drill to be perpendicular when it approaches a surface, the robot will then automatically orient the drill when it approaches another surface, letting the teleoperator concentrate on the position to drill by delegating the orientation tracking aspect to the robot. The robot will automatically react to perturbations transparently to the user (e.g., when reorienting the surface). The aim is to reduce the cognitive load of the teleoperator for repetitive or well structured tasks.

Project website: http://www.dexrov.eu/

References:

Havoutis, I. and Calinon, S. (2017). Supervisory teleoperation with online learning and optimal control. In Proc. of the IEEE Intl Conf. on Robotics and Automation (ICRA). info pdf

Tanwani, A.K. and Calinon, S. (2016). Learning Robot Manipulation Tasks with Task-Parameterized Semi-Tied Hidden Semi-Markov Model. IEEE Robotics and Automation Letters (RA-L), 1:1, 235-242. info pdf


Personalized assistance in dressing

We study how to transfer assistive skills to robots, by letting the robot acquire not only movements, but also force and compliance behaviors. We explore extensions of movement primitives frequently employed in robotics to a wider repertoire of skills composed of reactive, time-dependent and time-independent behaviors based on force, impedance, position and orientation information.

Assisting a person to dress is a challenging case study, in which the robot needs to adapt to different morphologies, pathologies or stages of recovery, with varied requirements for movement generation and physical interaction. Since this assistance is person-dependent, with preferences and requirements that can evolve with time, it cannot be pre-programmed. Dressing assistance is typically provided by healthcare workers, which is not always convenient. From the worker perspective, the activity takes time and is not particularly gratifying. From the patient perspective, such assistance is often viewed negatively because it reduces the sense of independence.

In this context, learning from demonstration could provide a solution to transfer dressing assistance skills to the robot. We explore if this could be achieved by means of kinesthetic teaching, where demonstrations are used to let the robot acquire person-specific requirements and preferences.

Project website: https://www.i-dress-project.eu/

Reference:

Pignat, E. and Calinon, S. (2017). Learning adaptive dressing assistance from human demonstration. Robotics and Autonomous Systems, 93, 61-75. info pdf

Adaptive robotic assistance to put on a coat (as shown here) or to put on shoes.

Handling of tensor-structured data for the control of prosthetic hands

Joint treatment of sEMG and tactile sensing data for the control of a prosthetic hand.

We apply Riemannian geometry approaches and tensor methods to the control of prosthetic hands by exploiting several sensor modalities. A tactile bracelet is providing sensory data organized as a cylindric grid, which are augmented with noisy surface electromyography (sEMG) signals typically processed as spatial covariance features. Instead of flattening or vectorizing the above data, we aim at treating the different sources statistically as a joint distribution, by keeping the original geometry and structure of the data.

Project website: http://www.idiap.ch/project/tact-hand/

Reference:

Jaquier, N., Castellini, C. and Calinon, S. (2017). Improving Hand and Wrist Activity Detection Using Tactile Sensors and Tensor Regression Methods on Riemannian Manifolds. Myoelectric Controls Symposium (MEC). info pdf


Intuitive user interfaces for the generation of natural movements

Many computer aided design applications require the generation of continuous traces, where the most common interface is to edit the control points of some form of interpolating spline. In this research, conducted in collaboration with Daniel Berio and Frederic Fol Leymarie at Goldsmiths University of London, we study how tools from computational motor control can be used to extend such methods to encapsulate movement dynamics, precision and coordination.

Our approach relies on a probabilistic formulation of model predictive control, by providing users with a simple interactive interface to generate multiple movements and traces at once by defining a distribution of trajectories rather then a single one. A dynamical system is then used to generate natural looking curves stochastically.

The method is applied to the generation of traces and patterns that are similar to the ones that can be seen in art forms such as calligraphy and graffiti.

Reference:

Berio, D., Calinon, S. and Fol Leymarie, F. (2017). Generating Calligraphic Trajectories with Model Predictive Control. In Proc. of the 43rd Conf. on Graphics Interface. info pdf

Alphabet letter edited by the proposed interface, enabling the definition of a path, together with additional variation and coordination information to generate naturally-looking variants of the path.