This page presents my research on robot learning and interaction. The descriptions of past research themes are available here.


Robot Learning & Interaction

My work focuses on human-centric robotic applications in which the robots can learn new skills by interacting with the end-users. From a machine learning perspective, the challenge is to acquire skills from only few demonstrations and interactions, with strong generalization demands. It requires: 1) the development of intuitive active learning interfaces to acquire meaningful demonstrations; 2) the development of models that can exploit the structure and geometry of the acquired data in an efficient way; 3) the development of adaptive control techniques that can exploit the learned task variations and coordination patterns.

The developed models must serve several purposes (recognition, prediction, online synthesis), and be compatible with different learning strategies (imitation, emulation, incremental refinement or exploration). For the reproduction of skills, these models need to be enriched with force and impedance information to enable human-robot collaboration and to generate safe and natural movements.

These models and algorithms can be applied to a wide range of robotic applications, with robots that are either close to us (assistive robots in I-DRESS), parts of us (prosthetic hands in TACT-HAND), or far away from us (manipulation skills in deep water with DexROV).

Research themes: Applications:

Research themes

An unconventional view of optimal control

Model predictive control (MPC) and linear quadratic tracking (LQT) are widespread techniques in robot control, but the core formulation of these problems and their associated algorithms can be extended to a wider range of problems, which is often underexploited in robot learning. In particular, the most simple form of MPC (unconstrained and linear, with a double integrator system) already has advantage for motion synthesis and planning problems, where it can be combined elegantly with probabilistic representations of movements. As a planning technique, this method allows the retrieval of smooth and natural trajectories in a fast and simple manner, by considering variation and coordination constraints. Instead of learning the trajectory directly, the approach allows the learning of an underlying controller. This can typically be exploited with torque-controlled robots to regulate the tracking gain and compliance required to reproduce a task in an adaptive manner.

Interestingly, when combined with mixture models, this basic form of MPC also has links with Bézier curves and the use of dynamic features in generative models such as hidden Markov models (trajectory-HMM).

References:

Calinon, S. (2016). A Tutorial on Task-Parameterized Movement Learning and Retrieval. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2016). Stochastic learning and control in multiple coordinate systems. Intl Workshop on Human-Friendly Robotics (HFR). info pdf

MPC combined with a statistical representation of the task to achieve. An MPC problem is typically composed of a tracking cost and a control cost (top part of the image), which is minimized to find a set of control commands or a set of tracking gains. The weighting terms and the target references in the cost function can be learned from demonstrations, with a compact generative model of the task (bottom part of the image). Solving the MPC problem with a double integrator system (center of the image) results in an analytical solution corresponding to a controller smoothly tracking the stepwise reference generated by the model, anticipating the step changes and modulating the tracking gains in accordance to the extracted precision and coordination patterns.

Probabilistic models of movements and skills

Statistical learning in multiple coordinate systems, and retrieval in a new situation.


Task-parameterized Gaussian mixture model (TP-GMM) combined with model predictive control (MPC).

In many robotics applications, demonstrations or experiences are sparse. In this situation, it is important to get as much information as possible from each demonstration. We explore approaches encoding each demonstration from the perspective of multiple coordinate systems. This is achieved by providing a list of observers that could potentially be relevant for the movement or the task to transfer. The collected datapoints are organized in a tensor form (multidimensional arrays). A statistical learning approach is then used to determine the variability and coordination patterns in the movement by considering the different frames of reference simultaneously, which then allows the orchestration of these different frames are then to reproduce an observed movement in a new situation (e.g., adaptation of movements to new positions of objects).

This approach provides better generalization capability than the conventional approach of associating task parameters (what describes the situation) with movement parameters (what describes the skill), which would consist of mapping the two as a joint distribution, and then use regression to retrieve a new motion for a given new situation. The proposed task-parameterized models instead exploit the structure of the task parameters, which can in many robotics problems be expressed in the form of coordinate systems. It was shown that such approach can provide extrapolation capability that could not be achieved by treating the problem as standard regression.

References:

Calinon, S. (2016). A Tutorial on Task-Parameterized Movement Learning and Retrieval. Intelligent Service Robotics (Springer), 9:1, 1-29. info pdf

Calinon, S. (2015). Robot learning with task-parameterized generative models. Intl Symp. on Robotics Research (ISRR) info pdf


Geometry-aware statistical learning and regression

Data in robot learning and control problems have various structures and symmetries that are often underexploited. We explore the use of Riemannian manifolds to provide geometry-aware statistical learning and regression techniques, which can be exploited in several sensing and actuation problems in robotics.

Examples of Riemannian manifolds in robotics with well known geometries include stiffness and damping gains, inertia, manipulability ellipsoids (symmetric positive definite matrices), orientations (unit quaternions), period movements (phase variable on circle manifold), or rotary joints (e.g., a 2-link planar robot forms a torus manifold).

We explore the use of differential geometry techniques to treat data of various forms in a uniform probabilistic manner (including data in standard Euclidean spaces). In particular, we exploit the potential that several Riemannian manifolds can be combined together to treat problems such as learning/controlling the pose of an end-effector (position and orientation), or learning/controlling both periodic and discrete movements.

It can also be exploited to revisit common optimization problems in robotics formulated in standard Euclidean spaces. This is typically achieved by removing the constraint of staying on a given manifold, and instead considering an unconstrained problems in spaces that inherently take into account this constraint.

Reference:

Zeestraten, M.J.A., Havoutis, I., Silvério, J., Calinon, S. and Caldwell, D.G. (2017). An Approach for Imitation Learning on Riemannian Manifolds. IEEE Robotics and Automation Letters (RA-L). info pdf

Minimization of geodesic distances as a gradient descent in the tangent space of the current solution estimate on the Riemannian manifold.


Gaussian mixture model (GMM) on a Riemannian manifold, where the centers of the Gaussians are defined on the manifold, and the covariances are defined in the tangent spaces of the corresponding centers.

Applications

Assistive teleoperation of a bimanual underwater robot

Assistive teleoperation by exploiting probabilistic models for both classification and synthesis purposes.


Minimal intervention controller able to exploit the task variations extracted from the set of demonstrations.

We develop approaches recasting teleoperation as a collaborative human-robot teamwork. The combination of model predictive control and task-parameterized probabilistic models is explored as a way to cope with teleoperation with long transmission delays (up to a second), and/or the simultaneous teleoperation of many degrees of freedom.

In this approach, the model parameters are first transmitted to both the teleoperator side and the robot side. The task parameters on the robot side can then update the model of the skill at fast pace by local sensing, without requiring the transmission of this change to the teleoperator.

As an example, if the robot has observed that the task of drilling requires the drill to be perpendicular when it approaches a surface, the robot will then automatically orient the drill when it approaches another surface, letting the teleoperator concentrate on the position to drill, by delegating the orientation tracking aspect to the robot. The robot will automatically react to perturbations transparently to the user (e.g., when reorienting the surface). The aim is to reduce the cognitive load of the teleoperator for repetitive or well structured tasks.

Project website: http://www.dexrov.eu/


Personalized assistance in dressing

We study how to transfer assistive skills to robots, by letting the robot aquire not only movements, but also force and compliance behaviors. We explore extensions of movement primitives frequently employed in robotics to a wider repertoire of skills composed of reactive, time-dependent and time-independent behaviors based on force, impedance, position and orientation information.

Assisting a person to dress is a challenging case study, in which the robot needs to adapt to different morphologies, pathologies or stages of recovery, with varied requirements for movement generation and physical interaction. Since this assistance is person-dependent, with preferences and requirements that can evolve with time, it cannot be pre-programmed. Dressing assistance is typically provided by healthcare workers, which is not always convenient. From the worker perspective, the activity takes time and is not particularly gratifying. From the patient perspective, such assistance is often viewed negatively because it reduces the sense of independence.

In this context, learning from demonstration could provide a solution to transfer dressing assistance skills to the robot. We explore if this could be achieved by means of kinesthetic teaching, where demonstrations are used to let the robot acquire person-specific requirements and preferences.

Project website: https://www.i-dress-project.eu/

Adaptive robotic assistance to put on a coat or to put on shoes.