Effective Dexterous ROV Operations in Presence of Communication Latencies

Project Information
Funding Agency EU Horizon2020
My Role Co-PI and Work Package Leader
Project Website
Duration March 2015 to August 2018
Coordinator Space Applications Services NV, Belgium
Partners Compagnie Maritime D'Expertises (COMEX) SA, France
Universita degli Studi di Genova, Italy
Jacobs University Bremen gGmbH, Germany
Fondation de l'Institut de Recherche IDIAP, Switzerland
Graal Tech SRL, Italy
EJR-Quartz BV, Netherlands

My role in the project

As co-proposer and work package leader (WP3, “Underwater perception and modeling”), my responsibilities include the supervision of Jacobs contributions to the project as well as inter-partner collaboration to reach the work package goals. My main research contributions are within the fields of visual odometry and SLAM with stereo cameras. Research and development work done under my supervision include advances in object recognition and localization as well as real-time simulation and integration. I am also responsible for producing technical and financial reports to the EC.

Executive Summary

DexROV focuses on assistive capabilities and high-latency telemanipulation in the context of offshore oil and gas inspection and maintenance. The effort aims to reduce the cognitive load on the operators and allow the operators to work from an onshore control center connected via satellite to the offshore vessel carrying the ROV (remotely operated vehicle), thus reducing personnel and logistics offshore. Jacobs focuses on underwater 3D perception and world modeling to facilitate semi-autonomous functions.

See also the project entry on Cordis. More news about DexROV published by the EC in late 2015.

What is it like to work on a collaborative project?

EJR-Quartz, one of the DexROV partners, produced a video showcasing our first review and interviews of a number of senior researchers from the consortium, including myself. We describe what our motivation is, and what collaborative projects and DexROV in particular mean to us.

Simulator Overview

We developed a simulation environment for use in integration and testing. This video shows the current state. Note that a hydrodynamic motion model (contributed by ISME) is used for the ROV motion.

Introduction Video

Project Objectives

Underwater operations (e.g. oil industry) are demanding and costly activities for which ROV based setups are often deployed in addition to deep divers – contributing to operations risks and costs cutting.

However the operation of a ROV requires significant off-shore dedicated manpower – such a setup typically requires a crew consisting of: (1) an intendant, (2) an operator, and (3) a navigator. This is a baseline, and extra staffing is often provisioned. Furthermore, customers representatives often wish to be physically present at the off-shore location in order to advise on, or to observe the course of the operations. Associated costs are high.

In order to reduce the burden of operations, DexROV will work out more cost effective and time efficient ROV operations, where manned support is in a large extent delocalized onshore (i.e. from a ROV control center), possibly at a large distance from the actual operations - thus with latencies in the communication. As a main strategy to mitigate them, DexROV will develop a real time simulation environment to accommodate operators’ requests on the onshore side with no delays. The simulated environment will exploit cm accuracy 3D models of the environment built online by the ROV, using data acquired with underwater sensors (3D sonar and vision based). A dedicated cognitive engine will analyse user’s control requests as done in the simulated environment, and will turn them into primitives that the ROV can execute autonomously in the real environment, despite the communication latencies.

Effective user interfaces will be developed for dexterous manipulation, including a double advanced arm and hand force feedback exoskeleton. The ROV will be equipped with a pair of new force sensing capable manipulators and dexterous end-effectors: they will be integrated within a modular skid.

The outcomes of the project will be integrated and evaluated in a series of tests and evaluation campaigns, culminating with a realistic offshore trial.

Related Papers

T. Łuczyński, M. Pfingsthorn and A. Birk. Image Rectification with the Pinax Camera Model in Underwater Stereo Systems with Verged Cameras. IEEE/MTS OCEANS 2017 Anchorage, IEEE Press, September 2017.
T. Fromm, C. Müller, M. Pfingsthorn, A. Birk and P. Di Lillo. Efficient Continuous System Integration and Validation for Deep-Sea Robotics Applications. IEEE/MTS OCEANS 2017 Aberdeen, IEEE Press, June 2017.
J. Gancet, P. Weiss, G. Antonelli, M. Pfingsthorn, S. Calinon, A. Turetta, C. Walen, D. Urbina, S. Govindaraj, C. Müller, X. Martinez, T. Fromm, B. Chemisky, G. Indiveri, G. Casalino, P. Di Lillo, E. Simetti, D. De Palma, A. Birk, A. Tanwani, I. Havoutis, A. Caffaz, L. Guilpain and P. Letier. Dexterous Undersea Interventions with Far Distance Onshore Supervision: The DexROV Project. 10th IFAC Conference on Control Applications in Marine Systems, CAMS, September 2016. Open Access
M. Pfingsthorn, R. Rathnam, T. Luczynski and A. Birk. Full 3D Navigation Correction using Low Frequency Visual Tracking with a Stereo Camera. IEEE/MTS OCEANS 2016 Shanghai, IEEE Press, April 2016.
J. Gancet, D. Urbina, P. Letier, M. Ilzkovitz, P. Weiss, F. Gauch, B. Chemisky, G. Antonelli, G. Casalino, G. Indiveri, A. Birk, M. Pfingsthorn, S. Calinon, A. Turetta, C. Walen and L. Guilpain. DexROV: Enabling effective dexterous ROV operations in presence of communication latency. IEEE/MTS OCEANS 2015 Genova, Italy, IEEE Press, May 2015.
J. Gancet, D. Urbina, P. Letier, M. Ilzkovitz, P. Weiss, F. Gauch, G. Antonelli, G. Casalino, G. Indiveri, A. Birk, M. Pfingsthorn, S. Calinon, A. Tanwani, A. Turetta, C. Walen and L. Guilpain. DexROV: Dexterous Undersea Inspection and Maintenance in Presence of Communication Latencies. 4th IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles, NGCUV, April 2015.
© 2015-2019 Max Pfingsthorn. Made with Hugo, Bootstrap, and FontAwesome.