Category archive - Uncategorized

Persistent Valve Turning

On December 2014, UdG team together with KCL, IIT and NTUA conducted experiments about persistent autonomy in the context of the valve turning scenario. We have combined different disciplines involving: visual-based detection and localization systems, robust control schemes, learning-by-demonstration techniques for intervention and a temporal planning architecture, to operate many times a valve panel mock-up in front of several disturbances: water currents, blocked valves, panel unknown position and panel occlusion. All these systems and methodologies have been validated individually and in an integrated long term experiment, in which the AUV locates an intervention panel and performs multiple valve turning operations while handling several failures (either spontaneous or induced) that take place along the mission time. Experiments have only been performed in water tank, in which exhaustive experiments were possible testing differents conditions and configurations. The positive results encourage the use of, in the short-term, autonomous robots operating in subsea facilities performing interventions with a cost benefit, in comparison with teleoperated vehicles.

The long-term mission was carried out in a water tank of 16x8x5 meters using the Girona 500 AUV equipped with a 4 Degrees of Freedom electrical manipulator, a Stereo Camera and a specifically designed end-effector, which had a camera in-hand and force and torque sensor. A mock-up panel of 0.8×0.5 meters with 4 valve handles was used to emulate a subsea panel from the offshore industry. To make the environment more realistic, two propellers, able to generate up to 14Kg of thrust each, were placed close to the panel in order to generate lateral water currents. Experiments were performed in a completely autonomous mode, the vehicle ran on its own batteries and all required processing was performed with the on-board computers.

Girona 500 AUV with the valve turning payload in the water tank, the valve panel mock-up and the external propellers for water current perturbation.

Girona 500 AUV with the valve turning payload in the water tank, the valve panel mock-up and the external propellers for water current perturbation.

In the experiment, the vehicle had to locate the intervention panel among different locations and modify the valve handles to achieve different panel configurations. The planning algorithm generated the inspection points to locate the panel, which was located by a vision-based detection system. The vision system was also determining the state of the valve handles, which was used by the planning system to generate the actions to turn the valves and to achieve the desired panel configuration. In the process of turning a valve, several systems were working: vision system for panel and valve detection; robust controller for vehicle and manipulator control; Learning By Demonstration for moving the AUV and the manipulator; reactive fuzzy decision maker for deciding if the task could be completed; and Force and Torque sensor processing for determining the contact and turning of the valves. After attempting a valve turning, the planning checked again the state of the valves, and decided new actions in case the action failed due to perturbations. During more than 3 hours, the AUV changed the panel to 9 different configurations, which required the turning of 29 valves. In order to accomplish these valve turnings, the planner generated 37 valve turning actions: 23 were successful; 10 failed because the valve was blocked; and 4 failed because the platform could not execute the action due to the high water current perturbations.

This table summarizes the results of the long-term experiment which tried to achieve 9 configurations, with 37 valve turning attempts. Some of them were accomplished (23) and the rest were not accomplished due to the fact that the valve was blocked (10) or because the platform could not face the perturbations (4).

This table summarizes the results of the long-term experiment which tried to achieve 9 configurations, with 37 valve turning attempts. Some of them were accomplished (23) and the rest were not accomplished due to the fact that the valve was blocked (10) or because the platform could not face the perturbations (4).

Watch the long-term autonomous intervention:

End of the project: some thoughts from our partner NTUA

The Final Year for the Pandora Project has ended successfully for the whole Consortium, but also for NTUA specifically. During the last Year the NTUA Team succesfully collaborated with all the Partnerns (HWU, UdG, KCL, IIT) for the last scientific and technical contributions, participated in the Final Integration Meeting in UdG (Girona, Spain) . A significant part of the work was dedicated to the fine tuning of the software implementations related to the vehicles (Girona500, Nessie VI) motion and interaction control schemes, which played a very important role in the three Demonstration Scenarios.
Moreover, during the Final Year NTUA worked towards transferring an important part of the PANDORA Project technological achievements to the local NTUA underwater infrastructure. More specific, the NTUA Team: i) developed a local ROS simulator based on the UWSim in order to test robust motion control schemes for underactuated underwater vehicles ii) submitted a journal paper entitled : “Trajectory Tracking with Prescribed Performance for Underactuated Underwater Vehicles with Model Uncertainties” at IEEE Transactions on Control Systems Technology Journal, which for the time being is condiatially accepted for publication iii) Designed and Implemented a small 4 DoF underwater manipulator which was installed on the NTUA Seabotix LBV, iv) Designed and implemeted an EKF based Navigation System for the NTUA UVMS (Seabotix LBV – 4 DoF manipulator), v) Designed and implemented various position, velocity and visual servo control schemes for the NTUA UVMS, which were succesfully tested in the local CSL new test tank.

Experiments at NTUA Local Infrastructure

 Experiments at NTUA Local Infrastructure

HWU Trials in Fort William

On April 2015 the HWU team visited The Underwater Centre site in Fort William (UK) to conduct the final trials with Nessie AUV. The task is the Autonomous Inspection of a human-made structure and in this case the Fort William site represent a perfect place to test our effort in real sea conditions.


Nessie AUV has visited the site during early trials showing its capabilities over a different set of tasks, among which data gathering and long-term navigation. The experience collected during previous trials allowed the HWU team to identify a suitable mission area, monitoring also the effect of tidal currents that can affect the performances of the AUV while operating in shallow waters.


Fort William Mission Area

After some initial preparation runs, the vehicle has been tasked to carefully inspect a portion of the marine pier in complete autonomy, leaving to the team monitoring the evolution of the missions through the use of the operator interfaces developed during the project.


Nessie AUV inspecting pillars and crossbeams at the Underwater Centre

In this scenario the HWU team is able to test the final integration of all the main components of the PANDORA’s software architecture. Contributions from our partners KCL (plannning) and NTUA (robust control), as well as the agile integration process, allowed a successful trial campaign, resulting in 5 days of operations, several gigabytes of collected data and a high number of successful inspections in the real sea environment.


HWU Team with Nessie AUV

Special Issue “Autonomous Robots”

Download the flyer: AURO CFP – Marine Robotoics

~Special Issue Call for Papers~

“Towards Long-Term Autonomy in Marine Robotics”

Guest Editors:
Marc Carreras, University of Girona, Spain
David Lane, Heriot-Watt University, United Kingdom
Francesco Maurelli, Heriot-Watt University, United Kingdom
Kanna Rajan, University of Porto, Portugal

In recent years, persistent autonomous operations have become a key area of interest for marine robotics researchers. As hardware costs have plummeted, sensors measuring various oceanographic properties have proliferated and the use of robotic platforms within the ocean science community has increased, the need for increased autonomy to perform tasks over large spatial and temporal durations. The challenge in doing so, is particularly severe in the context of the marine environment however, and especially for robotic assets to be observable and communicable over space and time. Over and beyond making time-series measurements marine robots have demonstrated their capability to respond to episodic events, perform targeted sample collection, track dynamic phenomenon in rough coastal environments and make quasi-synoptic observations in the meso-scale.

However, there continue to be significant challenges to marine robotic operations. While commercial deep-water oilfield inspection with autonomous vehicles is now a commercial reality, fielded robots continue to rely heavily on accurate a priori models of the subsea assets and expose limited capabilities for autonomous decision making.

Most autonomous vehicles in the marine environment are limited to preplanned missions, or to limited forms of autonomy involving script switching and re-parametrisation in response to pre-programmed events. Realizing the persistent autonomy that users in the ocean increasingly demand is involving a greater capability in understanding sensed events to detect failure and error, and more capable task planning approaches that can adapt behaviour and control in novel ways.

Topics of interest include, but are not limited to:
Autonomous long-term navigation, localization and SLAM
Automated dynamic re-planning, planning under uncertainty
Semantic-based world modelling, probabilistic approaches in ontologies
Architectures for long-term autonomy
Robust learning techniques
Probabilistic graphical models
Bio-inspired and bio-mimetic approaches
Multi-vehicle cooperation potentially in multiple domains (air, surface, underwater)

In this special issue of Autonomous Robots journal, we invite:
Research papers to report innovative work in the field (up to 20 pages)
Applied research case-studies to analyse industrial needs, current states and needs for current and future operations (up to 20 pages)

Important Dates:
Paper submission deadline: 15th October 2014
First reviews completed: 15th January 2015
Revised papers due: 15th February 2015
Potential publication date: Summer 2015

Manuscripts must be submitted to: Choose “Long-term autonomy in Marine Robotics” as the article type.

Path Planning and Energy Estimation

On the 5th of May, KCL visited NTUA to work on path planning, and to make some initial steps in energy estimation.

NTUA + KCL teams

NTUA + KCL teams

The first goal of the week was to generate a fast motion for the AUV (Nessie) and to explore the different types of motion capable by NTUA’s controllers. In response to the former objective, the team at NTUA developed a new controller for trajectory-tracking, to complement the existing waypoint-tracking controller. The new controller avoids lateral motions, and relies on surge. This makes it more energy efficient.

Using both controllers, the planner is able to combine the fast and controlled motions to achieve more complex, but reliable behaviour, such as slowing down to move through tight spaces or performing fly-by inspections. Most importantly, the AUV will move faster.

Nessie quickly converging to path in simulation

Nessie quickly converging to path in simulation

The integration on energy estimation, while still preliminary, is promising. The estimated thrust required for a trajectory was made available through a ROS service, estimated using the dynamics of the vehicle and the relevent motion control.

Next steps in this direction involve HWU: converting the thrust into energy estimations for KCL. Accessable from the knowledge-base and ontology, this information will be used to replan when energy costs for the current action grow too high, or to perform opportunistic planning when a task comes in under budget.

Integrating task planning and ontology

Researchers from HWU visited KCL on the 12th of March to continue integration of the planning and ontology nodes of the PANDORA architecture. The work produced a new node which collates and distributes information. By implanting an ontology in the new node, HWU are able to reason about the collected information and provide the planner with dynamic updates that affect the plan that is currently executing.

HWU + KCL teams

HWU + KCL teams

In order to test this link, a new scenario has been devised: the AUV will be placed in a completely unknown environment, and be asked to count the pillars. In reality the tank contains one pillar and one buoy. Pillars are recognised from sensor data, but only after inspection from multiple angles.

Using the integration between the ontology and the planner, new goals and inspection areas can be dynamically created and passed to the planner, as well as new knowledge about obstacles that invalidate the current plan. The planner will replan, when required, avoiding detected obstacles and carefully inspecting the objects.

The scenario will be run first in simulation, and then in the wave tank at HWU.

Testing planning capabilities with Girona 500 AUV

Researchers from KCL visited UdG between the 17th and 24th of January, successfully completing work on integrating the controllers of the valve turning scenario into the planning architecture. This involved running the scenario many times during the week, with the Girona 500 AUV.

The scenario begins with a panel, hidden somewhere in the sparsely decorated pool. The AUV is given a set of coordinates at which the panel might be. The planner directs the AUV to search for the panel, move close to inspect the valves, detect their orientation, and finally to turn the valves to the correct configuration.

The architecture was modeled to be robust — replanning when new information is discovered, or the environment does not meet expectations. This means that when the valves do not turn as expected, the Girona 500 AUV would try again and again, grasping persistently with its end effector.

KCL + UdG Teams

During the week, the two teams met to decide how their collaboration should continue. The next steps will integrate planning with the chain cleaning scenario; further improve the valve turning scenario; and combine both to form a longer, more elaborate mission.

Water Jet Control using the Girona500 AUV

From the 14th to the 21st of June, researchers from NTUA came to the University of Girona to do some tests with the Girona500 equipped with the manipulator.

Arnau Carrera (UdG), Narcís Palomeras (UdG), George Karras (NTUA) and Charalampos (Babis) Bechlioulis (NTUA) during the experiments in the UdG water tank
Arnau Carrera (UdG), Narcís Palomeras (UdG), George Karras (NTUA) and Charalampos (Babis) Bechlioulis (NTUA) during the experiments in the UdG water tank

During this period of time, the NTUA and UdG teams worked together and redesigned the control of the manipulator, obtaining a new, more precise and smooth controller. Additionally, the water jet was installed on top of the manipulator, and the control algorithm to keep the robot stable in front of the panel was improved, so as to be used while the water jet is working and being moved using the arm.

Teleoperated valve turning using Girona500 AUV

From the 3rd to the 7th of June, members of the team at IIT came to the University of Girona to do some tests with the Girona500 equipped with a manipulator.

Arnau Carrera (UdG), Matte Leonetti (IIT) and Nawid Jamal (IIT) during the experiments in the UdG water tank
Arnau Carrera (UdG), Matte Leonetti (IIT) and Nawid Jamali (IIT) during the experiments in the UdG water tank

In this short space of time, the two teams worked on integrating a haptic device to teleoperate the Girona500 AUV and the manipulator. Once the integration was completed, several trials were performed to demonstrate the task. These experiments have highlighted some weak points in the first theoretical approach and provides the opportunity to improve on this work.

There was also discussion relating to the integration of the force torque sensor, and a decision was made to add a camera to the hand on the manipulator. Furthermore, some experiments involving thruster failures in the Girona500 AUV were performed.

Postdoc opening in Machine Learning for Robotics

The Department of Advanced Robotics at the Italian Institute of Technology (an English-language research institute) is seeking to appoint a well-motivated full-time postdoctoral researcher in the area of machine learning applied to robotics in general, and in particular to Autonomous Underwater Vehicles (AUV).

The successful candidate will join an ongoing research project funded by the European Commission under FP7 in the category Cognitive Systems and Robotics called “PANDORA” (Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning) which started in January 2012. The project is a collaboration of five leading universities and institutes in Europe: Heriot Watt University (UK), Italian Institute of Technology (Italy), University of Girona (Spain), King’s College London (UK), and National Technical University of Athens (Greece). Details about the project can be found at:

The accepted candidate will contribute to the development and experimental validation of novel reinforcement learning and imitation learning algorithms for robot control, as well as their specific application to autonomous underwater vehicles. The research will be conducted at the Department of Advanced Robotics within the “Learning and Interaction Group” with project leader Dr. Petar Kormushev.

The research work will include conducting experiments with two different AUVs (Girona 500 and Nessie V) in water tanks in Spain and UK in collaboration with the other project partners. The developed machine learning algorithms can also be applied to other robots available at IIT, such as the compliant humanoid robot COMAN, the hydraulic quadruped robot HyQ, the humanoid robot iCub, two Barrett WAM manipulator arms, and a KUKA LWR arm robot.

Application Requirements:

  • PhD degree in Computer Science, Mathematics or Engineering
  • Excellent publication record
  • Strong competencies in: machine learning, reinforcement learning, imitation learning
  • Good programming skills, preferably in MATLAB and C/C++
  • Experience in robot control and ROS is a plus

International applications are encouraged. The successful candidate will be offered a fixed-term project collaboration contract for the remaining duration of the project due to end in December 2014 with a highly-competitive salary which will be commensurate with qualifications and experience. Expected starting date is as soon as possible, preferably before September 1st, 2013.

Application Procedure:

To apply please send a detailed CV, a list of publications, a statement of research interests and plans, degree certificates, grade of transcripts, the names of at least two referees, and other supporting materials such as reference letters to: Dr. Petar Kormushev (petar.kormushev(at), quoting [PANDORA-PostDoc] in the email subject. For consideration, please apply by June 21th, 2013.

For latest updates please check here.