The successful candidate will join an ongoing research project funded by the European Commission under FP7 in the category Cognitive Systems and Robotics called “PANDORA” (Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning) which started in January 2012. The project is a collaboration of five leading universities and institutes in Europe: Heriot Watt University (UK), Italian Institute of Technology (Italy), University of Girona (Spain), King’s College London (UK), and National Technical University of Athens (Greece). Details about the project can be found at: http://persistentautonomy.com/

The accepted candidate will contribute to the development and experimental validation of novel reinforcement learning and imitation learning algorithms for robot control, as well as their specific application to autonomous underwater vehicles. The research will be conducted at the Department of Advanced Robotics within the “Learning and Interaction Group” with project leader Dr. Petar Kormushev.

The research work will include conducting experiments with two different AUVs (Girona 500 and Nessie V) in water tanks in Spain and UK in collaboration with the other project partners. The developed machine learning algorithms can also be applied to other robots available at IIT, such as the compliant humanoid robot COMAN, the hydraulic quadruped robot HyQ, the humanoid robot iCub, two Barrett WAM manipulator arms, and a KUKA LWR arm robot.

Application Requirements:

• PhD degree in Computer Science, Mathematics or Engineering
• Excellent publication record
• Strong competencies in: machine learning, reinforcement learning, imitation learning
• Good programming skills, preferably in MATLAB and C/C++
• Experience in robot control and ROS is a plus

International applications are encouraged. The successful candidate will be offered a fixed-term project collaboration contract for the remaining duration of the project due to end in December 2014 with a highly-competitive salary which will be commensurate with qualifications and experience. Expected starting date is as soon as possible, preferably before September 1^st, 2013.

Application Procedure:

To apply please send a detailed CV, a list of publications, a statement of research interests and plans, degree certificates, grade of transcripts, the names of at least two referees, and other supporting materials such as reference letters to: Dr. Petar Kormushev (petar.kormushev(at)iit.it), quoting [PANDORA-PostDoc] in the email subject. For consideration, please apply by June 21^th, 2013.

For latest updates please check here.

Reproduction of the chain scenario at UdG’s water tank.

Before the first year review of the project, we conducted some experiments inside the UdG water tank to simulate inspection of the chain by means of sonar.

Girona-500, equipped with ARIS, was manually teleoperated along the chain gathering images at a short range to generate afterwards an acoustic mosaic of high resolution.

The following video summarizes the mosaicing process of the sonar images:

The figure below shows the obtained full chain mosaic:

Pandora partners from NTUA visited HW Januarty 2013 to try their robust model based control system on Nessie VI. In addition to testing and tuning the algorithm which provides 5DOF waypoint control to the vehicle, HW took the opportunity to test newly developed sonar analysis software. The overall trial objective was to integrate independently developed systems into a coherent whole. The new sonar wall pose estimator was connected to the new NTUA control system to create a wall following behaviour.

HW has a large 20x20x7m wave tank which was used for the experiment. This facility has the nice capability of generating waves. We had hoped these waves could test the robustness of the controller under current disturbances. However, the waves energy is largely on the surface of the tank, so it is questionable as to whether this was a good test or not. Nevertheless, control and integration testing was a success, with both the NTUA and HW team managing to achieve all their goals for the 5 day period of wave tank testing.

You can see the final day of results for yourself in the video. The AUV is extremely steady when performing a wall following routine. The AUV is also able to perform rapid movements between set waypoints. In the video some ringing is observed, particularly when the AUV is commanded to do extreme pitches. Pitch is the most unstable DOF on the AUV. We hope with further tuning we might improve the control, but the aim of the experiment was about getting the software working talking within the larger system correctly rather than absolute performance.

Last month the UdG team received a new piece of equipment for the PANDORA project: the ARIS Forward-Looking Sonar (FLS). This sonar generates high-resolution acoustic images at a near-video rate, and can play a key role on those underwater inspections where the water visibility does not allow the use of optical cameras. In the chain scenario of the PANDORA project, the process of cleaning the chain is prone to generate turbidity in the water which can difficult the algorithms taking control of the cleaning process itself as well as the subsequent inspection. By using a forward-looking sonar we plan to work with acoustic images and overcome this lack of visibility.
We have worked towards the development of an algorithm for the generation of acoustic mosaics and we have done some preliminar tests with the ARIS sonar in the water tank of the UdG. Although the sonar is still not integrated to Girona-500, it has been attached to the vehicle and we have used the ARIS commercial software to gather images of several small objects placed on the bottom of the water tank. The robot was driven in a zig-zag trajectory along three different tracklines gathering around 1500 different sonar frames. The developed mosaicing algorithm was able to register successfully a high number of frames, including many loop closures, achieving the consistent mosaic shown in the figure below.

Girona-500 with ARIS installed for the experiments.

Mosaic generated from the ARIS frames.

Panel template is taken from a close-up image of the panel. A mask is used to highlight the static marks on the panel and ignore the valve handles.

Valve detection is performed by first detecting the panel and making known of the rigid geometry to localise the valves with respect to the centre of the panel. To detect the orientation of the valves, a Hough transform is applied first to detect lines within a bounding box around each valve. The orientation of each valve is then obtained by searching for lines of specified minimum length within the bounding box.

The figures show the detected panel and valves highlighted with white lines at approximate distances 2m and 1m. The detection of valve orientation at large distances can be inaccurate and is only considered at short distances.

The NTUA-CSL team in collaboration with the IIT team have integrated a general identification scheme in ROS, based on AI learning techniques recently developed in IIT. Three nodes have been designed:

• /excitationG500 (implemented in Python): This node provides the appropriate system inputs that will excite the system in order to learn the unknown parameters.

• /learningMetricCalcG500 (implemented in Python): This node evaluates an estimated parameter set based on the measurements it collects from the /navigatorG500 node and the /excitationG500 node.

• /learningG500 (implemented in C++): This node implements the learning process and produces estimates of the unknown parameters based on the evaluation its gets from the /learningMetricCalcG500 node.

The whole process has been synchronized such that the /learningG500 node produces a new estimation of the unknown parameters only after it gets a feedback from the /learningMetricCalcG500 node with the metric of its previous parameter estimate. The aforementioned procedure is repeated until the learning algorithm has converged, based on certain predefined cretiria. The whole scheme is general in the sence that it can be applied to all kind of dynamical systems possessing parametric uncertainties no matter if the unknown parameters appear linearly or not, as long as we have state and input measurements.

What is SLAM?

Simultaneous Localisation and Mapping (SLAM) is a way of improving estimates of vehicle position in unknown environments. We estimate the position of landmarks based on the current vehicle position, and we can then use knowledge of these (stationary) landmarks to infer the position of the vehicle. By relying on a fixed reference, we can reduce the error due to drift.

PHD Filter and SLAM

The Probability Hypothesis Density (PHD) filter is a suboptimal Bayes filter used for multi-object estimation. Here, we are required to estimate the number and position of an unknown number of objects. The PHD filter performs this while eliminating the need for data association. We can combine this with SLAM by using the PHD filter to represent the landmarks. More technically, this forms a single cluster process, with the vehicle position as the parent state, and the landmarks as the daughter states conditioned on a vehicle position. This formulation is a form of feature-based SLAM since we approximate landmarks as point features.

Figure: Simulation of SLAM with a combination of detected and undetected landmarks

Detecting and Estimating Map Features

The PHD SLAM formulation only relies on a set of point observations. The algorithm does not change depending on whether we are using sonar or vision. Consequently, this offers the potential to combine these two sources using a single technique – as long as we can detect useful features from the sensors! Currently, we are relying on image feature extractors such as SURF and ORB to detect features from our stereo camera. In the coming months we will consider features from the forward looking sonar as well as apply PHD SLAM to real data.

Challenges For PANDORA

Computational resources are particularly constrained on AUVs. SLAM algorithms are notoriously computing intensive. One option available land robotics is the use of CUDA computing architectures to brute force around the problem, but for the underwater domain there are no suitable embedded CUDA systems. Therefore one big challenge for integration in PANDORA is adapting cutting edge SLAM algorithms to run on our embedded systems.
Another difficulty associated with the underwater domain is combining SLAM with sonar data. Standard forward looking sonars are unable to accurately localise in the depth dimension, thus observations are underconstrained. Furthermore, sonar pings do not have reliable high frequency components that optical vision does – this means that common feature extractors, such as SIFT, do not see anything on sonar data. In PANDORA we will be using next generation sonars to get better sonar data into the SLAM system, and developing new feature detectors that compliment SLAM in an underwater domain better.

On the other hand, the valve panel has been built and is ready to start working with it in the water tank at CIRS. The first steps will be focus on the visual detection of the valve panel and the valves positions. Also, the valve panel in the simulator has been updated to match the one built.

In this video we want to show the speed and kind of movement which the robotic arm is capable of. The movement will be quite different from the one done when the arm is installed in the robot because the control of the arm will have also take in consideration the DoF of the robots, which can make more movements possible. To control the arm for this video we have used the commercial software provided by the manufacturer.

The most important component of the learning layer is a learning algorithm. The learning algorithm search the parameter space in order to find a parameter vector that maximizes the agent’s performance. The parameters come from a class of parametric functions selected to represent the corrections to the control actions. The output of the parametric function is added to the given controller as correction. We devised a learning algorithm to perform iterative optimization of the parameters based on a series of experiments in the UWSim simulator of Girona500.

We simulated a very strong current, stronger than the thrusters. The controller we used was a simple PD controller. The task was to reach an object on the seabed and hover at less than 0.5 m above it for as long as possible, to take pictures. Since such a controller reacts to the current when the vehicle is already being carried away, it cannot by itself perform the task. The agent attempts to reach the target object in several trials, going out of the current to the initial location. It managed to correct the given controller by navigating out of the current as long as possible, and trying to resist the current at full power in the last part of the trajectory.