This article was first published on <a href="https://news.mit.edu/2022/battery-free-wireless-underwater-camera-0926" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>. &nbsp;<hr>Scientists estimate that more than 95 percent of Earth&rsquo;s oceans have never been observed, which means we have seen less of our planet&rsquo;s ocean than we have the far side of the moon or the surface of Mars.The high cost of powering an underwater camera for a long time, by tethering it to a research vessel or sending a ship to recharge its batteries, is a steep challenge preventing widespread undersea exploration.MIT researchers have taken a major step to overcome this problem by developing a battery-free, wireless underwater camera that is about 100,000 times more energy-efficient than other undersea cameras. The device takes color photos, even in dark underwater environments, and transmits image data wirelessly through the water.The autonomous camera is powered by sound. It converts mechanical energy from sound waves traveling through water into electrical energy that powers its imaging and communications equipment. After capturing and encoding image data, the camera also uses sound waves to transmit data to a receiver that reconstructs the image.&nbsp;Because it doesn&rsquo;t need a power source, the camera could run for weeks on end before retrieval, enabling scientists to search remote parts of the ocean for new species. It could also be used to capture images of ocean pollution or monitor the health and growth of fish raised in aquaculture farms.&ldquo;One of the most exciting applications of this camera for me personally is in the context of climate monitoring. We are building climate models, but we are missing data from over 95 percent of the ocean. This technology could help us build more accurate climate models and better understand how climate change impacts the underwater world,&rdquo; says Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab, and senior author of a new paper on the system.Joining Adib on the paper are co-lead authors and Signal Kinetics group research assistants Sayed Saad Afzal, Waleed Akbar, and Osvy Rodriguez, as well as research scientist Unsoo Ha, and former group researchers Mario Doumet and Reza Ghaffarivardavagh. The paper is <a href="https://www.nature.com/articles/s41467-022-33223-x" target="_blank">published today in Nature Communications</a>.<iframe src="https://www.youtube.com/embed/kyVZ1ll6_qY?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-gtm-yt-inspected-7="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 768px; height: 433px;"></iframe><h2 id="isPasted">Going battery-free</h2>To build a camera that could operate autonomously for long periods, the researchers needed a device that could harvest energy underwater on its own while consuming very little power.The camera acquires energy using transducers made from piezoelectric materials that are placed around its exterior. Piezoelectric materials produce an electric signal when a mechanical force is applied to them. When a sound wave traveling through the water hits the transducers, they vibrate and convert that mechanical energy into electrical energy.Those sound waves could come from any source, like a passing ship or marine life. The camera stores harvested energy until it has built up enough to power the electronics that take photos and communicate data.To keep power consumption as a low as possible, the researchers used off-the-shelf, ultra-low-power imaging sensors. But these sensors only capture grayscale images. And since most underwater environments lack a light source, they needed to develop a low-power flash, too.&ldquo;We were trying to minimize the hardware as much as possible, and that creates new constraints on how to build the system, send information, and perform image reconstruction. It took a fair amount of creativity to figure out how to do this,&rdquo; Adib says.They solved both problems simultaneously using red, green, and blue LEDs. When the camera captures an image, it shines a red LED and then uses image sensors to take the photo. It repeats the same process with green and blue LEDs.Even though the image looks black and white, the red, green, and blue colored light is reflected in the white part of each photo, Akbar explains. When the image data are combined in post-processing, the color image can be reconstructed.&ldquo;When we were kids in art class, we were taught that we could make all colors using three basic colors. The same rules follow for color images we see on our computers. We just need red, green, and blue &mdash; these three channels &mdash; to construct color images,&rdquo; he says.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1664333175160-MIT-Underwater-Camera-02.jpg" style="width: 766px;" class="fr-fic fr-dib">Fadel Adib (left) associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab, and Research Assistant Waleed Akbar display the battery-free wireless underwater camera that their group developed. Image: Adam Glanzman<h2>Sending data with sound</h2>Once image data are captured, they are encoded as bits (1s and 0s) and sent to a receiver one bit at a time using a process called underwater backscatter. The receiver transmits sound waves through the water to the camera, which acts as a mirror to reflect those waves. The camera either reflects a wave back to the receiver or changes its mirror to an absorber so that it does not reflect back.A hydrophone next to the transmitter senses if a signal is reflected back from the camera. If it receives a signal, that is a bit-1, and if there is no signal, that is a bit-0. The system uses this binary information to reconstruct and post-process the image.&ldquo;This whole process, since it just requires a single switch to convert the device from a nonreflective state to a reflective state, consumes five orders of magnitude less power than typical underwater communications systems,&rdquo; Afzal says.The researchers tested the camera in several underwater environments. In one, they captured color images of plastic bottles floating in a New Hampshire pond. They were also able to take such high-quality photos of an African starfish that tiny tubercles along its arms were clearly visible. The device was also effective at repeatedly imaging the underwater plant&nbsp;Aponogeton ulvaceus in a dark environment over the course of a week to monitor its growth.Now that they have demonstrated a working prototype, the researchers plan to enhance the device so it is practical for deployment in real-world settings. They want to increase the camera&rsquo;s memory so it could capture photos in real-time, stream images, or even shoot underwater video.They also want to extend the camera&rsquo;s range. They successfully transmitted data 40 meters from the receiver, but pushing that range wider would enable the camera to be used in more underwater settings.&ldquo;This will open up great opportunities for research both in low-power IoT devices as well as underwater monitoring and research,&rdquo; says Haitham Al-Hassanieh, an assistant professor of electrical and computer engineering at the University of Illinois Urbana-Champaign, who was not involved with this research.<hr>&quot;Reprinted with permission of&nbsp;<a href="https://news.mit.edu/2022/battery-free-wireless-underwater-camera-0926" id="isPasted" rel="noopener noreferrer" target="_blank">MIT News</a>&quot;&nbsp;

A battery-free, wireless underwater camera developed at MIT could have many uses, including climate modeling. “We are missing data from over 95 percent of the ocean. This technology could help us build more accurate climate models and better understand how climate change impacts the underwater world,” says Associate Professor Fadel Adib. Image: Adam Glanzman

The device could help scientists explore unknown regions of the ocean, track pollution, or monitor the effects of climate change.

Engineers build a battery-free, wireless underwater camera

Tests conducted by Cornell and the U.S. Navy used new algorithms to outperform state-of-the-art programming for autonomous underwater sonar imaging, significantly improving the speed and accuracy for identifying objects such as explosive mines, sunken ships, airplane black boxes, pipelines and corrosion on ship hulls.Sea reconnaissance is filled with challenges that include murky waters, unpredictable conditions and vast areas of subaquatic terrain. Sonar is the preferred imaging method in most cases, but acoustic waves can be difficult to decipher, often requiring different angles and views of an object before it can be identified.&ldquo;If you have a lot of targets and they&rsquo;re distributed over a large region, it takes a long time to classify them all,&rdquo; said Silvia Ferrari, the John Brancaccio Professor of Mechanical and Aerospace Engineering, who led the research published May 24 in the journal&nbsp;<a href="https://ieeexplore.ieee.org/document/9780576" target="_blank">IEEE Journal of Oceanic Engineering</a>. &ldquo;Sometimes an autonomous underwater vehicle won&#39;t be able to finish the mission because it has limited battery life.&rdquo;To improve the capability of these vehicles, Ferrari&rsquo;s research group teamed up with the Naval Surface Warfare Center, Panama City, and the Naval Undersea Warfare Center, Newport, Virginia. The team created and tested a new imaging approach called informative multi-view planning, which integrates information about where objects might be located with sonar processing algorithms that decide the optimal views, and the most efficient path to obtain those views. The planning algorithms take into account the sonar sensor&rsquo;s field-of-view geometry along with each target&rsquo;s position and orientation, and can make on-the-fly adjustments based on current sea conditions.In computer simulated tests, the research team&rsquo;s algorithms competed against state-of-the-art imaging methods to complete multi-target classification tasks. The new algorithms were able to complete the tasks in just half the time, and with a 93% improvement in accuracy of identifying targets. In a second test in which the targets were more randomly scattered, the new algorithms performed the imaging task more than 11% faster, and with 33% more accuracy.&ldquo;Until these algorithms, we were never able to account for the orientation and some of the more complicated automatic target variables that influence the quality of the images,&rdquo; Ferrari said. &ldquo;Now we can accomplish the same imaging tasks with higher accuracy and in less time.&rdquo;As a final test, the algorithms were programmed into a REMUS-100 autonomous underwater vehicle tasked with identifying 40 targets scattered within an area of St. Andrew Bay off the coast of Florida. Performing in its first undersea trial, the new algorithms achieved the same speed as the state-of-the-art algorithms, and with equal or superior classification performance.&ldquo;Demonstrating the developed algorithms using an actual vehicle in sea trials is a very exciting achievement,&rdquo; said Jane Jaejeong Shin, Ph.D. &rsquo;21, who is now an assistant professor of mechanical and aerospace engineering at the University of Florida. &ldquo;This result shows the potential of these algorithms to be extended and applied more generally in similar underwater survey missions.&rdquo;

Tests conducted by Cornell and the U.S. Navy used new algorithms to outperform state-of-the-art programming for autonomous underwater sonar imaging, significantly improving the speed and accuracy for identifying objects such as explosive mines, sunken ships, airplane black boxes, pipelines, etc.

Cornell, US Navy raise bar for autonomous underwater imaging

Wireless energy transmission, the dream of visionaries including Nikola Tesla with his Wardenclyffe Tower experiments, is today a reality. From completely-sealed electric toothbrushes to smartphones capable of charging when just thrown down onto a mat at the end of the day, there are plenty of examples of commercially-viable wireless power systems in everyday life &mdash; and still more on the horizon, including proposals for energising electric vehicles while they travel or robots as they roam.The majority of these, however, rely on electromagnetic transmission &mdash; an approach which works perfectly well over the air, but begins to encounter difficulties when faced with transmission through other media such as living tissue or water. What could be an ideal way to charge implanted electronics or undersea sensor networks becomes significantly more challenging &mdash; which is why a team of researchers from a number of Korean institutions have turned to acoustic energy transfer instead, switching from electromagnetic radiation to ultrasound as the energy source.<h1>The sound of power</h1>The team, led by the Korea Institute of Science and Technology (KIST), isn&rsquo;t the first to look at &mdash; or even succeed at &mdash; wireless energy transmission via ultrasound. Previous approaches, however, have been extremely limited in either the distance over which the energy can be transmitted or the amount of energy usable at the receiver &mdash; or, typically, both.The prototype developed by the researchers, by contrast, offers a considerable improvement over the state-of-the-art in both respects. Built using a five-layer design &mdash; a flexible electrode upper, a fixed area, a triboelectric layer, a novel ferroelectric layer, and a bottom electrode &mdash; the receiver sheets generate energy through movement induced by a 40kHz ultrasound signal, but represent only part of the gain in performance reported in the work.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUyODEzNjUzNzY0LWVzczQuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="A diagram of a prototype Acoustic Energy Transmission (AET) receiver, showing the five layers: Flexible electronics, fixed part, triboelectric layer, ferroelectric layer, and bottom electrode. A photograph of actual prototypes are seen to the right, marked with an outer fixed area frame and an inner active area.">The prototype AETs developed by the KIST-led team are made up of five layers, and produce energy from movement as triboelectric generators. The remainder, the team explains, comes from a switch from a sinusoidal acoustic wave to a square wave &mdash; an approach which, combined with the addition of piezoelectric PNM-PT crystals to the receiver as the ferroelectric layer, considerably boosts the output voltage available to connected electronics. Interestingly, the researchers conclude that the boosted voltage comes not from the piezoelectric effect directly but from a boost to the triboelectric effect.&ldquo;It can be seen that our device is mode-free,&rdquo; the team explains, &ldquo;and frequency independent. This is vital for its practical use, because unlike piezoelectric type receivers, our device does not necessitate a precisely matched ultrasound transducer.&rdquo;<h1>Switched on by sound</h1>To prove not only the core concept but also its applicability in simulations of various real-world use cases, the team built receiver prototypes and placed them in a range of materials: A lump of pork, used as a stand-in for human tissue in the case of recharging or directly powering implanted electronic devices; metal, acrylic, medium-density fiberboard (MDF), and plywood, to show its ability to transmit energy through a range of housing materials including those which would block electromagnetic radiation; and a tank of water, to showcase the system&rsquo;s potential for powering sub-aquatic drones or sensor networks ill-suited to other wireless energy transfer approaches.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUyODEzNjY3MDY4LWVzczIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Photographs of three prototypes: A trapezoidal 3D-printed prototype with five receiver faces connected to an electronic circuit and a CC2650 development board; the same receiver connected to two 200-LED matrices, with an ultrasonic transducer postioned 6cm away above a water tank; and the same receiver with wireless sensors fitted transmitting to a tablet PC over Bluetooth with the transducer 6cm away above the same water tank.">The team built several prototypes to prove the device&#39;s capabilities for useful energy transfer, including Bluetooth-connected sensor networks located under 6cm of water.The results are undeniably impressive. The body-implanted version proved capable of delivering sufficient power to drive implantable electronic devices without damage to the surrounding tissue and with the ultrasound emissions kept well within FDA-approved limits for use on patients; the system successfully transmitted though all the housing materials, representing the bulk of materials used in commercial and industrial electronic systems; and energy transmission worked perfectly through water.The latter experiments are worth further analysis. The team created a 3D-printed trapezoidal structure with prototype energy receiving sheets making up five of its faces, then fitted a range of wireless sensors designed for water quality monitoring &mdash; measuring metrics including temperature, carbon dioxide levels, pressure, and humidity. These were then powered using a transmitter 6cm (around 2.4&quot;) away &mdash; &ldquo;an order longer,&rdquo; the team notes, &ldquo;than the previous report.&rdquo;In further testing the team directly drove two matrices of 200 LEDs each at a brightness high enough to be observed under laboratory lighting, before concluding with a &ldquo;conclusive demonstration&rdquo; as to the approach&#39;s suitability for practical use: The replacement of the battery in a commercial Texas Instruments CC2650 Bluetooth Sensor Tag with an acoustic energy receiver which was capable of triggering Bluetooth transmission of sensor readings to a nearby tablet computer within a second of the ultrasound transmitter being energized.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjUyODEzNjc2Mzc0LWVzczMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="A line graph showing the voltage produced by the AET system with four design changes: Only PTFE and sinusoidal wave; PTFE, <111> PMN-PT (up) and sinusoidal wave; only PTFE and square wave; and PTFE, <111> PMN-PT (up) and square wave. With each change, the voltage can be seen increasing in turn.">The team&#39;s approach makes two key changes to the previous state-of-the-art, the use of square wave signals and the addition of a ferroelectric layer, with excellent results.&ldquo;This study demonstrated that electronic devices can be driven by wireless power charging via ultrasonic waves,&rdquo; says Hyun-Cheol Song, PhD, who led the project. &ldquo;If the stability and efficiency of the device are further improved in the future, this technology can be applied to supply power wirelessly to implantable sensors or deep-sea sensors, in which replacing batteries is cumbersome.&rdquo;The team&rsquo;s work has been published under closed-access terms in the journal <cite><a href="https://pubs.rsc.org/en/content/articlelanding/2022/EE/D1EE02623B" target="_blank">Energy &amp; Environmental Science</a></cite>.<h1>Reference</h1>Hyun Soo Kim, &nbsp;Sunghoon Hur, Dong-Gyu Lee, Joonchul Shin, Huimin Qiao, Seunguk Mun, Hoontaek Lee, Wonkyu Moon, Yunseok Kim, Jeong Min Baik, Chong-Yun Kang, Jong Hoon Jung, and Hyun-Cheol Song: <cite>Ferroelectrically augmented contact electrification enables efficient acoustic energy transfer through liquid and solid media, Energy &amp; Environmental Science, Iss. 3 2022</cite>. DOI <a href="https://doi.org/10.1039/D1EE02623B" target="_blank">10.1039/D1EE02623B</a>.

An artist's impression of the proposed AET system being used to drive both underwater sensor networks and autonomous underwater vehicles (AUVs), using nothing but ultrasound signals from a passing boat on the surface.

A team of researchers in Korea has been working on functional acoustic energy transmission (AET) systems, and their latest paper details a major breakthrough: Prototypes capable of powering LEDs, sensors, and Bluetooth transmitters at range and through materials include metal, tissue, and water.

A move to ultrasonic acoustics for wireless energy transmission could power implants, underwater sensors, and more

Microplastics are of growing concern, both for human health and the broader ecology. With millions of tonnes of plastic waste entering the marine ecosystem ever year, solutions are needed urgently &mdash; and while initiatives to prevent the waste from entering the ecosystem in the first place, by moving away from single-use plastics and implementing stronger recycling requirements, are a vital step, they do nothing for the waste already floating around our oceans.Autonomous underwater vehicles (AUVs), however, can help address the latter. Equipped with suitable grabbers, AUVs could roam the seas to locate plastic waste and remove it for recycling or more proper disposal &mdash; but only if they can recognize it. That&rsquo;s where a team of researchers, working with the United Kingdom&rsquo;s Natural Environment Research Council, aims to help: Improving computer vision systems to give AUVs better skills in recognizing plastic waste in underwater environments.<h1>Smarter vision</h1>The researchers envision the control system of a plastic-hunting AUV as three blocks: An image enhancement block, specifically designed for the murky low-light conditions found under the sea; a recognition block, capable of discerning plastic waste from other underwater objects and locating them accurately; and a guidance block, which navigates the vehicle to the waste and coordinates its collection.Seeing the potential for boosted performance, first author Federico Zocco and colleagues concentrated their efforts on the image enhancement and recognition blocks. The team started by applying a state-of-the-art object detector class, EfficientDets, to the task of recognizing plastic waste and differentiating it from other undersea objects like fish.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ5OTY2OTMxNDIxLWRlYnJpczMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="A picture showing a feed from an underwater camera with visible waste plastic, passing into a box marked &quot;underwater image enhancement&quot; before the enhanced images are passed to a box marked &quot;recognition&quot; and then the recognition outputs passed to a box marked &quot;guidance, navigation and control systems of both vehicle and gripper.&quot; A further box below shows the difference between three recognition systems: a classifier, with a class output; a detector, with class and position outputs; and a segmenter, with class, position, and shape outputs. A 3D model of an autonomous underwater vehicle is seen to the bottom-right.">Automated cleanup of plastic ocean waste could receive a boost from a new, more accurate version of EfficientDets running on autonomous underwater vehicles (AUVs). Created by a team working at Google Research, EfficientDets are designed to offer accurate object detection at a low latency &mdash; offering anything up to a ninefold size reduction and a forty-twofold drop in computational requirements than previous approaches. Zocco and colleagues, however, believed it possible to improve upon this &mdash; particularly in terms of specializing EfficientDets for the task at hand.Initially, the team concentrated on architectural modifications: Changing the number of weighted bi-directional feature pyramid network (BiFPN) layers, the number of convolutional layers, and scaling up parent detectors in order to find the best balance between latency and accuracy. Then, the resulting improved EfficientDets were trained on the Trash-ICRA19 dataset with biological, plastic, and remote-operated vehicle classes &mdash; before being trained again on a custom dataset, In-Water Plastic Bags and Bottles (WPBB), with 900 fully-annotated images of the target items recorded in a testing pool by a Chasing Dory drone.<h1>Better vision</h1>In addition to improving the object detection block, the team also investigated how the images themselves can be improved &mdash; taking the low-light captures of a AUV&rsquo;s camera feed and making them better-suited to object detection and classification.The team took the L2UWE framework detailed by Tunai Porto Marques and Alexandra Branzan Albu in 2020, designed for efficient enhancement of low-light underwater imagery, and applied it to a subset Trash-ICRA19 images which had been modified for simulated low-light conditions. While an improvement in accuracy was noted, it came at a considerable cost: &ldquo;L2UWE is too computationally demanding for real-time applications,&rdquo; the team noted, with the inference time per image reaching over 11 seconds when the framework was applied as a pre-processing step.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ5OTY2OTQ3OTM5LWRlYnJpczEuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="A graph showing a comparison between standard and improved EfficientDets: The improved version can be seen outperforming the standard version at each measurement, without an increase in latency. Below the graph is an example of a dark input image being processed through low-light image enhancement then through the detector; the final image shows an eel labelled &quot;bio: 91%&quot; and some debris labelled &quot;plastic: 80%.&quot;">The team&#39;s approach boosted the accuracy of EfficientDets measurably, without an increase in latency. A simpler approach was then tried: Just boosting the brightness of the image on a per-pixel basis, with no thought to enhancement at all. Doing so showed a notable increase in accuracy, though without analyzing each image and determining the amount of brightening required it&rsquo;s a rough approach at best.&ldquo;A more efficient approach,&rdquo; the team notes, &ldquo;is to tune the value of c [the increase in brightness applied to images] offline using genetic algorithms or particle swarm optimization and subsequently use it for low-light real-time marine debris detection within an automation pipeline.&rdquo;The team&rsquo;s overall results show promise: With EfficientDets already being considered state-of-the-art, a reported 1.2-2.6 per cent increase in average precision for the task of plastic waste recognition underwater with no increase in latency is a notable achievement &mdash; and the team has made it possible for others to improve the approach still further by publishing both the WPBB dataset and the project&rsquo;s source code <a href="https://github.com/fedezocco/MoreEffEffDetsAndWPBB-TensorFlow" target="_blank">on GitHub</a> under the permissive MIT license.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ5OTY2OTU3NzYzLWRlYnJpczIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib" alt="Images of plastic bottles and bags floating in an underwater tank, captured from an autonomous underwater vehicle. Each has been labelled &quot;plastic bottle&quot; or &quot;plastic bag&quot; by the improved EfficientDets with confidence ranging from 95 to 99 per cent, with one 43 per cent outlier.">The WPBB dataset, captured using a commercial AUV in a tank and with real examples of waste plastics, has been released under a permissive license along with full source code for the project.The team&rsquo;s next step: Implementing the proposed detection pipeline on a physical AUV, integrated with the navigation and gripper control system for automated waste collection.The paper detailing the team&rsquo;s work has been made available <a href="https://arxiv.org/abs/2203.07155" target="_blank">on Cornell&rsquo;s arXiv preprint server</a> under open-access terms.<h1>References</h1>Federico Zocco, Ching-I Huang, Hsueh-Cheng Wang, Mohammad Omar Khyam, and Mien Van: <cite>Towards More Efficient EfficientDets and Low-Light Real-Time Marine Debris Detection, to be submitted to IEEE Robotics and Automation Letters</cite>. Preprint DOI <a href="https://arxiv.org/abs/2203.07155" target="_blank">arXiv:2203.07155</a> [cs.CV].Mingxing Tan, Ruoming Pang, and Quoc V. Le: <cite>EfficientDet: Scalable and Efficient Object Detection, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020</cite>. DOI <a href="https://doi.org/10.1109/cvpr42600.2020.01079" target="_blank">10.1109/cvpr42600.2020.01079</a>.Tunai Porto Marques and Alexandra Branzan Albu: <cite>L2UWE: A Framework for the Efficient Enhancement of Low-Light Underwater Images Using Local Contrast and Multi-Scale Fusion, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020</cite>. DOI <a href="https://doi.org/10.1109/CVPRW50498.2020.00277" target="_blank">10.1109/CVPRW50498.2020.00277</a>.

Plastic waste running from rivers to the seas is a growing problem, which will take a major clean-up to solve. Enter the AUVs.

With growing concern over microplastics from ocean waste, autonomous underwater vehicles — AUVs — have been proposed as a tool for cleaning up our seas, but only if they can pick the plastics out from the fish: Enter these tweaked EfficientDets, boosting accuracy for the task.

Automated ocean waste retrival the target for new, specialized open-source EfficientDets

An ROV camera is any visual imaging system that can be used on <a href="https://blueprintlab.com/blog/what-is-an-underwater-rov/" target="_blank">underwater ROVs</a> to provide the operator with a perception of the operating environment. The depth rating for an <a href="https://blueprintlab.com/products/underwater-rov-camera/" target="_blank">ROV camera</a> usually starts at 300m.Most ROVs have a main forward-facing camera for control of the vehicle. Some ROVs will then have additional, auxiliary cameras to provide another point of view. These cameras are sometimes mounted on another point of the vehicle or extended out from the vehicle using a &lsquo;boom mount&rsquo; (e.g. a camera on a pole sticking out to the side). If an ROV manipulator is in use, this manipulator will sometimes have a camera at the end-effector providing yet another point of view of the workspace. One should remember that underwater vision systems are usually monocular and therefore do not provide distance information (try catching a ball with one eye closed); development in this area using stereo vision and other sensors is a hot topic in underwater robotics and is currently being pursued by Blueprint Lab to enable more advanced use of <a href="https://blueprintlab.com/products/manipulators/" target="_blank">manipulator arms</a> and <a href="https://blueprintlab.com/products/rov-grippers/" target="_blank">ROV grippers</a>.<h2><style type="text/css" id="isPasted"></style><style type="text/css"></style>What Are The Different Types Of ROV Camera?</h2>There are two main types of underwater or ROV camera:<h3>1. Main ROV Camera</h3>The main ROV camera is the primary means of providing visual information to the ROV operator. These cameras are generally selected to be High-Definition and to have exceptional performance in low-light conditions. They are the primary camera used to pilot the vehicle and are forward-facing. Depending on the size of the vehicle, this camera may also be used for looking at objects of interest or there may be an additional camera for that purpose to deconflict from piloting operations.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649419090826-image.png" style="width: 766px;" class="fr-fic fr-dib">An image from the forward-looking camera on the ROV SuBastian of the Schmidt Ocean InstituteSometimes they are fixed&hellip;&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649419133804-image-1.png" style="width: 766px;" class="fr-fic fr-dib">An HD camera by ROV Innovations suitable for forward-facing main ROV camera use&hellip;other times they are attached to mechanisms (<a href="https://blueprintlab.com/products/underwater-rotator/" target="_blank">rotators and pan/tilt units</a>) to enable them to move left/right and up/down, providing a greater field of view to the operator.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649419168373-image-2.png" style="width: 766px;" class="fr-fic fr-dib">The main ROV camera with tilt-function &ndash; the DTR-100Z from Bowtech&nbsp;<h3 id="isPasted">2. Auxiliary ROV Camera</h3>Auxiliary cameras are used to provide additional perspectives to that provided by the main, front ROV camera. Examples of where these are used are:<ul><li>A camera facing rearward to check the ROV tether condition or ensure the tether is not tangled during a complex manoeuvre.</li><li>A camera mounted on a corner of the ROV, or out on a boom and looking in towards the front of the ROV to provide a sense of distance when flying the ROV or using an ROV gripper or ROV manipulator.</li><li>A camera mounted on a manipulator arm to enable close vision of what the manipulator is being used for such as in <a href="https://blueprintlab.com/blog/top-uses-for-underwater-manipulators/" target="_blank">close visual inspections or complex interventions</a>.</li><li>A camera mounted on the side of the ROV to enable inspection of a structure while flying alongside it.</li></ul><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649419205930-image-3.png" style="width: 766px;" class="fr-fic fr-dib">One of the auxiliary cameras on the ROV SuBastian provides an additional perspective of the ROV workspace.&nbsp;Auxiliary cameras often do not need to meet the same specifications as the main ROV camera and may additionally be limited by size constraints. Hence, they are often selected to be <a href="https://blueprintlab.com/products/underwater-rov-camera/" id="isPasted" target="_blank">pencil-style subsea cameras</a>. These cameras have the advantage of lower cost and smaller size.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1649419238477-image-4.png" style="width: 766px;" class="fr-fic fr-dib">The Micro IP Camera from Blueprint Lab (Outer Diameter 2.4cm, length ~9cm)&nbsp;<h2 id="isPasted"><style type="text/css" id="isPasted"></style><style type="text/css"></style>Considerations In Selecting An ROV Camera </h2>If you are selecting a camera for your underwater ROV or ROV manipulator, you should consider the following:<ul><li>Quality</li><li>Size (check out our <a href="https://blueprintlab.com/products/underwater-rov-camera/" target="_blank">miniature subsea camera</a> if this is a factor in your design)</li><li>Water Weight</li><li>Connector</li><li>Protocol</li><li>Performance in Low Light (sometimes additional illumination should be considered by adding <a href="https://blueprintlab.com/products/subsea-light/" target="_blank">ROV lights</a>, or selecting a camera with integrated LEDs)</li><li>Viewing Angle</li><li>Whether actuation (pan, tilt, complex motion) is required (if so, take a look at Blueprint Lab&rsquo;s<a href="https://blueprintlab.com/products/underwater-rotator/" target="_blank">&nbsp;rotator</a> and <a href="https://blueprintlab.com/products/manipulators/" target="_blank">manipulator</a> solutions)</li></ul>

An ROV camera is any visual imaging system that can be used on underwater ROVs to provide the operator with a perception of the operating environment. The depth rating for an ROV camera usually starts at 300m.

What is an ROV camera?

Doing things underwater has always been challenging. Human divers brave the pressures of the deep subsea environment, dealing with low light, poor visibility, and little room for error. Since the 1970s, technology developments gave rise to ROVs with robotic arms (aka &lsquo;<a href="https://blueprintlab.com/products/manipulators/" target="_blank">underwater manipulator arms&rsquo;</a>) to start accomplishing tasks underwater.Industries that use ROV manipulators include offshore energy, military, search and rescue, and marine science.Whilst not as dexterous as humans, these ROV manipulator systems are stronger and can go deeper than humans can dive. This ushered in a new era for offshore oil companies who could now install and maintain deeper installations.Originally, ROV manipulators were large and hydraulically powered. Today, whilst large hydraulic manipulators are still the workhorse of the ROV industry, electric systems have started to be released to the market. In particular, smaller electric ROV manipulators that are still highly dexterous have opened up new possibilities for the ROV community.Here are the top 9 uses of underwater ROV manipulators based on working with our clients here at Blueprint Lab.<h2>1. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Non-Destructive Testing (NDT)</h2><a href="https://en.wikipedia.org/wiki/Nondestructive_testing" rel="noreferrer noopener nofollow" target="_blank">Non-Destructive Testing</a> (NDT), and sometimes called NDE or NDI for examination and inspection respectively, is the process of using non-invasive means to inspect a component or structure to detect cracking, corrosion, or other defects within the material.Offshore oil rigs will sit in the water, braving the elements, salt corrosion, and marine growth for 20-30 years. To ensure that these structures and other water-based assets remain safe, there are regulations that govern their Inspection, Maintenance, and Repair (IMR). Disasters such as what happened to the<a href="https://www.offshore-technology.com/features/feature-the-worlds-deadliest-offshore-oil-rig-disasters-4149812/" rel="noreferrer noopener nofollow" target="_blank">&nbsp;Alexander L Kiellande oil platform in 1980</a> or more recent <a href="https://www.cbsnews.com/news/gulf-of-mexico-fire-ocean-burst-pipeline" rel="noreferrer noopener nofollow" target="_blank">near misses in the Gulf of Mexico</a> have been caused by cracking and corrosion of subsea structures left unchecked.Manipulators and human divers routinely carry out inspections of structures for cracking or corrosion using<a href="https://en.wikipedia.org/wiki/Ultrasonic_thickness_measurement" rel="noreferrer noopener nofollow" target="_blank">&nbsp;Ultrasonic Thickness</a> (UT),<a href="https://en.wikipedia.org/wiki/Cathodic_protection" rel="noreferrer noopener nofollow" target="_blank">&nbsp;Cathodic Protection</a> (CP), or<a href="https://en.wikipedia.org/wiki/Alternating_current_field_measurement" rel="noreferrer noopener nofollow" target="_blank">&nbsp;Alternating Current Field Measurement</a> instruments. ROV manipulators that are dexterous, more compact, and have advanced software control (including being able to precisely control the position and orientation of the instrument) have enabled small ROVs to <a href="https://blueprintlab.com/blog/degrees-of-freedom-vs-functions-of-a-robotic-arm/" target="_blank">carry out these tasks completely remotely</a>. Companies such as Fugro are<a href="https://www.fugro.com/about-fugro/our-expertise/remote-and-autonomous-solutions/remote-and-autonomous-vessels" rel="noreferrer noopener nofollow" target="_blank">&nbsp;pioneering remote operations</a> where an Unmanned Surface Vehicle (USV) hosting an ROV can deploy and inspect infrastructure using dexterous manipulators.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2Njk5OTc0LXBhc3RlZCBpbWFnZSAwLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">2. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Dexterous (Human-Like) Intervention&nbsp;</h2>Achieving the same (or better) level of dexterity as a human is the holy grail of dexterous robotic manipulation. Huge progress has been made in recent years in terms of the size and joint dexterity of robotic manipulators above and below the water. One current limitation is the ability to achieve precise, small-scale dexterity. Land robots have demonstrated the use of robotic hands with fingers; successfully manufacturing similar technologies underwater is far more difficult and yet to be commercially solved.Some dexterous tasks such as tying a knot, attaching a carabiner, or fastening a nut to a bolt have all been demonstrated using underwater manipulator technology. This has allowed industries to use robotic manipulators for net repair in aquaculture settings, and conduct complex military tasks.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2NzM2MTM1LUFscGhhLTUtQ2xpcHBpbmctQ2FyYWJpbmVyLTc2OHg1NzAuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">Alpha 5 Manipulator attaching a carabiner underwater&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2Nzg0NjE3LXBhc3RlZCBpbWFnZSAwICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">3. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Construction And Repair</h2>With hydraulic and electric actuators able to exceed the capability of humans, underwater manipulator arms are used for moving large tools and instruments around to aid in the construction, maintenance, repair, and decommissioning of a whole range of subsea infrastructure. In the offshore energy industry, manipulators move large hooks attached to cables for deployment and recovery of subsea infrastructure and wield torque tools or welding torches.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2ODE1NTExLU1pY3JvLUlQLUNhbWVyYS1CYW5uZXItMi03Njh4NDMyLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">4. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Close Visual Inspection</h2>Like NDT, Close Visual Inspection (CVI) is concerned with looking at infrastructure with <a href="https://blueprintlab.com/products/underwater-rov-camera/" target="_blank">HD subsea cameras</a> to detect any issues that need repair or follow up. Manipulators with good reach (length) and dexterity enable CVI in hard-to-reach areas such as inside pipe inlets and behind propellers.This ability for confined space access is a value-add offered by more modern, compact, electric manipulators like Blueprint Lab&rsquo;s Reach Alpha and Reach Bravo. Leveraging HD CVI allows for Underwater Inspection in Lieu of Dry-docking (UWILD). UWILD, as the name suggests, is the CVI/NDT of a ship&rsquo;s hull that avoids needing to take the vessel out of the water; a huge cost and efficiency saving.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2ODQ1NDc1LUJyYXZvLVdyaXN0LUNhbWVyYS03Njh4NDI4LnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 766px;" class="fr-fic fr-dib"><h2 id="isPasted">5. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Cleaning/Anti-Fouling&nbsp;</h2>Over time, subsea infrastructure can become coated in marine growth (barnacles etc) and other biofouling substances. To properly inspect the condition of the infrastructure (using NDT or CVI methods), surfaces must be cleaned of any marine growth. This is generally done by using high-power water pressure, a cavitation blaster, or a rotary brush.In some scenarios, this will be done using a fixed system on the ROV. In other cases, the dexterity of a manipulator is required or beneficial to adequately reach all areas that need to be inspected. This is especially relevant where the geometry of the inspection area is more complex, such as at the weld nodes of a jack-up rig.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2ODc1MDg5LU9pbF9SaWctNzY4eDU3Ni5qcGVnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 766px;" class="fr-fic fr-dib">Subsea steel joints&nbsp;<h2 id="isPasted">6. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Sea Mine And UIED Intervention&nbsp;</h2>Military clearance divers carry out dangerous tasks including deploying a shaped charge to disarm a mine or examining and disrupting a Waterborne Improvised Explosive Device (WBIED). These tasks are still done by humans because they require high fidelity perception and dexterous intervention capabilities not previously available through a remote solution. Recently, with the release of more dexterous, compact manipulator arms like the Reach Alpha, military organisations are investigating ways for ROVs to start to do some of these more dangerous tasks.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2OTEzMTM1LTEwMDU1MzczMDc1XzIxNmRkZTc3NWFfYy5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Image Source: Flickr<h2 id="isPasted">7. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Special Recovery Operations (e.g. Downed Aircraft, Drowning Victims)</h2>Special Recovery Operations include scenarios such as recovering a downed aircraft from the seafloor or drowning victims from a lakebed. These sensitive tasks have historically been conducted by highly skilled military or civilian diver teams. Recently ROVs have been able to assist and in some cases take-over, these high-risk tasks.Dexterous manipulators can be used to tie knots or attach carabiners to recover objects or for the recovery of forensic items without disturbing the lake/sea bed.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ4NzE2OTYxMjYwLUhFQVJULXNpbXVsYXRpb24tNzY4eDQ4OS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 766px;" class="fr-fic fr-dib">Alpha Grabber during a simulated recovery mission &ndash; See the full video story below!&nbsp;<iframe src="https://www.youtube.com/embed/19V9YdC-ZOM?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" style="width: 768px; height: 430px;"></iframe><h2 id="isPasted">8. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Oceanography And Hydrography (Sampling And Measurement)</h2>Understanding the condition of our marine environment is critical for building a global picture of the Earth&rsquo;s ecosystem. Scientists regularly take samples from the sea bed for analysis and collect marine organisms for biological study. In some cases, sampling and organism collection requires precise positioning of a sampling device as well as the ability to retrieve a sample without stirring up sediment from the seafloor. Additionally, measuring devices (rulers and callipers) are often used to properly characterize an object of interest. In these cases, a dexterous manipulator allows for the ROV to sit on the seafloor and for confined areas to be reached (such as amongst a coral reef)<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1648717362455-EGM_5604_Credit_AIMS-EricMatson-2048x1367.jpeg" style="width: 766px;" class="fr-fic fr-dib">Image source: Scimex<h2 id="isPasted">9. <style type="text/css" id="isPasted"></style><style type="text/css"></style>Autonomy And Custom Control&nbsp;</h2>Robotics and Autonomous Systems researchers around the world are pioneering the use of underwater robots in a variety of autonomous applications. Some examples of this are using manipulators to autonomously interact with a subsea control panel or using multiple robots to lift, carry, and place an object, thereby utilising the lift capacity of more than one underwater vehicle.<h2 id="isPasted"><style type="text/css" id="isPasted"></style><style type="text/css"></style>Final Thoughts </h2>The uses of underwater ROV manipulators are varied and constantly expanding. As a company, Blueprint Lab is seeking not just to make incredible robotic arms, but to actually solve industry problems. We constantly discuss and explore the problems that our clients are trying to solve and have multiple projects in work regarding underwater manipulation, perception and control.

The aim of this article is to provide a high-level answer to the question: “What do people use underwater robotic arms for?” If you’ve worked with Remotely Operated Vehicles (ROVs) before, this article probably isn’t for you. For everyone else, let’s go.

Top Uses of Underwater ROV Manipulators

A Quick Start Guide to Underwater ROVs. In this guide, you’ll learn about what defines a remotely operated vehicle, different types of ROVs, and what they are used for. Underwater ROVs are used to observe and interact underwater across a range of industries.

What are underwater ROVs & What can they be used for?

Is there life on Jupiter&#39;s moon Europa? An ocean suspected to be under miles of ice gives science hope. But how could it be reached and explored? Researchers at the DFKI Robotics Innovation Center have been investigating this question since 2012 in the Europa Explorer project series. The focus is on the development of robotic systems that can penetrate the water, explore it autonomously and remain there for a longer period of time. At the end of the third phase of the project, the autonomous underwater robot DeepLeng will now be put to the test in a frozen lake in northern Sweden.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1647686785744-csm_Deep_Leng-9963_b8f4a16cf7.jpg" style="width: 766px;" class="fr-fic fr-dib">DeepLeng in the Maritime Exploration Hall of the DFKI in Bremen &copy; DFKI, Annemarie PoppAn average of 600 million kilometers separates the earth from the icy moon Europa, one of the four large moons of Jupiter. Underneath its mighty shell of water ice an ocean is assumed about 100 km deep with hydrothermal vents at its bottom, which release hot and mineral-rich water into their surroundings, creating the basis for living organisms. As part of the German Aerospace Center&#39;s (DLR) Explorer Initiatives, the Robotics Innovation Center has been investigating how this highly interesting place for science could be explored. To this end, the researchers developed autonomous underwater vehicles (AUVs), which can navigate and explore its surroundings under an ice sheet autonomously, in three successive phases of the project. In the third phase of the Europa Explorer project for long-range under-ice navigation (EurEx-LUNa), the final field tests are ahead, in which the AUV DeepLeng will be tested for the first time outside the laboratory under realistic conditions: For this purpose, five DFKI scientists will travel to the small northern Swedish town of Abisko in the national park of the same name on Lake Tornetr&auml;sk from March 20 to April 2, 2022. Here, north of the Arctic Circle, with outside temperatures expected to be around minus 10 degrees, the underwater robot is to autonomously explore the frozen lake, which is up to 168 meters deep. Despite the freezing cold, it is a baptism of fire for the researchers, who are housed in a research station in Abisko for the duration of the field trials, to prove the feasibility of the concept for autonomous under-ice navigation. Once launched by the mobile crane, the hole in the ice allows only a brief glimpse of the three-meter-long system with its eye-catching yellow cover before it dives down and disappears completely under the ice shell. Here, the robot has to find its way around completely on its own in murky conditions, gather as much information as possible from its surroundings and then navigate back to the docking station, where it can transmit the data, it has collected and recharge its batteries.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1647686861679-csm_auv_detaches_from_dockingport_3771a69fbc.png" style="width: 766px;" class="fr-fic fr-dib">Animation of autonomous docking under ice. &copy; DFKI, Meltem FischerFor the success of this challenging mission, the researchers further developed both the AUV&#39;s hardware and software in the EurEx-LUNa project. Among other things, they equipped DeepLeng with an USBL underwater navigation system that can transmit, receive, and process acoustic signals. They also used machine learning to train the system to find its way even under the most adverse conditions and to dock safely at the docking interface. Now, the preliminary work in the project, as well as in the entire Europa Explorer project series, should pay off in the field tests in Sweden. As a continuation of the Europa Explorer and EurEx-SiLaNa projects, EurEx-LUNa is funded since March 1, 2020, until May 31, 2022, by the German Federal Ministry of Economics and Climate Protection (BMWK) via the German Aerospace Center (DLR). Note: For those who want to get a picture of the field trials in Sweden, a live camera will be installed on site. The images will be broadcast via YouTube. There will also be a field test blog where project leader Dr. Marc Hildebrandt will report daily on what has happened over the past day. The links to the live stream and the blog will be announced here in the next few days. More information: Project EurEx-LUNa: <a href="https://robotik.dfki-bremen.de/en/research/projects/eurex-luna/" rel="noreferrer" target="_blank">https://robotik.dfki-bremen.de/en/research/projects/eurex-luna/</a> System DeepLeng: <a href="https://robotik.dfki-bremen.de/en/research/robot-systems/deepleng/" rel="noreferrer" target="_blank">https://robotik.dfki-bremen.de/en/research/robot-systems/deepleng/</a> &nbsp;

The predecessor version of DeepLeng dives autonomously in the icy Stadtwaldsee in Bremen, Germany.  © DFKI, Annemarie Popp

Is there life on Jupiter's moon Europa? An ocean suspected to be under miles of ice gives science hope. But how could it be reached and explored? Researchers at the DFKI Robotics Innovation Center have been investigating this question since 2012 in the Europa Explorer project series.

On an icy mission: DFKI researchers test extraterrestrial under-ice navigation in northern Swedish National Park

Our seas and oceans currently contain somewhere between 26 and 66 million tonnes of plastic waste, most of which is lying on the seafloor. This represents an enormous threat to marine plants and animals and to the ecological balance of the seas.But removing waste from the waters is a complex and expensive process. It is often dangerous, too, because the work is generally done by scuba divers. The cleanup operations are also usually limited to the water surface. In the <a href="https://seaclear-project.eu/" rel="noreferrer" target="_blank">SeaClear Project</a>, a team at TUM is working with eight European partner institutions to develop a robotic system capable of collecting underwater litter. &nbsp;<h2>Four robots working together</h2>The system combines four robotic components: an autonomous surface vehicle performs an initial scan of the sea bottom and localizes large litter pockets. Next an observation robot is lowered into the water to detect undersea litter and transmit additional information to the computers such as close-up images of the sea bottom.In clear water and with good visibility, an aerial drone is also used to identify further litter objects. The resulting data are combined to generate a virtual map. A collection robot then visits defined points on the map and picks up litter. It uses a gripper to place larger pieces in a basket that is towed to shore by the autonomous boat.<div class="fr-img-space-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1641200382603-Seaclear_Foto.jpg" style="width: 766px;" class="fr-fic fr-dib">A system consisting of four robots ensures clean sea floors. Image: The SeaClear Project&nbsp;</div><h2 id="isPasted">The challenge of currents</h2>“Developing autonomous robots for underwater applications is a unique challenge” says Dr. Stefan Sosnowski, the technical director of the SeaClear project at the Chair of Information-oriented Control at TUM. That is because, in contrast to land-based applications, very special conditions prevail in the water. “When a piece of litter is identified and located, the robot needs to get close to it. To do so, it may need to overcome strong currents. The task of TUM in the SeaClear project is to enable the robot to move in the right direction.”<h2>Efficient machine learning</h2>To achieve this, the team is using machine learning methods. An artificial intelligence (AI) module performs calculations and learns the conditions under which the robot will move in certain ways. This makes it possible to predict its behavior precisely.“Another challenge is that we don’t have the computing power at our disposal that we would on dry land,” says Prof. Sandra Hirche, director of the chair and SeaClear principal investigator. “We do not have links to large data centers with supercomputers. So we need highly efficient algorithms that run with limited resources. We are therefore working with high-efficiency sampling methods that arrive at precise predictions with minimal data. The AI system simply discards unnecessary information.”<h2>90% success rate</h2>When the SeaClear system is fully operational, it is expected to achieve 80% accuracy in classifying underwater litter and to successfully collect 90% of it. This is comparable to the results produced by scuba divers. The initial trials with the prototype were carried out in October 2021 in Dubrovnik, Croatia, where the water is clear and visibility is excellent. Further trials are scheduled in the port of Hamburg in May 2022.<div class="fr-img-space-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1641200417136-First_Litter_Grabbed_1.jpeg" style="width: 766px;" class="fr-fic fr-dib">The research group of the SeaClear project observes the underwater activities of the robot on the monitor. Image: The SeaClear Project&nbsp;</div><hr><h3 id="isPasted">Publications:</h3>P. Bevanda, S. Sosnowski, S. Hirche, "Koopman operator dynamical models: Learning, analysis and control", Annual Reviews in Control , 2021, 52, 197-212 ,&nbsp;<a href="https://www.sciencedirect.com/science/article/abs/pii/S1367578821000729?via%3Dihub" rel="noreferrer" target="_blank">DOI:10.1016/j.arcontrol.2021.09.002</a>

The robot of the SeaClear Project is able to detect and collect underwater litter. Image: The SeaClear Project

Removing litter from oceans and seas is a costly and time-consuming process. As part of a European cooperative project, a team at the Technical University of Munich (TUM) is developing a robotic system that uses machine learning methods to locate and collect waste under water.

Robots collect underwater litter

Autonomous Underwater Vehicles (AUVs) play a vital role in fields such as hydrocarbon drilling and extraction, marine surveying and aquatic environmental monitoring. An AUV can perform functions in which it is difficult for divers to accomplish, and can descend to depths which would put human operators in danger.Most AUV products on the market today are large, capable of carrying heavy payloads, and fitted with a large battery to enable them to stay underwater for long periods of time. This type of product suits customers such as oil companies, which can afford the substantial cost not only to buy a large AUV, but to transport it to where it is needed, to manoeuvre it into position, and to operate it.UK-based marine products supplier <a href="https://planet-ocean.co.uk/wp/" rel="noopener" target="_blank">Planet Ocean</a> spotted that this left a gap in the market for a smaller, more affordable AUV. This realization spurred the development of <a href="https://planet-ocean.co.uk/wp/?page_id=4509" rel="noopener" target="_blank">ecoSUB</a>, a range of small, low-cost AUVs which can be easily carried, launched, operated and retrieved from the water by a single person.Its goal was to democratize the use of AUVs: to bring them within reach of all research institutes, universities, commercial users, users in less developed countries, and of other users who have limited resources.&nbsp;It turned out that the design of this innovative AUV required a motion tracking sensor which was both compact and accurate.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1619774171204-1619774171204.png" alt="Capture 2" width="1289" srcset="https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=645&amp;name=Capture%202.png 645w, https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=1289&amp;name=Capture%202.png 1289w, https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=1934&amp;name=Capture%202.png 1934w, https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=2578&amp;name=Capture%202.png 2578w, https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=3223&amp;name=Capture%202.png 3223w, https://www.xsens.com/hs-fs/hubfs/Capture%202.png?width=3867&amp;name=Capture%202.png 3867w" sizes="(max-width: 1289px) 100vw, 1289px" class="fr-fic fr-dii" style="width: 777px;">Xsens makes accurate underwater navigation possibleThe efficient operation of the ecoSUB AUV calls for accurate location data derived by the dead reckoning technique. On the surface of the water, geolocation is easy: satellite positioning signals determine position to a typical accuracy of ±5m.The satellites’ RF signals do not penetrate water, however, so the ecoSUB AUV determines its position by continuously tracking its motion relative to its launch point on the surface.&nbsp;For this motion tracking function, ecoSUB uses the <a href="https://www.xsens.com/inertial-sensor-modules" rel="noopener" target="_blank">Xsens MTi-3 AHRS</a> (Attitude and Heading Reference System), together with a discrete pressure sensor for depth measurement. The MTi-3 sensor provides the ecoSUB with highly accurate measurements of roll, pitch, heading and acceleration. The ecoSUB’s on-board computer continuously converts these measurements into position coordinates.<iframe src="https://www.youtube.com/embed/7EEqPm0ylWk?&amp;t=6s&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 788px; height: 501px;"></iframe>The especially challenging requirements of designing a small and light AUV made the Xsens motion tracking sensor the ideal fit.&nbsp;‘The fundamental design objective for the ecoSUB was to make an AUV that was small and light enough for an average person to lift, while still having enough load-carrying capacity and range to perform useful tasks. Our design for the ecoSUB fits an “A” size sonar buoy tube – it is 1m long, and weighs just 4kg. To optimize the ecoSUB’s range and dynamic characteristics, we needed to apportion as much of the internal space as possible to the battery and the motor. So the motion sensor and the rest of the control electronics system have to be packed in a small space, in the presence of the strong magnetic fields generated by the motor system. And of course the accuracy of the AUV’s position data is absolutely crucial. When the AUV is being used for instance to detect an oil leak or to monitor variations in the oxygen content of sea water, users need to know exactly where in three dimensions it has taken its measurements.’Jeremy Sitbon, ecoSUB Chief Robotics EngineerXsens MTi-3 motion tracker: Small, light, reliableThe constraints applying to the design of the ecoSUB meant the motion sensor had to offer three essential characteristics:<ul><li>High accuracy in the sensor’s motion tracking data&nbsp;</li><li>Dependability, with high immunity to magnetic interference</li><li>Small size and weight</li></ul>The Xsens MTi-3 AHRS delivers on all three counts. The MTi-3 integrates multiple sensor functions including 3D accelerometer, gyroscope and magnetometer plus a processor.The accuracy of the sensor’s static measurements of roll and pitch is ±0.5°, and for yaw the value is ±2°. In use, the ecoSUB typically achieves positioning accuracy of ±5% of the distance travelled when using only the MTi-3 motion sensor and the pressure sensor measurements.The accuracy of the sensor’s measurements remains stable regardless of changes in the magnetic field around it. The ecoSUB designers have made full use of the sophisticated filtering capabilities built into the MTi-3 sensor: the sensor can be configured for operation in various types of operating environments, including for continuous strong magnetic interference. This is ideal for the ecoSUB, in which the propellers are coupled to the motor magnetically.All functionality is squeezed into a compact package which measures just 12.1 x 12.1 x 2.55mm and weighs less than 1g. Typical power consumption is less than 100mW, helping the ecoSUB’s battery to last longer and extend the vehicle’s range.Opening up the use of AUVs to whole new market segmentsAfter completing sea trials, the innovative ecoSUB AUV went into full commercial production in 2020. Jeremy Sitbon says that Planet Ocean is excited about the possibility of introducing a new generation of users to the utility of AUVs. The ecoSUB’s capabilities include acoustic (sonar) communication, enabling multiple ecoSUB AUVs to be used concurrently in ‘shoals’ to give a 3D picture of subsea conditions. The ecoSUB can act like a marine ‘truck’ – a carrier for almost any type of sensor or other payload, so the potential for its use is almost limitless.<blockquote>‘The success of the product comes down to our ability to pack a lot of functionality and performance into a very small space. Our decision to use the MTi-3 played a really important part in our ability to make this design successful. I’m very happy that we made the right choice!’</blockquote>Jeremy Sitbon, ecoSUB Chief Robotics Engineer&nbsp;Learn more about the <a href="https://www.xsens.com/mti-3" rel="noopener" target="_blank">Xsens MTi-3 AHRS</a>&nbsp;here.Choosing your productWant to know the right inertial measurement unit for your application? Try our recommendation tool and find out what MTi you need by answering just 3 questions.<a href="https://www.xsens.com/cs/c/?cta_guid=e67ae0db-e496-4b1b-aaae-0f33c66a97bb&signature=AAH58kERCATymf5UB4b7WHUo8PJHsSLMlQ&pageId=42748316968&placement_guid=0e0cf390-8244-48ac-aa9c-cc22448b3988&click=38fa5093-baba-426c-92e2-244e329bd831&hsutk=26cd9f0233255f079c3a3909dd662892&canon=https%3A%2F%2Fwww.xsens.com%2Fcases%2Fsmall-light-and-accurate-xsens-mti-3-sensor-helps-planet-ocean-open-up-new-markets-for-autonomous-underwater-vehicles&utm_referrer=https%3A%2F%2Fwww.xsens.com%2Fexplore%3Ffilter%3Dcases&portal_id=3446270&redirect_url=APefjpHFi12gvYuHqoWSZfHrIGqiyim6_ONHhAlUSf5P2fSu9LX3yPDWqIEBNMTq7zPbzoK-WfoIZg8Bfa15xkMpbz1GkbkzFFTzUwGCgcPUASxo3Mq8GkOyNHKblRAbv48KApkSyYPNJwvTLiTVMI_w5QhewNpuAjHGGtTfSn2xG8nR_UcoJBph_ito-8xtKFpTW9QoAoROG3GmG8mI-y8TDzizVxJguxXZRuSkE0Jr3QNx52alDInkfDxEm_QGTqfCxOEdsmfaCQUt-NiFKNTC_MtmiftGKw&__hstc=81749512.26cd9f0233255f079c3a3909dd662892.1612181589013.1619708666522.1619773674521.185&__hssc=81749512.8.1619773674521&__hsfp=1627031933&contentType=blog-post" target="_blank" title="Recommendation tool">Recommendation tool</a>&nbsp;

with Xsens MTi

Xsens AHRS Enables Planet Ocean To Open Up New Markets For Autonomous Underwater Vehicles

In this episode, we talk about Charm Industrial’s novel approach for carbon offsetting, a concept fitness drone from Hongik University that could be the first to commercialize, and a joint effort between MIT and the US Navy to understand how sound waves are altered by water conditions in the North Pole. As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/e71ade31-1622-42a8-93a9-a489df82c655?dark=false" class="fr-draggable"></iframe><h3>EPISODE NOTES</h3>(0:50) -&nbsp;<a href="https://www.wevolver.com/article/a-charm-ing-approach-for-carbon-offsetting" target="_blank">A Charming Response To The Carbon Offsetting Controversy</a>:&nbsp;Approximately 85% of all carbon offsetting efforts do not actually remove any additional carbon dioxide from the atmosphere and Charm wants to change that by injecting bio oil into wells that stored crude oil at one point.(8:10) -&nbsp;<a href="https://www.wevolver.com/article/drones-as-fitness-trainers" target="_blank">Drones As Personal Fitness Instructors</a><a href="https://www.wevolver.com/article/organic-material-may-help-design-energy-efficient-heating-and-cooling-systems" target="_blank">:</a>Drones are finally making their debut in the fitness scene with a concept design from Hongik University that could serve as the first commercially viable fitness drone and pave the way for the future of the industry.&nbsp;(14:20) -&nbsp;<a href="https://www.wevolver.com/article/navigating-beneath-the-arctic-ice" target="_blank">MIT &amp; ONR’s Trip To The North Pole</a>:Acoustic signals get distorted in regions of the ocean where warm water meets cold water - especially in the Beaufort Sea - so MIT has collaborated with ONR to understand these distortions in hopes of allowing exploration in these regions without risking loss of communication.--About the podcast:Every day, some of the most innovative universities, companies, and individual technology developers share their knowledge on Wevolver. To ensure we can also provide this knowledge for the growing group of podcast listeners, we started a collaboration with two young engineers, Daniel Scott Mitchell &amp; Farbod Moghaddam who discuss the most interesting content in this podcast series.&nbsp;To learn more about this show, please visit the&nbsp;<a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">shows page</a>. By following the page, you will get automatic updates by email when a new show is published.Be sure to give us a follow and review on <a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" rel="nofollow" target="_blank">Apple</a> podcasts, <a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" rel="nofollow" target="_blank">Spotify</a>, and most of your favorite podcast platforms!

In this episode, we talk about Charm Industrial’s novel approach for carbon offsetting, a concept fitness drone from Hongik University that could be the first to commercialize, and a joint effort between MIT and US Navy to understand how sound waves are altered by water conditions in the North Pole.

Podcast: Charm's Carbon Offsets, Fitness Trainer Drones, MIT Studies Sound Waves In The North Pole

The deployment of divers to make underwater video recordings is difficult and expensive. It calls for professionals who are trained in two disciplines: diving and video camera operation. There are even some video inspection tasks – such as recording pictures of the inside of an undersea pipe – which are impossible for divers because the location is inaccessible to a person, or too dangerous.Now a Russian manufacturer, <a href="https://gnomrov.com/" rel="noopener" target="_blank">Underwater Robotics</a>, has developed a Remotely Operated underwater Vehicle (ROV) which can carry out video photography missions under the control of an operator who remains safe and dry on the surface of the water. Underwater Robotics’ GNOM-PRO, the world’s smallest and lightest ROV designed for video recording missions, enables users to perform underwater visual inspections without the need for a diver. It also provides a safe way to view small or hazardous locations underwater.Offering outstanding underwater dynamic characteristics, its motion is controlled simply with a joystick. And the key to its stability and ease of use is a <a href="https://www.xsens.com/inertial-sensor-modules" rel="noopener" target="_blank">3D motion sensor</a> from Xsens.Complete motion-tracking module speeds development of ROVXsens’ MTi-3 is a complete AHRS (Attitude and Heading Reference System) including gyroscopes, accelerometers and magnetometer in a 12.1 x 12.1 mm module. It includes Xsens’ industry-leading sensor fusion algorithm, which maintains high accuracy over the sensor’s lifetime, and performs sensor auto-calibration. The use of an integrated Xsens module enabled Underwater Robotics to develop the motion control system in the GNOM-PRO quickly. The module’s calculated measurements of roll, pitch and heading are also easy to integrate into the GNOM-PRO’s control software.When choosing the motion sensor for the GNOM-PRO, Underwater Robotics also carefully considered its size and weight. An important feature of the ROV is that it can be carried by hand: the complete system including the ROV and its maximum 400m-long tether fits in two portable cases. The ROV itself measures 520 x 440 m x 347 mm and weighs 25 kg: this means that it can easily be transported to the site of a mission, and launched and operated from almost any type of boat. Its compact size also means that it can be used in constricted spaces such as pipelines in which a conventional ROV will not fit.The MTi-3 sensor’s small size and its light weight, less than 1g, were therefore highly attractive. But Underwater Robotics’ choice of the MTi-3 was as much about the sensor’s dynamic characteristics. The GNOM-PRO is intended for use in monitoring and maintenance of marine and underwater installations such as oil production platforms and hydroelectric generation plants, for the inspection of ships’ hulls, and for monitoring functions in hazardous locations such as nuclear power plants. Its use as a video camera platform calls for high dynamic stability to ensure that video images are not impaired by blur, shake or other image artefacts.Maintaining stability in waterThe GNOM-PRO’s small size and light weight, however, mean that it is more strongly affected by the motion of waves and currents than a larger vehicle is. The MTi-3’s high performance, including roll and pitch accuracy of ±0.5° under static conditions (±0.8° under dynamic conditions), enables the operator to maintain stable orientation. The GNOM-PRO also has an Auto-Heading Mode to enable the operator to set its direction of travel via an automatic setting controlled by the heading output from the MTi-3. In addition, the MTi-3 sensor’s measurements of velocity, roll, pitch and yaw enable accurate dead reckoning from a known starting position, to establish the geolocation of the ROV.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1618320084977-1618320084977.png" alt="352" width="814" srcset="https://www.xsens.com/hs-fs/hubfs/352.jpg?width=407&amp;name=352.jpg 407w, https://www.xsens.com/hs-fs/hubfs/352.jpg?width=814&amp;name=352.jpg 814w, https://www.xsens.com/hs-fs/hubfs/352.jpg?width=1221&amp;name=352.jpg 1221w, https://www.xsens.com/hs-fs/hubfs/352.jpg?width=1628&amp;name=352.jpg 1628w, https://www.xsens.com/hs-fs/hubfs/352.jpg?width=2035&amp;name=352.jpg 2035w, https://www.xsens.com/hs-fs/hubfs/352.jpg?width=2442&amp;name=352.jpg 2442w" sizes="(max-width: 814px) 100vw, 814px" class="fr-fic fr-dii">Underwater Robotics decided to embed the MTi-3 in the GNOM-PRO ROV after exhaustive evaluation of the motion sensor market.&nbsp;‘On every parameter which we considered important, our evaluation pointed us towards the MTi motion tracker. It is a full-function motion tracker including sensor fusion software, but Xsens fits all the functionality into a small and light package, which is very important for a portable product such as the GNOM-PRO. But the small size does not compromise performance – the accuracy of the MTi-3’s measurements are very good, and in conjunction with the depth measurements provided by a separate pressure sensor, allows the user to tag each frame of video with an exact geolocation.The performance of the GNOM-PRO ROV with the MTi-3 sensor has brought us great success – the product is now in use around the world in countries including France, Italy, Germany, Spain, the US and India as well as Russia. That is why I am sure that we made the right decision to use the MTi-3.’Boris Rozman, CEO of Underwater RoboticsLearn more about the Xsens MTi-3 AHRS&nbsp;The MTi-3 stands out for its small form factor (12.1mm x 12.1mm), easy integration, low cost and robustness – as well as its strapdown integration algorithm.Want to know more? Go to the <a href="https://www.xsens.com/mti-3" rel="noopener noreferrer" target="_blank">MTi-3 page</a>.

ROV With Xsens MTi-3 Sensor Inside Performs Geotagged Visual Inspections Underwater

In this webinar (July 2020) we take the first public in-depth look at the Reach Bravo underwater robotic arm system and answer attendees questions.

WEBINAR Reach Bravo - A New Underwater Robotic Arm - Demo and Q&A

Whether in murky waters, cramped ship tanks or caves with little light – poor visibility conditions have a huge impact on the environmental perception of autonomous robots. How can they still operate safely and reliably? In the new DeeperSense project, an international consortium led by the German Research Center for Artificial Intelligence (DFKI) is working on technologies that combine the strengths of visual and acoustic sensors with the help of artificial intelligence. The aim is to significantly improve the perception of robotic underwater vehicles in three use cases from the maritime sector. The project is funded by the European Union (EU) with around 3 million euros.&nbsp;The possible applications of robotic systems are numerous. However, in practice, there is often a lack of functional technologies that enable autonomous robots to comprehensively perceive complex environments. Visual sensors such as cameras, which are used in robotics for tasks such as autonomous navigation, manipulation, mapping and object recognition, provide detailed information about the environment. However, their performance is highly dependent on the prevailing light and visibility conditions. In contrast, acoustic sensors for distance measurement are independent of visibility conditions. They also generate image-like data, but with a significantly lower resolution than cameras. In addition, their functionality is severely limited at close range.<h2>Intelligent technologies for practicable robot systems in complex environments</h2>This is where the DeeperSense (Deep-Learning for Multimodal <a href="https://www.wevolver.com/article/sensor-fusion-everything-you-need-to-know-" rel="noopener noreferrer" target="_blank">Sensor Fusion</a>) project comes into play, which started on January 1, 2021. The overall objective of the project, which is funded under the EU's Horizon 2020 research framework program, is to significantly improve the environmental perception of service robots, especially in complex and unstructured environments such as underwater. To achieve this, the project pursues an innovative approach: by using artificial intelligence (AI), in particular deep learning as a data-driven machine learning method, it aims to combine the strengths of non-visual and visual sensors to optimize their environment perception capabilities beyond those of the individual sensors. This should not only significantly increase the performance and reliability of autonomous systems, but also open up new functionalities and applications for robotics. <img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE1NTk4MTM0NDEwLWNzbV9EZWVwZXJTZW5zZV9EaXZlcjJfd2ViX2FkZTM5NmUwNWUuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib">Mission scenario: AUV monitors diver working under water. © Technisches Hilfswerk (THW)&nbsp; The DeeperSense concept will be demonstrated in three use cases from the particularly demanding underwater sector: diver monitoring in turbid waters, exploration of coral reefs, and seabed mapping. For this purpose, the project consortium brings together leading experts in the fields of maritime robotics, artificial intelligence and underwater sensor technology – the DFKI Robotics Innovation Center, the University of Girona, the University of Haifa and Kraken Robotik GmbH – with end users from the three different application areas – the German Federal Agency for Technical Relief (THW), the Israel Nature and Parks Authority and Tecnoambiente SL.<h2>Intersensory learning for improved environmental perception in underwater applications</h2>DeeperSense is based on the overarching concept of intersensory learning, in which a sensor modality B learns from a sensor modality A. This way, sensor B can provide output that is similar to that of sensor A – in terms of accuracy as well as the type of output and interpretation of the data. For this purpose, sensor A, e.g. a camera, and sensor B, e.g. a sonar, observe the same scene simultaneously. The collected low-resolution sonar data serves as input to an artificial neural network, while the high-resolution camera data serve as output. By gradually adapting the network to the desired output, it learns about the relationships between the input and output data. The result is an algorithm that, once sufficiently trained, is capable of generating a camera-like image based only on the low-resolution sonar data. In the project, this concept will be realized in three core algorithms that meet the specific requirements of the use cases.In the first use case, an autonomous underwater vehicle (AUV) is used to monitor divers at work in turbid waters. Usually, a remote-controlled robot with a camera is used for this purpose, which transmits its data to a control station. However, since optical sensors can only be used to a limited extent underwater, the project partners are also equipping the AUV with a sonar sensor. In DeeperSense, they want to teach this sensor to deliver camera-like images that can be easily interpreted by human personnel at the control station.The second use case deals with the robotic exploration of complex and sensitive underwater structures such as coral reefs. In order to create detailed environmental maps for reliable obstacle detection, the project relies on a combination of visual and acoustic sensors. Here, the challenge will be to reconcile the differing image perspectives. For this purpose, the planned algorithm must be able to recognize objects identified in the data from one sensor in the data of the other. This way, autonomous robots will be able to navigate through a coral reef instead of just going over it. <div class="fr-img-space-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE1NTk4MjI1NDIxLURlZXBlclNlbnNlX0NvcmFsUmVlZl93ZWIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib">With DeeperSense AUVs will be able to explore coral reefs at close range. © Israel Nature and Parks Authority (INPA)&nbsp;Finally, in the third use case, the partners want to enable an AUV to generate precise maps of the seafloor. These surveys are typically carried out by ships, which is very expensive. Cameras usually provide supplementary information to verify acoustic data. However, this method is not practical in the darkness of deep waters. In the project, the AUV will use various acoustic sensors to classify sediments on the seafloor. With the help of the DeeperSense concept, the acoustic sensors learn to produce annotated maps of different types of seafloor, just as detailed as it would be possible with cameras under good visibility conditions.</div> <h2>Algorithm development and optimization in the laboratory and under real conditions</h2>During the project, the partners will generate comprehensive training data for the three use cases. The algorithms will be trained using this data and tested under real conditions both in the laboratory and in extensive field tests – in inland waters, in the Mediterranean and the Red Sea. Eventually, the trained algorithms will be optimized for on-board use by underwater vehicles in order to enable real-time execution to support the autonomous robot’s behaviour. The consortium intends to make the results available to the robotics community in online repositories embedded in European research infrastructures.

In the new DeeperSense project, an international consortium led by the German Research Center for Artificial Intelligence (DFKI) is working on technologies that combine the strengths of visual and acoustic sensors with the help of artificial intelligence.

Artificial Intelligence To Optimize The Environmental Perception Of Underwater Robots

The work blurs the boundary between materials and robots. In the self-propelled devices, the material itself makes the machine function. "Our examples show that we can use structured materials that deform in response to environmental cues, to control and propel robots," says Daraio, professor of mechanical engineering and applied physics in Caltech's Division of Engineering and Applied Science, and corresponding author of a paper unveiling the robots that appears in the Proceedings of the National Academy of Sciences&nbsp;on May 15.The new propulsion system relies on strips of a flexible polymer that is curled when cold and stretches out when warm. The polymer is positioned to activate a switch inside the robot's body, that is in turn attached to a paddle that rows it forward like a rowboat.The switch is made from a bistable element, which is a component that can be stable in two distinct geometries. In this case, it is built from strips of an elastic material that, when pushed on by the polymer, snaps from one position to another.<iframe src="https://www.youtube.com/embed/tBxb1A6boTI?&amp;wmode=opaque" allowfullscreen="" class="fr-draggable" style="width: 767px; height: 439px;" frameborder="0"></iframe>When the cold robot is placed in warm water, the polymer stretches out, activates the switch, and the resulting sudden release of energy paddles the robot forward. The polymer strips can also be "tuned" to give specific responses at different times: that is, a thicker strip will take longer to warm up, stretch out, and ultimately activate its paddle than a thinner strip. This tunability allows the team to design robots capable of turning and moving at different speeds.The research builds on previous work by Daraio and Dennis Kochmann, professor of aerospace at Caltech. They used chains of <a href="http://www.caltech.edu/news/utility-instability-51629" target="_blank">bistable elements to transmit signals and build computer-like logic gates</a>.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1564499883153-Screenshot_2019-07-30+Harnessing+bistability+for+directional+propulsion+of+soft%2C+untethered+robots+-+5698+full+pdf%281%29.png" style="width: 766px;" class="fr-fic fr-dib">In the latest iteration of the design, Daraio's team and collaborators were able to link up the polymer elements and switches in such a way to make a four-paddled robot propel itself forward, drop off a small payload (in this case, a token with a Caltech seal emblazoned on it), and then paddle backward.&nbsp;"Combining simple motions together, we were able to embed programming into the material to carry out a sequence of complex behaviors," says Caltech postdoctoral scholar Osama R. Bilal, co-first author of the PNASpaper. In the future, more functionalities and responsivities can be added, for example using polymers that respond to other environmental cues, like pH or salinity. Future versions of the robots could contain chemical spills or, on a smaller scale, deliver drugs, the researchers say.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1564499825448-1.jpg" style="width: 766px;" class="fr-fic fr-dib">Currently, when the bistable elements snap and release their energy, they must be manually reset in order to work again. Next, the team plans to explore ways to redesign the bistable elements so that they are self-resetting when water temperature shifts again—making them potentially capable of swimming on indefinitely, so long as water temperature keeps fluctuating.The PNAS&nbsp;paper is titled <a href="http://resolver.caltech.edu/CaltechAUTHORS:20180515-101708023" target="_blank">"Harnessing bistability for directional propulsion of soft, untethered robots."</a> Daraio and Bilal collaborated with Tian Chen and Kristina Shea from ETH in Zurich. This research was supported by the Army Research Office and an ETH postdoctoral fellowship to Bilal.

 An artist's rendering of the new design for a robot that uses material deformation to propel itself through water. 

Engineers have developed robots capable of self-propulsion without using any motors, servos, or power supply. Instead, these first-of-their-kind devices paddle through water as the material they are constructed from deforms with temperature changes.

autonomous underwater vehicles (AUVs)

Profiles