<h2 dir="ltr" id="isPasted">FREE REPORT:&nbsp;Reducing Human Driver Error and Setting Realistic Expectations with Advanced Driver Assistance Systems</h2><p dir="ltr">Thousands die or are injured each year in automobile crashes. Bringing automated driving systems technologies into the advanced driver assist systems (ADAS) and connected vehicle space will help humans drive more safely and better prepare us for automated vehicles (AVs).</p><p dir="ltr">Learn more about ADAS and ADS implementation with the goal of reaching zero deaths and serious injuries by downloading this FREE report from SAE International (valued at $50).&nbsp;</p><p dir="ltr"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyMzIxNDMwMTE3LXNhZSBmcmVlIHJlcG9ydC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"></p><p dir="ltr"><span contenteditable="false" draggable="true" class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable fr-active"><iframe src="https://c8056756.sibforms.com/serve/MUIFAIm7QnvQQ9ZyzkX--H5jNS-TSb9GWr5SQBXi_SUjRPpt67CBveq_yevZYMSeiJ-O1dwYw05f2kl-Q7sfNDRem1Y-03GXYZV4qkTbJeR_K_VpJzgMk_8Ry8fjyfgJ7fAMrCmcRHc1KXsSTuhIU5p5pBa3F1-TGDgOdQgcF-pU8cSTrl8JF2j2b9gl6-5e66Ro-rltcIrGxsRG" frameborder="0" scrolling="auto" allowfullscreen="" style="display: block; margin-left: auto; margin-right: auto; max-width: 100%; width: 540px; height: 740px;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"><span class="fr-mk" style="display: none;">&nbsp;</span><span class="fr-mk" style="display: none;">&nbsp;</span></iframe></span></p><hr><h2>Provizio promises its 5D Perception stack can safely compete with expensive lidar sensors at a fraction of the cost.</h2><p>"Safety first” is more than a catchphrase. For sensing company Provizio, it’s the only way the transportation industry should introduce autonomous vehicles. In Provizio’s view, using AV building blocks – technology such as automatic emergency braking and lane-keep assist – can be valuable in ADAS systems, but they should not be used to drive vehicles until the perception problem has been solved.</p><p><span class="fr-img-caption fr-fic fr-dib" style="width: 302px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzAyMjgyNDM3MTM3LXByb3ZpemlvLXJhZGFyLXN1cGVyLXJlc29sdXRpb24tZ2FsbGVyeS5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">An example image using data taken from Provizio’s “super-resolution” radar sensor.</span></span></span></p><blockquote><p>“It’s not that we’re skeptical about autonomous driving, it’s just that we strongly believe that the industry has taken this wrong path,” Dane Mitrev, machine learning engineer at Provizio, told SAE at September 2023’s AutoSens Brussels conference.</p></blockquote><p>“The industry has looked at things the other way around. They tried to solve autonomy first, without looking at accident prevention and simpler ADAS systems. We are building a perception technology which will first eliminate road fatalities, then deliver next-generation ADAS – and then solve autonomy. We believe that’s the right order.”</p><p>Obviously, the industry is in favor of AVs. A group of 11 mobility and automotive companies released a 157-page paper in 2019 called “Safety First for Automated Driving.” Those companies — Aptiv, Audi, Baidu, BMW, Continental, Daimler, FCA US LLC, HERE, Infineon, Intel and Volkswagen — used the paper to propose a non-binding organized framework focused on “the development, testing and validation of safe automated passenger vehicles,” but did not attempt to establish any actual limits on what companies could do as they developed AVs. In the years since, AV technology has continued to advance, but not with the mission to solve sensing and important safety issues before teaching the vehicles how to maneuver.</p><p>Provizio’s solution is its 5D Perception driving platform, named for the way it uses five different inputs to sense the environment. There’s the 3D radar information of an object in space, plus velocity information, plus the fifth dimension: perception, which means AI embedded in the hardware. Mitrev said the final vision of this technology will be powered by radar and cameras, not light detection and ranging (lidar). Radar’s benefits are well known: the sensors are smaller (and thus simpler to integrate) and less expensive than lidar and can see in all weather conditions. For now, though, Provizio is working with lidar, especially around its hometown.</p><h3><strong>Irish sensors are smiling</strong></h3><p>Provizio is headquartered in Limerick, Ireland, and has offices in North Ireland and Pittsburgh, Pennsylvania. In late 2022, Provizio signed a deal with micromobility company Voi to test 5D perception on Voi’s scooters. Alongside potential OEM and Tier 1 partnerships he couldn’t discuss, Mitrev said Provizio also has a partnership with Jaguar Land Rover, supporting tests at a future mobility campus in Ireland.</p><p>These real-world tests involve collecting data from lidar to train the new radar AI. Provizio synchronizes the lidar and radar sensors, then extracts value from the lidar data at training time, embedding it into the neural net and teaching the neural networks based on these data streams. Imagine lidar as an old professor teaching a young radar sensor a few things, but sooner or later, the student – Provizio’s radar – won’t need the teacher anymore.</p><blockquote><p>“We lean fully towards a lidar-less path,” he said. “Lidars should not be involved, even in [SAE Level 5] vehicles. They’re not necessary.”</p></blockquote><p>The same is not true for cameras. To understand semantic information such as traffic signs, Provizio’s system uses cameras to augment the radar data.</p><p>“Provizio is a perception company, first of all,” Mitrev said. “We are not a radar company. We’re not a car manufacturer. We’re a perception company, we have this radar and we have a perception platform that fuses radars and cameras and provides this low-cost solution.”</p><p>Provizio was founded by Barry Lunn following the acquisition of his company Arralis, which focused on millimeter-wave communication technology for satellite communications and aerospace markets. Provizio’s radar technology grew out of radar work Lunn was familiar with in the aerospace industry. Instead of a commercial, off-the-shelf radar unit, Provizio is “redefining radars,” Mitrev said, by adding AI neural networks to the radar GPUs so all critical decisions can be made on the radar itself with less than five milliseconds latency to make those decisions. He said Provizio’s goal for its 5D radar is to see up to 600 m (1,969 feet) in 360 degrees in all weather conditions.</p><blockquote><p>“Once we have this 5D perception, there is literally no reason to build any other system that will do ADAS or autonomy,” he said, adding that accident prevention is “something tangible that we can solve in the next one, three-to-five, years.”</p></blockquote><p>At CES 2023, Provizio said it planned to deliver the first of its 5D Perception platforms by 2025. Provizio’s commercial strategy relies on licensing its technology to Tier 1 suppliers and OEMs. Once better sensors are in more vehicles that accurately understand the world around them, that’s when we can start letting vehicles drive themselves, Mitrev said.</p><p>“The end goal is mass adoption, to have our 5D perception on all vehicles or a lot of vehicles worldwide, then we bring down the road deaths,” he said. “We believe that this is the right path. The industry has taken this the other way around. [Starting by building a self-driving car] is just too complex, let alone with only cameras, as some companies claim.”</p>

Provizio offered demo drives of its technology on the road in Las Vegas during CES in January 2023. These images show some of what the sensors saw during the drive, along with some of Provizio’s identifying data. (Provizio)

"Safety first” is more than a catchphrase. For sensing company Provizio, it’s the only way the transportation industry should introduce autonomous vehicles.

Autonomus safety with radar, not LiDAR

<p id="isPasted">Knowing the distance between two short-range transceivers is useful for many applications. Examples include proximity measurement for making sure people maintain a set distance apart for safety reasons, indoor navigation, and tracking items in a warehouse.</p><p>There are several techniques for measuring distance from straightforward received signal strength indicator (RSSI) and time-of-flight (ToF), through more complex phase-based measurements to advanced techniques such as Inverse Fast Fourier Transform (IFFT) and estimation of multipath channel impulse response from frequency spectrum measurements. Each trades off some combination of accuracy, resolution, calculation time, processing resource, and energy.</p><p>It&#39;s possible to try out each of these techniques to find out which is best for your applications using the Nordic Distance Toolbox (NDT). NDT is a proprietary technology that&rsquo;s an easy-to-use development tool for measuring the distance between two Nordic short-range radios. It is included in Nordic&rsquo;s <a href="https://www.nordicsemi.com/Products/Development-software/nRF-Connect-SDK?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">nRF Connect SDK</a>, the company&rsquo;s scalable and unified software development kit. Let&rsquo;s take a closer look at these various distance measuring techniques.</p><h2>Using signal intensity to estimate distance</h2><p>While not able to focus a signal nor determine the direction from which it has come, a short-range radio such as a Bluetooth LE transceiver with a single antenna can measure signal strength, the frequency of the incoming signal, its phase, and its time of arrival. This information is useful for estimating the distance to the transmitter.</p><p>When transmitting, the short-range radio broadcasts a 2.402 to 2.480 GHz signal that forms a sphere expanding at the speed of light. As the sphere propagates, the intensity (power per unit area) of the signal in a vacuum and without obstructions drops off according to the formula 1/r2 (where &ldquo;r&rdquo; is the radius of the sphere). With knowledge of the original transmission power and incoming signal strength, the receiver can therefore work out approximately how far it is from the transmitter.</p><p>But a vacuum without obstructions is not the usual operating environment for a short-range radio. In typical use, for example, in a house or an office, the intensity drop will be greater. To compensate and improve distance measurement accuracy, engineers instead use the formula 1/rn where &ldquo;n&rdquo; is the &ldquo;path loss factor&rdquo;. For example, in a home with walls made of wood, n is around 4.5.</p><p>RSSI is a cheap and inexpensive way to estimate the distance between two transceivers. The downside is that the path loss factor introduces errors even if it has been carefully assessed for an anticipated operating scenario. Nordic conducted some tests in an office environment that compared the distance between two short-range radios measured very precisely using LIDAR with those obtained from RSSI measurements. RSSI proved precise up to a few meters, but beyond that distance estimates became increasingly inaccurate. When people were moving through the measurement area, RSSI precision deteriorated markedly, and the useable range dropped to half a meter.</p><h2>Time-of-Flight and phase measurements</h2><p>A second technique that brings greater precision to distance measurement is ToF. The technique measures the time it takes for a signal to travel from the transmitter to the receiver and back, and then calculates the distance based on the speed of light. (ToF (seconds)/2 x c (meters per second) = distance to object (meters).) There is a small correction, because the receiving radio will require a short but finite time to switch from receive to transmit and &ldquo;reflect&rdquo; the signal back. However, such a correction is easily included in the distance calculation.</p><p>A third technique that also offers greater accuracy than RSSI is phase measurement. It&rsquo;s a technique that has been used for many years in radar installations. The math is complex, but in essence the measurement relies on different radio frequencies having different wavelengths. A 2.4 GHz signal, for example, has a wavelength of 12.5 cm, whereas a 2.48 GHz signal has a slightly shorter wavelength of 12.1 cm.</p><p>Over two meters, for example, a 2.4 GHz signal will oscillate exactly (2/0.125) 16 times so its phase at the receiver will be the same as when it left the transmitter. However, a 2.48 GHz signal will oscillate (2/0.121) 16.53 times. That extra 0.53 of a cycle will mean that the received signal is (2&pi; x 0.53) 1.06&pi; out of phase compared with the transmitted signal.</p><p>For a given distance, phase measurements across a range of frequencies results in a linear phase variation with a measurable gradient. Changing the distance alters the gradient of the phase variation with a steeper gradient representing a greater range. With knowledge of the gradient change for a set of reference distances, in a practical situation, it is relatively simple to correlate a measured gradient to the actual distance between the two transmitters. &nbsp; &nbsp;</p><p>Nordic&rsquo;s tests using the LIDAR comparison showed that both ToF and phase measurement offer reasonable accuracy and are not affected by the signal attenuation that plagues RSSI measurements. For shorter distances (up to ten meters), phase measurement offers greater precision. Over long distances (up to several hundred meters), phase measurement tends to become less precise and ToF is a better way to measure the distance between the radios. Note that both the ToF and phase change techniques require a two-way connection between transmitter and receiver, unlike RSSI.</p><h2>Greater precision but increased complexity</h2><p>If the precision offered by ToF or phase measurement is still not adequate for an application, there are two more methods supported by Nordic Distance Toolbox that offer even greater accuracy. One uses Inverse Fast Fourier Transform (IFFT) techniques and the other uses an estimation of multipath channel impulse response from frequency spectrum measurements.</p><p>These techniques are complex, and both rely on sophisticated analysis of the transmitted and received signals. However, the result is a precise and repeatable measurement of the distance between the short range radios. The trade-off with both techniques is that they demand a high level of computing resources and long processing time that consumes energy and hence impacts battery life.</p><p>Nordic&#39;s tests revealed that IFFT produces a good correlation to the actual distance measured by LIDAR and is more accurate than RSSI, ToF and phase measurement, although some outlier filtering is likely to be required to get the best results. The estimation of multipath channel impulse response measurement is even better than IFFT, producing high precision results; but it also requires outlier filtering and features longer processing times than the other techniques. However, if the highest precision distance measurement is needed for the application, then the energy trade-off could be worth it.</p><h2>Nordic Distance Toolbox does all the work</h2><p>The Nordic Distance Toolbox included with Nordic&rsquo;s nRF Connect SDK is useful for testing various distance measurement techniques with Nordic short-range radio SoCs and includes a quality indicator to provide a degree of confidence in how accurate the measurement data is likely to be. Distance is computed based on all intensity, frequency, and phase information available to the transceiver and all calculations are completed inside the toolbox.</p><p>Nordic Distance Toolbox currently offers experimental support for <a href="https://www.nordicsemi.com/Products/nRF52833?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">nRF52833</a>, <a href="https://www.nordicsemi.com/Products/nRF52840?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">nRF52840</a>, and <a href="https://www.nordicsemi.com/Products/nRF5340?utm_campaign=2021%20wevolver" rel="noopener noreferrer" target="_blank">nRF5340</a> and includes measurement support for: RSSI; ToF; phase slope estimation; IFFT; and estimation of multipath channel impulse response from frequency spectrum measurements.</p><hr id="isPasted"><p>This article was first published on Nordic&#39;s <a href="https://blog.nordicsemi.com/getconnected" target="_blank" rel="nofollow"></a><a href="https://blog.nordicsemi.com/getconnected?utm_campaign=2021%20wevolver" target="_blank" rel="nofollow">Get Connected Blog</a>. &nbsp;</p>

Knowing the distance between two short-range transceivers is useful for many applications. Examples include proximity measurement for making sure people maintain a set distance apart for safety reasons, indoor navigation, and tracking items in a warehouse.

Using the Nordic Distance Toolbox to measure the distance between short-range radios

In this article, we will dive deep into the world of simultaneous localization and mapping using Lidar technology. Lidar SLAM has been gaining popularity in recent years, thanks to its versatility and applications across various domains, including autonomous vehicles, mobile robotics, and indoor mapping.

Lidar SLAM: The Ultimate Guide to Simultaneous Localization and Mapping

Explore the key differences, advantages, and applications of lidar and radar technologies in modern engineering projects.

Lidar vs. Radar: Comprehensive Comparison and Analysis

<p>In this episode, we talk about how Toyota and Stanford are collaborating to utilize AI for drifting controls and the efforts behind a team at TUM which has resulted in an autonomous race car performing at the capacity of an amateur F1 driver.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/3793ff13-dbcd-4d01-bee8-121b58085bba?dark=false" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><hr><p>This podcast is sponsored by <a href="https://www2.mouser.com/" id="isPasted" rel="nofollow" target="_blank">Mouser Electronics</a>. &nbsp; &nbsp; &nbsp;</p><p><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1662459182875-Mouser-sponsor-1-a.jpg" style="width: 766px;" class="fr-fic fr-dib"></p><hr><h3 id="isPasted">EPISODE NOTES</h3><p>(3:55) -&nbsp;<a href="https://www.wevolver.com/article/researchers-program-an-autonomous-vehicle-to-drift-around-obstacles-using-nonlinear-model-predictive-control-nmpc" target="_blank">Drifting Autonomous Vehicle</a>🚗</p><p>(16:50) -&nbsp;<a href="https://www.wevolver.com/article/formula-1-could-see-driverless-race-cars-as-early-as-2025" target="_blank">Autonomous F1 Races By 2025</a>🏎</p><p>Episode 86 was brought to you by Mouser Electronics, Farbod &amp; Daniel&rsquo;s favorite electronics distributor. Click <a href="https://resources.mouser.com/automotive/formula-e-racers-help-advance-ev-design" target="_blank">here</a> to read about the Mouser technical resource discussing Formula racing!</p><hr><p id="isPasted">As always, you can find these and other interesting &amp; impactful engineering articles on Wevolver.com.</p><p>To learn more about this show, please visit our<a href="https://www.wevolver.com/profile/the.next.byte" target="_blank">&nbsp;shows page</a>. By following the page, you will get automatic updates by email when a new show is published. Be sure to give us a follow and review on<a href="https://podcasts.apple.com/nl/podcast/the-next-byte/id1548255861?l=en" target="_blank">&nbsp;Apple&nbsp;</a>podcasts,<a href="https://open.spotify.com/show/6lPPsRtegnivsRUEsQ09R7?si=IPMiah7xT_apJtPjqvXMlA" target="_blank">&nbsp;Spotify</a>, and most of your favorite podcast platforms!</p>

In this episode, we talk about how Toyota and Stanford are collaborating to utilize AI for drifting controls and the efforts behind a team at TUM which has resulted in an autonomous race car performing at the capacity of an amateur F1 driver.

Podcast: The Autonomous Race & Drift Cars Of The Not So Distant Future

<p id="isPasted">Erbium-doped fiber amplifiers (EDFAs) are devices that can provide gain to the optical signal power in optical fibers, often used in long-distance communication fiber optic cables and fiber-based lasers. Invented in the 1980s, EDFAs are arguably one of the most important inventions, and have profoundly impacted our information society enabling signals to be routed across the Atlantic and replacing electrical repeaters.</p><p>What is interesting about erbium ions in optical communications is that they can amplify light in the 1.55 mm wavelength region, which is where silica-based optical fibers have the lowest transmission loss. The unique electronic intra-4-f shell structure of erbium &ndash; and rare-earth ions in general &ndash; enables long-lived excited states when doped inside host materials such as glass. This provides an ideal gain medium for simultaneous amplification of multiple information-carrying channels, with negligible cross-talk, high temperature stability and low noise figure.</p><p>Optical amplification is also used in virtually all laser applications, from fiber sensing and frequency metrology, to industrial applications including laser-machining and LiDAR. Today, optical amplifiers based on rare-earth ions have become the workhorse for optical frequency combs (2005 Nobel Prize in Physics), which are used to create the world&rsquo;s most precise atomic clocks.</p><p>Achieving light amplification with rare-earth ions in photonic integrated circuit can transform integrated photonics. Already in the 1990s, Bell Laboratories were looking into erbium-doped waveguide amplifiers (EDWAs), but ultimately abandoned them because their gain and output power could not match fiber-based amplifiers, while their fabrication doesn&rsquo;t work with contemporary photonic integration manufacturing techniques.</p><p>Even with the recent rise of integrated photonics, renewed efforts on EDWAs have only been able to achieve less than 1 mW output power, which is not enough for many practical applications. The problem here has been high waveguide background loss, high cooperative upconversion &ndash; a gain-limiting factor at high erbium concentration, or the long-standing challenge in achieving meter-scale waveguide lengths in compact photonic chips.</p><p>Now, researchers at EPFL, led by Professor Tobias J. Kippenberg, have built an EDWA based on silicon nitride (Si<sub>3</sub>N<sub>4</sub>) photonic integrated circuits of a length up to half meter on a millimeter-scale footprint, generating a record output power of more than 145 mW and providing a small-signal net gain above 30 dB, which translates to over 1000-fold amplification in the telecommunication band in continuous operation. This performance matches the commercial, high-end EDFAs, as well as state-of-the-art heterogeneously integrated III-V semiconductor amplifiers in silicon photonics.</p><p>﻿&ldquo;We overcame the longstanding challenge by applying ion implantation &ndash; a wafer-scale process that benefits from very low cooperative upconversion even at a very high ion concentration &ndash; to the ultralow-loss silicon nitrideintegrated photonic circuits,&rdquo; says Dr Yang Liu, a researcher in Kippenberg&rsquo;s lab, and the study&rsquo;s lead scientist.</p><p>&ldquo;This approach allows us to achieve low loss, high erbium concentration, and a large mode-ion overlap factor in compact waveguides with meter-scale lengths, which have previously remained unsolved for decades,&rdquo; says Zheru Qiu, a PhD student and co-author of the study.</p><p>﻿&ldquo;Operating with high output power and high gain is not a mere academic achievement; in fact, it is crucial to the practical operation of any amplifier, as it implies that any input signals can reach the power levels that are sufficient for long-distance high-speed data transmission and shot-noise limited detection; it also signals that high-pulse-energy femtosecond-lasers on a chip can finally become possible using this approach,&rdquo; says Kippenberg.</p><p>The breakthrough signals a renaissance of rare-earth ions as viable gain media in integrated photonics, as applications of EDWAs are virtually unlimited, from optical communications and LiDAR for autonomous driving, to quantum sensing and memories for large quantum networks. It is expected to trigger follow-up studies that cover even more rare-earth ions, offering optical gain from the visible up to the mid-infrared part of the spectrum and even higher output power.</p><hr><p><strong id="isPasted">References</strong></p><p>Yang Liu, Zheru Qiu, Xinru Ji, Anton Lukashchuk, Jijun He, Johann Riemensberger, Martin Hafermann, Rui Ning Wang, Junqiu Liu, Carsten Ronning, and Tobias J. Kippenberg. A photonic integrated circuit based erbium-doped amplifier. Science, 17 June 2022. DOI:&nbsp;<a href="https://dx.doi.org/10.1126/science.abo2631" rel="noopener" target="_blank">10.1126/science.abo2631</a></p>

An erbium-doped waveguide amplifier on a photonic integrated chip. Credit: EPFL Laboratory of Photonics and Quantum Measurements/Niels Ackerman

EPFL scientists have built a compact waveguide amplifier by successfully incorporating rare-earth ions into integrated photonic circuits. The device produces record output power compared to commercial fiber amplifiers, a first in the development of integrated photonics over the last decades.

Boosting light power revolutionizes communications and autopilot

<p>This article was discussed in our <a data-saferedirecturl="https://www.google.com/url?q=https://www.wevolver.com/profile/the.next.byte&source=gmail&ust=1662506724055000&usg=AOvVaw3ePqI00MDVgWOHTO-3eNvi" href="https://www.wevolver.com/profile/the.next.byte" id="isPasted" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/3793ff13-dbcd-4d01-bee8-121b58085bba?dark=false" class="fr-draggable" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true"></iframe></span></p><p>The full article will continue below.</p><hr><header id="isPasted"><p itemprop="description"><strong>A team at the Technical University of Munich (TUM) has developed software which lets race cars compete in motor sports without a driver. The TUM Autonomous Motorsport Team was able to take 1st and 2nd place at the Autonomous Challenges in Indianapolis and most recently at the CES in Las Vegas. Does this technology have the potential to revolutionize racing? Markus Lienkamp, Professor for automotive technology, tells us the answer.</strong></p></header><section><h2><strong>Prof. Lienkamp, what gave you the idea of entering your team in the two Autonomous Challenges in the USA?</strong></h2><p>The story began back in 2005, as information scientist Sebastian Thrun from Stanford University won the DARPA Grand Challenge, which was until then highly renowned as the only race for autonomous vehicles, held in the Nevada desert. At the time I was working at Volkswagen and built the car for him. Then came the 2007 Urban Challenge in California, a race for autonomous cars on a paved course under realistic traffic conditions, where once again three Volkswagen vehicles from various universities made the finals. Sebastian Thrun then had the idea of staging speed races similar to Formula 1 on a real race track: That was the birth of the <a href="https://www.indyautonomouschallenge.com/" rel="noreferrer" target="_blank">Autonomous Challenge</a> concept. And it went without saying that I wanted to participate together with a TUM team.</p><h2><strong>The autonomous vehicles which are currently being tested for street use already contain an enormous amount of AI. What&#39;s the difference between the software you use in the race cars and the systems already in use?</strong></h2><p>The race track doesn&#39;t have traffic laws or points of reference like marked lanes, traffic lights or traffic signs. Ultimately we&#39;re dealing with &quot;unpredictable&quot; objects, in our case the other race cars. We have to detect them and make predictions about where they&#39;ll move, and that all at speeds of over 250 km/h. We&#39;re able to do this because our software doesn&#39;t concentrate on strictly complying with traffic regulations, as is the case with other vendors. Instead, our software calculates the probable locations of the other objects which it then uses to calculate the optimum solution for our own movements.</p><h2><strong>How exactly does your software do that?</strong></h2><p>Our approach uses conventional sensor technologies with lasers, cameras and radar. The software knows the race course and detects the other vehicles. That means it can predict the most probable trajectory, i.e. what location on the course the other vehicle will move to at what point in time, and can then plan its own movements accordingly. Here the computing speed of the software plays an important role. That&#39;s decisive when it comes to safely executing aggressive maneuvers and reacting spontaneously to critical situations.</p><p>The fact that we made the software generally available as open source code and thus let other teams work on it was enormously helpful as well. That speeds up the development process so that soon the number of people around the world who will be able to work on our code will surpass the development capacities of all the automobile manufacturers put together. Ultimately, though, the software and the car are only as good as the teams behind them. I have an extraordinarily good team whose members worked very hard and well beyond the call of their normal everyday duties to make this success happen. 15 doctoral candidates moved this entire project ahead over the course of four years, and four team leaders made sure the teams succeeded in the individual phases.</p><h2><strong>Formula 1 is the best-known auto racing format in the world. Do you think it&#39;s possible that Formula 1 will be completely driverless sometime in the future?</strong></h2><p>That&#39;s both a technical and emotional discussion which affects the spectators more than anything else. Fans always like to bond with particular driver personalities and identify with them in competition. When I see the constantly growing enthusiasm for E-Sports, I think at some point we&#39;ll see both. In technical terms of course we always have to ask when it will be realistic to actually replace the driver. After our experience in Indianapolis, we believe this could be possible by 2025. We can even imagine sending one of our autonomous cars to a real Formula 1 race to compete against human professionals.</p><h2><strong>How high do you think the chances are that an autonomously driven car could beat a human driver?</strong></h2><p>In its present form our software has already reached the level of an amateur driver. When we talk about professional-class races, experience shows we&#39;re about half a second behind. So it will probably take a couple more years before our autonomous vehicles are able to beat racing pros. You can compare it to a chess game against the computer: In the beginning the AI was only able to beat hobby players. It took quite a while before it was possible to beat the chess world champion. But it&#39;s definitely possible.</p><h2><strong>After these great successes in Indianapolis and Las Vegas &ndash; What comes next?</strong></h2><p>The competition will pretty much continue to CES next year. We&#39;d like to compete once again. We want to put what we&#39;ve learned in the races back on the street &ndash; for example with an autonomous shuttle during the Oktoberfest. And we&#39;ve founded our own software company &ldquo;driveblocks&rdquo; which is industrializing the whole thing and making it ready for series production. That means our expertise will soon be for sale.</p></section>

Prof. Markus Lienkamp believes in the future of autonomous motorsports. Image: Jacob Kepler

A team at the Technical University of Munich (TUM) has developed software which lets race cars compete in motor sports without a driver. The TUM Autonomous Motorsport Team was able to take 1st and 2nd place at the Autonomous Challenges in Indianapolis and most recently at the CES in Las Vegas.

"Formula 1 could see driverless race cars as early as 2025"

<p>The technology, developed by researchers from the University of Cambridge, the University of Oxford and University College London (UCL), is based on LiDAR (light detection and ranging), and uses LiDAR data to create ultra-high-definition holographic representations of road objects which are beamed directly to the driver’s eyes, instead of 2D windscreen projections used in most head-up displays.</p><p>While the technology has not yet been tested in a car, early tests, based on data collected from a busy street in central London, showed that the holographic images appear in the driver’s field of view according to their actual position, creating an augmented reality. This could be particularly useful where objects such as road signs are hidden by large trees or trucks, for example, allowing the driver to ‘see through’ visual obstructions. The <a href="https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-29-9-13681&id=450306" target="_blank">results</a> are reported in the journal <em>Optics Express</em>.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/pa7o2Gd4ITU?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 786px; height: 442px;"></iframe></span></p><p>“Head-up displays are being incorporated into connected vehicles, and usually project information such as speed or fuel levels directly onto the windscreen in front of the driver, who must keep their eyes on the road,” said lead author <a href="http://www.eng.cam.ac.uk/profiles/js2459" target="_blank">Jana Skirnewskaja</a>, a PhD candidate from Cambridge’s Department of Engineering. “However, we wanted to go a step further by representing real objects as panoramic 3D projections.”</p><p>Skirnewskaja and her colleagues based their system on LiDAR, a remote sensing method that works by sending out a laser pulse to measure the distance between the scanner and an object. LiDAR is commonly used in agriculture, archaeology and geography, but it is also being trialled in autonomous vehicles for obstacle detection.</p><p><span class="fr-video fr-deletable fr-fvc fr-dvb fr-draggable" contenteditable="false" draggable="true"><iframe src="https://www.youtube.com/embed/9kgweRItSFU?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 787px; height: 466px;"></iframe></span></p><p>Using LiDAR, the researchers scanned Malet Street, a busy street on the UCL campus in central London. Co-author Phil Wilkes, a geographer who normally uses LiDAR to scan tropical forests, scanned the whole street using a technique called terrestrial laser scanning. Millions of pulses were sent out from multiple positions along Malet Street. The LiDAR data was then combined with point cloud data, building up a 3D model.</p><p>“This way, we can stitch the scans together, building a whole scene, which doesn’t only capture trees, but cars, trucks, people, signs, and everything else you would see on a typical city street,” said Wilkes. “Although the data we captured was from a stationary platform, it’s similar to the sensors that will be in the next generation of autonomous or semi-autonomous vehicles.”</p><p>When the 3D model of Malet St was completed, the researchers then transformed various objects on the street into holographic projections. The LiDAR data, in the form of point clouds, was processed by separation algorithms to identify and extract the target objects. Another algorithm was used to convert the target objects into computer-generated diffraction patterns. These data points were implemented into the optical setup to project 3D holographic objects into the driver’s field of view.</p><p></p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 788px;"><span class="fr-img-wrap"><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1621597362621-Projection+3_CENTRE.jpg" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner">Image based on LiDAR data (left), converted to a hologram (right).&nbsp;</span></span></span><p class="fr-img-space-wrap2">The optical setup is capable of projecting multiple layers of holograms with the help of advanced algorithms. The holographic projection can appear at different sizes and is aligned with the position of the represented real object on the street. For example, a hidden street sign would appear as a holographic projection relative to its actual position behind the obstruction, acting as an alert mechanism.</p><p>In future, the researchers hope to refine their system by personalising the layout of the head-up displays and have created an algorithm capable of projecting several layers of different objects. These layered holograms can be freely arranged in the driver’s vision space. For example, in the first layer, a traffic sign at a further distance can be projected at a smaller size. In the second layer, a warning sign at a closer distance can be displayed at a larger size.</p><p>“This layering technique provides an augmented reality experience and alerts the driver in a natural way,” said Skirnewskaja. “Every individual may have different preferences for their display options. For instance, the driver’s vital health signs could be projected in a desired location of the head-up display.</p><p>“Panoramic holographic projections could be a valuable addition to existing safety measures by showing road objects in real time. Holograms act to alert the driver but are not a distraction.”</p><p>The researchers are now working to miniaturise the optical components used in their holographic setup so they can fit into a car. Once the setup is complete, vehicle tests on public roads in Cambridge will be carried out.</p><p>Skirnewskaja is a PhD candidate at the <a href="https://www.ceps-cdt.org/" target="_blank">EPSRC Centre for Doctoral Training (CDT) in Connected Electronic and Photonic Systems</a>, a joint centre with the University of Cambridge and UCL. She also is a fellow of the Foundation of German Business (SDW).</p><hr><p><strong>Reference:</strong><br><em>Jana Skirnewskaja et al. ‘<a href="https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-29-9-13681&id=450306" target="_blank">LiDAR-Derived Digital Holograms for Automotive Head-Up Displays</a>.’ Optics Express (2021). DOI: 10.1364/oe.420740</em></p></div><p></p>

Image based on LiDAR data (left), converted to a hologram (right).

Researchers have developed the first LiDAR-based augmented reality head-up display for use in vehicles. Tests on a prototype version of the technology suggest that it could improve road safety by ‘seeing through’ objects to alert of potential hazards without distracting the driver.

3D holographic head-up display could improve road safety

<h2 dir="ltr">Introduction</h2><p dir="ltr">LiDAR and RADAR are both remote sensing devices that are often used for the detection, tracking, and imaging of various objects.</p><p dir="ltr">Both sensors have assisted measurement in a lot of fields, including environmental research, space exploration, and medical discovery.</p><p dir="ltr">While they&rsquo;re used across a broad range of industries, LiDAR and RADAR are stirring up a buzz for their functionality in the autonomous vehicle space. The discussion around which one is better has plagued the industry for a while.</p><p dir="ltr">Most manufacturers choose LiDAR sensors as their top sensing device. On the other hand, Elon Musk, Tesla&rsquo;s CEO and a central figure in the industry have <a href="https://arstechnica.com/cars/2019/08/elon-musk-says-driverless-cars-dont-need-lidar-experts-arent-so-sure/" target="_blank">shown his preference for RADAR</a> on multiple occasions.</p><p dir="ltr">Self-driving vehicles need to be aware of their surrounding environment. Often, the system that enables object detection in autonomous vehicles includes a variety of sensing devices connected through a <a href="https://www.wevolver.com/article/sensor-fusion-everything-you-need-to-know-" target="_blank">sensor fusion algorithm</a>, including LiDAR, RADAR, ultrasonic sensors, and cameras.</p><p dir="ltr">While they have overlapping goals, the simple change of the type of wave used makes their capabilities vastly different from one another.</p><h2 dir="ltr">What is LiDAR?</h2><p dir="ltr">LiDAR stands for light detection and ranging or laser imaging, detection, and ranging. While it&rsquo;s a fairly new concept compared to RADAR, it&rsquo;s been around since the discovery of lasers in the 1960s.</p><p dir="ltr">There are two types of LiDAR sensing devices, airborne- and ground-based.</p><p dir="ltr">Ground-based LiDAR uses lasers with a wavelength of 500 - 600 nm and is used in various applications, such as landslide studies or obstacle detection in autonomous vehicles. Airborne-based LiDAR, on the other hand, uses lasers with a wavelength of 1000 - 1600 nm and mostly used to collect terrain data. (Dong and Qi Chen, 2017)</p><p dir="ltr">LiDAR systems are dependable because they&rsquo;re capable of measuring an environment with high accuracy and producing a 3D image based on the results.</p><h3 dir="ltr">How does LiDAR work?</h3><p dir="ltr">The lasers used to detect objects in a LiDAR can either be discrete or continuous.</p><p dir="ltr">LiDAR sensors that use continuous waves use the phase difference of the return signal to determine the distance and characteristic of the object. For pulsed waves, we&rsquo;re more interested in the amplitude of the transmitted and received signals to create a point of cloud that reflects the detected object. (Beland, 2019)</p><p dir="ltr">In measurement, full-waveform signals are often used in forestry, while pulsed signals have more use in other fields.</p><p dir="ltr">A LiDAR system mainly consists of a light source and a receiver or sensor. The source beams out light or laser, which bounces off the target and back towards the LiDAR system, where the sensor catches the pulse.</p><p dir="ltr">To determine the precise distance of an object, the LiDAR system calculates the time interval between the emittance of the laser and when it&rsquo;s received, taking the speed of light into account.</p><p dir="ltr">On its own, LiDAR can catch the position, size, and shape of an object relative to the LiDAR system. However, LiDAR sensors are often paired with a GPS, an IMU sensor, or a camera to enhance their capabilities.</p><h2 dir="ltr">What is RADAR?</h2><p dir="ltr">RADAR stands for Radio Detection and Ranging. Just as its name suggests, RADAR&rsquo;s working principle is almost identical to LiDAR, except it uses radio waves instead of lasers or light.</p><p dir="ltr">Radio waves have a much longer wavelength compared to light waves, making radars able to cover longer distances compared to LiDAR devices. The frequency and type of radio waves used depend on the requirement of your measurement device.</p><table style="width: 100%;"><tbody><tr><td style="width: 33.3333%; text-align: center;"><strong>RADAR Band</strong></td><td style="width: 33.2487%; text-align: center;"><strong>Frequency (GHz)</strong></td><td style="width: 33.3333%; text-align: center;"><strong>Wavelength (cm)</strong></td></tr><tr><td style="width: 33.3333%; text-align: center;">Millimeter</td><td style="width: 33.2487%; text-align: center;">40-100</td><td style="width: 33.3333%; text-align: center;">0.75-0.30</td></tr><tr><td style="width: 33.3333%; text-align: center;">Ka</td><td style="width: 33.2487%; text-align: center;">26.5-40</td><td style="width: 33.3333%; text-align: center;">1.1-0.75</td></tr><tr><td style="width: 33.3333%; text-align: center;">K</td><td style="width: 33.2487%; text-align: center;">18-26.5</td><td style="width: 33.3333%; text-align: center;">1.7-1.1</td></tr><tr><td style="width: 33.3333%; text-align: center;">Ku</td><td style="width: 33.2487%; text-align: center;">12.5-18</td><td style="width: 33.3333%; text-align: center;">2.4-1.7</td></tr><tr><td style="width: 33.3333%; text-align: center;">X</td><td style="width: 33.2487%; text-align: center;">8-12.5</td><td style="width: 33.3333%; text-align: center;">3.75-2.4</td></tr><tr><td style="width: 33.3333%; text-align: center;">C</td><td style="width: 33.2487%; text-align: center;">4-8</td><td style="width: 33.3333%; text-align: center;">7.5-3.75</td></tr><tr><td style="width: 33.3333%; text-align: center;">S</td><td style="width: 33.2487%; text-align: center;">2-4</td><td style="width: 33.3333%; text-align: center;">15-7.5</td></tr><tr><td style="width: 33.3333%; text-align: center;">L</td><td style="width: 33.2487%; text-align: center;">1-2</td><td style="width: 33.3333%; text-align: center;">30-15</td></tr><tr><td style="width: 33.3333%; text-align: center;">UHF</td><td style="width: 33.2487%; text-align: center;">0.3-1</td><td style="width: 33.3333%; text-align: center;">100-30</td></tr></tbody></table><p dir="ltr"><em>RADAR frequency bands (Parker, 2017)</em></p><p dir="ltr">The discovery of RADAR preceded LiDAR, with the first known application of RADAR dating back to the early 1900s. RADAR systems are mainly used for tracking, detection, and 2D imaging in various industries, such as <a href="https://link.springer.com/book/10.1007%2F978-3-030-05093-1" target="_blank">environmental studies</a>,&nbsp;military defenses, and the development of smart buildings.</p><h3 dir="ltr">How does RADAR work?</h3><p dir="ltr">Similar to LiDAR, RADAR systems have two main components: a transmitter and a receiver. Here&rsquo;s a schematic of a modern RADAR system. (Richards, 2014)</p><p><br></p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 589px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE4ODIxODAxNjE1LVJhZGFyX3NjaGVtYXRpYyAoMSkucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 589px;" class="fr-fic fr-dib" alt="Schematic of a RADAR system"><span class="fr-inner" spellcheck="false">Schematic of a RADAR system. Reproduced with permission from Richards, 2014&nbsp;</span></span></span></div><p dir="ltr">The radio waves used by a radar system can also be continuous or pulsed, which are the more frequent option.</p><p dir="ltr">The transmitter and antenna in radar work together. The transmitter generates an electromagnetic wave, and the antenna sends out the wave through a medium.&nbsp;</p><p dir="ltr">Then, the signal travels through the medium and is reflected by the target back to the RADAR system, which accepts the reflected radio wave through a receiver. The RADAR processes the radio wave and determines the distance relative to the system by using the time interval between the transmission of the signal and the time when the reflected signal is received.</p><p dir="ltr">A RADAR system uses an antenna to transmit radio signals.</p><p dir="ltr">Depending on the application of the RADAR, the frequency of the signal transmitted will vary (Parker, 2017). The frequency of your RADAR decides the limitations and capabilities of your RADAR system, such as range, wavelength, and antenna size.</p><p dir="ltr">Systems with higher frequencies have lower power, higher attenuation, and more granular detection, which makes them perfect for short-range applications that need a higher resolution, such as in autonomous vehicles.</p><h2 dir="ltr">LiDAR vs RADAR: Which technology is better?</h2><p dir="ltr">While LiDAR and RADAR have identical goals and principles, the type of wave used by both systems created a whole new level of difference between them.</p><p dir="ltr">Rather than which one is better, LiDAR and RADAR systems have their own place in your system with their advantages and disadvantages.</p><h3 dir="ltr">Accuracy</h3><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 636px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE4ODE0NjgyNTk0LWxpZGFyX3ZzX3JhZGFyLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 636px;" class="fr-fic fr-dib" alt="LiDAR vs RADAR accuracy comparison"><span class="fr-inner" spellcheck="false">A comparison of LiDAR&#39;s and RADAR&#39;s accuracy</span></span></span><p class="fr-img-space-wrap2">As you can see from the image above, images formed through a LiDAR system are crisper and much more detailed than ones formed through RADAR.</p></div><p dir="ltr">LiDAR uses light signals that work in the nanometer range, making LiDAR systems much more accurate and precise than RADAR systems. A lower wavelength also means that a LiDAR system can detect smaller objects, the size of targets, and create 3D images of the target. (Ryde and Hillier, 2009)</p><p dir="ltr">On the other hand, the resolution of RADAR images is limited by the size of the antenna used. Antennas with a bigger size produce waves with higher frequencies, resulting in images with higher resolution. Keep in mind that the requirements for your antenna also depend on the limitations of your measurement system.</p><p dir="ltr">In short, LiDAR is your best choice when you need to create a detailed image, while RADAR is better when constantly detecting the distance to the target is more important for your use case.</p><h3 dir="ltr">Reliability</h3><p dir="ltr">While RADAR&rsquo;s accuracy leaves much to be desired, it&rsquo;s much more reliable than LiDAR.</p><p dir="ltr">LiDAR uses light waves as a medium and is easily affected by the medium itself. For example, moisture in the atmosphere affects the performance of a LiDAR system. LiDAR systems don&rsquo;t perform as well in bad weather, such as in the rain, fog, or snowstorm.</p><p dir="ltr">Furthermore, most LiDAR used in autonomous vehicles uses a rotating device to shoot laser pulses, which means regular maintenance is necessary to keep it working well. However, recently there are more and more options for solid-state LiDAR systems, so this might not be a concern for much longer.</p><p dir="ltr">RADAR, on the other hand, uses radio waves that have a much bigger wavelength, so it has enough power to cover longer distances than LiDAR.</p><p dir="ltr">Radio waves also have lower attenuation, which allows them to travel minimally disturbed even in poor weather. When sensors that rely on the principle of optics are compromised, RADAR is an excellent choice as a substitute, even with its low resolution.</p><h3 dir="ltr">Cost</h3><p dir="ltr">For many years, LiDAR sensors are not an option for most manufacturers thanks to its high cost.</p><p dir="ltr">A high-end automotive LiDAR by Velodyne used to cost $75,000 for one car.</p><p dir="ltr">However, in recent years, LiDAR has been undergoing a significant price reduction with many companies working to make LiDAR systems more affordable.</p><p dir="ltr">In early 2020, Velodyne released <a href="https://velodynelidar.com/press-release/velodyne-lidar-introduces-velabit/" target="_blank">a solid-state LiDAR with no moving parts for $100</a> called Velabit on CES 2020.</p><p dir="ltr">RADAR systems, on the other hand, have always been a cheaper option compared to LiDAR, with a price as low as $50 for an automotive millimeter-wave RADAR sensor module.</p><p dir="ltr">The price of LiDAR sensors is partially responsible for the high prices of autonomous vehicles. Most include the use of LiDAR to detect the environment for their self-driving functionality, such as Waymo, Toyota, and Uber, while Tesla continues to avoid LiDAR and <a href="https://electrek.co/2021/01/13/tesla-millimeter-wave-radar-electric-cars/" target="_blank">develop RADAR</a> for its self-driving cars.</p><h2 dir="ltr">Conclusion</h2><p dir="ltr">Except for the type of wave used, LiDAR and RADAR are almost identical in principle. Both use a transmitter to emit a wave and use the reflected wave received by the sensor to decide the precise distance to the target.</p><p dir="ltr">To decide which one is better for your project, you need to decide the requirements and limitations of your measurement unit.</p><p dir="ltr">LiDAR uses lasers with a much lower wavelength than the radio waves used by RADAR. Thanks to this, LiDAR has better accuracy and precision, which allows it to detect smaller objects, in more detail, and create 3D images based on the high-resolution image it creates.</p><p dir="ltr">On the other hand, RADAR is much more robust than LiDAR with a lower starting price. While you don&rsquo;t get as many details, it works in worse operating conditions and has a wider range than LiDAR.</p><p dir="ltr">Alternatively, you can combine RADAR and LiDAR with other sensors using data fusion algorithms to create a system with better capabilities.</p><h2 dir="ltr">Takeaways&nbsp;</h2><p dir="ltr">So, here&rsquo;s what you need to remember about LiDAR and RADAR.</p><ol><li dir="ltr">LiDAR and RADAR are remote sensing devices that use light waves and radio waves respectively to detect objects. They use the interval between the time the waves are transmitted and the time the wave reflected from the object is received.</li><li dir="ltr">Light waves have better precision and accuracy compared to RADAR. However, RADAR sensors are more robust. It performs well in bad weather when LiDAR often fails and has a longer range.</li><li dir="ltr">RADAR systems are more affordable in comparison to LiDAR sensors. However, this might change as many companies continue to develop solid-state LiDARS at a lower price range.</li></ol><hr><p><br></p><h2 dir="ltr">References&nbsp;</h2><p dir="ltr">Dong, Pinliang, and Qi Chen. &ldquo;LiDAR Remote Sensing and Applications.&rdquo; 2017, doi:10.4324/9781351233354.</p><p dir="ltr">Beland, Martin, et al. &ldquo;On Promoting the Use of Lidar Systems in Forest Ecosystem Research.&rdquo;&nbsp;Forest Ecology and Management, vol. 450, 2019, p. 117484., doi:10.1016/j.foreco.2019.117484.</p><p dir="ltr">Richards, Mark A. Principles of Modern Radar Volume 1: Basic Principles. Scitech Publishing Inc,, 2010.</p><p dir="ltr">Parker, Michael Alan.&nbsp;Digital Signal Processing 101: Everything You Need to Know to Get Started. Newnes an Imprint of Elsevier, 2017.</p><p dir="ltr">Ryde, Julian, and Nick Hillier. &ldquo;Performance of Laser and Radar Ranging Devices in Adverse Environmental Conditions.&rdquo;&nbsp;Journal of Field Robotics, vol. 26, no. 9, 2009, pp. 712&ndash;727., doi:10.1002/rob.20310.</p><p><br></p>

Find out the difference between two of the most used remote sensing devices: LiDAR and RADAR.

LiDAR vs RADAR: Detection, Tracking, and Imaging

<h1>Introduction</h1><p>The automotive industry is going through a rapid transformation driven by electrification, autonomous vehicles &amp; driver support systems, and changes in mobility patterns and city infrastructure. These trends will influence the physical design and capability of commercial, public, and individual vehicles and the broader social and economic impacts of changes to car/fleet ownership and use. The increased need for sensors in vehicles, in the electric/hybrid drivetrains and advanced driver-assistance systems (ADAS) up to autonomous driving systems, creates new significant opportunities for the integrated photonics industry.</p><p>Combustion engine vehicles are facing increasing emission legislation demands. This is leading to accelerated innovation in electric vehicles (EV) and a <a href="https://www.eenewsautomotive.com/news/number-automotive-ecus-continues-rise" target="_blank">very strong increase of electronics</a> in vehicles in two key areas: Power electronics and sensors in EV and Sensors and computing/networking for ADAS in autonomous vehicles.</p><p dir="ltr">Integrated photonics offers <a href="https://www.wevolver.com/article/key-application-areas-for-integrated-photonics" rel="noopener noreferrer" target="_blank">promising solutions</a> in both areas. Integrated photonics is the rapidly <a href="https://www.wevolver.com/article/an-introduction-to-integrated-photonics" rel="noopener noreferrer" target="_blank">emerging field of photonics</a> that works to integrate multiple photonic functions into integrated circuits for communication, sensing, and, going forward, computing applications. Complex photonic circuits can now generate, process, and detect light, in analogy to how electronic integrated circuits do for electronic signals by way of connecting multiple discrete single-function components (e.g. dedicated lasers, photodetectors) and relevant optical interfaces.</p><p dir="ltr">This article will give an overview of the applications in automotive that hold the most potential for Integrated Photonic potential. Additionally, it will introduce the PhotonDelta Integrated Photonic Automotive roadmap.&nbsp;</p><h1 dir="ltr">Automotive applications with potential Integrated photonic content&nbsp;</h1><p>As the amount of electronic content in cars continues to increase the potential for PIC also rises rapidly. &nbsp;There is a wide range of applications within the future motor vehicle suitable for PIC.</p><p><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE3MTc4NTM3MTkxLUlNQUdFIDgtMDEuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"></p><h1>FMCW lidar and Fiber Bragg Grating Sensors</h1><p>Two areas with the most potential for PIC are FMCW Lidar and Fiber Bragg Grating interrogators.</p><h2>FMCW (Frequency-modulated coherent wave) lidar</h2><p>Lidar is a complementary sensor technology to radar and cameras for ADAS and autonomous vehicles. It works by actively scanning its environment with a laser beam and detecting the return signal to measure distance and speed (for some versions). Different operating principles and many different technical solutions exist, and the technology is developing rapidly under the increased investment from major automotive manufacturers.</p><p>FMCW lidar is gaining traction because it can offer higher sensitivity and measure the speed at the same time. This principle is very well suited for using Integrated Photonics. Future electric vehicle designs are expected to host multiple Lidar, particularly vehicles with elevated autonomous driving functions.&nbsp;</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 788px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE3MDEzMjU5ODQ5LUlNQUdFIDItMDEuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Possible sensor set up for autonomous vehicles.&nbsp;</span></span></span><h3 class="fr-img-space-wrap2">FMCW lidar using integrated photonics</h3></div><p>The challenge to bringing an FMCW solution to market has been the lack of low-cost, high-volume manufacturing of the required high-performance components. Affordable FMCW lidar solutions require all-optical functionalities to be integrated into a single silicon chip. PIC offers the level of manufacturing control and accuracy needed for integrated circuits to achieve superior performance and the functionality needed to enable a robust, cost-effective, and high-quality product.</p><p>A low-noise, low-loss, and polarization-independent technology will enable very high signal-to-noise ratios (SNRs) so even lower optical powers can be used. Thus, even when humans are close to the sensor, it will remain eye-safe for the detection of distant objects. Measuring objects a few centimeters in dimension &gt;200 m away should be within the capability of integrated circuit-based FMCW lidar.</p><p>Improvements in lidar bring deliver better driver assist systems and will ultimately be essential to widespread autonomous vehicle design. However, FMCW lidar also has massive potential for use in other types of ‘vehicles’ such as autonomous robots for use in manufacturing, healthcare, and infrastructure monitoring.&nbsp;</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 788px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE3MDEyOTE5MTI3LWNsZWFycGF0aC5qcGciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">Lidar will be employed across robotics applications. Image: Clearpath</span></span></span><h3 class="fr-img-space-wrap2">Fiber Bragg Grating sensors using integrated photonics</h3></div><p>Fiber Bragg Grating (FBG) sensors work on the principle that light is reflected or absorbed in a fiber where on specific positions a frequency tuned grating is made. This is dependent on force, pressure, temperature, and strain, all of which can be simultaneously measured with extreme accuracy.</p><p>Most fiber optic sensor systems today make use of FBG technology. FBG-based sensing offers advantages, such as small size, fast response, distributed sensing, and immunity to the electromagnetic field. All attributes relevant to application in automotive. These systems can be miniaturized using integrated photonics into very compact interrogator systems that have multiple applications for electric vehicles.</p><p>This miniaturization isn’t just a ‘nice to have' but rather essential for the continued innovation of vehicles to occur. As described above the rapid electrification of all forms of transport means also means more ‘stuff’ is needed inside the vehicles. &nbsp;Wiring, sensors, and processors increase contributing to the vehicle’s overall weight and in turn a reduction in its efficiency. Integrated photonics offers a huge reduction in the overall size of systems. Combined with other communication technologies such as wireless and single pair ethernet, vehicles will get lighter and more efficient.</p><div class="fr-img-space-wrap"><span class="fr-img-caption fr-fic fr-dib" style="width: 788px;"><span class="fr-img-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE3MDEzMjcyODExLUlNQUdFIDMtMDEuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"><span class="fr-inner" spellcheck="false">The sensor market is rapidly expanding&nbsp;</span></span></span><h2 class="fr-img-space-wrap2">The future of vehicles</h2></div><p>We are at a precipice of change within the automotive industry. Multiple technologies, including integrated photonics, are rapidly coming to maturity that has the potential to radically change the way we understand vehicles. At this moment almost all imagined scenarios could become reality. It will be a complex combination of market demand and adoption, resource management, and imagination that will ultimately decide the outcome.</p><h2>Integrated Photonic Automotive Roadmap</h2><p>Dutch growth Accelerator PhotonDelta has created the Integrated Photonic Automotive Roadmap, which provides a clear overview of the automotive industry’s current trends and market conditions and identifies the key opportunity areas for integrated photonics. The roadmap details which technology developments need to occur to support PIC applications and links this with technology roadmaps. Finally, the roadmap proposes an approach for the industry to ensure new applications are not delayed due to roadblocks caused by industry or manufacturing capabilities. <a href="http://eepurl.com/huN8Fj" rel="noopener noreferrer" target="_blank">Sign up here</a> to receive a notification of the roadmap's official release.&nbsp;</p><h1>Summary</h1><p>Integrated photonics opens a wide range of innovation opportunities in automotive. As the electronic components of cars increase and the move to full electrification continue, PIC offers low-power and low-cost solutions that directly contribute to vehicle efficiency. PIC will follow a similar path of decreasing cost as mass manufacturing expands. Future automotive solutions are likely to heavily rely on the advantages of PIC.</p><hr><h2><em>About the sponsor: PhotonDelta</em></h2><p><a href="https://www.wevolver.com/article/about%3Ablank" target="_blank"><em>PhotonDelta</em></a><em>&nbsp;is a growth accelerator for the Dutch integrated photonics industry. Its objective is to promote the adoption of integrated photonics and support companies to get prioritized access to the Dutch Integrated Photonics supply chain. Furthermore, PhotonDelta runs a new business creation program that provides companies, ranging from startups to SMEs and big organizations, with access to strategy, market, and funds.</em></p><p><em>Through its wide network within the industry, PhotonDelta has gathered insights on market trends, challenges, and product roadmaps. This information is used to drive business creation, investments, and strategic technology roadmaps. Furthermore, PhotonDelta actively supports start-ups and SMEs to facilitate their involvement with the Dutch integrated photonic supply chain. This includes facilitating options for establishing a solid presence in the Netherlands.</em></p><p><em><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE3MTAxMzgwMDU0LXBob3RvbiBkZWx0YSBhcnRpY2xlIGJvdHRvbSBiYW5uZXIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 788px;" class="fr-fic fr-dib"></em><br></p><p><br></p>

Integrated photonics offers promising solutions in a range of automotive applications.

The Road Ahead for Integrated Photonics in Automotive

<p>LiDARs have grown in popularity in recent years as better manufacturing techniques and mass production have lowered the price of this technology.</p><p>But since no single LiDAR solution is best for all applications, choosing the right LDAR can be tricky, a &gt;$50k LiDAR is great for autonomous vehicles but it may not be able to properly detect an obstacle less than a meter away, a &lt;$1k LiDAR can do that job effectively but may not be able to look very far.&nbsp;</p><p>If you are considering using a LiDAR for any of your projects keep reading to find out how to choose the right LiDAR for your application.</p><h2>What is a LiDAR used for?</h2><p>LiDARs have been used in surveying, forestry, geology, mining, architecture, and mapping for many years, but they are now becoming a popular component of autonomous vehicle and robot applications.</p><p>In architecture, for example, you can get a handheld LiDAR device that you carry while walking around a building. The device creates a 3D point cloud, so you can match the scan of the building to the CAD file of the construction design to see if the real building measures up and also to assist in the as-built process and survey.</p><p>In mining, you can use LiDAR for safety and surveillance. Instead of putting humans in danger, you can send a LiDAR-equipped robot down mines to detect anomalies like cracks in the walls or other physical objects that may be a hazard. Although you could do this with cameras as well, cameras aren’t great at picking up depth, even if you can see a crack in the wall, a camera can't tell you how deep it is. Cameras are also heavily influenced by lighting conditions, which may not be ideal in some environments.&nbsp;</p><h2>Long range vs. short &amp; close range</h2><p>First, you have to look at the LiDAR range. As a rule of thumb, long-ranging LiDARs with a range of 100m or more are more expensive.</p><p>For example, long range LiDARs work work well for navigating vehicles that are moving quickly., but they usually only detect objects at mid- to long-range; they don’t work at close range, such as for detecting objects within 1 m.</p><p>If your application involves a fast-moving vehicle, long-range LiDARs can help , because you have to detect objects from far away and take necessary actions before the objects come closer.</p><p>Indoors or with slower-moving vehicles, you typically don’t need long-range. &nbsp;If you’re operating a slow moving robot you typically need to see up to 25m, you can work with a short-range LiDAR.</p><p>Close-range LiDARs are used to detect objects in close vicinity. For example you may need a close range LiDAR on a vehicle to detect obstacles while opening the doors. Usually you would combine a long or short range LiDAR with a close range one.</p><h2>2D vs. 3D</h2><p>2D LiDARs will detect an object. 3D LiDARs will produce a more detailed point cloud which can be used to determine &nbsp;the shape and depth of the object, and determine what it is, rather than just telling the robot or vehicle “there’s something there.” 3D gives you better understanding of the surroundings, so your robot or vehicle can distinguish between a tree, a sign on the highway, or an oncoming truck.</p><p>The difference is really in the information that can be extracted from them. For instance, using them in a building, 2D LiDARs can only give you a floor plan whereas, with 3D LiDAR, you can get the 3D map and location and height of every object in the building. So which one to use depends on the application.</p><p>If you are navigating in a flat environment and your robot is not tall, perhaps a 2D LiDAR is enough. However, for 3D outdoor environments where there are obstacles like hanging tree branches or pipes, you need 3D perception. 3D LiDARs are also useful in other applications like terrain classification and segmentation.</p><p>More considerations for different applications</p><ul><li><strong>Field of View (FoV):</strong> Field of View is a critical feature of any LiDAR. It is measured separately along the horizontal and vertical axis. An autonomous vehicle is usually equipped with one or more LiDARs with 360° horizontal FOV, or a larger number of directional LiDARs to ensure the 360° degree view is available. It may not need a larege vertical FoV, however, as it may only need to look at the road and perhaps 30 to 40 degrees above the road surface. <br><br>If your application is obstacle avoidance within a constrained space you may not need a 360 degree LiDAR. The 360 degree surround capability is not an asset in all cases. If you only need to look straight ahead a 360 degree LiDAR would give you extra and non-essential data and hence put unnecessary load on system processing.<br><br></li><li><strong>Number of channels:</strong> A LiDAR with more channels will produce a denser, more detailed point cloud. The amount you choose to get may boil down to your budget, processing capabilities, size of the objects, distance from the objects.<br><br></li><li><strong>Vertical separation between adjacent channels:</strong> If you are trying to detect small objects you need to make sure that the separation is as small as possible, otherwise there is a chance that the object may fall within the two adjacent channels and go undetected.<br><br>This can, however, be mitigated by moving the LiDAR so that the adjacent lasers fall on different points and are thus able to detect the object. Some LIDARs have smaller separation near the center of the vertical FOV and bigger separation away from the center. This helps detect objects near the focus of the LiDAR.<br><br></li><li><strong>Detecting Low Reflectivity Objects:</strong> The range and accuracy of LiDARs varies considerably depending on the object’s reflectivity or how much light is reflected back from the object. Known as Remission, it is expressed in percentage. A white dry wall would have close to 90% remission while something like coal would have around 5% remission.<br><br>A LiDAR’s range is typically expressed at 80% remission, it can go down to 1/4th or even less at 10% remission. If your project involves looking at or detecting darker objects or objects that do not reflect a lot of light you may want to ask the LiDAR manufacturer to provide you with the range and accuracy at different remissions… unless of course you are looking at a white drywall all the time!<br><br></li><li><strong>Points Per Second:&nbsp;</strong>PPS or Points Per Second is another good spec to look at. It shows how many laser measurements are being taken per second. More points will result in a more detailed, denser and accurate point cloud.<br><br>PPS is primarily the function of the frequency of the LiDAR in Hz, number of channels and number of returns. A LiDAR operating in dual return mode would provide double the points per second. Similarly a LiDAR operating at a higher frequency would yield higher PPS.<br><br></li><li><strong>Safety-certified LiDAR:&nbsp;</strong>If your robot or autonomous guided vehicle needs to operate amongst humans, you might want something which is safety-rated. There are some companies that offer safety-rated 2D LiDARs but this option is very limited at this point in time. The safety rated LiDAR would be just one component of the overall system and other considerations would need to be made to make the entire system safety certified.<br><br></li><li><strong>LiDAR or a Complete Solution?&nbsp;</strong>You might need a complete mapping solution, not just LiDAR. Your technical resources will determine how easy it is to put an extensive ecosystem into place. The LiDAR can provide you with a point cloud but to make sense out of it you may need localization, time stamping and other valuable data.&nbsp;you may have to consider complete solutions that consist of LiDARs, IMUs, GPS, Cameras and software.&nbsp;You may also need to do some kind of post processing to use the data in a meaningful way.</li></ul>

LiDARs have grown in popularity in recent years as better manufacturing techniques and mass production have lowered the price of this technology.

How to Choose the Right LiDAR Sensor for Your Project

<p>Article written by <a href="https://europe.naverlabs.com/people_user/martin-humenberger/" rel="noopener noreferrer" target="_blank">Martin Humenberger</a></p><hr><p><em>Visual localization</em>, which estimates the position and orientation of a camera in a map based on query images, and <em>structure from motion</em> (<em>SFM</em>), which is one of the most popular ways to build this visual localization map, are fundamental components in the development of technologies such as autonomous robots and self-driving vehicles.</p><p>However, a major barrier to research in visual localization and SfM lies in the format of the data itself. Although many good public datasets for evaluation now exist, these datasets are all structured using different formats. As a result, data importers and exporters must often be modified and coordinate systems or camera parameters almost always must be transformed. Even more data conversion is necessary if you want to use a combination of multiple tools in a single pipeline. Many times, existing data formats don’t include the types of data needed for a specific application, especially when it comes to data such as WiFi or other non-image sensors.</p><p>To overcome this research barrier, we created the kapture format. Kapture includes a wide range of data to store and comes with several format converters and data processing tools. On the one hand, it will make public datasets easier to use and on the other hand, it will allow processed data (such as local or global features or matches) to be shared easily (again in one common format). By sharing our kapture format with the research community, our aim is to facilitate future research and development in topics such as visual localization, SfM, VSLAM and <a href="https://www.wevolver.com/article/sensor-fusion-everything-you-need-to-know-" rel="noopener noreferrer" target="_blank">sensor fusion</a>.</p><p>The release contains the following main features:</p><ul><li><strong>Data converters:&nbsp;</strong>To convert other datasets from and to kapture, we provide a set of converters for popular formats (e.g., COLMAP, OpenMVG, OpenSfM, bundler, nvm, and more).</li><li><strong>Example pipelines:&nbsp;</strong>As an example, we provide two full visual localization pipelines based on COLMAP. The first uses COLMAP SIFT features and COLMAP vocabulary tree matching, and the second uses our own custom features and matches, which obtained <a href="https://europe.naverlabs.com/blog/one-method-one-pipeline-naver-labs-europe-ranks-high-across-three-visual-localization-challenges-at-cvpr-2020/" target="_blank">state-of-the-art results</a> in the <a href="https://sites.google.com/view/vislocslamcvpr2020/home" target="_blank">VisLocOdomMapCVPR2020</a></li><li><strong>Converted datasets:&nbsp;</strong>To help kapture users get started, we provide several <a href="https://github.com/naver/kapture/blob/master/doc/datasets.adoc" target="_blank">ready-to-use datasets</a>. These datasets are used for the localization challenge sponsored by NAVER LABS and held at the <a href="https://sites.google.com/view/ltvl2020/challenges" target="_blank">Workshop on Long-Term Visual Localization under Changing Conditions</a> at <a href="https://eccv2020.eu/" target="_blank">ECCV 2020</a>.</li></ul><h3><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1596641259021-ezgif-7-7e871522cdf7.gif" style="width: 788px;" class="fr-fic fr-dib"></h3><h2>Details of the format</h2><p>Given a known 3D space representation, <em>visual localization</em> is the problem of estimating the position and orientation of a camera using query images. Structure from Motion (<em>SfM)</em> is one of the most popular ways to reconstruct a 3D scene from an unordered set of images.</p><p>Kapture<strong>&nbsp;</strong>can be used to store many types of data collected for visual localization and SfM. Examples include:</p><ul><li>sensor parameters such as intrinsic and extrinsic camera parameters</li><li>raw sensor data such as camera images or LiDAR data</li><li>other sensor data such as GPS or WiFi signals</li></ul><p>It can also store data computed during various stages of the process, such as</p><ul><li>2D local features (keypoints and descriptors)</li><li>2D–2D matches between local features</li><li>global features (e.g. for image retrieval)</li><li>3D reconstructions consisting of 3D points and keypoint observations</li></ul><h2>Data converters</h2><p>Kapture also includes a set of Python tools to load, save, and convert datasets to and from kapture. General formats such as COLMAP, openmvg, OpenSfM, bundler, image_folder, image_list, and nvm, as well as a few formats specific to particular datasets such as IDL_dataset_cvpr17, RobotCar_Seasons, ROSbag cameras+trajectory, SILDa, and virtual_gallery are all supported.</p><h2>Example pipelines</h2><p>Our <a href="https://github.com/naver/kapture/blob/master/doc/tutorial.adoc" target="_blank">example pipelines</a> take you through the process of localizing query images on a map, which consists of two major parts: i) building the map and ii) localizing a query. In each pipeline, we first show you how to build the map using SfM and known poses, and then we show you how to localize query images. We also explain how to use kapture to evaluate the precision of the obtained localization against the ground truth.</p><ul><li><strong>Standard COLMAP pipeline:&nbsp;</strong>In the first example, we use <a href="https://colmap.github.io/" target="_blank">COLMAP</a>, which is a general-purpose SfM and multi-view stereo pipeline with both a graphical and command-line interface. We use a downloaded vocabulary tree that uses SIFT local features and vocabulary tree matching to build the map and perform the localization in COLMAP. Then, we import the results into kapture to evaluate them.</li><li><strong>Custom features and matches:&nbsp;</strong>Using our custom <a href="http://openaccess.thecvf.com/content_ICCV_2019/html/Revaud_Learning_With_Average_Precision_Training_Image_Retrieval_With_a_Listwise_ICCV_2019_paper.html" target="_blank">R2D2&nbsp;</a>and <a href="http://openaccess.thecvf.com/content_ICCV_2019/html/Revaud_Learning_With_Average_Precision_Training_Image_Retrieval_With_a_Listwise_ICCV_2019_paper.html" target="_blank">AP-GeM</a> features pre-extracted from the sample database, we use kapture scripts to determine potential image pairs through image retrieval, compute the 2D–2D matches, perform the query matching, and evaluate the results. To build the initial map and visualize the results, we use COLMAP again to take advantage of its graphical interface. In this example, data can be easily ported between kapture and COLMAP at different stages thanks to kapture’s conversion tools.</li></ul><h2>Datasets</h2><p>We’ve already converted a number of datasets to the kapture format. For instance, kapture can be used to process all the datasets included in ECCV 2020’s <a href="https://www.visuallocalization.net/workshop/eccv/2020/" target="_blank">Visual Localization Challenge</a> (the Aachen Day-Night, Inloc, RobotCar Seasons, Extended CMU-Seasons, and SILDa Weather and Time of Day datasets). The images in these datasets are meant to include the many challenging scenarios that arise in real driving situations and include changes in the time of day, season of the year, and outdated reference representations. The datasets also include images with occlusion, motion blur, extreme viewpoint changes, and low-texture areas.</p><p>If you already have your SfM or visual localization processing tools up and running, you just need to integrate kapture support once, after which you can use all the datasets without any additional conversion or glue code writing. Additional details and how to obtain an updated list of datasets can be found in the <a href="https://github.com/naver/kapture/blob/master/doc/tutorial.adoc" target="_blank">kapture tutorial</a>.</p><h2>Contribute to kapture</h2><p>If you find kapture useful, we encourage contributions!</p><p>You are welcome to provide your own dataset in kapture format (we’re happy to help), write new data converters, report bugs and suggest improvements, provide processed data (e.g., extracted features or matches) in kapture format, and add support for additional data types.</p><hr><p>This article was first published on the blog of <a href="https://europe.naverlabs.com/blog/" rel="nofollow" target="_blank">NAVER LABS Europe</a>.</p><hr><h3>Further Resources</h3><p>More information, news, and updates can be found <a href="https://europe.naverlabs.com/research/3d-vision/kapture/" rel="noopener noreferrer" target="_blank">on this website.</a></p><p><strong>GitHub Repository:</strong> <a href="https://github.com/naver/kapture" target="_blank">https://github.com/naver/kapture</a></p><p><strong>License:&nbsp;</strong><a href="https://github.com/naver/kapture/blob/master/LICENSE" target="_blank">BSD 3-Clause</a></p><p><strong>Paper:</strong> M Humenberger, Y Cabon, N Guerin, J Morat, J Revaud, P Rerole, N Pion, C de Souza, V Leroy, and G Csurka: <a href="https://europe.naverlabs.com/research/publications/robust-image-retrieval-based-visual-localization-using-kapture/" target="_blank"><strong>Robust Image Retrieval-based Visual Localization using kapture</strong></a></p>