The guide to understanding the state of the art in hardware & software for self-driving vehicles

About The Contributors

About Nexperia

About Wevolver

References

Introduction

Sensing

Thinking & Learning

Acting

User Experience

Communication & Connectivity

Use Case: Autonomous Racing

Summary

2020 Autonomous Vehicle Technology Report

Automotive semiconductors are getting smaller and lighter. Image: Nexperia

Reliable, small, and lightweight hardware that meets increasing performance requirements is today’s engineering challenge in the automotive industry.

Putting high-performance computing into cars: automotive discrete semiconductors for autonomous driving

This article is part of a series about ADAS and Autonomous Vehicles, sponsored by <a href="https://www.nexperia.com/" rel="noopener noreferrer" target="_blank">Nexperia</a>. We'll start the series by unpacking the concept of ADAS and exploring its impact on the world of semiconductors. The article is written by Ahmed Faizan Sheikh and co-authored by Ian Kennedy and Wayne Lawson. &nbsp;The second article will dive into new semiconductor products that can increase reliability and performance in electrical automotive systems. Find it&nbsp;<a href="https://www.wevolver.com/article/putting.highperformance.computing.into.cars.automotive.discrete.semiconductors.for.autonomous.driving" rel="noopener noreferrer" target="_blank">here</a>.&nbsp;<hr>Advanced Driver-Assistance Systems, also known as ADAS, are electronic systems that help with monitoring, warning, braking, and steering tasks in a vehicle. These technologies exist at different levels. For example, infotainment systems, rear-view cameras, blind spot, and lane departure warnings are applications of this technology which empower the driver with crucial information but also let them exercise control over the vehicle at all times.&nbsp;Next are semi-autonomous systems (SAS), like lane-keep assistance, which enable the vehicle to control itself for a brief period but allow the driver to override the automatic control when needed.&nbsp;Lastly, highly autonomous systems are the ones that take full control of the vehicle at times, like during automatic parking.All these subsystems come together to realize automated and semi-autonomous systems. From electronic stability control systems which help drivers regain control of their vehicles as needed, adaptive headlights which adjust their direction and intensity as per the road conditions, a rearview video system which allows the driver to see the area behind the rear bumper, adaptive cruise control which determines the speed of the vehicle based on the position of the driver’s vehicle with respect to the vehicle in front, and so on.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNTczNTcwMzEyOTExLUFEQVMgb3ZlcnZpZXcgMDMuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==">Examples of ADAS systems (blue) and Sensors (black)<h1>Components of ADAS</h1>Quite predictably, ADAS has played an important role in the development of autonomous cars and will play an even critical role in the years to come. But before we delve deeper into this, let’s understand how ADAS works and what makes up an ADAS system.<h2>Sensors</h2>Sensors capture real-time actionable data from the environment that provide input for the various Advanced Driver-Assistance Systems in a vehicle. Commonly used ADAS sensors include cameras, ultrasonic, radar, and Lidar sensors. They’re often located in the rear and front bumpers of the car and side-view mirrors.&nbsp;<h2>Software &amp; Data</h2>Most automotive companies are employing advanced software to process the incoming sensor data, learn and make decisions, and actuate the hardware for a specific response. Machine learning and artificial intelligence are ruling the industry and feeding the algorithms to decide how to behave in varied situations.&nbsp;The mapping functions of ADAS are used to collect and store geographical and infrastructure data related to vehicle sensors.<h2>Processors</h2>Electronic Control Units (ECUs) and Microcontroller Units (MCUs) are the hardware components (the electronic circuitry) that the software runs on. They are often dedicated to a single or narrow range of tasks, so with increasing ADAS applications, the number of processors is likely to increase as well. However, increasingly vehicle manufacturers are changing their approach from embedded software that is running on ADAS-specific ECUs to higher-level software platforms running on centralized ECUs or MCUs.Recommended reading: <a href="https://www.wevolver.com/article/putting-high-performance-computing-into-cars-automotive-discrete-semiconductors-for-autonomous-driving" rel="noopener noreferrer" target="_blank">Putting high-performance computing into cars: automotive discrete semiconductors for autonomous driving</a>ActuatorsActuators execute the decisions made by the system. They come in many forms, depending on the ADAS system. For example, displays that provide information, or actuators that control the engine and steering for adaptive cruise control or lane-keep assistance. <img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNTczMjMwMzE4MzE3LUFEQVMgS2V5IENvbXBvbmVudHMgMDIuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==">Four control points in Advanced Driver Assitance Systems are key for autonomous driving and product differentiation. &nbsp; | &nbsp; Image courtesy McKinsey&amp;Company<h1>Centralized vs. Decentralized ADAS System Architecture</h1>As you can sense, with increasing capabilities and a larger number of sub-systems comes increasing complexity. One of the key questions on how to achieve highly effective sensing &amp; processing evolves around the concept of sensor fusion and the question of implementing a centralized versus decentralized architecture for fusion.Centralized ADAS System ArchitecturesIn a centralized architecture, all the Remote Control Units (RCUs) are directly connected to a central control unit. For instance, Lidar and ultrasonic sensors will be located in two different locations of a vehicle and will perform two different functions. These sensors will gather input and send it to a central control unit which processes the data.&nbsp;Decentralized ADAS System&nbsp; ArchitecturesA decentralized fusion system consists of a network of sensing units, each with an ability to process its raw data and communicate with the adjacent nodes.&nbsp;Which architecture is the right one?It is tough to determine as to which architecture is the right way to design a self driving car. Centralized architecture could sound like a better option as the measurements from different sensors are independent quantities and not influenced by other nodes; there are two drawbacks. There should be enough communication bandwidth to transport the raw data to the centralized node. This essentially means high-power processing, and this could prove to be costly.&nbsp;On the other hand, decentralized architecture is complex. Although the demand related to bandwidth and centralized processing is reduced greatly in such an architecture, it introduces latency between actuation and sensing phases and increases the challenges related to the sharing of misinformation.<h1>Engineering Challenges in Designing ADAS Systems&nbsp;</h1>ADAS technology development affects many parts of vehicle engineering. Vital requirements for future developments are:&nbsp;<ol><li>Increasing redundancy.</li><li>Reducing weight and size of components (such as electronics, sensors and cameras).</li><li>Assimilating exclusive functions into more comprehensive systems. Additionally, accomplishing more multifaceted ADAS functions involve not only input from more cameras and sensors but also the fusion of data from those sensor elements.&nbsp;</li></ol>RedundancyIncreasing automation and (semi-)autonomy in vehicles increases the importance of the reliability of automotive electronics both at the system- and component level. Therefore the concept of dual redundancy of key systems has been introduced to improve reliability and safety. For instance, in the event of failure of part of a vehicle’s ADAS system you still need to be able to control the steering and brakes; a so-called limp home mode. Examples of systems that need redundancy are: drive by wire, steering, and braking systems. &nbsp;To achieve hardware redundancy extra components are included, which allow the continuing operation of a system in the event of a partial failure, thus increasing the availability and reliability of the system. Take for example a traditional brushless motor with three phases; if one of these phases is lost, then the motor will fail to turn. &nbsp;However, if the motor is designed to have another 3-phase set of windings, then if one phase fails, the motor will still operate. &nbsp;The requirements for high levels of system availability are driven by standards such as ISO26262 for critical safety functions where Automotive Safety Integrity Level (ASIL) C or D is required. Modern mission-critical automotive systems now feature redundancy and high-reliability automotive power MOSFETs that adhere to AEC Q101 qualifications from the Automotive Electronics Council. &nbsp;Increased safety and reliability are great. However, dual redundancy has the potential to increase the size, weight, and complexity of electronic subsystems, which already have become bigger, heavier, and more complex with an increasing amount of ADAS systems.Semiconductor manufacturers are innovating to overcome these challenges around the weight, size, and reliability of their components. Examples of solutions are MOSFETS with two independent channels inside a compact footprint, enabling both space and weight savings for designers.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNTczNTA3Nzc5ODI5LVJTMTA3N19uZXhwZXJpYV9hYWEtMDE2MDA5LmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=">Dual LFPAK56D MOSFETS which include 2 independent channels inside a compact footprint. | Image courtesy NexperiaA future article in this series will particularly address reducing the size of electronic components by the use of automotive qualified highly reliable small size Discrete Flat No Leads (DFN) packages.InterferenceAlong the same lines, another important concern is related to interference and shortcuts. The automotive software industry must meet the strict safety requirements and follow the proper certifications to ensure that everything is going as intended. Freedom from interference is an important requirement. The electronic currents passing through a component of an electronic system should not impact a nearby component through interference.Besides adhering to standards and qualifications, something called Failure Mode and Effects Analysis (FMEA) can play an imperative role in discovering potential failures and studying their impact. FMEA can be an extremely powerful tool to lessen the error rate in an automotive system and to recognize the underlying complications in any situation of failure.<h1>Conclusion</h1>ADAS systems have tremendous potential to increase the safety, comfort, and effectiveness of our vehicles and transportation systems. By many ADAS is seen as a stepping stone to fully autonomous vehicles.&nbsp;As we’ve seen, more systems and more advanced systems, increase weight, seize, and complexity both on a systems level as well as on the overall vehicle architecture level. Semiconductor companies are responding to above mentioned challenges by innovating their components. The technological expertise of semiconductor companies is more impactful than ever in the future development of vehicle autonomy.&nbsp;<hr>This article is written by Ahmed Faizan Sheikh and co-authored by Ian Kennedy and Wayne Lawson.This article is part of a series about ADAS and Autonomous Vehicles, sponsored by <a href="https://www.nexperia.com/" rel="noopener noreferrer" target="_blank">Nexperia</a>. In the next articles we will dive more deeply into the new semiconductor technologies that reliably enable you to bring high-performance computing, 5G connectivity, and sensing into cars. The second article will dive into how new semiconductor products can increase reliability and performance in electrical automotive systems. It's called&nbsp;“<a href="https://www.wevolver.com/article/putting.highperformance.computing.into.cars.automotive.discrete.semiconductors.for.autonomous.driving" rel="noopener noreferrer" target="_blank">Putting high-performance computing into cars – automotive highly reliable miniaturized Discretes for autonomous driving</a>” and written by Jan Birger Preibisch.More articles on autonomous vehicle technology can be found&nbsp;<a href="https://www.wevolver.com/?articles=true&category=autonomous%20vehicles" rel="noopener noreferrer" target="_blank">here</a>. To read more about how the electrification of cars' powertrains impact semiconductors; read this article by Ian Kennedy. References<a href="https://www.mckinsey.com/industries/semiconductors/our-insights/advanced-driver-assistance-systems-challenges-and-opportunities-ahead" rel="noopener noreferrer" target="_blank">• &nbsp; Advanced driver-assistance systems: Challenges and opportunities ahead &nbsp; - &nbsp; Mckinsey &amp; Company</a><a href="https://www.sciencedirect.com/science/article/pii/B9780128037294000040" rel="noopener noreferrer" target="_blank">• &nbsp; Motion Planning for Intelligent Transportation Systems, Chapter 4 Advanced Driver Assistance Systems - Rahul KalaRobotics and Artificial Intelligence Laboratory, Indian Institute of Information Technology.</a><a href="https://ieeexplore.ieee.org/document/6681152" rel="noopener noreferrer" target="_blank">• &nbsp; S. Sommer et al., "RACE: A Centralized Platform Computer Based Architecture for Automotive Applications," 2013 IEEE International Electric Vehicle Conference (IEVC)</a><a href="https://www.nexperia.com/applications/automotive/adas-radar-sensor-module.html" rel="noopener noreferrer" target="_blank">• &nbsp; ADAS radar sensor module design &nbsp; - &nbsp; Nexperia</a>

Vehicles increasingly contain electronic systems that assist drivers with monitoring, warning, braking, and steering tasks. Semiconductor companies are innovating to increase safety, while reducing weight, size, and complexity.

How Advanced Driver-Assistance Systems (ADAS) Impact Automotive Semiconductors

Autonomous driving is no longer sci-fi, it&rsquo;s become a reality and soon to be hitting our streets. Every week, Autotech companies are announcing their plan for self-driving tech. But, no two autonomous driving technologies are exactly alike.<blockquote class="lk ll lm">&ldquo;What we actually mean when we say a vehicle is self-driving, fully automated, and so on?Do we mean that the vehicle can drive itself anywhere at anytime, with no-assistance inside it?Or, that the vehicle always needs someone inside ready to take over just in case?Or, that the vehicle can drive itself, as long as it is within certain constraints, such as good weather and in a defined area of operations?&rdquo;</blockquote>To bring clarity to such situations, the Society for Automotive Engineers (SAE) outlined 6 levels of automation for automakers, suppliers, and policymakers to use to classify a system&rsquo;s sophistication via their standard titled&nbsp;<a class="de bv lo lp lq lr" href="https://www.sae.org/misc/pdfs/automated_driving.pdf" rel="noopener" target="_blank">J3016</a>.<h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="173a">SAE&rsquo;s Six Levels of Autonomy</h1><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1570500234379-1_tVu-Og0xM0bL3IuVk7QP1Q.png" style="width: 766px;" class="fr-fic fr-dib">The driver assistance systems of level 1 are very common today. Some cars even offer steering and lane-keeping assistance, as well as remote-controlled parking &mdash; all systems defined as level 2 &ldquo;Partly Automated Driving (PAD)&rdquo;. A good example is Tesla&rsquo;s &ldquo;Autopilot&rdquo;, an industry-leading driver assistance system.A crucial shift occurs between Levels 3 and 4 when the driver releases responsibility for monitoring the driving environment to the system. Level 3 &ldquo;Highly Automated Driving&rdquo;, level 4 &ldquo;Fully Automated Driving&rdquo; and level 5 &ldquo;Full Automation&rdquo; are still in the testing phase.<h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="3184">Level 0: No Automation</h1>This is where the human driver controls the car completely without any support from a driver assistance system.<h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="2ffb">Level 1: Driver Assistance</h1>SAE&rsquo;s Level 1 is an automated system on the vehicle that can sometimes assist the human driver to conduct some parts of the driving task. In fact, many top-of-the-range car models offer level one automation. You&rsquo;re still the driver with full-control on driving however, you may call upon technology like adaptive cruise control for active safety.At Level 1, a computer can control either steering or acceleration/braking, but it is not programmed to do both at the same time. To sum it up, you still have full responsibility to monitor road situations and assume all driving functions if the assistance system cannot do so for any reason.<h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="c93c">Level 2: Partly Automated Driving</h1>Functions that make partial automation possible are already a reality and in practice. Semi-autonomous driving assistance systems, such as the Steering and Lane Control Assistant including Traffic Jam Assistant, make daily driving much easier. They can brake automatically, accelerate and, unlike level 1, take over steering.With the remote-controlled parking function, it&rsquo;s possible to pull into tight spots without a driver for the first time. In level 2, the driver continues to remain in control of the car and must always pay attention to traffic.Here&rsquo;s the list of some driving assistance systems,<ol><li class="kw kx cj aq ky b kz la lb lc ld le lf lg lh li lj nm nn no" data-selectable-paragraph="" id="594c">Adaptive Cruise Control automatically adapts speed to maintain a safe distance from vehicles in front.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nm nn no" data-selectable-paragraph="" id="ae0d">Autonomous Emergency Breaking detects an obstacle, warn the driver or automatically brake to avoid or mitigate a crash.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nm nn no" data-selectable-paragraph="" id="fb01">Lane Detection using a forward camera to detect lane markings on the road.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nm nn no" data-selectable-paragraph="" id="4273">Lane Keeping Assist combines a forward-facing camera to detect lane markings with an electric steering system, keeping the vehicle in the center of the lane.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nm nn no" data-selectable-paragraph="" id="7855">Parking Assistance systems are designed to help a driver park. Some perform the entire job automatically, while others simply provide advice so that the driver knows when to turn the steering wheel and when to stop.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nm nn no" data-selectable-paragraph="" id="cfa5">Parking Line Detection system that detects markers on the road surface in order to determine the exact location of parking lots.</li></ol><h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="1767">Level 3: Highly Automated Driving</h1>In the third development stage, drivers gain more freedom to completely turn their attention away from the road under certain conditions. In other words, they will be able to hand over complete control to the car. The driver, however, must be able to take over control within a few seconds, such as at&nbsp;<a class="de bv lo lp lq lr" href="https://www.wired.com/2017/02/self-driving-cars-cant-even-construction-zones/" rel="noopener" target="_blank">road construction sites</a>.<h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="ad46">Level 4: Fully Automated Driving</h1>Level 4 is considered to be fully autonomous driving, although a human driver can still request control, and the car still has a cockpit. In level 4, the car can handle the majority of driving situations independently. The technology in level 4 is developed to the point that a car can handle highly complex urban driving situations, such as the sudden appearance of construction sites, without any driver intervention.The driver, however, must remain fit to drive and capable of taking over control if needed, yet the driver would be able to sleep temporarily. If the driver ignores a warning alarm, the car has the authority to move into safe conditions, for example by pulling over. While level 4 still requires the presence of a driver, cars won&rsquo;t need drivers at all in the next, final level of autonomous driving. However, their operations will be constrained to certain situations,<ul><li class="kw kx cj aq ky b kz la lb lc ld le lf lg lh li lj nv nn no" data-selectable-paragraph="" id="7d84">Limited to a geo-fenced area, such as paved streets in a defined area of town.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nv nn no" data-selectable-paragraph="" id="5e30">Limited by adverse weather, such as falling snow, snow-packed roads, intense rain, thick fog, etc.</li><li class="kw kx cj aq ky b kz nq lb nr ld ns lf nt lh nu lj nv nn no" data-selectable-paragraph="" id="dff8">Limited to a maximum speed, e.g., vehicles limited to 35 mph may be able to travel on most streets, but not on roads posted at 40 mph and above.</li></ul><h1 class="ls lt cj aq ap cb lu lv lw lx ly lz ma mb mc md me" data-selectable-paragraph="" id="f8b9">Level 5: Full Automation (Driverless)</h1>Unlike levels 3 and 4, the &ldquo;Full Automation&rdquo; of level 5 is where true autonomous driving becomes a reality.Here drivers don&rsquo;t really even need to have a license. and everyone in the car is a passenger. Cars at this level will clearly need to meet stringent safety demands, and will only drive at relatively low speeds within populated areas. They are also able to drive on highways but initially, they will only be used in defined areas of city centres.

Autonomous driving is no longer sci-fi, it’s become a reality and soon to be hitting our streets. Every week, Autotech companies are announcing their plan for self-driving tech. But, no two autonomous driving technologies are exactly alike.

Autonomous Driving Levels 0-5 Explained

Even though it has the right of way, the connected and automated Lincoln MKZ stops at an intersection because another vehicle is running the light. Vehicle-to-vehicle communication enabled this safety feature. Photo by Michigan Photography

Mcity demonstrates the powerful pairing of connectivity and automation.

How self-driving cars can be safer with connected technology

<div class="textimage basecomponent"><div class="textimage ">The future of mobility is measured in milestones: this February, Google subsidiary Waymo announced that its fleet of self-driving cars had covered over 8 million kilometres on public roads. This came shortly after Uber&rsquo;s announcement that it had completed 3 million kilometres of autonomous driving. If industry has its way, then we will soon be sharing all our streets with vehicles controlled by algorithms instead of drivers. But is that a realistic scenario? Or simply a rose-tinted vision of a tech-driven future?We asked one of the leading experts in this field, Italian researcher Emilio Frazzoli, Professor of Dynamic Systems and Control at ETH Zurich since October 2016. &ldquo;It all depends what level of autonomous mobility you have in mind,&rdquo; he replies. &ldquo;I would say it will be at least another 15 years before you can buy a self-driving car from a dealer. But if you mean a limit- ed kind of car sharing, then it&rsquo;s already happening.&rdquo; In fact, this latter concept is a core part of Frazzoli&rsquo;s own research. In mid-2018, the ride sharing company Lyft launched a service on the Las Vegas Strip that allows anyone to book one of a fleet of 30 BMWs through its app. The driverless cars are controlled by algorithms developed by vehicle technology company Aptiv, which acquired NuTonomy &ndash; the start-up founded by Frazzoli &ndash; in October 2017.<h2>Rethinking urban mobility</h2>Before joining ETH, Frazzoli spent ten years as a professor at the prestig- ious MIT in Boston. Autonomous systems &ndash; initially aircraft and drones &ndash; were the focus of his work right from the start. &ldquo;The technical side of that was generally pretty cool, but it wasn&rsquo;t really doing much to help solve the challenges society is facing.&rdquo; In 2009, he found himself pondering a fundamental question: &ldquo;Back then, the main argument for conducting research into self-driving cars was the idea that they would make road traffic safer.&rdquo; While acknowledging the truth of that statement, at least in the long term, Frazzoli realised there was potentially a much larger, medium-term benefit to be gained from completely rethinking the issue of individual mobility for city dwellers.&ldquo;The goal of my research group is a form of mobility that combines the convenience of a private car with the sustainability of public transport.&rdquo; In other words, a kind of Uber, but driverless and therefore much more economical and available. Plus &ndash; thanks to electrification and better capacity utilisation &ndash; a solution that offers significantly lower energy consumption and CO2 emissions. Right now, people use private cars, on average, just 5 percent of the time, which means the cars spend the remaining 95 percent of the time standing idle in car parks and garages or on the street. This makes no sense in terms of sustainability, urban development or resource efficiency.Frazzoli&rsquo;s start-up NuTonomy, which develops control software for autonomous vehicles, began drawing up plans to test self-driving cars in Singapore back in 2014. At around the same time, the professor published an article in which he investigated how replacing all the private vehicles in the 719-square-kilometre city-state with shared, self-driving vehicles would affect traffic volumes. His results showed that the mobility needs of Singapore&rsquo;s entire population could be met with some 40 percent of the vehicles (350,000 instead of 800,000).One year later, Prime Minister Lee Hsien Loong unveiled his vision of a &ldquo;car-lite future&rdquo; based on autonomous vehicles, the expansion of public transport and the fostering of slow traffic such as walking and cycling. With 5.5 million inhabitants and a population density of 7,697 people per square kilometre &ndash; compared with Switzerland&rsquo;s figure of 203 &ndash; Singapore is more dependent on efficient transport than any other major metropolitan area.That is why Singapore has spent years trying to crack down on demand for private cars by imposing high taxes and charging up to 70,000 dollars for the certificates of entitlement required to own a vehicle. More than ten companies are current- ly testing their systems in a 2-hectare test facility at Nanyang Technological University in the western part of Singapore Island. And plans are already in place to operate the first self- driving buses outside rush hours in three of the city&rsquo;s suburbs starting in 2022.</div><h2>Simulating transformation</h2>Pieter Fourie&rsquo;s &ldquo;laboratory&rdquo; is located in the western part of Singapore. He works in a sunlit office on the sixth floor of the CREATE Tower, a building encased in vertical foliage at the National University of Singapore (NUS). Here, he conducts research into the cities of the future on behalf of ETH Zurich&rsquo;s Future Cities Laboratory. Fourie heads up the Engaging Mobility project, which brought together government authorities and universities at a preliminary workshop in July 2017. The goal was to define the basic conditions required to implement city-wide, on-demand mobility using autonomous cars and buses.The researchers used the results of the workshop to formulate key research questions such as: What do we do with the current supply of parking spaces if the majority of vehicles are constantly on the road? Do we need to redefine the layout of our roads? And what effect will auto- mated, electrified transport have on existing public transport, energy requirements and safety?Fourie explores these and similar issues using the MATSim simulation platform developed by a group led by Professor Kay Axhausen at ETH Zurich&rsquo;s Institute for Transport Planning and Systems. MATSim is agent-based, which means the simulation is driven by the behaviour of individual agents rather than by overarching rules. &ldquo;On the basis of Singapore&rsquo;s most recent demographics, we are modelling a synthetic population that is as close as possible to the real one,&rdquo; Fourie says.Within this population, each individual agent exhibits a certain mobility behaviour and has a specific destination based on real-life traffic data. Fourie is now at the stage of tinkering with the underlying conditions, including the number of vehicles employed, their size, the maximum permissible waiting times for passengers, the availability of parking spaces and a variety of different traffic flows. He then lets the synthetic population loose on the simulation for 24 hours. The system automatically evaluates how efficiently the individual agents were able to reach their destinations in each scenario.Right now, Fourie&rsquo;s team is programming these kinds of simulations for the waterfront area of Tanjong &shy;Pagar, a district of some 2 square kilometres in the western part of Singapore. This site is currently being converted from a container terminal into a residential and commercial area. Fourie has already simulated more than 200,000 trips involving 60,000 individual agents. This included calculating how big the fleet of autonomous vehicles would need to be and how many kilometres the vehicles would have to cover to achieve an equivalent level of service in three different street typologies.The researchers also simulated four different parking strategies for a fleet of 4-, 10- and 20-seater vehicles. Preliminary results suggest that the transport system is at its most efficient if the shared vehicles are allowed to park in the street when they stop receiving requests for pick-ups. That holds true even if it means temporarily reducing the roadway capacity by one lane. The researchers&rsquo; findings also suggest that having fewer, but correspondingly larger, pick-up and drop-off stations has a favourable impact on traffic flow by reducing the detours cars have to take to collect passengers. The stations also need to be big enough to accommodate different vehicle sizes. Fourie is hoping to have these kinds of simulations up and running for the entire island as early as next year.<h2>Decision-making dilemmas</h2>Despite these rapid developments in Singapore and the fledgling services coming online in Las Vegas, Emilio Frazzoli still sees plenty of challenges ahead, especially when it comes to dealing with chaotic environments. &ldquo;We still don&rsquo;t know exactly how autonomous vehicles should behave in traffic,&rdquo; he says, explaining that this involves dozens of decision-making dilemmas that are an integral part of everyday traffic situations. For example, should a self-driving car cross a double line in order to avoid a potential collision? And what if an innocent road user is injured as a result of a manoeuvre designed to save a culpable driver from a fatal crash? These are the kinds of decisions that have to be defined when programming control algorithms. One key focus of Frazzoli&rsquo;s current research is therefore the &ldquo;rulebooks&rdquo; that should be used to prioritise these various decision- making criteria in control algorithms. At the top of the hierarchy are rules designed to ensure road users&rsquo; safety. At the bottom are rules designed to enhance passenger comfort.In a recent article, Frazzoli and his team estimated that it would take 200 rules in 12 hierarchy groups to prepare vehicles for every possible scenario, including low-priority rules such as not frightening animals on the edge of the road. Frazzoli feels the time has come for a broader public &shy;debate on autonomous driving: &ldquo;The coding of safety and liability rules is not something we should simply leave in the hands of engineers working for private companies&rdquo;. Ultimately, he argues, it is in everyone&rsquo;s interest to incorporate our new, virtual drivers into urban traffic as smoothly as possible &ndash; much like we do with new human drivers, but with the greater levels of safety, predictability and efficiency that autonomous vehicles offer.&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;</div>

Emilio Frazzoli’s start-up NuTonomy develops control software for autonomous vehicles and uses Singapore as a test bed. (Photograph: NuTonomy)

Car sharing with autonomous vehicles could improve cities in many different ways. Singapore is taking a pioneering role, working with ETH researchers to explore the potential of personalised, electrified and automated public transport.

Algorithms take the wheel

What sensors do we need for building a fully autonomous robot?Autonomous Robots are equipped with numerous sensors to perceive its surroundings as well as be aware of its own movements. The first step for achieving autonomy of robots i.e., robots moving from point A to point B on its own without colliding with anything, is awareness of the surrounding environment. Autonomous robots have a stack of sensors, among them, Wheel Odometers, IMU, GPS, Lidar, and Multiple Cameras being the most common. The current developments in the autonomous vehicles space also make use of sensors like Radar, Stereo Cameras in combination with the above stack.&nbsp;What is localization?<a href="http://www.cs.cmu.edu/~rasc/Download/AMRobots5.pdf" rel="noopener noreferrer" target="_blank">Localization</a> means to determine the current position and orientation of a body with respect to some coordinate system.How humans localize, is it a difficult task?Humans very naturally tend to determine their &nbsp;current position with respect to the different environmental landmarks/features around them. So, when we localize ourselves we tend to understand we are at some given distance and at certain angle from some house/tree/lamp post or any other landmarks. Given an empty featureless space even humans cannot localize. Thus, features are important for this localization task and thus we need a map filled with features/landmark to localize ourselves in that map, else it is an impossible task to perform.Then how do robots do it?The SLAM algorithm has two parts; the first is mapping and the second is localization. Autonomous ground robots have visual sensors like Lidar and Camera which can pretty well map its environment around. Then with 3D reconstruction from their data, robots generate something called a HD map. HD maps differ from normal maps, the former has a lot more features than the later. Once this map is ready, the robot starts localizing itself in this map. <a href="http://robots.stanford.edu/papers/thrun.pf-in-robotics-uai02.pdf" rel="noopener noreferrer" target="_blank">Particle Filter</a>, <a href="https://www-users.cs.umn.edu/~isler/pub/tase10.pdf" rel="noopener noreferrer" target="_blank">Triangulation</a>, <a href="http://www.cs.toronto.edu/~urtasun/courses/CSC2541/03_odometry.pdf" rel="noopener noreferrer" target="_blank">Visual Odometry</a> are some methods used for this purpose.&nbsp;Another widely used method for localization is <a href="https://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf" rel="noopener noreferrer" target="_blank">Extended Kalman Filter</a>. An EKF is a non-linear version of the linear Kalman Filter. It's a state-space estimator, what that means is that it can help in estimating the current state given input of the previous state. It's is used as a data fusion filter for localization tasks in robotics. It takes inputs from the IMU, Wheel Odometers and GPS and performs its computation based on a CTRV (Constant Turn Rate Velocity) model for a wheeled ground vehicle, to estimate the current position and orientation of the vehicle. Often this localization information is fused with visual odometry to achieve an accuracy of 100mm in the localization task.Once the robots know where they are in a map they can now start planning their path for the destination point B. Thus, invoking another interesting field of research called Path Planning. &nbsp; More resources on SLAM :1. <a href="https://blog.cometlabs.io/teaching-robots-presence-what-you-need-to-know-about-slam-9bf0ca037553" rel="noopener noreferrer" target="_blank">Teaching robots presense</a>2. <a href="https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping" rel="noopener noreferrer" target="_blank">SLAM</a>3. <a href="https://ieeexplore.ieee.org/abstract/document/1638022" rel="noopener noreferrer" target="_blank">Simultaneous Localization and Mapping</a>&nbsp;

Image: Highway image using LIDAR - Oregon State University (CC BY-SA 2.0)

A brief introduction to localization and mapping, and how these two functions are performed simultaneously during SLAM (Simultaneous Localization and Mapping).

Autonomous Vehicles

ORGANIZATIONS.

SHAPING THE INDUSTRY.

Ansys

Nexperia

Keysight Test and Measurements

PhotonDelta

Wiferion GmbH

Latest Posts