Some call it a marvel of technology. Some call it a fad. Self-driving cars are constantly making the headlines. These vehicles, designed to carry passengers from point A to B without a human maneuver, are promised to bring greater mobility, reduce street congestion and fuel consumption, and create safer roads.

Driverless vehicles are hardly a novel concept anymore, but what is really going on in the autonomous car industry? How do self-driving cars work? Who are making these cars? Why is it taking this long? This article will answer these questions.

How do self-driving cars work

To reach a destination, a driverless car needs to know the route, understand its surrounding, observe traffic rules, and make correct judgments when interacting with other vehicles and pedestrians on the road. To accomplish all this, it relies on the following key technologies:

• Camera: to see things like lanes and traffic lights
• Radar: to sense objects, especially the big ones, and determine their distances
• Lidar: also to sense objects as Radar, but it’s much better in detecting small objects and mapping 3D objects. Unfortunately, it is expensive and can still be a little unreliable at times.
• AI: the brain of the car. It combines the car’s sensors and camera visions to understand the path ahead and determine how the car should be maneuvered.

Leveling up

The progression of driverless technology doesn’t go from no automation at all to full-on self-driving with no human intervention. In fact, SAE International, an automotive standardization organization classified autonomous cars to 5 levels:

• Level 0 — No Automation: Human driver controls everything — steering, braking, and accelerating.
• Level 1 — Driver Assistance: The car provides some help with either steering OR accelerating. Cars with cruise controls fall into this category.
• Level 2 — Partial Automation: The car can control both steering and accelerating at the same time, with human drivers monitoring the environment and keeping their hands on the wheel. Tesla’s autopilot is an example of Level 2 automation.
• Level 3 — Conditional Automation: The car will control the car and monitor the environment, but may request human drivers to intervene in certain situations. Audi A8’s Traffic Jam Pilot was able to achieve level 3.
• Level 4 — High Automation: Compared to level 3, the autonomous car can handle more complex driving tasks. It may still ask for human intervene in rare situations such as extreme weather.
• Level 5 — Full Automation: A truly self-sufficient car. It will not ask for human intervene and can carry out all driving tasks. Google’s Waymo is an example of that.

Key players in the market

Many companies recognized that whether they like it or not, the disruption is inevitable. Hoping to get ahead of the competition, they have invested billions of dollars in the R&D of key technologies behind self-driving cars.

The transportation revolution attracted both big corporations and startups. Over 260 startups are ready to tackle every aspect of the ecosystem while forming strategic partnerships with others. Big corporations are also investing billions of dollars to aggressively establish and implement roadmaps. Some of the leading and high-profile companies include the following.

General Motors

As one of the leading car manufacturer, GM has spent a lot of money to maintain its lead position in driverless cars. In 2016, it acquired self-driving tech company Cruise Automation for over $1billion. This unit received $2.25 billion investment from SoftBank and an additional $1.1 billion from GM itself in 2018. Furthermore, eager to dominate the market, GM bought its own LIDAR manufacturer so that it can rely less on partnerships. GM has been testing in San Francisco with a plan to expand to New York City. It plans to commercialize its self-driving car with ride-hailing services in 2019.

Waymo

Believe it or not, Waymo was established a decade ago. However, it is still considered to be the cool kid in the block. Valued at $175 billion, Waymo has accumulated over 8 million self-driving miles with a fleet of Chryslers, Hondas, and Jaguars. Just recently, Waymo announced its plan of purchasing 62,000 Fiat Chrysler vans for its ride-hailing service. Pledged to commercialize by the end of 2018, Waymo is leading the self-driving car race.

Uber

From the high-profile lawsuit against Waymo to the operation halt in Arizona and Pittsburgh, Uber has had a bumpy ride. However, Uber has not given up. With partners like Volvo and Daimler, Uber scored a $500 million investment from Toyota this year. Back on the streets of Pittsburg, Uber’s autonomous vehicle is driven manually to recalibrate its HD map of the city. It is also pumping money to its driverless car engineering hub in Toronto to continue its mission.

Lyft

Compared to Uber’s aggressive development and expansion strategy, Lyft’s approach is more controlled. Lyft partnered with Aptiv, a formerly bankrupted company that turned its tide by investing in driverless technology. Together they have completed over 5000 paid rides with 20 vehicles in Las Vegas, servicing popular destinations on the strip. When requesting Lyft rides, users can opt-in for driverless cars, and for now, there’s still a safety driver in the vehicle to monitor the ride.

Tesla

Tesla takes a completely different approach to driverless technologies. Instead of using LIDAR, Elon Musk believes the advancement of camera and image recognition technology are sufficient for automation. Although its current vehicles are equipped with auto-driving features, they still require human attention, and we have seen accidents reported due to the lack of human interventions. It’s yet to be seen how far Tesla can push its self-driving technology.

Baidu

Baidu leads the biggest driverless car effort in China since 2014. In 2017, it announced the Apollo, an open source project to facilitate the research and development of driverless cars. Baidu aimed to mass produce autonomous vehicles by 2019 to 2020, but their ability to successfully carry out this plan becomes doubtful when it lost multiple key AI talents and COO Lu Qi.

What’s taking so long?

Waymo was first established in 2009, and almost a decade later, its cars are finally getting ready to become commercialized. That’s quite a long time to release a product. Why does it take so long? Although eager to advance as quickly as possible, the self-driving car industry is facing key some challenges that they need to overcome in order to truly become common on streets. These issues include:

Lidar

Driverless car experts agree that Lidar plays a key role in ensuring a safe trip. Similar to Radar, Lidar is also used to detect surrounding objects’ distance and shape. However, instead of using radio waves, Lidar sends infrared lights, which can better detect smaller objects (like rocks, or a cyclist and ) and objects in closer distance compared to Radar.

So is Radar unnecessary for self-driving cars? Definitely not. Radar works better when detecting objects that are further away, and it’s much less sensitive to extreme weather than Lidar.

Lidar measures distance to a target by illuminating the target with laser light and measuring the reflected light with a sensor. Image by Luminar

As you have probably figured, Radar and Lidar are a complementary pair and they can accomplish so much together for the driverless future.

However, even though Lidar is not a new technology, it still needs further development. The bulky equipment comes with a hefty price tag of $75k per piece and is not ready for mass production. Driverless cars will also benefit greatly from higher resolutions and greater distance range than what the current Lidar technology can offer.

Lidar’s room for improvement can be translated into a huge business opportunity. The trade secrets regarding Lidar technology was in the eye of the Waymo v. Uber storm in 2017. Now, over 50 companies are trying to improve Lidar and make it cheaper for mass production.

AI

Driverless cars rely on artificial intelligence to make judgments. AI identifies objects on the streets and determines how the car should behave in different situations. To train such intelligence, scientists use a huge amount of data so that the algorithm can learn objects on the road. For example, the algorithm can identify a bicyclist on the street because it has seen images of tens and thousands of bicyclists.

We all have high expectations for what AI can accomplish, but it’s still far from perfect. For example, in the fatal accident caused by Uber’s self-driving car, the algorithm failed to recognize the victim, causing a delay in reactions. Furthermore, there are so many moving parts (literally) on the street, producing millions of possible scenarios. A tremendous amount of data is needed for the system to learn how to handle these scenarios, and we are just simply not there yet.

Mapping

Another race is happening in the mapping world. Driverless cars need more than the regular map we get on our phone. Although a self-driving car’s sensors and cameras can pick up objects on the street, it would be a lot safer to have an accurate 3D map of the route so that the car is familiar with all aspects of the street and can focus on detecting other elements like pedestrians. 3D maps also provide tremendous value during extreme weathers, when the data cameras and sensors provide are inadequate.

Mapping is difficult because it requires a lot of data to produce an accurate representation of the world. Even Google map has low fidelity for rural areas. What’s more challenging is the constant need for continuously data update. Companies like Here, TomTom, DeepMap, and Google are all in the race to produce maps that can be an irreplaceable part of the ecosystem.

Infrastructure

Just as how new infrastructure was needed when automobile replaced horse carriages, self-driving cars also require updated infrastructure. Worn-off pavement markings and unclear street signages will all need to be upgraded for self-driving cars so that they can clearly see the road. This may sound easy, but the implementation calls for the creation of a uniformed standard, and collaboration between federal, state, and local agencies. We also need to build connected infrastructure as a way for driverless cars to communicate with nearby infrastructure. For example, in construction zones, rather than relying on worker’s hand gestures to direct traffic, devices should be installed so that signals can be transmitted to driverless vehicles.

The infrastructure overhaul comes with a hefty price tag, but it is crucial to the safety of driverless cars. States like Michigan have begun testing new infrastructure with its modernization projects, and we will see more of them coming.

Human Communication

Autonomous vehicles may have programmed behaviors backed by AI to handle various scenarios on the street, but humans will also need to learn how to interact with autonomous vehicles.

Frequently, drivers, pedestrians, bicyclists, and motorcyclists use gestures and eye contacts on streets to signal their intentions. But how will that work with driverless cars? One of the self-driving technology company Drive.ai tackled this problem by having these cars “talk to” pedestrians. Embedded with LED screens on the front, rear and sides of the vehicle, these screens can display texts such as “waiting for you to cross”, providing a mean for the vehicle to demonstrates its intention.

Human Trust

While the industry has slowly got a grasp of human-vehicle communication, human trust is a more complex problem to solve, and it directly affects the adoption rate. In a research conducted by the Reuters and Ipsos, only 38% men and 17% women surveyed said they would feel comfortable riding a driverless car. Another Research shows that to gain people’s trust in autonomous vehicles, the industry needs to demonstrate the cars’ reliability, performance, privacy (e.g., location tracking), and security (e.g., hacking).

The wariness is not surprising, given the novelty of the technology and the potential catastrophic consequence if things go wrong. These research results demonstrate that besides improving vehicle safety and performance, self-driving car manufacturers must establish trust with consumers by educating and promoting. Designing a compelling rider experience that addresses their needs and pain points is also important, as it highlights the benefits of riding self-driving cars.

Buckle up for the future

Human species have the exceptional talent of imagination. What further sets us apart from other animals is the ability to build nested scenarios in our minds, share our fantasies with each other, and eventually work on a common goal that is based on imaginations. The rise of self-driving vehicles is precisely an outcome of collaborative imagination. We are witnessing an evolution based on rapid technology advancement, and sooner or later, human mobility may never be the same. We don’t know how many twists and turns there are ahead of us, but the journey to our destination sure is exciting.