How Do Self-Driving Cars 'See' The Road? A Simple Guide for beginners

 HOW DO SELF DRIVING CARS OPERATE?

You’ve probably heard about Tesla's self-driving cars and must have been curious to know  how they can  navigate roads on their own. It’s intriguing to think about cars "driving on their own", to a beginner it might seem like magic as it's hard for him to believe. Imagine being told to blindfold and try driving...feels almost impossible, right ? Because for us navigating through complex roads and traffic without human vision seems near to impossible( as we sometimes face difficulty to navigate through traffic with our vision ).

But how exactly these self driving cars (also referred to as "Autonomous Vehicles") operate. What makes them work is a  combination of varoius technologies which includes advanced sensors technology, powerful computers, and intelligent software working together seamlessly. Although all of this seems so complex to understand but in this blog everyhting about self driving cars will be  explained in simple terms without the use of so-called technical terms that leaves you scratching your head, we'll see how these autonomous vehicles perceive and understand their environment. Let's dive into it....

1. COMPUTER VISION  : The most important concern when we hear the term "self driving cars" is that how are they going to SEE without any human vision. This where computer vision comes into play, this uses sensor technology, that allows the car to "see " and " analyze" it's surrounding. This serves as the eyes of the cars. It has basically three things :

(a) CAMERAS:  Cameras are very important in a self-driving car, they serve  like human eyes to cars. These cameras, mounted around the vehicle, capture visual details of the road in real-time, providing a 360-degree view. Modern autonomous vehicles typically use between 8 to 12 cameras. These are not like old cameras they're specialized devices which are capable of processing up to 60 images every second. They can cover a very wide range of view. Cameras cover the following tasks: 

i ) Cameras basically detect the colors, shapes, and symbols on signs and signals.

ii)  Recognizing the boundaries of roads, lanes is essential to navigate properly, which cameras help to identify.

iii) Cameras detect nearby objects, including moving ones, allowing the vehicle to avoid collisions and make safe navigation choices.

It seems like cameras solely do the job of providing  vision to cars, but there's a lot more to it, after all it's about providing the car better-than-human vision.

There are certain limitations to cameras also, they can not work efficiently in bad weathers like fog or rain and most importantly they generate 2-D images which makes it challenging to measure distance accurately, to overcome this we have other devices like radars and lidars.

 You must be wondering, it's fine that cameras detect the colour, shapes of the object, roads boundaries in the way of car but how the cars come to know the distance of other cars or any other obstacle from it , how exactly it gets to know when to slow down or apply brakes... There comes radars into play.

(b) RADARS:  Radar technology can be percieved as the car's ability to feel the distance and speed of objects around it. To it in simple terms in autonomous vehicles, radar systems continuously sends out invisible radio waves that bounce off objects and returns back to the sensor. By measuring how long these waves take to return, the car can calculate exact distances to multiple objects simultaneously. This helps the car to navigate through traffic as car can detect the distance of multiples cars from it, simultaneously.

Radars also accurately detect the speed and  direction of  other vehicles or pedestrians. Radars have an edge over cameras as they are wheather resilient sensors, they can function reliably in all weather conditions like foggy or rainy conditions in which cameras struggle to work efficiently.

(c) LiDARS: Light Detection and Ranging, this technology basically uses lasers. These lasers help to create an incredibly detailed 3-D map of the car's surrounding. Lidars emit millions of laser beams every second and measure how long each takes to bounce back.  Lidars are very precise, they can detect from large to very small objects and their position accurately which makes them invaluable for navigating complex environments.

Although Lidars are invaluable for navigation in self driving cars but Lidar sensors are very expensive, and weather conditions like rain and snow might make them less efficient.

Now, after collecting all the data from cameras, radars and Lidars, it is all blended to create a single view of the car's surrounding, this technique is called ' sensor fusion'. 

 We have talked about the eyes of the self driving cars and how those eyes see and capture the surrounding information. But seeing only, is not sufficient, right? We need brain to comprehend the data that eyes see. So there must be a brain for self driving cars also so that can interpret the data collected by the sensors.

(d) Machine Learning and Artificial Intelligence: It serves as the brain of self driving cars. AI helps to identify nearby vehicles, pedestrians and road signs etc. It also helps in making decisions as when to accelerate, apply brake, change lanes, or take any other action based on the data collected by the sensors. You must be wondering these cars are mere machines how do they differentiate between cars and trucks, pedestrians and animals or whether it's a raod sign or just an advertisement,  so this achieved through " Deep Learning " (also a type of AI). Deep learning works similar to how human brains learn from experience, it  involves training AI systems(which are there in self driving cars) on millions of real world driving scenarios. So when a self driving car see the road, it scans every minute detail about its surrounding. 

It might seem like that it is a time taking process, first objects are detected then a 3-D map is formed then it identifies the object and it's speed and then AI takes decision of what action to be performed by the car, But all this, is in coordination, all this  happens in fractions of a second. 

Some examples of famous self driving cars:

1) Tesla's self driving cars : These are the most popular one's. They uses 8 cameras to have a 360 degree view of car's surroundings and they don't use Lidars, they rely on high quality cameras and AI. They use Autopilot and Full-self- driving  (FSD) systems.

(2) Google's Waymo: These are fully autonomous taxis that are actually operating in Phoenix, Arizona. They use advanced Lidars, radars and cameras to create accurately detailed 3-D maps of car's surroundings. Here's how google's Waymo looks.

With the advent of technology these self driving cars are becoming mainstream they're now a reality on our roads thanks to this sophisticated blend of sensors, cameras, and artificial intelligence. The next time you see a self-driving car, you won't be wondering how these are navigating so perfectly through the crowd. As now you know, what fascinating technology makes it possible for a car to navigate our complex world all on its own.