Applied Deep Learning and Computer Vision for Self-Driving Cars

Build autonomous vehicles using deep neural networks and behavior-cloning techniques

Sumit Ranjan, Dr. S. Senthamilarasu

• Editore: Packt Publishing
• Anno: 2020
• ISBN:
  9781838647025
• Edizione: 1
• Argomento: Computer Science
• Categoria: Neural Networks

Explore self-driving car technology using deep learning and artificial intelligence techniques and libraries such as TensorFlow, Keras, and OpenCV

Key Features

• Build and train powerful neural network models to build an autonomous car
• Implement computer vision, deep learning, and AI techniques to create automotive algorithms
• Overcome the challenges faced while automating different aspects of driving using modern Python libraries and architectures

Book Description

Thanks to a number of recent breakthroughs, self-driving car technology is now an emerging subject in the field of artificial intelligence and has shifted data scientists' focus to building autonomous cars that will transform the automotive industry. This book is a comprehensive guide to use deep learning and computer vision techniques to develop autonomous cars.

Starting with the basics of self-driving cars (SDCs), this book will take you through the deep neural network techniques required to get up and running with building your autonomous vehicle. Once you are comfortable with the basics, you'll delve into advanced computer vision techniques and learn how to use deep learning methods to perform a variety of computer vision tasks such as finding lane lines, improving image classification, and so on. You will explore the basic structure and working of a semantic segmentation model and get to grips with detecting cars using semantic segmentation. The book also covers advanced applications such as behavior-cloning and vehicle detection using OpenCV, transfer learning, and deep learning methodologies to train SDCs to mimic human driving.

By the end of this book, you'll have learned how to implement a variety of neural networks to develop your own autonomous vehicle using modern Python libraries.

What you will learn

• Implement deep neural network from scratch using the Keras library
• Understand the importance of deep learning in self-driving cars
• Get to grips with feature extraction techniques in image processing using the OpenCV library
• Design a software pipeline that detects lane lines in videos
• Implement a convolutional neural network (CNN) image classifier for traffic signal signs
• Train and test neural networks for behavioral-cloning by driving a car in a virtual simulator
• Discover various state-of-the-art semantic segmentation and object detection architectures

Who this book is for

If you are a deep learning engineer, AI researcher, or anyone looking to implement deep learning and computer vision techniques to build self-driving blueprint solutions, this book is for you. Anyone who wants to learn how various automotive-related algorithms are built, will also find this book useful. Python programming experience, along with a basic understanding of deep learning, is necessary to get the most of this book.

Section 1: Deep Learning Foundation and SDC Basics

In this section, we will learn about the motivation behind becoming a self-driving car engineer, and the associated learning path, and we will get an overview of the different approaches and challenges found in the self-driving car field. It covers the foundations of deep learning, which are necessary, so that we can take a step toward the implementation of self-driving cars. This section provides a step-by-step explanation to enable you to understand deep neural network libraries such as Keras. It also covers the implementation of deep learning models from scratch by using Keras.

This section comprises the following chapters:

• Chapter 1, The Foundation of Self-Driving Cars
• Chapter 2, Deep Dive into Deep Neural Networks
• Chapter 3, Implementing a Deep Learning Model Using Keras

The Foundation of Self-Driving Cars

The driverless car is popularly known as an self-driving car (SDC), an autonomous vehicle, or a robot car. The purpose of an autonomous car is to drive automatically without a driver. The SDC is the sleeping giant that might improve everything from road safety to universal mobility, while dramatically reducing the costs of driving. According to McKinsey & Company, the widespread use of robotic cars in the US could save up to $180 billion annually in healthcare and automotive maintenance alone based on a realistic estimate of a 90% reduction in crash rates.

Although self-driving automotive technologies have been in development for many decades, it is only in recent years that breakthroughs have been achieved. SDCs have proved to be much safer than human drivers, and automotive firms as well as other tech firms are investing billions in bringing this technology into the real world. They struggle to find great engineers to contribute to the field. This book will teach you what you need to know to kick-start a career in the autonomous driving industry. Whether you're coming from academia, or from within the industry, this book will provide you with the foundational knowledge and practical skills you will need to help build a future with Advanced Driver-Assistance Systems (ADAS) engineers or SDC engineers. Throughout this book, you will study real-world data and scenarios from recent research in autonomous cars.

This book can also help you learn and implement the state-of-the-art technologies for computer vision that are currently used in the automotive industry in the real world. By the end of this book, you will be familiar with different deep learning and computer vision techniques for SDCs. We'll finish this book off with six projects that will give you a detailed insight into various real-world issues that are important to SDC engineers.

In this chapter, we will cover the following topics:

• Introduction to SDCs
• Advancement in SDCs
• Levels of autonomy
• Deep learning and computer vision approaches for SDCs

Let's get started!

Introduction to SDCs

The following is an image of an SDC by WAYMO undergoing testing in Los Altos, California:

Fig 1.1: A Google SDC

You can check out the image at https://en.wikipedia.org/wiki/File:Waymo_Chrysler_Pacifica_in_Los_Altos,_2017.webp.

The idea of the autonomous car has existed for decades, but we saw enormous improvement from 2002 onward when the Defense Advanced Research Projects Agency (DARPA) announced the first of its grand challenges, called the DARPA Grand Challenge (2004). That would forever change the world's perception of what autonomous robots can do. The first event was held in 2004 and DARPA offered the winners a one-million-dollar prize if they could build an autonomous vehicle that was able to navigate 142 miles through the Mojave Desert. Although the first event saw only a few teams get off the start line (Carnegie Mellon's red team took first place, having driven only 7 miles), it was clear that the task of driving without any human aid was indeed possible. In the second DARPA Grand Challenge in 2005, five of the 23 teams smashed expectations and successfully completed the track without any human intervention at all. Stanford's vehicle, Stanley, won the challenge, followed by Carnegie Mellon's Sandstorm, an autonomous vehicle. With this, the era of driverless cars had arrived.

Later, the 2007 installment, called the DARPA Urban Challenge, invited universities to show off their autonomous vehicles on busy roads with professional stunt drivers. This time, after a harrowing 30-minute delay that occurred due to a jumbotron screen blocking their vehicle from receiving GPS signals, the Carnegie Mellon team came out on top, while the Stanford Junior vehicle came second.

Collectively, these three grand challenges were truly a watershed moment in the development of SDCs, changing the way the public (and more importantly, the technology and automotive industries) thought about the feasibility of full vehicular autonomy. It was now clear that a massive new market was opening up, and the race was on. Google immediately brought in the team leads from both Carnegie Mellon and Stanford (Chris Thompson and Mike Monte-Carlo, respectively) to push their designs onto public roads. By 2010, Google's SDC had logged over 140 thousand miles in California, and they later wrote in a blog post that they were confident about cutting the number of traffic deaths by half using SDCs. blog by Google: What we're driving at — Sebastian Thrun (https://googleblog.blogspot.com/2010/10/what-were-driving-at.html)

According to a report by the World Health Organization, more than 1.35 million lives are lost every year in road traffic accidents, while 20-50 million end up with non-fatal injuries (https://www.who.int/health-topics/road-safety#tab=tab_1).

As per a study released by the Virginia Tech Transportation Institute (VTTI) and the National Highway Traffic Safety Administration (NHTSA), 80% of car accidents involve human distraction (https://seriousaccidents.com/legal-advice/top-causes-of-car-accidents/driver-distractions/). An SDC can, therefore, become a useful and safe solution for the whole of society to reduce these accidents. In order to propose a path that an intelligent car should follow, we require several software applications to process data using artificial intelligence (AI).

Google succeeded in creating the world's first autonomous car 2 years ago (at the time of writing). The problem with Google's car was its expensive 3D RADAR, which is worth about $75,000.

The 3D RADAR is used for environmental identification, as well as the development of a high-resolution 3D map.

The solution to this cost is to use multiple, cheaper cameras that are mounted to the car to capture images that recognize the lane lines on the road, as well as the real-time position of the car.

In addition, a driverless car can reduce the distance between cars, thereby reducing the degree of road loads, reducing the number of traffic jams. Furthermore, they greatly reduce the capacity for human errors to occur while driving and allow people with disabilities to drive long distances.

A machine as a driver will never make a mistake; it will be able to calculate the distance between cars very accurately. Parking will be more efficiently spaced, and the fuel consumption of cars will be optimized.

The driverless car is a vehicle equipped with sensors and cameras for detecting the environment, and it can navigate (almost) without any real-time input from a human. Many companies are investing billions of dollars in order to advance this toward an accessible reality. Now, a world where AI takes control of driving has never been closer.

Nowadays, self-driving car engineers are exploring several different approaches in order to develop an autonomous system. The most successful and popularly used among them are as follows:

• The robotics approach
• The deep learning approach

In reality, in the development of SDCs, both robotics and deep learning methods are being actively pursued by developers and engineers.

The robotic approach works by fusing output from a set of sensors to analyze a vehicle's environment directly and prompt it to navigate accordingly. For many years, self-driving automotive engineers have been working on and improving robotic approaches. However, more recently, engineering teams have started developing autonomous vehicles using a deep learning approach.

Deep neural networks enable SDCs to learn how to drive by imitating th...

Indice dei contenuti

• Title Page
• Copyright and Credits
• About Packt
• Contributors
• Preface
• Section 1: Deep Learning Foundation and SDC Basics
• The Foundation of Self-Driving Cars
• Dive Deep into Deep Neural Networks
• Implementing a Deep Learning Model Using Keras
• Section 2: Deep Learning and Computer Vision Techniques for SDC
• Computer Vision for Self-Driving Cars
• Finding Road Markings Using OpenCV
• Improving the Image Classifier with CNN
• Road Sign Detection Using Deep Learning
• Section 3: Semantic Segmentation for Self-Driving Cars
• The Principles and Foundations of Semantic Segmentation
• Implementing Semantic Segmentation
• Section 4: Advanced Implementations
• Behavioral Cloning Using Deep Learning
• Vehicle Detection Using OpenCV and Deep Learning
• Next Steps
• Other Books You May Enjoy