AI for Cars

Josep Aulinas, Hanky Sjafrie

• Publisher: Chapman and Hall/CRC
• Year: 2021
• ISBN:
  9781000417197
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Artificial Intelligence (AI) is undoubtedly playing an increasingly significant role in automobile technology. In fact, cars inhabit one of just a few domains where you will find many AI innovations packed into a single product.

AI for Cars provides a brief guided tour through many different AI landscapes including robotics, image and speech processing, recommender systems and onto deep learning, all within the automobile world. From pedestrian detection to driver monitoring to recommendation engines, the book discusses the background, research and progress thousands of talented engineers and researchers have achieved thus far, and their plans to deploy this life-saving technology all over the world.

1

AI for Advanced Driver Assistance Systems

Advanced Driver Assistance Systems (ADAS) can be defined as a collection of electronic systems that aid drivers by improving safety, comfort and efficiency while driving and parking. Even if you’ve never heard the term ADAS before, it’s quite possible that you – as a car user or owner – have been using some ADAS functions, for instance, an Anti-lock Braking System (ABS) or Electronic Stability Control (ESC), without noticing. ABS and ESC are just two popular examples of many ADAS applications that find their ways from niche innovations to standard car safety features.

In contrast to AD, which is generally placed at one of the levels 3–5 (conditional, high or full automation) on the scale created by the Society for Automotive Engineers (SAE) International, ADAS is commonly located at SAE level 2 (partial automation). This means that the driver is assisted by ADAS but remains the primary actor controlling the vehicle.

ADAS functions vary from purely informative, such as speed limit information, to safety-critical ones, such airbag deployment. Similar to safety-related systems in other domains such as aerospace or medical devices, the whole development process of ADAS needs to comply with industry-wide standards as well as other governing regulations. One such standard is the ISO 26262 “Road Vehicles – Functional Safety” standard [3]. ISO’s Automotive Safety Integrity Levels (ASILs) for each ADAS function are determined according to the levels of severity, exposure and controllability of hazardous events that might arise from the system’s malfunction. The ISO 26262 defines four ASIL levels, ranging from ASIL A (the least stringent level) to ASIL D (the most stringent level). As such, ASIL D functions require more comprehensive safety requirements and measures than ASIL A, B and C. Functional safety and ISO 26262 will be covered in more detail in the last chapter of this book.

Developing safe ADAS functions that work reliably everywhere is undoubtedly challenging. In the age of digital connectivity, we’re living in, however, they are not the only variables in the equation. For ADAS to promote safety as intended, a car’s systems must be securely protected from unwanted influences and even cyber-attacks. The famous “Jeep attack” case showed how serious the consequences of unaddressed security holes (which the automaker Chrysler later corrected) can be [4]. We will discuss car security and some of the AI-driven cybersecurity measures toward the end of this book.

Other challenges in ADAS include the necessity of reliable functionality wherever motor vehicles may be used. Traffic laws and signs are not globally standardized; in addition, there can be temporary and long-term changes in speed limits and other posted restrictions due to construction and spontaneously arising conditions such as accidents and weather. ADAS must also be able to work dependably in widely differing and sometimes very harsh physical environments, ranging from desert heat to arctic winters.

Yet in the face of all these challenges, AI’s contribution to the progress in ADAS is already making driving both easier and safer. To look a bit more closely at these practical benefits, in this chapter we will consider the advances made with AI’s help in three ADAS applications, namely automatic parking, traffic sign recognition and Driver Monitoring Systems (DMS).

Automatic Parking

For a lot of people, parking is probably the least enjoyable aspect of driving a car. Finding an empty parking space in big cities can be challenging, and trying to park in a tight slot on a busy street can be downright stressful. But maybe especially in less stressful cases, parking assistant systems can be very helpful.

Over the past two decades, parking assistant systems have evolved from purely informational to fully automatic systems. Whereas the early versions of such systems could only visualize the rear environment of the vehicle with the help of a rear-view or backup camera, modern automatic parking systems park and unpark the vehicle autonomously without requiring any human inside the vehicle.

The transition from parking assistant systems (which were non-automatic) to semi-automatic parking systems for series-production cars was pioneered by Toyota through the introduction of Intelligent Parking Assist in its Prius hybrid model in Japan in 2003. The initial version of the system supported only the reverse parallel parking scenario. The required parking maneuver was automatically calculated by the system, so that the driver could take their hands off the steering wheel and control the maneuver using the brake pedal. The introduction of remote-control parking in the BMW 7 Series and the Mercedes E-Class during the years 2015/2016 marked the transition from semi-automatic to fully automatic systems for series-production cars. This automatic parking system enabled a vehicle to park and unpark itself autonomously within a parking lot or a garage without requiring any human assistance from inside the vehicle. The user just needed to be in proximity to the car, to control the automatic parking maneuver by pressing a button in the car key or in a smartphone app. In 2019, an automated valet parking system in Stuttgart, Germany became the first system of its kind to receive official approval for daily use from the local transport authority [5]. Using the automated valet parking system, the driver can just leave the vehicle at the drop-off area, after which it will park itself inside the intelligent parking building without any human involvement. Likewise, the vehicle can unpark and navigate itself back to the drop-off area when the owner summons it.

Modern automatic parking systems typically support common parking scenarios. These especially include parallel parking (where the parking space is parallel to the road), angle parking (where the space is arranged at an acute angle – smaller than 90° – to the road) and perpendicular parking (where the space is situated perpendicular – at a 90° angle – to the road).

In terms of automation level, parking systems can be generally divided into two categories: semi-automatic and fully automatic. Semi-automatic parking systems require the driver to press the gas pedal to start or proceed with the parking maneuver, shift gears (into forward or reverse) and if necessary, press the brake pedal to interrupt the maneuver. The system controls the steering wheel automatically, based on the calculated trajectory for the parking space.

By contrast, fully automatic parking systems do not require manual control of the transmission nor of the gas and brake pedals, as the vehicle is capable of performing the parking maneuver autonomously from start to finish. For this reason, it is not even necessary for the driver to be inside the car while the vehicle is parking in a parking garage. However, for safety reasons, it is usually still required for the driver or an operator to keep pressing the “hold-to-run button” on the car key or on the smartphone app during the parking maneuver. If the button is no longer pressed, the parking maneuver stops immediately to prevent accidents from arising due to unattended operation. The automatic parking system is designed to be automatically interruptible whenever there could be a collision with humans or inanimate obstacles.

The more advanced form of fully automatic parking is automated valet parking. The concept is similar to conventional valet parking service, in which the driver and all passengers alight at a drop-off area and a person parks the vehicle for them and returns it when they need it back. The only difference in the automated valet parking scenario is that there is no valet. The vehicle parks autonomously, driving itself from the drop-off area into the parking space and later returning autonomously at the customer’s request. Automated valet parking typically requires additional sensors or other equipment installed in the parking infrastructure to help vehicle navigation as well as to provide other important information for the vehicle, such as the location of a free parking space together with a high-definition map of the parking area. The additional “help” from the parking infrastructure is particularly important for automated valet parking in an indoor parking garage, since Global Navigation Satellite Systems (GNSS) such as Global Positioning System (GPS) are often not available to help with localization.

Automatic parking systems are made possible through well-orchestrated actions involving several ADAS functions such as parking space detection, parking spot marking lane detection, object detection, localization, path planning and path tracking. Depending on the vehicle architecture and the complexity of the parking scenarios, these systems require various amounts of data from multiple sensors and good coordination of several ADAS Electronic Control Units (ECUs).

For a vehicle to detect a suitable parking space according to present technology, the driver usually first activates the automatic parking system and slowly passes by a potential space. While passing, the onboard sensors scan the parking lot, measure the free space and determine whether the space is suitable for automatic parking. This step might not be necessary in the case of the automated valet parking scenario, as the parking infrastructure should be able to provide information about available parking spots directly to the vehicle. Although the ultrasonic sensor is the most common sensor type used for parking space detection, several studies have also showed promising results for parking space detection using a camera [6] or a combination of ultrasonic sensor and camera [7]. The lower cost of cameras and the richer information they provide make them an attractive sensor for parking space detection; however, their usage can be limited in poor lighting environments.

To ensure that vehicles are parked within the defined parking space boundaries, lane-marking detection is necessary. We should note here that this form of detection is also important for automatic parking scenarios when there is no adjacent vehicle. This is due to the fact that ultrasonic sensors can only sense when an ultrasonic wave is reflected by an adjacent object and returns to the sensor. If there is no adjacent object (like another car) and no other physical structure (such as walls) to help define the limits of the parking space, the vehicle might park anywhere – irrespective of the markings on the floor. For this reason, cameras are still the predominant sensors for lane-marking detection. Lidars have also been investigated for lane-marking detection, although at present their relatively high price tag still hinders the broad adoption of this technology in cars. Lidar-based detection works by analyzing the intensity of the point clouds reflected by the lane markings on the surface of the floor [8]. Laser beams reflected by markings have a different intensity than those from unmarked surfaces. Because lidar sensors work in all lighting conditions, they could become a viable option to compensate camera limitations in dark parking environments.

Throughout the parking maneuver, the vehicle’s surrounding environment is permanently monitored with object detection to avoid collisions with any obstacles, whether static (such as walls, objects on the ground etc.) or moving (humans, animals etc.). Whereas path planning algorithms such as Rapidly-Exploring Random Trees (RRT) [29] and Hybrid A-Star (Hybrid A*) [9] calculate the necessary trajectory of a vehicle to reach the desired parking location, path tracking ensures that the vehicle follows the planned trajectory properly, using for instance the pure pursuit [10] or the Stanley method [11].

For automated valet parking use cases, the vehicle must also localize itself along the travel path from the drop-off point to the destination parking space or vice versa. Depending on the parking space environment (indoor/outdoor), the vehicle sensor configuration and the availability of additional sensors/equipment installed in the parking infrastructure, there are several approaches to perform this localization: combining the Inertial Navigation System (INS) and GNSS, using external references (beacons, magnets, visual markers etc.) or using the on-board lidar or camera sensor only [12]. In the latter case, the parking environment is previously mapped using a Simultaneous Localization and Mapping (SLAM) algorithm and the localization is then applied on the generated high-resolution map of the parking environment. SLAM algorithms can generally be divided into filtering and optimization approaches [13]. The filtering approach, for example, the Extended Kalman Filter SLAM (EKF-SLAM) [14] or the FastSLAM algorithm [15], summarizes information from past observations and iteratively (i.e. through repetition) improves its internal belief (its appraisal of what it has perceived through the car’s sensors) by incorporating new observations as they are gathered. By contrast, the optimization approach, such as the graph-based SLAM algorithm [16], keeps track of all poses (the car’s positions and the directions in which it has been pointed) and measurements from the beginning until the current observation and finds the most likely overall trajectory or the one most consistent with the whole observation set.

Automatic parking systems not only increase drivers’ comfort level; they also have the potential to reduce traffic accidents and to result in more effective parking lot use in the case of automated valet parking. The latter is undoubtedly an attractive way to address parking space issues in cities and other areas experiencing a parking space shortage. However, the current technology for automated valet parking systems still requires some high-tech investment in the parking infrastructures and possibly also in the vehicles. Hence, the wide adoption of such systems depends not only on the maturity level of the technology but also on the economic aspects of its application.

Traffic Sign Recognition

Have you ever missed or forgotten the last speed limit sign because you were so engaged in a discussion while driving or maybe were simply not attentive enough? Or have you ever driven on rural roads abroad while wondering what the actual speed limit is, since there was no speed limit sign to be seen?

Thanks to affordable personal navigation devices and the navigation apps on our smartphones nowadays, we may not be completely lost in these situations as our devices or apps could show us the valid speed limit throughout our journey. However, the reliability of this information is heavily dependent on the quality of the map material referred to and the availability of satellite navigation systems such as the GPS.

If the map material is outdated or incomplete, the speed limit shown might be incorrect – or entirely absent in the case of new roads. Speed limits might also not be shown correctly when one is driving through a long tunnel with more than one-speed limit posted inside it, since the navigation assistants would lose track of the vehicle’s position due to the unavailability of the GPS signals.

Another drawback of map-based speed limit information is the lack of consideration of what are known as variable speed limits. Typically shown on electronic signs or portable signposts and only valid temporarily, these flexible restrictions take priority over the permanent ones on specific road segments.

Variable speed limits are used by local transportation or law enforcement authorities to regulate traffic flow, for instance, to avoid or reduce the impact of road congestion or to improve safety for road workers. Map-based speed limit information is not able to take these variations into account and will therefore show an incorrect speed limit.

To overcome the above limitations, modern cars typically use a combination of camera-based traffic sign recognition and map input to provide highly accurate speed limit information. In the case of variable speed limit situations, the camera-detected speed limit takes priority over the information embedded in the map material. On the other hand, the information from the map material will take priority if no speed limit signs are detected for a long time, or when camera recognition is temporarily unavailable or unreliable, e.g. due to glare or heavy rain.

Due to advances in ADAS, speed limit signs are far from being the only traffic signs that cars are able to recognize nowadays. AI has for example empowered vehicles to recognize no-entry/wrong-way, no-passing, yield and stop signs and even the color of traffic lights, from a distance of up to 150 meters [17]. The information is typically shown using pictograms displayed in the cluster-instrument, head-unit or head-up display. Traffic sign recognition is primarily done using cameras. Some studies have also proposed a sensor fusion of camera and lidar (light dete...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Contents
• Foreword
• Preface
• 1 AI for Advanced Driver Assistance Systems
• 2 AI for Autonomous Driving
• 3 AI for In-Vehicle Infotainment Systems
• 4 AI for Research & Development
• 5 AI for Services
• 6 The Future of AI in Cars
• Further Reading
• References
• Index