Python Deep Learning

Exploring deep learning techniques and neural network architectures with PyTorch, Keras, and TensorFlow, 2nd Edition

Ivan Vasilev, Daniel Slater, Gianmario Spacagna, Peter Roelants, Valentino Zocca

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781789349702
• Topic: Informatica
• Subtopic: Reti neurali

Learn advanced state-of-the-art deep learning techniques and their applications using popular Python libraries

Key Features

• Build a strong foundation in neural networks and deep learning with Python libraries
• Explore advanced deep learning techniques and their applications across computer vision and NLP
• Learn how a computer can navigate in complex environments with reinforcement learning

Book Description

With the surge in artificial intelligence in applications catering to both business and consumer needs, deep learning is more important than ever for meeting current and future market demands. With this book, you'll explore deep learning, and learn how to put machine learning to use in your projects.This second edition of Python Deep Learning will get you up to speed with deep learning, deep neural networks, and how to train them with high-performance algorithms and popular Python frameworks. You'll uncover different neural network architectures, such as convolutional networks, recurrent neural networks, long short-term memory (LSTM) networks, and capsule networks. You'll also learn how to solve problems in the fields of computer vision, natural language processing (NLP), and speech recognition. You'll study generative model approaches such as variational autoencoders and Generative Adversarial Networks (GANs) to generate images. As you delve into newly evolved areas of reinforcement learning, you'll gain an understanding of state-of-the-art algorithms that are the main components behind popular games Go, Atari, and Dota.By the end of the book, you will be well-versed with the theory of deep learning along with its real-world applications.

What you will learn

• Grasp the mathematical theory behind neural networks and deep learning processes
• Investigate and resolve computer vision challenges using convolutional networks and capsule networks
• Solve generative tasks using variational autoencoders and Generative Adversarial Networks
• Implement complex NLP tasks using recurrent networks (LSTM and GRU) and attention models
• Explore reinforcement learning and understand how agents behave in a complex environment
• Get up to date with applications of deep learning in autonomous vehicles

Who this book is for

This book is for data science practitioners, machine learning engineers, and those interested in deep learning who have a basic foundation in machine learning and some Python programming experience. A background in mathematics and conceptual understanding of calculus and statistics will help you gain maximum benefit from this book.

]]>

Reinforcement Learning Theory

You may have read sci-fi novels from the 50s and 60s; they are full of visions of what life in the 21st century would look like. These stories imagined a world of people with personal jet packs, underwater cities, intergalactic travel, flying cars, and truly intelligent robots capable of independent thought. The 21st century has arrived now; sadly, we are not going to get those flying cars, but thanks to deep learning, we may get the robot.

In Chapter 9, Deep Reinforcement Learning for Games, and Chapter 10, Deep Learning in autonomous Vehicles, we'll talk about Reinforcement learning (RL) – a way to make machines interact with an environment, similar to the way we people interact with the physical world. As with many of the algorithms discussed so far, RL is not a new concept. But, recently, the field has seen something of a resurgence, in no small part thanks to the successes of deep learning. Indeed, we'll later see how integrating deep networks in RL frameworks can produce great results. In this section, we'll talk about the main paradigms and algorithms of RL. Then, we'll see how to combine them with deep networks to teach the computer to navigate a dynamic environment, such as a computer game. Games act as a great playing field for testing RL algorithms. They give us an environment of large, but manageable, possibilities. This is unlike the physical world, where even simple a task, such as getting a robot arm to pick up objects, requires analyzing huge amounts of sensory data and controlling many continuous-response commands for the arm's movement. Furthermore, we can create and simulate different training and evaluation scenarios more easily in a virtual environment, compared to a physical one.

When it comes to computer games, we know that humans can learn to play a game just from the pixels visible on the screen and minimal instructions. If we input the same pixels plus an objective into a computer agent, we know we have a solvable problem, given the right algorithm. This is why so many researchers are looking at games as a great place to start developing true AI self-learning machines that can operate independently of humans. Also, if you like games, it's lots of fun.

In this chapter, we will cover the following topics:

• RL paradigms
• RL as a Markov decision process
• Finding optimal policies with Dynamic Programming
• Monte Carlo methods
• Temporal difference methods
• Value function approximation
• Experience replay
• Q-learning in action

RL paradigms

In this section, we'll talk about the main paradigms of RL. We first mentioned some of them in Chapter 1, Machine Learning: an Introduction, but it's worth discussing them here to refresh our memory and for the sake of completeness. To help us with this task, we'll use a maze game as an example. The maze is represented by a rectangular grid, where grid cells with a value of 0 represent the walls, and the cells with a value of 1 are the paths. Some locations contain intermediate rewards. An agent in the maze can use the paths to move between locations. Its objective is to navigate its way to the other end of the maze and to get the largest possible reward while doing so. The following is a diagram describing the basic principles of how RL works:

Reinforcement learning scenario

Here are some elements of an RL system:

• Agent: The entity for which we are trying to learn actions. In the game, this is the player who tries to find their way through the maze.
• Environment: The world in which the agent operates. Here, this is the maze (grid) itself.
• State: All of the information available to the agent about its current envi...