Reinforcement Learning with TensorFlow

Sayon Dutta

• Verlag: Packt Publishing
• Jahr: 2018
• ISBN:
  9781788830713
• Auflage: 1
• Thema: Informatique
• Thema: Réseaux de neurones

Leverage the power of the Reinforcement Learning techniques to develop self-learning systems using TensorflowAbout This Book• Learn reinforcement learning concepts and their implementation using TensorFlow• Discover different problem-solving methods for Reinforcement Learning• Apply reinforcement learning for autonomous driving cars, robobrokers, and moreWho This Book Is ForIf you want to get started with reinforcement learning using TensorFlow in the most practical way, this book will be a useful resource. The book assumes prior knowledge of machine learning and neural network programming concepts, as well as some understanding of the TensorFlow framework. No previous experience with Reinforcement Learning is required.What You Will Learn• Implement state-of-the-art Reinforcement Learning algorithms from the basics• Discover various techniques of Reinforcement Learning such as MDP, Q Learning and more• Learn the applications of Reinforcement Learning in advertisement, image processing, and NLP• Teach a Reinforcement Learning model to play a game using TensorFlow and the OpenAI gym• Understand how Reinforcement Learning Applications are used in roboticsIn DetailReinforcement Learning (RL), allows you to develop smart, quick and self-learning systems in your business surroundings. It is an effective method to train your learning agents and solve a variety of problems in Artificial Intelligence—from games, self-driving cars and robots to enterprise applications that range from datacenter energy saving (cooling data centers) to smart warehousing solutions.The book covers the major advancements and successes achieved in deep reinforcement learning by synergizing deep neural network architectures with reinforcement learning. The book also introduces readers to the concept of Reinforcement Learning, its advantages and why it's gaining so much popularity. The book also discusses on MDPs, Monte Carlo tree searches, dynamic programming such as policy and value iteration, temporal difference learning such as Q-learning and SARSA. You will use TensorFlow and OpenAI Gym to build simple neural network models that learn from their own actions. You will also see how reinforcement learning algorithms play a role in games, image processing and NLP.By the end of this book, you will have a firm understanding of what reinforcement learning is and how to put your knowledge to practical use by leveraging the power of TensorFlow and OpenAI Gym.Style and approachAn Easy-to-follow, step-by-step guide to help you get to grips with real-world applications of Reinforcement Learning with TensorFlow.

Deep Learning – Architectures and Frameworks

Artificial neural networks are computational systems that provide us with important tools to solve challenging machine learning tasks, ranging from image recognition to speech translation. Recent breakthroughs, such as Google DeepMind's AlphaGo defeating the best Go players or Carnegie Mellon University's Libratus defeating the world's best professional poker players, have demonstrated the advancement in the algorithms; these algorithms learn a narrow intelligence like a human would and achieve superhuman-level performance. In plain speech, artificial neural networks are a loose representation of the human brain that we can program in a computer; to be precise, it's an approach inspired by our knowledge of the functions of the human brain. A key concept of neural networks is to create a representation space of the input data and then solve the problem in that space; that is, warping the data from its current state in such a way that it can be represented in a different state where it can solve the concerned problem statement (say, a classification or regression). Deep learning means multiple hidden representations, that is, a neural network with many layers to create more effective representations of the data. Each layer refines the information received from the previous one.

Reinforcement learning, on the other hand, is another wing of machine learning, which is a technique to learn any kind of activity that follows a sequence of actions. A reinforcement learning agent gathers the information from the environment and creates a representation of the states; it then performs an action that results in a new state and a reward (that is, quantifiable feedback from the environment telling us whether the action was good or bad). This phenomenon continues until the agent is able to improve the performance beyond a certain threshold, that is, maximizing the expected value of the rewards. At each step, these actions can be chosen randomly, can be fixed, or can be supervised using a neural network. The supervision of predicting action using a deep neural network opens a new domain, called deep reinforcement learning. This forms the base of AlphaGo, Libratus, and many other breakthrough research in the field of artificial intelligence.

We will cover the following topics in this chapter:

• Deep learning
• Reinforcement learning
• Introduction to TensorFlow and OpenAI Gym
• The influential researchers and projects in reinforcement learning

Deep learning

Deep learning refers to training large neural networks. Let's first discuss some basic use cases of neural networks and why deep learning is creating such a furore even though these neural networks have been here for decades.

Following are the examples of supervised learning in neural networks:

Inputs(x)	Output(y)	Application domain	Suggested neural network approach
House features	Price of the house	Real estate	Standard neural network with rectified linear unit in the output layer
Ad and user info Click on ad ?	Yes(1) or No(0)	Online advertising	Standard neural network with binary classification
Image object	Classifying from 100 different objects, that is (1,2,.....,100)	Photo tagging	Convolutional neural network (since image, that is, spatial data)
Audio	Text transcript	Speech recognition	Recurrent neural network (since both input-output are sequential data)
English	Chinese	Machine translation	Recurrent neural network (since the input is a sequential data)
Image, radar information	Position of other cars	Autonomous driving	Customized hybrid/complex neural network

We will go into the details of the previously-mentioned neural networks in the coming sections of this chapter, but first we must understand that different types of neural networks are used based on the objective of the problem statement.

Supervised learning is an approach in machine learning where an agent is trained using pairs of input features and their corresponding output/target values (also called labels).

Traditional machine learning algorithms worked very well for the structured...

Inhaltsverzeichnis

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Deep Learning – Architectures and Frameworks
• Training Reinforcement Learning Agents Using OpenAI Gym
• Markov Decision Process
• Policy Gradients
• Q-Learning and Deep Q-Networks
• Asynchronous Methods
• Robo Everything – Real Strategy Gaming
• AlphaGo – Reinforcement Learning at Its Best
• Reinforcement Learning in Autonomous Driving
• Financial Portfolio Management
• Reinforcement Learning in Robotics
• Deep Reinforcement Learning in Ad Tech
• Reinforcement Learning in Image Processing
• Deep Reinforcement Learning in NLP
• Further topics in Reinforcement Learning
• Other Books You May Enjoy