Mastering PyTorch

Ashish Ranjan Jha

• Publisher: Packt Publishing
• Year: 2021
• ISBN:
  9781789616408
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Master advanced techniques and algorithms for deep learning with PyTorch using real-world examples

Key Features

• Understand how to use PyTorch 1.x to build advanced neural network models
• Learn to perform a wide range of tasks by implementing deep learning algorithms and techniques
• Gain expertise in domains such as computer vision, NLP, Deep RL, Explainable AI, and much more

Book Description

Deep learning is driving the AI revolution, and PyTorch is making it easier than ever before for anyone to build deep learning applications. This PyTorch book will help you uncover expert techniques to get the most out of your data and build complex neural network models.

The book starts with a quick overview of PyTorch and explores using convolutional neural network (CNN) architectures for image classification. You'll then work with recurrent neural network (RNN) architectures and transformers for sentiment analysis. As you advance, you'll apply deep learning across different domains, such as music, text, and image generation using generative models and explore the world of generative adversarial networks (GANs). You'll not only build and train your own deep reinforcement learning models in PyTorch but also deploy PyTorch models to production using expert tips and techniques. Finally, you'll get to grips with training large models efficiently in a distributed manner, searching neural architectures effectively with AutoML, and rapidly prototyping models using PyTorch and fast.ai.

By the end of this PyTorch book, you'll be able to perform complex deep learning tasks using PyTorch to build smart artificial intelligence models.

What you will learn

• Implement text and music generating models using PyTorch
• Build a deep Q-network (DQN) model in PyTorch
• Export universal PyTorch models using Open Neural Network Exchange (ONNX)
• Become well-versed with rapid prototyping using PyTorch with fast.ai
• Perform neural architecture search effectively using AutoML
• Easily interpret machine learning (ML) models written in PyTorch using Captum
• Design ResNets, LSTMs, Transformers, and more using PyTorch
• Find out how to use PyTorch for distributed training using the torch.distributed API

Who this book is for

This book is for data scientists, machine learning researchers, and deep learning practitioners looking to implement advanced deep learning paradigms using PyTorch 1.x. Working knowledge of deep learning with Python programming is required.

Section 1: PyTorch Overview

This section includes a refresher on deep learning concepts, as well as PyTorch essentials. Upon completing this section, you will be able to identify how to train your own PyTorch models, as well as how to build a neural network model that generates text/captions as output when given images as input using PyTorch.

This section comprises the following chapters:

• Chapter 1, Overview of Deep Learning Using PyTorch
• Chapter 2, Combining CNNs and LSTMs

Chapter 1: Overview of Deep Learning using PyTorch

Deep learning is a class of machine learning methods that has revolutionized the way computers/machines are used to perform cognitive tasks in real life. Based on the mathematical concept of deep neural networks, deep learning uses large amounts of data to learn non-trivial relationships between inputs and outputs in the form of complex nonlinear functions. Some of the inputs and outputs, as demonstrated in Figure 1.1, could be the following:

• Input: An image of a text; output: Text
• Input: Text; output: A natural voice speaking the text
• Input: A natural voice speaking the text; output: Transcribed text

And so on. Here is a figure to support the preceding explanation:

Figure 1.1 – Deep learning model examples

Deep neural networks involve a lot of mathematical computations, linear algebraic equations, complex nonlinear functions, and various optimization algorithms. In order to build and train a deep neural network from scratch using a programming language such as Python, it would require us to write all the necessary equations, functions, and optimization schedules. Furthermore, the code would need to be written such that large amounts of data can be loaded efficiently, and training can be performed in a reasonable amount of time. This amounts to implementing several lower-level details each time we build a deep learning application.

Deep learning libraries such as Theano and TensorFlow, among various others, have been developed over the years to abstract these details out. PyTorch is one such Python-based deep learning library that can be used to build deep learning models.

TensorFlow was introduced as an open source deep learning Python (and C++) library by Google in late 2015, which revolutionized the field of applied deep learning. Facebook, in 2016, responded with its own open source deep learning library and called it Torch. Torch was initially used with a scripting language called Lua, and soon enough, the Python equivalent emerged called PyTorch. Around the same time, Microsoft released its own library – CNTK. Amidst the hot competition, PyTorch has been growing fast to become one of the most used deep learning libraries.

This book is meant to be a hands-on resource on some of the most advanced deep learning problems, how they are solved using complex deep learning architectures, and how PyTorch can be effectively used to build, train, and evaluate these complex models. While the book keeps PyTorch at the center, it also includes comprehensive coverage of some of the most recent and advanced deep learning models. The book is intended for data scientists, machine learning engineers, or researchers who have a working knowledge of Python and who, preferably, have used PyTorch before.

Due to the hands-on nature of this book, it is highly recommended to try the examples in each chapter by yourself on your computer to become proficient in writing PyTorch code. We begin with this introductory chapter and subsequently explore various deep learning problems and model architectures that will expose the various functionalities PyTorch has to offer.

This chapter will review some of the concepts behind deep learning and will provide a brief overview of the PyTorch library. We conclude this chapter with a hands-on exercise where we train a deep learning model using PyTorch.

The following topics will be covered in this chapter:

• A refresher on deep learning
• Exploring the PyTorch library
• Training a neural network using PyTorch

Technical requirements

We will be using Jupyter notebooks for all of our exercises. And the following is the list of Python libraries that shall be installed for this chapter using pip. For example, run pip install torch==1.4.0 on the command line:

jupyter==1.0.0

torch==1.4.0

torchvision==0.5.0

matplotlib==3.1.2

All code files relevant to this chapter are available at https://github.com/PacktPublishing/Mastering-PyTorch/tree/master/Chapter01.

A refresher on deep learning

Neural networks are a sub-type of machine learning methods that are inspired by the structure and function of the human brain....