Python Deep Learning

Valentino Zocca, Gianmario Spacagna, Daniel Slater, Peter Roelants

• Editore: Packt Publishing
• Anno: 2017
• ISBN:
  9781786460660
• Edizione: 1
• Argomento: Ciencia de la computación
• Categoria: Minería de datos

Take your machine learning skills to the next level by mastering Deep Learning concepts and algorithms using Python.About This Book• Explore and create intelligent systems using cutting-edge deep learning techniques• Implement deep learning algorithms and work with revolutionary libraries in Python• Get real-world examples and easy-to-follow tutorials on Theano, TensorFlow, H2O and moreWho This Book Is ForThis book is for Data Science practitioners as well as aspirants who have a basic foundational understanding of Machine Learning concepts and some programming experience with Python. A mathematical background with a conceptual understanding of calculus and statistics is also desired.What You Will Learn• Get a practical deep dive into deep learning algorithms• Explore deep learning further with Theano, Caffe, Keras, and TensorFlow• Learn about two of the most powerful techniques at the core of many practical deep learning implementations: Auto-Encoders and Restricted Boltzmann Machines• Dive into Deep Belief Nets and Deep Neural Networks• Discover more deep learning algorithms with Dropout and Convolutional Neural Networks• Get to know device strategies so you can use deep learning algorithms and libraries in the real worldIn DetailWith an increasing interest in AI around the world, deep learning has attracted a great deal of public attention. Every day, deep learning algorithms are used broadly across different industries.The book will give you all the practical information available on the subject, including the best practices, using real-world use cases. You will learn to recognize and extract information to increase predictive accuracy and optimize results.Starting with a quick recap of important machine learning concepts, the book will delve straight into deep learning principles using Sci-kit learn. Moving ahead, you will learn to use the latest open source libraries such as Theano, Keras, Google's TensorFlow, and H20. Use this guide to uncover the difficulties of pattern recognition, scaling data with greater accuracy and discussing deep learning algorithms and techniques.Whether you want to dive deeper into Deep Learning, or want to investigate how to get more out of this powerful technology, you'll find everything inside.Style and approachPython Machine Learning by example follows practical hands on approach. It walks you through the key elements of Python and its powerful machine learning libraries with the help of real world projects.

Python Deep Learning

---

Table of Contents

Python Deep Learning

Credits

About the Authors

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Customer Feedback

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Machine Learning – An Introduction

What is machine learning?

Different machine learning approaches

Supervised learning

Unsupervised learning

Reinforcement learning

Steps Involved in machine learning systems

Brief description of popular techniques/algorithms

Linear regression

Decision trees

K-means

Naïve Bayes

Support vector machines

The cross-entropy method

Neural networks

Deep learning

Applications in real life

A popular open source package

Summary

2. Neural Networks

Why neural networks?

Fundamentals

Neurons and layers

Different types of activation function

The back-propagation algorithm

Linear regression

Logistic regression

Back-propagation

Applications in industry

Signal processing

Medical

Autonomous car driving

Business

Pattern recognition

Speech production

Code example of a neural network for the function xor

Summary

3. Deep Learning Fundamentals

What is deep learning?

Fundamental concepts

Feature learning

Deep learning algorithms

Deep learning applications

Speech recognition

Object recognition and classification

GPU versus CPU

Popular open source libraries – an introduction

Theano

TensorFlow

Keras

Sample deep neural net code using Keras

Summary

4. Unsupervised Feature Learning

Autoencoders

Network design

Regularization techniques for autoencoders

Denoising autoencoders

Contractive autoencoders

Sparse autoencoders

Summary of autoencoders

Restricted Boltzmann machines

Hopfield networks and Boltzmann machines

Boltzmann machine

Restricted Boltzmann machine

Implementation in TensorFlow

Deep belief networks

Summary

5. Image Recognition

Similarities between artificial and biological models

Intuition and justification

Convolutional layers

Stride and padding in convolutional layers

Pooling layers

Dropout

Convolutional layers in deep learning

Convolutional layers in Theano

A convolutional layer example with Keras to recognize digits

A convolutional layer example with Keras for cifar10

Pre-training

Summary

6. Recurrent Neural Networks and Language Models

Recurrent neural networks

RNN — how to implement and train

Backpropagation through time

Vanishing and exploding gradients

Long short term memory

Language modeling

Word-based models

N-grams

Neural language models

Character-based model

Preprocessing and reading data

LSTM network

Training

Sampling

Example training

Speech recognition

Speech recognition pipeline

Speech as input data

Preprocessing

Acoustic model

Deep belief networks

Recurrent neural networks

CTC

Attention-based models

Decoding

End-to-end models

Summary

Bibliography

7. Deep Learning for Board Games

Early game playing AI

Using the min-max algorithm to value game states

Implementing a Python Tic-Tac-Toe game

Learning a value function

Training AI to master Go

Upper confidence bounds applied to trees

Deep learning in Monte Carlo Tree Search

Quick recap on reinforcement learning

Policy gradients for learning policy functions

Policy gradients in AlphaGo

Summary

8. Deep Learning for Computer Games

A supervised learning approach to games

Applying genetic algorithms to playing games

Q-Learning

Q-function

Q-learning in action

Dynamic games

Experience replay

Epsilon greedy

Atari Breakout

Atari Breakout random benchmark

Preprocessing the screen

Creating a deep convolutional network

Convergence issues in Q-learning

Policy gradients versus Q-learning

Actor-critic methods

Baseline for variance reduction

Generalized advantage estimator

Asynchronous methods

Model-based approaches

Summary

9. Anomaly Detection

What is anomaly and outlier detection?

Real-world applications of anomaly detection

Popular shallow machine learning techniques

Data modeling

Detection modeling

Anomaly detection using deep auto-encoders

H2O

Getting started with H2O

Examples

MNIST digit anomaly recognition

Electrocardiogram pulse detection

Summary

10. Building a Production-Ready Intrusion Detection System

What is a data product?

Training

Weights initialization

Parallel SGD using HOGWILD!

Adaptive learning

Rate annealing

Momentum

Nesterov's acceleration

Newton's method

Adagrad

Adadelta

Distributed learning via Map/Reduce

Sparkling Water

Testing

Model validation

Labeled Data

Unlabeled Data

Summary of validation

Hyper-parameters tuning

End-to-end evaluation

A/B Testing

A summary of testing

Deployment

POJO model export

Anomaly score APIs

A summary of deployment

Summary

Index

Python Deep Learning

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2017

Production reference: 1270417

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78646-445-3

www.packtpub.com

Credits

Authors

Valentino Zocca

Gianmario Spacagna

Daniel Slater

Peter Roelants

Reviewer

Max Pumperla

Commissioning Editor

Akram Hussain

Acquisition Editor

Vinay Argekar

Content Development Editor

Mayur Pawanikar

Technical Editor

Vivek Arora

Copy Editor

Safis Editing

Project Coordinator

Nidhi Joshi

Proofreader

Safis Editing

Indexer

Francy Puthiry

Graphics

Tania Dutta

Production Coordinator

Arvindkumar Gupta

About the Authors

Valentino Zocca graduated with a PhD in mathematics from the University of Maryland, USA, with a dissertation in symplectic geometry, after having graduated with a laurea in mathematics from the University of Rome. He spent a semester at the University of Warwick. After a post-doc in Paris, Valentino started working on high-tech projects in the Washington, D.C. area and played a central role in the design, development, and realization of an advanced stereo 3D Earth visualization software with head tracking at Autometric, a company later bought by Boeing. At Boeing, he developed many mathematical algorithms and predictive models, and using Hadoop, he has also automated several satellite-imagery visualization programs. He has since become an expert on machine learning and deep learning and has worked at the U.S. Census Bureau...

Indice dei contenuti

• Python Deep Learning