![]()
Table of Contents
Python Deep Learning
Credits
About the Authors
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and more
Why subscribe?
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Machine Learning â An Introduction
What is machine learning?
Different machine learning approaches
Supervised learning
Unsupervised learning
Reinforcement learning
Steps Involved in machine learning systems
Brief description of popular techniques/algorithms
Linear regression
Decision trees
K-means
NaĂŻve Bayes
Support vector machines
The cross-entropy method
Neural networks
Deep learning
Applications in real life
A popular open source package
Summary
2. Neural Networks
Why neural networks?
Fundamentals
Neurons and layers
Different types of activation function
The back-propagation algorithm
Linear regression
Logistic regression
Back-propagation
Applications in industry
Signal processing
Medical
Autonomous car driving
Business
Pattern recognition
Speech production
Code example of a neural network for the function xor
Summary
3. Deep Learning Fundamentals
What is deep learning?
Fundamental concepts
Feature learning
Deep learning algorithms
Deep learning applications
Speech recognition
Object recognition and classification
GPU versus CPU
Popular open source libraries â an introduction
Theano
TensorFlow
Keras
Sample deep neural net code using Keras
Summary
4. Unsupervised Feature Learning
Autoencoders
Network design
Regularization techniques for autoencoders
Denoising autoencoders
Contractive autoencoders
Sparse autoencoders
Summary of autoencoders
Restricted Boltzmann machines
Hopfield networks and Boltzmann machines
Boltzmann machine
Restricted Boltzmann machine
Implementation in TensorFlow
Deep belief networks
Summary
5. Image Recognition
Similarities between artificial and biological models
Intuition and justification
Convolutional layers
Stride and padding in convolutional layers
Pooling layers
Dropout
Convolutional layers in deep learning
Convolutional layers in Theano
A convolutional layer example with Keras to recognize digits
A convolutional layer example with Keras for cifar10
Pre-training
Summary
6. Recurrent Neural Networks and Language Models
Recurrent neural networks
RNN â how to implement and train
Backpropagation through time
Vanishing and exploding gradients
Long short term memory
Language modeling
Word-based models
N-grams
Neural language models
Character-based model
Preprocessing and reading data
LSTM network
Training
Sampling
Example training
Speech recognition
Speech recognition pipeline
Speech as input data
Preprocessing
Acoustic model
Deep belief networks
Recurrent neural networks
CTC
Attention-based models
Decoding
End-to-end models
Summary
Bibliography
7. Deep Learning for Board Games
Early game playing AI
Using the min-max algorithm to value game states
Implementing a Python Tic-Tac-Toe game
Learning a value function
Training AI to master Go
Upper confidence bounds applied to trees
Deep learning in Monte Carlo Tree Search
Quick recap on reinforcement learning
Policy gradients for learning policy functions
Policy gradients in AlphaGo
Summary
8. Deep Learning for Computer Games
A supervised learning approach to games
Applying genetic algorithms to playing games
Q-Learning
Q-function
Q-learning in action
Dynamic games
Experience replay
Epsilon greedy
Atari Breakout
Atari Breakout random benchmark
Preprocessing the screen
Creating a deep convolutional network
Convergence issues in Q-learning
Policy gradients versus Q-learning
Actor-critic methods
Baseline for variance reduction
Generalized advantage estimator
Asynchronous methods
Model-based approaches
Summary
9. Anomaly Detection
What is anomaly and outlier detection?
Real-world applications of anomaly detection
Popular shallow machine learning techniques
Data modeling
Detection modeling
Anomaly detection using deep auto-encoders
H2O
Getting started with H2O
Examples
MNIST digit anomaly recognition
Electrocardiogram pulse detection
Summary
10. Building a Production-Ready Intrusion Detection System
What is a data product?
Training
Weights initialization
Parallel SGD using HOGWILD!
Adaptive learning
Rate annealing
Momentum
Nesterov's acceleration
Newton's method
Adagrad
Adadelta
Distributed learning via Map/Reduce
Sparkling Water
Testing
Model validation
Labeled Data
Unlabeled Data
Summary of validation
Hyper-parameters tuning
End-to-end evaluation
A/B Testing
A summary of testing
Deployment
POJO model export
Anomaly score APIs
A summary of deployment
Summary
Index
![]()
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: April 2017
Production reference: 1270417
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78646-445-3
www.packtpub.com
![]()
Authors
Valentino Zocca
Gianmario Spacagna
Daniel Slater
Peter Roelants
Reviewer
Max Pumperla
Commissioning Editor
Akram Hussain
Acquisition Editor
Vinay Argekar
Content Development Editor
Mayur Pawanikar
Technical Editor
Vivek Arora
Copy Editor
Safis Editing
Project Coordinator
Nidhi Joshi
Proofreader
Safis Editing
Indexer
Francy Puthiry
Graphics
Tania Dutta
Production Coordinator
Arvindkumar Gupta
![]()
Valentino Zocca graduated with a PhD in mathematics from the University of Maryland, USA, with a dissertation in symplectic geometry, after having graduated with a laurea in mathematics from the University of Rome. He spent a semester at the University of Warwick. After a post-doc in Paris, Valentino started working on high-tech projects in the Washington, D.C. area and played a central role in the design, development, and realization of an advanced stereo 3D Earth visualization software with head tracking at Autometric, a company later bought by Boeing. At Boeing, he developed many mathematical algorithms and predictive models, and using Hadoop, he has also automated several satellite-imagery visualization programs. He has since become an expert on machine learning and deep learning and has worked at the U.S. Census Bureau...