Python: Advanced Guide to Artificial Intelligence

Expert machine learning systems and intelligent agents using Python

Giuseppe Bonaccorso, Armando Fandango, Rajalingappaa Shanmugamani

• Editore: Packt Publishing
• Anno: 2018
• ISBN:
  9781789951721
• Edizione: 1
• Argomento: Computer Science
• Categoria: Neural Networks

Demystify the complexity of machine learning techniques and create evolving, clever solutions to solve your problems

Key Features

• Master supervised, unsupervised, and semi-supervised ML algorithms and their implementation
• Build deep learning models for object detection, image classification, similarity learning, and more
• Build, deploy, and scale end-to-end deep neural network models in a production environment

Book Description

This Learning Path is your complete guide to quickly getting to grips with popular machine learning algorithms. You'll be introduced to the most widely used algorithms in supervised, unsupervised, and semi-supervised machine learning, and learn how to use them in the best possible manner. Ranging from Bayesian models to the MCMC algorithm to Hidden Markov models, this Learning Path will teach you how to extract features from your dataset and perform dimensionality reduction by making use of Python-based libraries.

You'll bring the use of TensorFlow and Keras to build deep learning models, using concepts such as transfer learning, generative adversarial networks, and deep reinforcement learning. Next, you'll learn the advanced features of TensorFlow1.x, such as distributed TensorFlow with TF clusters, deploy production models with TensorFlow Serving. You'll implement different techniques related to object classification, object detection, image segmentation, and more.

By the end of this Learning Path, you'll have obtained in-depth knowledge of TensorFlow, making you the go-to person for solving artificial intelligence problems

This Learning Path includes content from the following Packt products:

• Mastering Machine Learning Algorithms by Giuseppe Bonaccorso
• Mastering TensorFlow 1.x by Armando Fandango
• Deep Learning for Computer Vision by Rajalingappaa Shanmugamani

What you will learn

• Explore how an ML model can be trained, optimized, and evaluated
• Work with Autoencoders and Generative Adversarial Networks
• Explore the most important Reinforcement Learning techniques
• Build end-to-end deep learning (CNN, RNN, and Autoencoders) models

Who this book is for

This Learning Path is for data scientists, machine learning engineers, artificial intelligence engineers who want to delve into complex machine learning algorithms, calibrate models, and improve the predictions of the trained model.

You will encounter the advanced intricacies and complex use cases of deep learning and AI. A basic knowledge of programming in Python and some understanding of machine learning concepts are required to get the best out of this Learning Path.

EM Algorithm and Applications

In this chapter, we are going to introduce a very important algorithmic framework for many statistical learning tasks: the EM algorithm. Contrary to its name, this is not a method to solve a single problem, but a methodology that can be applied in several contexts. Our goal is to explain the rationale and show the mathematical derivation, together with some practical examples. In particular, we are going to discuss the following topics:

• Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP) learning approaches
• The EM algorithm with a simple application for the estimation of unknown parameters
• The Gaussian mixture algorithm, which is one the most famous EM applications
• Factor analysis
• Principal Component Analysis (PCA)
• Independent Component Analysis (ICA)
• A brief explanation of the Hidden Markov Models (HMMs) forward-backward algorithm considering the EM steps

MLE and MAP learning

Let's suppose we have a data generating process p_data, used to draw a dataset X:

In many statistical learning tasks, our goal is to find the optimal parameter set θ according to a maximization criterion. The most common approach is based on the likelihood and is called MLE. In this case, the optimal set θ is found as follows:

This approach has the advantage of being unbiased by wrong preconditions, but, at the same time, it excludes any possibility of incorporating prior knowledge into the model. It simply looks for the best θ in a wider subspace, so that p(X|θ) is maximized. Even if this approach is almost unbiased, there's a higher probability of finding a sub-optimal solution that can also be quite different from a reasonable (even if not sure) prior. After all, several models are too complex to allow us to define a suitable prior probability (think, for example, of reinforcement learning strategies where there's a huge number of complex states). Therefore, MLE offers the most reliable solution. Moreover, it's possible to prove that the MLE of a parameter θ converges in probability to the real value:

On the other hand, if we consider Bayes' theorem, we can derive the following relation:

The posterior probability, p(θ|X), is obtained using both the likelihood and a prior probability, p(θ), and hence takes into account existing knowledge encoded in p(θ). The choice to maximize p(θ|X) is called the MAP approach and it's often a good alternative to MLE when it's possible to formulate trustworthy priors or, as in the case of Latent Dirichlet Allocation (LDA), where the model is on purpose based on some specific prior assumptions.

Unfortunately, a wrong or incomplete prior distribution can bias the model leading to unacceptable results. For this reason, MLE is often the default choice even when it's possible to formulate reasonable assumptions on the structure of p(θ). To understand the impact of a prior on an estimation, let's consider to have observed n=1000 binomial distributed (θ corresponds to the parameter p) experiments and k=800 had a successful outcome. The likelihood is as follows:

For simplicity, let's compute the log-likelihood:

If we compute the derivative with respect to θ and set it equal to zero, we get the following:

So the MLE for θ is 0.8, which is coherent with the observations (we can say that after observing 1000 experiments with 800 successful outcomes, p(X|Success)=0.8). If we have only the data X, we could say that a success is more likely than a failure because 800 out of 1000 experiments are p...

Indice dei contenuti

• Title Page
• Copyright and Credits
• About Packt
• Contributors
• Preface
• Machine Learning Model Fundamentals
• Introduction to Semi-Supervised Learning
• Graph-Based Semi-Supervised Learning
• Bayesian Networks and Hidden Markov Models
• EM Algorithm and Applications
• Hebbian Learning and Self-Organizing Maps
• Clustering Algorithms
• Advanced Neural Models
• Classical Machine Learning with TensorFlow
• Neural Networks and MLP with TensorFlow and Keras
• RNN with TensorFlow and Keras
• CNN with TensorFlow and Keras
• Autoencoder with TensorFlow and Keras
• TensorFlow Models in Production with TF Serving
• Deep Reinforcement Learning
• Generative Adversarial Networks
• Distributed Models with TensorFlow Clusters
• Debugging TensorFlow Models
• Tensor Processing Units
• Getting Started
• Image Classification
• Image Retrieval
• Object Detection
• Semantic Segmentation
• Similarity Learning
• Other Books You May Enjoy