Machine Learning for OpenCV

Michael Beyeler

• Éditeur: Packt Publishing
• Année: 2017
• ISBN:
  9781783980291
• Édition: 1
• Sujet: Computer Science
• Sous-sujet: Computer Vision & Pattern Recognition

Expand your OpenCV knowledge and master key concepts of machine learning using this practical, hands-on guide.About This Book• Load, store, edit, and visualize data using OpenCV and Python• Grasp the fundamental concepts of classification, regression, and clustering• Understand, perform, and experiment with machine learning techniques using this easy-to-follow guide• Evaluate, compare, and choose the right algorithm for any taskWho This Book Is ForThis book targets Python programmers who are already familiar with OpenCV; this book will give you the tools and understanding required to build your own machine learning systems, tailored to practical real-world tasks.What You Will Learn• Explore and make effective use of OpenCV's machine learning module• Learn deep learning for computer vision with Python• Master linear regression and regularization techniques• Classify objects such as flower species, handwritten digits, and pedestrians• Explore the effective use of support vector machines, boosted decision trees, and random forests• Get acquainted with neural networks and Deep Learning to address real-world problems• Discover hidden structures in your data using k-means clustering• Get to grips with data pre-processing and feature engineeringIn DetailMachine learning is no longer just a buzzword, it is all around us: from protecting your email, to automatically tagging friends in pictures, to predicting what movies you like. Computer vision is one of today's most exciting application fields of machine learning, with Deep Learning driving innovative systems such as self-driving cars and Google's DeepMind.OpenCV lies at the intersection of these topics, providing a comprehensive open-source library for classic as well as state-of-the-art computer vision and machine learning algorithms. In combination with Python Anaconda, you will have access to all the open-source computing libraries you could possibly ask for.Machine learning for OpenCV begins by introducing you to the essential concepts of statistical learning, such as classification and regression. Once all the basics are covered, you will start exploring various algorithms such as decision trees, support vector machines, and Bayesian networks, and learn how to combine them with other OpenCV functionality. As the book progresses, so will your machine learning skills, until you are ready to take on today's hottest topic in the field: Deep Learning.By the end of this book, you will be ready to take on your own machine learning problems, either by building on the existing source code or developing your own algorithm from scratch!Style and approachOpenCV machine learning connects the fundamental theoretical principles behind machine learning to their practical applications in a way that focuses on asking and answering the right questions. This book walks you through the key elements of OpenCV and its powerful machine learning classes, while demonstrating how to get to grips with a range of models.

Using Deep Learning to Classify Handwritten Digits

Let's now return to supervised learning and discuss a family of algorithms known as artificial neural networks. Early studies of neural networks go back to the 1940s when Warren McCulloch and Walter Pitts first described how biological nerve cells (or neurons) in the brain might work. More recently, artificial neural networks have seen a revival under the buzzword deep learning, which powers state-of-the-art technologies, such as Google's DeepMind and Facebook's DeepFace algorithms.

In this chapter, we want to wrap our heads around some simple versions of artificial neural nets, such as the McCulloch-Pitts neuron, the perceptron, and the multilayer perceptron. Once we have familiarized ourselves with the basics, we will be ready to implement a more sophisticated deep neural net in order to classify handwritten digits from the popular MNIST database (short for Mixed National Institute of Standards and Technology database). For this, we will be making use of Keras, a high-level neural network library, which is also frequently used by researchers and tech companies.

Along the way, we want to get answers to the following questions:

• How do I implement perceptrons and multilayer perceptrons in OpenCV?
• What is the difference between stochastic and batch gradient descent, and how does it fit in with backpropagation?
• How do I know what size my neural net should be?
• How can I use Keras to build sophisticated deep neural networks?

Excited? Then let's go!

Understanding the McCulloch-Pitts neuron

In 1943, Warren McCulloch and Walter Pitts published a mathematical description of neurons as they were believed to operate in the brain. A neuron receives input from other neurons through connections on its dendritic tree, which are integrated to produce an output at the cell body (or soma). The output is then communicated to other neurons via a long wire (or axon), which eventually branches out to make one or more connections (at axon terminals) on the dendritic tree of other neurons. An example neuron is shown in the following figure:

Schematic of a neuron (nerve cell). Adapted from a figure by Looxix at French Wikipedia (CC BY-SA 3.0)

McCulloch and Pitts described the inner workings of such a neuron as a simple logic gate that would be either on or off, depending on the input it receives on its dendritic tree. Specifically, the neuron would sum up all of its inputs, and if the sum exceeded a certain threshold, an output signal would be generated and passed on by the axon.

However, today we know that real neurons are much more complicated than that. Biological neurons perform intricate nonlinear mathematical operations on thousands of inputs and can change their responsiveness dynamically depending on the context, importance, or novelty of the input signal. You can think of real neurons being as complex as computers and of the human brain being as complex as the internet.

Let's consider a simple artificial neuron that receives exactly two inputs, x₀ and x₁. The job of the artificial neuron is to calculate a sum of the two inputs (usually in the form of a weighted sum), and if this sum exceeds a certain threshold (often zero), the neuron will be considered active and output a one; else it will be considered silent and output a minus one (or zero). In more mathematical terms, the output y of this McCulloch-Pitts neuron can be described as follows:

In the preceding equation, w₀ and w₁ are weight coefficients, which, together with x₀ and x₁, make up the weighted sum. In textbooks, the two different scenarios where the output y is either +1 and -1 would often be masked by an activation function ϕ, which could take on two different values:

Here, we introduce a new variable z (the so-called net input), which is equivalent to the weighted sum: z = w₀x₀ + w₁x₁. The weighted sum is then compared to a threshold θ to determine the value of ϕ and subsequently the value of y. Apart from that, these two equations say exactly the same thing as the preceding one.

If these equations look strangely familiar, you might be reminded of Chapter 3, A Taste of Machine Learning, when we were talking about linear classifiers.

And you are right, a McCulloch-Pitts neuron is essentially a linear, binary classifier!

You can think of it this way: x₀ and x₁ are the input features, w₀ and w₁ are weights to be learned, and the classification is performed by the activation function ϕ. If we do a good job of learning the weights, which we would do with the help of a suitable training set, we could classify data as positive or negative samples. In this scenario, ϕ(z)=θ would act as the decision boundary.

This might all make more sense with the help of the following figure:

An artificial neuron's activation function (left) used to classify some example data (right)

On the left, you can see the neuron's activation function ϕ plotted against z. Remember that z is nothing more than the weighted sum of the two inputs x₀ and x_1. The rule is that as long as the weighted sum is below some threshold θ, the output of the neuron is -1; above θ, the output is +1. On the right, you can see the decision boundary denoted by ϕ(z)=θ, which splits the data into two regimes, ϕ(z)<θ (where all data points are predicted to be negative samples) and ϕ(z)>θ (where all data points are predicted to be positive samples).

The decision boundary does not need to be vertical or horizontal, it can be tilted as shown in the preceding figure. But in the case of a single McCulloch-Pitts neuron, the decision boundary will always be a straight line.

Of course, the magic lies with learning the weight coefficients w₀ and w₁ such that the decision boundary comes to lie right between all positive and all negative data points. To train a neural network, we generally need three things:

• Training data: It is no surprise to learn that we need some data samples with which the effectiveness of our classifier can be verified.
• Cost function (also known as loss function): A cost function provides a measure of how good the current weight coefficients are. There is a wide range of cost functions available, which we will talk about towards the end of this chapter. One solution is to count the number of misclassifications. Another one is to calculate the su...

Table des matières

• Title Page
• Copyright
• Credits
• Foreword
• About the Author
• About the Reviewers
• www.PacktPub.com
• Customer Feedback
• Dedication
• Preface
• A Taste of Machine Learning
• Working with Data in OpenCV and Python
• First Steps in Supervised Learning
• Representing Data and Engineering Features
• Using Decision Trees to Make a Medical Diagnosis
• Detecting Pedestrians with Support Vector Machines
• Implementing a Spam Filter with Bayesian Learning
• Discovering Hidden Structures with Unsupervised Learning
• Using Deep Learning to Classify Handwritten Digits
• Combining Different Algorithms into an Ensemble
• Selecting the Right Model with Hyperparameter Tuning
• Wrapping Up