Azure Data Scientist Associate Certification Guide

Andreas Botsikas, Michael Hlobil

• Publisher: Packt Publishing
• Year: 2021
• eBook ISBN: 9781800561267
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Develop the skills you need to run machine learning workloads in Azure and pass the DP-100 exam with easeKey Features• Create end-to-end machine learning training pipelines, with or without code• Track experiment progress using the cloud-based MLflow-compatible process of Azure ML services• Operationalize your machine learning models by creating batch and real-time endpointsBook DescriptionThe Azure Data Scientist Associate Certification Guide helps you acquire practical knowledge for machine learning experimentation on Azure. It covers everything you need to pass the DP-100 exam and become a certified Azure Data Scientist Associate.Starting with an introduction to data science, you'll learn the terminology that will be used throughout the book and then move on to the Azure Machine Learning (Azure ML) workspace. You'll discover the studio interface and manage various components, such as data stores and compute clusters.Next, the book focuses on no-code and low-code experimentation, and shows you how to use the Automated ML wizard to locate and deploy optimal models for your dataset. You'll also learn how to run end-to-end data science experiments using the designer provided in Azure ML Studio.You'll then explore the Azure ML Software Development Kit (SDK) for Python and advance to creating experiments and publishing models using code. The book also guides you in optimizing your model's hyperparameters using Hyperdrive before demonstrating how to use responsible AI tools to interpret and debug your models. Once you have a trained model, you'll learn to operationalize it for batch or real-time inferences and monitor it in production.By the end of this Azure certification study guide, you'll have gained the knowledge and the practical skills required to pass the DP-100 exam.What you will learn• Create a working environment for data science workloads on Azure• Run data experiments using Azure Machine Learning services• Create training and inference pipelines using the designer or code• Discover the best model for your dataset using Automated ML• Use hyperparameter tuning to optimize trained models• Deploy, use, and monitor models in production• Interpret the predictions of a trained modelWho this book is forThis book is for developers who want to infuse their applications with AI capabilities and data scientists looking to scale their machine learning experiments in the Azure cloud. Basic knowledge of Python is needed to follow the code samples used in the book. Some experience in training machine learning models in Python using common frameworks like scikit-learn will help you understand the content more easily.

Section 1: Starting your cloud-based data science journey

An Azure Data Scientist Associate is a subject matter expert with applied data science and machine learning knowledge, able to run machine learning workloads on Azure. The responsibilities for this role include planning and creating a suitable working environment for data science workloads on Azure. In this section, you will get an overview of the data science and machine learning terminology used throughout the book. You will also learn how to provision and configure the Azure Machine Learning services, the de facto environment for running machine learning experiments in the Azure cloud.

This section comprises of the following chapters:

• Chapter 1, An Overview of Modern Data Science
• Chapter 2, Deploying Azure Machine Learning Workspace Resources
• Chapter 3, Azure Machine Learning Studio Components
• Chapter 4, Configuring the Workspace

Chapter 1: An Overview of Modern Data Science

Data science has its roots in the early eighteenth century and has gained tremendous popularity during the last couple of decades.

In this book, you will learn how to run a data science project within Azure, the Microsoft public cloud infrastructure. You will gain all skills needed to become a certified Azure Data Scientist Associate. You will start with this chapter, which gives some foundational terminology used throughout the book. Then, you will deep dive into Azure Machine Learning (AzureML) services. You will start by provisioning a workspace. You will then work on the no-code, low-code experiences build in the AzureML Studio web interface. Then, you will deep dive into the code-first data science experimentation, working with the AzureML Software Development Kit (SDK).

In this chapter, you will learn some fundamental data science-related terms needed for the DP 100 exam. You will start by understanding the typical life cycle of a data science project. You will then read about big data and how Apache Spark technology enables you to train machine learning models against them. Then, you will explore what the DevOps mindset is and how it can help you become a member of a highly efficient, multi-disciplinary, agile team that builds machine learning-enhanced products.

In this chapter, we are going to cover the following main topics:

• The evolution of data science
• Working on a data science project
• Using Spark in data science
• Adopting the DevOps mindset

The evolution of data science

If you try to find the roots of the data science practices, you will probably end up discovering evidence at the beginning of civilization. In the eighteenth century, governments were gathering demographic and financial data for taxation purposes, a practice called statistics. As years progressed, the use of this term was expanded to include the summarization and analysis of the data collected. In 1805, Adrien-Marie Legendre, a French mathematician, published a paper describing the least squares to fit linear equations, although most people credit Carl Friedrich Gauss for the complete description he published a couple of years later. In 1900, Karl Pearson published in the Philosophical Magazine his observations on the chi-square statistic, a cornerstone in data science for hypothesis testing. In 1962, John Tukey, the scientist famous for the fast Fourier transformation and the box plot, published a paper expressing his passion for data analysis and how statistics needed to evolve into a new science.

On the other hand, with the rise of informatics in the middle of the twentieth century, the field of Artificial Intelligence (AI) was introduced in 1955 by John McCarthy as the official term for thinking machines. AI is a field of computer science that develops systems that can imitate intelligent human behavior. Using programming languages such as Information Processing Language (IPL) and LISt Processor (LISP), developers were writing programs that could manipulate lists and various other data structures to solve complex problems. In 1955, Arthur Samuel's checkers player was the first piece of software that would learn from the games it has already played by storing board states and the chance of winning if ending up in that state in a cache. This checkers program may have been the first example of machine learning, a subfield of AI that utilizes historical data and the patterns encoded in the data to train models and enable systems to mimic human tasks without explicitly coding the entire logic. In fact, you can think of machine learning models as software code that is generated by training an algorithm against a dataset to recognize certain types of patterns.

In 2001, William S. Cleveland published the first article in which the term data science was used in the way we refer to it today, a science at the intersection of statistics, data analysis, and informatics that tries to explain phenomena based on data.

Although most people correlate data science with machine learning, data science has a much broader scope, which includes the analysis and preparation of data before the actual machine learning model training process, as you will see in the next section.

Working on a data science project

A data science project aims to infuse an application with intelligence extracted from data. In this section, you will discover the common tasks and key considerations needed within such a project. There are quite a few well-established life cycle processes, such as Team Data Science Process (TDSP) and Cross-Industry Standard Process for Data Mining (CRISP-DM), that describe the iterative stages executed in a typical project. The most common stages are shown in Figure 1.1:

Figure 1.1 – The iterative stages of a data science project

Although the diagram shows some indicative flows between the phases, you are free to jump from one phase to any other if needed. Moreover, this approach is iterative, and the data science team should go through multiple iterations, improving its business understanding and the resulting model until the success criteria are met. You will read more about the benefits of an iterative process in this chapter's Adopting the DevOps mindset section. The data science process starts from the business understanding phase, something you will read more about in the next section.

Understanding of the business problem

The first stage in a data science project is that of business understanding. In this stage, the data science team collaborates with the business stakeholders to define a short, straightforward question that machine learning will try to answer.

Figure 1.2 shows the five most frequent questions that machine learning can answer:

Figure 1.2 – Five questions machine learning can answer

Behind each of those questions, there is a group of modeling techniques you will use:

• Regression models allow you to predict a numeric value based on one or more features. For example, in Chapter 8, Experimenting with Python Code, you will be trying to predict a numeric value based on 10 measurements that were taken one year before the value you are trying to predict. Training a regression model is a supervised machine learning task, meaning that you need to provide enough sample data to train the model ...

Table of contents

• Azure Data Scientist Associate Certification Guide
• Contributors
• Preface
• Section 1: Starting your cloud-based data science journey
• Chapter 1: An Overview of Modern Data Science
• Chapter 2: Deploying Azure Machine Learning Workspace Resources
• Chapter 3: Azure Machine Learning Studio Components
• Chapter 4: Configuring the Workspace
• Section 2: No code data science experimentation
• Chapter 5: Letting the Machines Do the Model Training
• Chapter 6: Visual Model Training and Publishing
• Section 3: Advanced data science tooling and capabilities
• Chapter 7: The AzureML Python SDK
• Chapter 8: Experimenting with Python Code
• Chapter 9: Optimizing the ML Model
• Chapter 10: Understanding Model Results
• Chapter 11: Working with Pipelines
• Chapter 12: Operationalizing Models with Code
• Other Books You May Enjoy