Hands-On Data Analysis with NumPy and pandas

Implement Python packages from data manipulation to processing

Curtis Miller

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN: 9781789530797
• eBook ISBN: 9781789534245
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Modelling & Design
• Index: Computer Science

Get to grips with the most popular Python packages that make data analysis possible

Key Features

• Explore the tools you need to become a data analyst
• Discover practical examples to help you grasp data processing concepts
• Walk through hierarchical indexing and grouping for data analysis

Book Description

Python, a multi-paradigm programming language, has become the language of choice for data scientists for visualization, data analysis, and machine learning.

Hands-On Data Analysis with NumPy and Pandas starts by guiding you in setting up the right environment for data analysis with Python, along with helping you install the correct Python distribution. In addition to this, you will work with the Jupyter notebook and set up a database. Once you have covered Jupyter, you will dig deep into Python's NumPy package, a powerful extension with advanced mathematical functions. You will then move on to creating NumPy arrays and employing different array methods and functions. You will explore Python's pandas extension which will help you get to grips with data mining and learn to subset your data. Last but not the least you will grasp how to manage your datasets by sorting and ranking them.

By the end of this book, you will have learned to index and group your data for sophisticated data analysis and manipulation.

What you will learn

• Understand how to install and manage Anaconda
• Read, sort, and map data using NumPy and pandas
• Find out how to create and slice data arrays using NumPy
• Discover how to subset your DataFrames using pandas
• Handle missing data in a pandas DataFrame
• Explore hierarchical indexing and plotting with pandas

Who this book is for

Hands-On Data Analysis with NumPy and Pandas is for you if you are a Python developer and want to take your first steps into the world of data analysis. No previous experience of data analysis is required to enjoy this book.

Operations on NumPy Arrays

Now that we know how to create NumPy arrays, we can discuss the important topic of slicing NumPy arrays in order to access and manipulate subsets of array data. In this chapter, we will cover what every NumPy user should know about array slicing, arithmetic, linear algebra with arrays, and employing array methods and functions.

Selecting elements explicitly

If you know how to select subsets of Python lists, you know most of what you need to know about ndarray slicing. The elements of the array being indexed that correspond to the elements of the indexing object are returned in a new array. The most important aspect of indexing is to remember that there is more than one dimension, and the indexing method should be able to handle these other dimensions.

Remember the following points while selecting elements explicitly:

Separate the indexing objects for different dimensions with a comma; the object before the first comma shows how the first dimension is indexed. After the first comma comes the index for the second dimension, after the second comma comes the index for the third dimension, and so on.

Slicing arrays with colons

Indexing ndarray objects using colons works like indexing lists using colons. Just remember there are multiple dimensions now. Remember that when the spot before or after the colon is left blank, Python treats the index as extending to either the beginning or the end of the dimension. A second colon can be specified to instruct Python to, say, skip every other row or reverse the order of rows, depending on the number under the second colon.

The following points need to be remembered when slicing arrays with colons:

Let's see an example. First we load in NumPy and create an array:

Notice that what we created is a three-dimensional array. Now, this array is a bit complicated, so let's work with a two-dimensional 3 x 3 array instead:

We used the copy method here. A new object was returned, but that object isn't a new copy of the array; it is a view of the array's contents. So if we wish to create an independent copy, we will need to use the copy method when slicing as well, as we have seen before.

If we want to change an entry in this new array, say the second row and the second column's contents to Atilla, then we change this new array:

But we have not changed the original contents:

So, these are two independent copies of the data in the first array. Now let's explore some other slicing schemes.

Here, we see indexing using lists. What we do is create a list that corresponds to the first coordinate of every element from the object we wish to capture, and then we have a list for the second coordinate. So 1 and 0 correspond to one element that we wish to select; if this were a three-dimensional object, we would need a third list for the third coordinate:

We select elements from the upper-left corner using slicers:

Now, let's select elements from the middle column:

And, let's select elements from the middle column but we will not flatten the matrix and we'll keep its shape:

This is a one-dimensional object, but here we want a two-dimensional object. While it has only one column, it has one column and one row, as opposed to having only one row and columns don't make sense. Now let's select the last two rows of the middle column:

We reverse the row order:

If you look at the original object, you will see that these rules are happening in reverse order (compared to how they originally were ordered) and this means selecting odd number columns:

We can go to a more complex three-dimensional array and see si...

Table of contents

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Setting Up a Python Data Analysis Environment
• Diving into NumPY
• Operations on NumPy Arrays
• pandas are Fun! What is pandas?
• Arithmetic, Function Application, and Mapping with pandas
• Managing, Indexing, and Plotting
• Other Books You May Enjoy