Mastering IPython 4.0
eBook - ePub

Mastering IPython 4.0

  1. 382 pages
  2. English
  3. ePUB (mobile friendly)
  4. Available on iOS & Android
eBook - ePub

Mastering IPython 4.0

Book details
Book preview
Table of contents
Citations

About This Book

Get to grips with the advanced concepts of interactive computing to make the most out of IPython

About This Book

  • Most updated book on Interactive computing with IPython 4.0;
  • Detailed, example-rich guide that lets you use the most advanced level interactive programming with IPython;
  • Get flexible interactive programming with IPython using this comprehensive guide

Who This Book Is For

This book is for IPython developers who want to make the most of IPython and perform advanced scientific computing with IPython utilizing the ease of interactive computing.

It is ideal for users who wish to learn about the interactive and parallel computing properties of IPython 4.0, along with its integration with third-party tools and concepts such as testing and documenting results.

What You Will Learn

  • Develop skills to use IPython for high performance computing (HPC)
  • Understand the IPython interactive shell
  • Use XeroMQ and MPI to pass messages
  • Integrate third-party tools like R, Julia, and JavaScript with IPython
  • Visualize the data
  • Acquire knowledge to test and document the data
  • Get to grips with the recent developments in the Jupyter notebook system

In Detail

IPython is an interactive computational environment in which you can combine code execution, rich text, mathematics, plots, and rich media.

This book will get IPython developers up to date with the latest advancements in IPython and dive deep into interactive computing with IPython. This an advanced guide on interactive and parallel computing with IPython will explore advanced visualizations and high-performance computing with IPython in detail.

You will quickly brush up your knowledge of IPython kernels and wrapper kernels, then we'll move to advanced concepts such as testing, Sphinx, JS events, interactive work, and the ZMQ cluster. The book will cover topics such as IPython Console Lexer, advanced configuration, and third-party tools.

By the end of this book, you will be able to use IPython for interactive and parallel computing in a high-performance computing environment.

Style and approach

This is a comprehensive guide to IPython for interactive, exploratory and parallel computing. It will let the IPython get up to date with the latest advancements in IPython and dive deeper into interactive computing with IPython

Frequently asked questions

Simply head over to the account section in settings and click on “Cancel Subscription” - it’s as simple as that. After you cancel, your membership will stay active for the remainder of the time you’ve paid for. Learn more here.
At the moment all of our mobile-responsive ePub books are available to download via the app. Most of our PDFs are also available to download and we're working on making the final remaining ones downloadable now. Learn more here.
Both plans give you full access to the library and all of Perlego’s features. The only differences are the price and subscription period: With the annual plan you’ll save around 30% compared to 12 months on the monthly plan.
We are an online textbook subscription service, where you can get access to an entire online library for less than the price of a single book per month. With over 1 million books across 1000+ topics, we’ve got you covered! Learn more here.
Look out for the read-aloud symbol on your next book to see if you can listen to it. The read-aloud tool reads text aloud for you, highlighting the text as it is being read. You can pause it, speed it up and slow it down. Learn more here.
Yes, you can access Mastering IPython 4.0 by Thomas Bitterman in PDF and/or ePUB format, as well as other popular books in Ciencia de la computación & Arquitectura de sistemas. We have over one million books available in our catalogue for you to explore.

Information

Publisher
Packt Publishing
Year
2016
ISBN
9781785888410
Edition
1
Topic
Ciencia de la computación
Subtopic
Arquitectura de sistemas

Mastering IPython 4.0

Table of Contents

Mastering IPython 4.0
Credits
About the Author
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and more
Why subscribe?
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Using IPython for HPC
The need for speed
FORTRAN to the rescue – the problems FORTRAN addressed
Readability
Portability
Efficiency
The computing environment
Choosing between IPython and Fortran
Fortran
IPython
Object-orientation
Ease of adoption
Popularity – Fortran versus IPython
Useful libraries
The cost of building (and maintaining) software
Requirements and specification gathering
Development
Execution
Testing and maintenance
Alternatives
Cross-language development
Prototyping and exploratory development
An example case – Fast Fourier Transform
Fast Fourier Transform
Fortran
Python
Performance concerns
Software engineering concerns
Complexity-based metrics
Size-based metrics
Where we stand now
High Performance Computing
The HPC learning curve
Cloudy with a chance of parallelism (or Amazon's computer is bigger than yours)
HPC and parallelism
Clouds and HPC
Going parallel
Terminology
A parallel programming example
A serial program
A parallel equivalent
Discussion
Summary
2. Advanced Shell Topics
What is IPython?
Installing IPython
All-in-one distributions
Package management with conda
Canopy Package Manager
What happened to the Notebook?
Starting out with the terminal
IPython beyond Python
Shell integration
History
Magic commands
Creating custom magic commands
Cython
Configuring IPython
Debugging
Post-mortem debugging
Debugging at startup
Debugger commands
Read-Eval-Print Loop (REPL) and IPython architecture
Alternative development environments
Spyder
Canopy
PyDev
Others
Summary
3. Stepping Up to IPython for Parallel Computing
Serial processes
Program counters and address spaces
Batch systems
Multitasking and preemption
Time slicing
Threading
Threading in Python
Example
Limitations of threading
Global Interpreter Lock
What happens in an interpreter?
CPython
Multi-core machines
Kill GIL
Using multiple processors
The IPython parallel architecture
Overview
Components
The IPython Engine
The IPython Controller
The IPython Hub
The IPython Scheduler
Getting started with ipyparallel
ipcluster
Hello world
Using map_sync
Asynchronous calls
Synchronizing imports
Parallel magic commands
%px
%%px
%pxresult
%pxconfig
%autopx
Types of parallelism
SIMD
SPMD
ipcluster and mpiexec/mpirun
ipcluster and PBS
Starting the engines
Starting the controller
Using the scripts
MapReduce
Scatter and gather
A more sophisticated method
MIMD
MPMD
Task farming and load balancing
The @parallel function decorator
Data parallelism
No data dependence
External data dependence
Application steering
Debugging
First to the post
Graceful shutdown
Summary
4. Messaging with ZeroMQ and MPI
The storage hierarchy
Address spaces
Data locality
ZeroMQ
A sample ZeroMQ program
The server
The client
Messaging patterns in ZeroMQ
Pairwise
Server
Client
Discussion
Client/server
Server 1
Server 2
Client
Discussion
Publish/subscribe
Publisher
Subscriber
Discussion
Push/Pull
Ventilator
Worker
Sink
Discussion
Important ZeroMQ features
Issues using ZeroMQ
Startup and shutdown
Discovery
MPI
Hello World
Rank and role
Point-to-point communication
Broadcasting
Reduce
Discussion
Change the configuration
Divide the work
Parcel out the work
Process control
Master
Worker
ZeroMQ and IPython
ZeroMQ socket types
IPython components
Client
Engine(s)
Controller
Hub
Scheduler
Connection diagram
Messaging use cases
Registration
Heartbeat
IOPub
Summary
5. Opening the Toolkit – The IPython API
Performance profiling
Using utils.timing
Using %%timeit
Using %%prun
The AsyncResult class
multiprocessing.pool.Pool
Blocking methods
Nonblocking methods
Obtaining results
An example program using various methods
mp.pool.AsyncResult
Getting results
An example program using various methods
AsyncResultSet metadata
Metadata keys
Other metadata
The Client class
Attributes
Methods
The View class
View attributes
Calling Python functions
Synchronous calls
Asynchronous calls
Configurable calls
Job control
DirectView
Data movement
Dictionary-style data access
Scatter and gather
Push and pull
Imports
Discussion
LoadBalancedView
Data movement
Imports
Summary
6. Works Well with Others – IPy...

Table of contents

  1. Mastering IPython 4.0