The Pythonic Way

An Architect's Guide to Conventions and Best Practices for the Design, Development, Testing, and Management of Enterprise Python Code

Sonal Raj

• Publisher: BPB Publications
• Year: 2021
• ISBN:
  9789391030124
• Topic: Informatik
• Subtopic: Programmierung

Learn to build and manage better software with clean, intuitive, scalable, maintainable, and high-performance Python code.

Key Features
? Comparative analysis of regular and Pythonic coding constructs.
? Illustrates application design paradigms for Python projects.
? Detailed pointers on optimal data processing and application design.
? Highlights accepted conventions for testing and managing production code.

Description
'The Pythonic Way' acquaints you with Python's capabilities beyond basic syntax. This book will help you understand widely accepted Pythonic constructs and procedures, thus enabling you to write reliable, optimized, and modular applications.You'll learn about Pythonic data structures, class and object creation, and more. The book then delves into some of Python's lesser-known but incredibly powerful functionalities such as meta-programming, decorators, context managers, generators, and iterators. Additionally, you'll learn how to accelerate computations by using Pandas Series and Dataframes. You will be introduced to various design patterns that work well with Python applications. Finally, we'll discuss testing frameworks and best practices for testing, packaging, launching, and publishing applications in production environments.This book will empower you as you transition from beginner or competitive Python coding to industry-standard Python software development. Intermediate Python developers will gain a deeper understanding of the language's nuances, enabling them to create better software.

What you will learn
? Understand common practices for writing scalable and legible Python code.
? Create robust and maintainable production codebases for time and space performant applications.
? Master effective data processing practices and features like generators and decorators to improve complex computations on large datasets.
? Get familiar with Pythonic design patterns for secure, large-scale applications.

Who this book is for
This book is a valuable reference manual for novice and intermediate programmers and data scientists to learn about Pythonic standards and conventions. For beginners, this book will get you started with Pythonic thinking. This book will serve as a guide to fine-tune your skills beyond syntax and help build robust Python applications for intermediate Python coders.

Table of Contents
1. Introduction to Pythonic Code
2. Pythonic Data Structures
3. Classes and OOP Conventions
4. Python Modules and Metaprogramming
5. Pythonic Décorators and Context Managers
6. Data Processing Done Right
7. Iterators, Generators, and Coroutines
8. Python Descriptors
9. Pythonic Application Design and Architecture
10. Effective Testing for Python Code
11. Production Code Management

About the Authors
Sonal Raj is an engineer, mathematician, data scientist, and Python evangelist from India, who has carved a niche in the financial services domain. He is a Goldman Sachs and D.E. Shaw alumnus who currently heads the data analytics and research efforts for a high-frequency trading firm.He holds a dual master's degree in Computer Science and Business Management and is a former research fellow of the Indian Institute of Science. His areas of research range from image processing, real-time graph computations to electronic trading algorithms and data science. He is a doctoral candidate at the Swiss School of Business Management, Geneva. Over the years, he has implemented low latency platforms, trading strategies, and market signal models. With more than a decade of hands-on experience, he is a community speaker and a Python and data science mentor to newcomers in the field. LinkedIn Profile: https://www.linkedin.com/in/sonalraj/
Blog Link: https://www.sonalraj.com/

CHAPTER 1

Introduction to Pythonic Code

“Do the right thing. It will gratify some people and astonish the rest.”

— Mark Twain

As is the case with most programming languages, there exists a set of community-accepted guidelines and conventions, which help in the promotion of applications that are maintainable, crisp, uniform, and concise, and basically written in the way the language had intended them to be written. The guidelines provide suggestions on how to create proper conventional names for classes, functions, variables, and modules, to constructing loops, and how to correctly wrap your lines in the Python scripts. The term “Pythonic” is generally used in the context of programs, methods, and code blocks, which adhere to these guidelines to make the best of Python’s powerful features.

Structure

This chapter will broadly cover the following topics:

• Importance of clean Python code
• Writing better comments, docstrings, and annotations.
• Naming and code layout in Python
• Control structures in Python
• Clean Python code patterns
• Code quality enforcement tools

Further, we aim to introduce you to what is widely accepted as ‘Pythonic’ conventions, which bring out the ease and power of the Python language.

Objectives

You will understand that just like there is a difference in knowing how to drive and knowing how to drive a Ferrari – similarly knowing how to best use Python is different from simply knowing how to code. The main objectives of this chapter include the following:

• Understanding the common practices of writing scalable and legible code.
• Using constructs like docstrings, comments, and basic tools for maintainable production codebases.
• Understanding some of the lesser-known features about structuring Python code.

The importance of clean Python code

When working with Python in production, a cleanly written codebase is important from the view of creating a maintainable and successfully managed project, thereby leading to a reduction in the technical debt and creating effective agile development.

For a continuous, successful delivery of features, with a steady, constant, and predictable pace of work, it is paramount that the code base is readable, clean, and maintainable. Otherwise, every time you start working or estimating the work on a new project, which is a little out of the way, there will be tons of refactoring involved, thereby leading to technical debt.

All the compromises and bad decisions accumulate problems in the software. This is referred to as Technical Debt. One instance of technical debt would be currently faced problems that result from the decisions from the past or historical bad code. On the other hand, taking shortcuts in the current code, without giving proper time to design or review, can lead up to issues in the future. It is aptly called debt, since it will be more complicated and expensive to handle in the future than now, and even more with passing time. Every time work is delayed because your team gets blocked on a bug fix or code restructuring is how you can quantify the cost of technical debt. Technical debt is slow and progressive, and hence not easily identified or explicitly alarming. Hence, the underlying problem remains hidden.

PEP 8 is the standard set of conventions, which we follow in Python (more discussions on that, later in the chapter). It can be followed as is, or extended to suit your use case or project. These conventions already incorporate several Python syntax particulars (which you may not have encountered in other languages), and was created by the developers of the language itself. The improvements that come with following these conventions include the following:

• Consistent code: With time, you get used to reading through code of a certain format. Having uniform formats of code within an organisation is of utmost importance for scalability in the long term, and making the onboarding of new developers relatively easy. Learning and maintenance becomes simpler when the code layout, naming convention, and documentation, among other features, is identical across all the repositories maintained across the firm.
• Greppable code: Imagine a scenario where you are trying to grep a variable. Now the variable can occur in assignment statements, or as named arguments in function arguments. If we want to differentiate between the two in order to refine our search, what options do we have?

The PEP 8 conventions say that named argument assignments should not have spaces around the operator, whereas block assignments are preferred to have spaces around the operator. So, when searching for arguments, you can use the following:

$ grep -nr "name=" .

./validate.py:23: name=user_name,

When searching for variable assignments, you can use the following:

$ grep -nr "name =" .

./validate.py:20: user_name = get_username()

This makes the search efficient. Following the conventions can make a difference and lead to higher code quality. Getting used to a structured code base gradually reduces the time taken to understand such code. Over time, you will be able to identify bugs at a glimpse and reduce the number of mistakes. It is simple muscle memory. Additionally, code quality is ensured with advanced static code analysis and repair tools, which are to be discussed later.

Uniform coding standards help an organization scale its code base, even though they may not conform to the globally accepted standards. If your organization does not have a set of well-defined conventions, it is still not late to create one.

Adding comments to your code

A clean and well-documented code base goes a long way in ensuring production reliability for any company. You reduce bugs and errors, and ensure that handing over ownership of the code base to future generations is a seamless process, and you do not have to rewrite your software every time someone leaves.

Comments should be added, preferably at the time of the initial development. Personally, I prefer to have people start their code structure with elaborate comments and then fill in the gaps with functional code. Not only does that reduce the future burden of adding documentation, it also helps in ensuring a well thought out first draft of the software.

Some key points to keep in mind in the process of adding comments are as follows:

• A maximum of 72 – 80 characters should be the limit of the comment line.
• Complete sentences in camel-case is preferred.
• Developers and Reviewers should ensure that comments are updated when code is changed to avoid future confusion.

Let us take a glimpse of the types of comments that are popular in Python and their nuances.

Block comments

The aggregated comments for a sub-section of the code can be documented using block comments. They can be a single line or multi-line in nature, and are useful when you have to explain specifics of a functionality or the use of a section of ...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Dedication Page
• About the Author
• About the Reviewer
• Acknowledgements
• Preface
• Errata
• Table of Contents
• 1. Introduction to Pythonic Code
• 2. Pythonic Data Structures
• 3. Classes and OOP Conventions
• 4. Python Modules and Metaprogramming
• 5. Pythonic Decorators and Context Managers
• 6. Data Processing Done Right
• 7. Iterators, Generators, and Coroutines
• 8. Python Descriptors
• 9. Pythonic Design and Architecture
• 10. Effective Testing for Python Code
• 11. Production Code Management
• Index