Linux Shell Scripting Cookbook - Third Edition

Clif Flynt, Sarath Lakshman, Shantanu Tushar

• Publisher: Packt Publishing
• Year: 2017
• ISBN:
  9781785882388
• Edition: 3
• Topic: Computer Science
• Subtopic: System Administration
• Index: Computer Science

Do amazing things with the shellAbout This Book• Become an expert in creating powerful shell scripts and explore the full possibilities of the shell• Automate any administrative task you could imagine, with shell scripts• Packed with easy-to-follow recipes on new features on Linux, particularly, Debian-based, to help you accomplish even the most complex tasks with easeWho This Book Is ForIf you are a beginner or an intermediate Linux user who wants to master the skill of quickly writing scripts and automate tasks without reading the entire man pages, then this book is for you. You can start writing scripts and one-liners by simply looking at the relevant recipe and its descriptions without any working knowledge of shell scripting or Linux. Intermediate / advanced users, system administrators / developers, and programmers can use this book as a reference when they face problems while coding.What You Will Learn• Interact with websites via scripts• Write shell scripts to mine and process data from the Web• Automate system backups and other repetitive tasks with crontab• Create, compress, and encrypt archives of your critical data.• Configure and monitor Ethernet and wireless networks• Monitor and log network and system activity• Tune your system for optimal performance• Improve your system's security• Identify resource hogs and network bottlenecks• Extract audio from video files• Create web photo albums• Use git or fossil to manage revision control and interact with FOSS projects• Create and maintain Linux containers and Virtual Machines• Run a private Cloud serverIn DetailThe shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish.Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more.This book will show you how to do all this and much more.This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates.Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world issues. From there, you'll learn text processing, web interactions, network and system monitoring, and system tuning.Software engineers will learn how to examine system applications, how to use modern software management tools such as git and fossil for their own work, and how to submit patches to open-source projects.Finally, you'll learn how to set up Linux Containers and Virtual machines and even run your own Cloud server with a free SSL Certificate from letsencrypt.org.Style and approachThis book will take you through useful real-world recipes designed to make your daily life easier when working with the shell.

The Old-Boy Network

In this chapter, we will cover the following recipes:

• Setting up the network
• Let us ping!
• Tracing IP routes
• Listing all available machines on a network
• Running commands on a remote host with SSH
• Running graphical commands on a remote machine
• Transferring files through the network
• Connecting to a wireless network
• Password-less auto-login with SSH
• Port forwarding using SSH
• Mounting a remote drive at a local mount point
• Network traffic and port analysis
• Measuring network bandwidth
• Creating arbitrary sockets
• Building a bridge
• Sharing an Internet connection
• Basic firewall using iptables
• Creating a Virtual Private Network

Introduction

Networking is the act of connecting computers to allow them to exchange information. The most widely used networking stack is TCP/IP, where each node is assigned a unique IP address for identification. If you are already familiar with networking, you can skip this introduction.

TCP/IP networks work by passing data packets from node to node. Each data packet contains the IP address of its destination and the port number of the application that can process this data.

When a node receives a packet, it checks to see if it is this packet's destination. If so, the node checks the port number and invokes the appropriate application to process the data. If this node is not the destination, it evaluates what it knows about the network and passes the packet to a node that is closer to the final destination.

Shell scripts can be used to configure the nodes in a network, test the availability of machines, automate execution of commands at remote hosts, and more. This chapter provides recipes that introduce tools and commands related to networking, and shows how to use them effectively.

Setting up the network

Before digging through recipes based on networking, it is essential to have a basic understanding of setting up a network, terminologies, and commands for assigning IP address, adding routes, and so on. This recipe provides an overview of commands used in GNU/Linux networks.

Getting ready

A network interface physically connects a machine to a network, either with a wire or a Wi-Fi link. Linux denotes network interfaces using names such as eth0, eth1, or enp0s25 (referring to Ethernet interfaces). Other interfaces, namely usb0, wlan0, and tun0, are available for USB network interfaces, wireless LAN, and tunnels, respectively.

In this recipe, we will use these commands: ifconfig, route, nslookup, and host.

The ifconfig command is used to configure and display details about network interfaces, subnet mask, and so on. It should be available at /sbin/ifconfig.

How to do it...

1. List the current network interface configuration:

  $ ifconfig  lo Link encap:Local Loopback  inet addr:127.0.0.1 Mask:255.0.0.0  inet6addr: ::1/128 Scope:Host  UP LOOPBACK RUNNING MTU:16436 Metric:1  RX packets:6078 errors:0 dropped:0 overruns:0 frame:0  TX packets:6078 errors:0 dropped:0 overruns:0 carrier:0  collisions:0 txqueuelen:0  RX bytes:634520 (634.5 KB) TX bytes:634520 (634.5 KB)  wlan0 Link encap:EthernetHWaddr 00:1c:bf:87:25:d2  inet addr:192.168.0.82 Bcast:192.168.3.255 Mask:255.255.252.0  inet6addr: fe80::21c:bfff:fe87:25d2/64 Scope:Link  UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1  RX packets:420917 errors:0 dropped:0 overruns:0 frame:0  TX packets:86820 errors:0 dropped:0 overruns:0 carrier:0  collisions:0 txqueuelen:1000  RX bytes:98027420 (98.0 MB) TX bytes:22602672 (22.6 MB) 

The leftmost column in the ifconfig output lists the names of network interfaces, and the right-hand columns show the details related to the corresponding network interface.

2. To set the IP address for a network interface, use the following command:

  # ifconfig wlan0 192.168.0.80 

You will need to run the preceding command as root

192.168.0.80 is defined as the address for the wireless device, wlan0

To set the subnet mask along with the IP address, use the following command:

  # ifconfig wlan0 192.168.0.80 netmask 255.255.252.0 

3. Many networks use Dynamic Host Configuration Protocol (DHCP) to assign IP addresses automatically when a computer connects to the network. The dhclient command assigns the IP address when your machine is connected to a network that assigns IP addresses automatically. If addresses are assigned via DHCP, use dhclient instead of manually choosing an address that might conflict with another machine on the network. Many Linux distributions invoke dhclient automatically when they sense a network cable connection

  # dhclient eth0 

There's more...

The ifconfig command can be combined with other shell tools to produce specific reports.

Printing the list of network interfaces

This one-line command sequence displays network interfaces available on a system:

 $ ifconfig | cut -c-10 | tr -d ' ' | tr -s 'n' lo wlan0 

The first ten characters of each line in ifconfig output is reserved for writing name...

Table of contents

• Title Page
• Copyright
• Credits
• About the Authors
• About the Reviewer
• www.PacktPub.com
• Customer Feedback
• Preface
• Shell Something Out
• Have a Good Command
• File In, File Out
• Texting and Driving
• Tangled Web? Not At All!
• Repository Management
• The Backup Plan
• The Old-Boy Network
• Put On the Monitors Cap
• Administration Calls
• Tracing the Clues
• Tuning a Linux System
• Containers, Virtual Machines, and the Cloud