Mastering SaltStack

Joseph Hall

• Publisher: Packt Publishing
• Year: 2015
• ISBN:
  9781785282164
• Edition: 1
• Topic: Informatica
• Subtopic: Reti di computer

Take charge of SaltStack to automate and configure enterprise-grade environments

About This Book

• Automate tasks effectively, so that your infrastructure can run itself
• Take advantage of cloud-based services that can expand the capabilities of your own data centers
• Tackle real-world problems that appear in everyday situations

In Detail

SaltStack is known as a popular configuration management system, but that barely scratches the surface. It is, in fact, a powerful automation suite, which is designed not only to help you manage your servers, but to help them manage themselves. SaltStack is used worldwide by organizations ranging from just a few servers, to tens of thousands of nodes across data centers in multiple continents. This award-winning software is fast becoming the standard for systems management in the cloud world.

This book will take you through the advanced features of SaltStack, bringing forward capabilities that will help you excel in the management of your servers.

You will be taken through the the mind of the modern systems engineer, and discover how they use Salt to manage their infrastructures, and why those design decisions are so important. The inner workings of Salt will be explored, so that as you advance your knowledge of Salt, you will be able to swim with the current, rather than against it.

Various subsystems of Salt are explained in detail, including Salt SSH, Salt Cloud, and external pillars, filesystems, and job caches.

You will be taken through an in-depth discussion of how to effectively scale Salt to manage thousands of machines, and how to troubleshoot issues when things don't go exactly the way you expect them to.

You will also be taken through an overview of RAET, Salt's new transport protocol, and given an insight into how this technology improves Salt, and the possibilities that it brings with it.

What You Will Learn

• Learn how the pros are managing their infrastructures, and what techniques they use to keep everything running smoothly with Salt
• Understand what makes Salt tick, and how that affects the way you use it
• Take a look at familiar features in a new light, so that you have a better handle on how to approach tasks
• Use Salt SSH to manage servers that don't fit Salt's traditional use cases
• Besides automating your configuration, you will be able to automate your servers, and make them behave more intelligently
• Make better use of cloud-based services, including compute clouds such as EC2, Azure and Openstack
• Learn about the new RAET protocol, and how it changes the way automation works

Who This Book Is For

This book is ideal for professionals who have been managing groups of servers, and want to learn how to add functionality and expand their tool set. This book will also explain some of the more advanced features of Salt, and explore how to use them to bring additional power to the fundamentals that the professionals have already been using.

Style and approach

This book speaks informally, sometimes almost conversationally, to the user. Topics are covered in detail, using examples that should be comfortable to most users.

Mastering SaltStack

---

Table of Contents

Mastering SaltStack

Credits

Foreword

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. Reviewing a Few Essentials

Executing commands remotely

Master and Minions

Targeting Minions

Glob

Perl Compatible Regular Expression (PCRE)

List

Subnet

Grain

Grain PCRE

Pillar

Compound

Nodegroup

Using module functions

test.ping

test.echo

test.sleep

test.version

pkg.install

pkg.remove

file.replace

Other file functions

Various user and group functions

sys.doc

SLS file trees

SLS files

Tying things together with top files

Organizing the SLS directories

Using States for configuration management

Using include blocks

Ordering with requisites

require

watch

onchanges

onfail

use

prereq

Inverting requisites

Extending SLS files

The basics of Grains, Pillars, and templates

Using Grains for Minion-specific data

Centralizing variables with Pillars

Managing files dynamically with templates

A quick Jinja primer

Summary

2. Diving into Salt Internals

Understanding the Salt configuration

Following the configuration tree

Looking inside /etc/salt/

Managing Salt keys

Exploring the SLS directories

Examining the Salt cache

The Master job cache

The Master-side Minion cache

The external file server cache

The Minion-side proc/ directory

External modules

The Renderer

Rendering SLS files

Render pipes

Serving templated files

Understanding the Loader

Dynamic modules

Execution modules

Cloud modules

Plunging into the State compiler

Imperative versus declarative

Requisites

High and Low States

High States

Low States

Enforcing statefulness

name

result

changes

comment

Summary

3. Exploring Salt SSH

Grappling with SSH

Remote shells

Using rosters

The flat roster

host

port

user

passwd

sudo

priv

timeout

thin_dir

Other built-in rosters

scan

cache

cloud

ansible

Building dynamic rosters

Using Salt SSH

Using a Saltfile

Salt versus Salt SSH

Architecture

Performance

Understanding the salt-thin agent

Building the thin package

Including extra modules

Deploying the thin package

Executing the thin package

The Salt SSH shim

Preparing for Salt States

Running Salt

Salt's running data

Using the raw SSH mode

Caching SSH connections

Summary

4. Managing Tasks Asynchronously

Looking at the event system

Reviewing the basics

The structure of event data

Watching event data

Installing the event listener

Using the event listener

Firing custom data

Namespacing events

Namespacing guidelines

Some common events

salt/auth

salt/key

salt/minion/<minion_id>/start

salt/job/<job_id>/new

salt/job/<job_id>/ret/<minion_id>

salt/presence/present

salt/presence/change

Common cloud events

salt/cloud/<vm_name>/creating

salt/cloud/<vm_name>/requesting

salt/cloud/<vm_name>/querying

salt/cloud/<vm_name>/waiting_for_ssh

salt/cloud/<vm_name>/deploying

salt/cloud/<vm_name>/created

salt/cloud/<vm_name>/destroying

salt/cloud/<vm_name>/destroyed

Salt API events

salt/netapi/<url_path>

Building Reactors

Configuring Reactors

Writing Reactors

Calling execution modules

Calling runner modules

Calling wheel modules

Writing more complex Reactors

Sending out alerts

Using webhooks

Reactors calling Reactors

Using the queue system

Learning how queues work

Adding to the queue

Listing queues

Listing items in a queue

Processing queue items

Deleting items from a queue

Using queues with the Reactor

Spreading out State runs

Dividing tasks among Minions

Summary

5. Taking Salt Cloud to the Next Level

Examining the Salt Cloud configuration

Global configurations

The provider and profile configuration

Providers

Profiles

Extending configuration blocks

Building custom deploy scripts

Understanding the Salt Bootstrap script

Installing from prebuilt packages

Installing from Git

Looking back at legacy deploy scripts

Writing your own deploy scripts

Passing arguments to scripts

Using file maps

Taking a look at cloud maps

Working with autoscale Reactors

The cloud cache

Using cloud cache events

Setting up a schedule

Catching cloud cache events

Summary

6. Using Salt with REST

Looking at Salt's HTTP library

Why a Salt-specific library?

Using the http.query function

GET versus POST

Decoding return data

Using the http.query state

Using http.query with Reactors

Understanding Salt API

What is Salt API?

Setting up Salt API

CherryPy

Tornado

WSGI

Creating SSL certificates

Configuring authentication

The external authentication module

Taking your first steps with Salt API

Issuing one-off commands

Working with webhooks

Security considerations

More complex authentication

Summary

7. Understanding the RAET Protocol

Comparing RAET and ZeroMQ

Starting with HTTP

SSH, the old favorite

Using ZeroMQ

ZeroMQ and security

The need for RAET

Flow-based programming

The pieces of the puzzle

Black boxes

Shared storage

Concurrent scheduling

Driving with RAET

Configuring RAET

The RAET architecture

The basics

The RAET scheduler

Estates and yards

Summary

8. Strategies for Scaling

All about syndication

Different folks, different strokes

No need for micro-managing

Configuring syndication

High availability with multiple Masters

Built-in high availability configuration

Old-school high availability

The round robin DNS

IP-based load balancing

Synchronizing files

Base configuration files

Synchronizing the nonexternal files

Using rsync

Using the event Reactor

Incorporating external data sources

The external job cache

Using Returners on the Minions

Using the Master job cache

External filesystems

GitFS

base

root

mountpoint

user and password

insecure_auth

pubkey, privkey, and passphrase

Other source control backends

SVNFS

root and mountpoint

trunk

branches

tags

HGFS

S3FS

One environment per bucket

Multiple environments per bucket

AzureFS

External Pillars

cmd_yaml/cmd_json

git

redis

mysql

Using the Master API

The Salt keys

Configuration

The file and Pillar roots

Using the Wheel Re...

Table of contents

• Mastering SaltStack