Snowflake Cookbook

Techniques for building modern cloud data warehousing solutions

Hamid Mahmood Qureshi, Hammad Sharif

• Publisher: Packt Publishing
• Year: 2021
• ISBN:
  9781800560185
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Warehousing
• Index: Computer Science

Develop modern solutions with Snowflake's unique architecture and integration capabilities; process bulk and real-time data into a data lake; and leverage time travel, cloning, and data-sharing features to optimize data operations

Key Features

• Build and scale modern data solutions using the all-in-one Snowflake platform
• Perform advanced cloud analytics for implementing big data and data science solutions
• Make quicker and better-informed business decisions by uncovering key insights from your data

Book Description

Snowflake is a unique cloud-based data warehousing platform built from scratch to perform data management on the cloud. This book introduces you to Snowflake's unique architecture, which places it at the forefront of cloud data warehouses.

You'll explore the compute model available with Snowflake, and find out how Snowflake allows extensive scaling through the virtual warehouses. You will then learn how to configure a virtual warehouse for optimizing cost and performance. Moving on, you'll get to grips with the data ecosystem and discover how Snowflake integrates with other technologies for staging and loading data.

As you progress through the chapters, you will leverage Snowflake's capabilities to process a series of SQL statements using tasks to build data pipelines and find out how you can create modern data solutions and pipelines designed to provide high performance and scalability. You will also get to grips with creating role hierarchies, adding custom roles, and setting default roles for users before covering advanced topics such as data sharing, cloning, and performance optimization.

By the end of this Snowflake book, you will be well-versed in Snowflake's architecture for building modern analytical solutions and understand best practices for solving commonly faced problems using practical recipes.

What you will learn

• Get to grips with data warehousing techniques aligned with Snowflake's cloud architecture
• Broaden your skills as a data warehouse designer to cover the Snowflake ecosystem
• Transfer skills from on-premise data warehousing to the Snowflake cloud analytics platform
• Optimize performance and costs associated with a Snowflake solution
• Stage data on object stores and load it into Snowflake
• Secure data and share it efficiently for access
• Manage transactions and extend Snowflake using stored procedures
• Extend cloud data applications using Spark Connector

Who this book is for

This book is for data warehouse developers, data analysts, database administrators, and anyone involved in designing, implementing, and optimizing a Snowflake data warehouse. Knowledge of data warehousing and database and cloud concepts will be useful. Basic familiarity with Snowflake is beneficial, but not necessary.

Chapter 1: Getting Started with Snowflake

Snowflake is a cloud analytics solution that uses the cloud's flexibility and cost benefits but does not compromise on providing the user with a database-like environment. It allows the user to manage data using familiar concepts. Moreover, it allows the user to use SQL to query data. This chapter will provide you with an introduction to Snowflake's architecture and its different components. Then, you'll learn how to connect to Snowflake via its Web UI. You will also be introduced to the concept of a Virtual warehouse and how a Virtual warehouse can be used elastically according to the complexity of user queries. Later, we move onto the ecosystem support for Snowflake, explaining how Snowflake can be connected to and used with diverse tools using different interfaces.

The following recipes will be covered in this chapter:

• Creating a new Snowflake instance
• Creating a tailored multi-cluster virtual warehouse
• Using the Snowflake WebUI and executing a query
• Using SnowSQL to connect to Snowflake
• Connecting to Snowflake with JDBC
• Creating a new account admin user and understanding built-in roles

Let's get started!

Technical requirements

For this chapter, you will need access to a modern internet browser (Chrome, Microsoft Edge, Firefox, and so on) and access to the internet in order to provision and connect to your Snowflake instance in the cloud. Since we will be installing the SnowSQL client, administrative (root) access on your machine will be required.

To set up, configure, and compile a sample program that uses the Snowflake JDBC driver, you will need to have the following installed and configured:

• OpenJDK 1.8+
• Maven 2.4+

You may use Visual Studio IDE for Linux or another IDE of your choice to compile the Java code.

The code for this chapter can be found at https://github.com/PacktPublishing/Snowflake-Cookbook/tree/master/Chapter01.

Creating a new Snowflake instance

There are a number of decisions that you need to make when setting up a new Snowflake instance via the Snowflake website at www.Snowflake.com. Among those decisions is also the question of what Snowflake edition you should get and which public cloud to host your instance on. Let's get answers to these as we go through the process of creating a new Snowflake instance.

Getting ready

Before starting this process, please have your contact details (name, email, company/business name, country, and so on) ready as you will be using them during the sign-up process. Your email address will be used to send a link to your Snowflake instance, so make sure you have the correct email address ready. You do not need payment information at this stage, but as soon as your trial expires (after 30 days), you will need to provide this.

How to do it…

The steps for this recipe are as follows:

1. Let's start by creating a Snowflake account. Snowflake's website offers an easy-to-use UI for creating and managing your account. It also offers a free-to-use (for 30 days) account that has $400 credit for compute.
2. Navigate to the Snowflake website at www.Snowflake.com and find the START FOR FREE button. Then, click on the START FOR FREE button on that page to start the provisioning process.
3. You will be navigated to a signup page, prompting you to start your 30-day trial with $400 worth of credit to use. Please fill in the required contact details on this page and continue.
4. The next steps show you the different options that are available regarding the type of instance and choice of the underpinning cloud platform. These two decisions are critical both from a cost perspective as well as in terms of efficiency.
5. On the next screen, you will be prompted to select your Snowflake edition and public cloud provider, as well as the region where you want to host your Snowflake instance. The Snowflake edition you choose mainly depends on your use case. The Standard edition is the entry-level edition and provides all the necessary SQL warehouse functionality. However, it does not support multi-cluster virtual warehouses or materialized views and provides only 1 day of time travel (Snowflake's way of maintaining versions of data as a result of merging as new data arrives). The Enterprise edition is a good choice for most organizations as it supports multi-cluster virtual warehouses, materialized views, and up to 90 days of time travel. As shown in the following screenshot, we have selected Enterprise:
   
   Figure 1.1 – Types of instance
   Business Critical provides several additional security features and enhanced failovers, thus delivering you better business continuity.
6. A key thing to note here is that Snowflake is a Software as a Service (SaaS), so regardless of which public cloud platform you choose, you will be accessing your Snowflake instance using a URL and will not necessarily need to log into your cloud provider console. Snowflake offers its service on the three major public cloud vendors: AWS, Microsoft Azure and Google Cloud Platform (GCP). These three platforms offer largely similar capabilities with slight constraints due to the platform architecture.
7. Which public cloud you choose is important when you take into consideration which public cloud the rest of your applications and data is hosted on. Usually, it would be best to choose the same public cloud and the same region...