Kali Linux Intrusion and Exploitation Cookbook

Ishan Girdhar, Dhruv Shah

• Publisher: Packt Publishing
• Year: 2017
• ISBN:
  9781783982172
• Edition: 1
• Topic: Computer Science
• Subtopic: Cyber Security
• Index: Computer Science

Over 70 recipes for system administrators or DevOps to master Kali Linux 2 and perform effective security assessmentsAbout This Book• Set up a penetration testing lab to conduct a preliminary assessment of attack surfaces and run exploits• Improve your testing efficiency with the use of automated vulnerability scanners• Work through step-by-step recipes to detect a wide array of vulnerabilities, exploit them to analyze their consequences, and identify security anomaliesWho This Book Is ForThis book is intended for those who want to know more about information security. In particular, it's ideal for system administrators and system architects who want to ensure that the infrastructure and systems they are creating and managing are secure. This book helps both beginners and intermediates by allowing them to use it as a reference book and to gain in-depth knowledge.What You Will Learn• Understand the importance of security assessments over merely setting up and managing systems/processes• Familiarize yourself with tools such as OPENVAS to locate system and network vulnerabilities• Discover multiple solutions to escalate privileges on a compromised machine• Identify security anomalies in order to make your infrastructure secure and further strengthen it• Acquire the skills to prevent infrastructure and application vulnerabilities• Exploit vulnerabilities that require a complex setup with the help of MetasploitIn DetailWith the increasing threats of breaches and attacks on critical infrastructure, system administrators and architects can use Kali Linux 2.0 to ensure their infrastructure is secure by finding out known vulnerabilities and safeguarding their infrastructure against unknown vulnerabilities.This practical cookbook-style guide contains chapters carefully structured in three phases – information gathering, vulnerability assessment, and penetration testing for the web, and wired and wireless networks. It's an ideal reference guide if you're looking for a solution to a specific problem or learning how to use a tool. We provide hands-on examples of powerful tools/scripts designed for exploitation.In the final section, we cover various tools you can use during testing, and we help you create in-depth reports to impress management. We provide system engineers with steps to reproduce issues and fix them.Style and approachThis practical book is full of easy-to-follow recipes with based on real-world problems faced by the authors. Each recipe is divided into three sections, clearly defining what the recipe does, what you need, and how to do it. The carefully structured recipes allow you to go directly to your topic of interest.

Building a Classification Model with Spark *

In this chapter, you will learn the basics of classification models, and how they can be used in a variety of contexts. Classification generically refers to classifying things into distinct categories or classes. In the case of a classification model, we typically wish to assign classes based on a set of features. The features might represent variables related to an item or object, an event or context, or some combination of these.

The simplest form of classification is when we have two classes; this is referred to as binary classification. One of the classes is usually labeled as thepositive class (assigned a label of 1), while the other is labeled as thenegative class (assigned a label of -1, or, sometimes, 0). A simple example with two classes is shown in the following figure. The input features in this case have two dimensions, and the feature values are represented on thex andy axes in the figure. Our task is to train a model that can classify new data points in this two-dimensional space as either one class (red) or the other (blue).

A simple binary classification problem

If we have more than two classes, we would refer to multiclass classification, and classes are typically labeled using integer numbers starting at 0 (for example, five different classes would range from label 0 to 4). An example is shown in the following figure. Again, the input features are assumed to be two-dimensional for ease of illustration.

A simple multiclass classification problem

Classification is a form of supervised learning, where we train a model with training examples that include known targets or outcomes of interest (that is, the model is supervised with these example outcomes). Classification models can be used in many situations, but a few common examples include the ones listed next:

• Predicting the probability of Internet users clicking on an online advert; here, the classes are binary in nature (that is, click or no click)
• Detecting fraud; again, in this case, the classes are commonly binary (fraud or no fraud)
• Predicting defaults on loans (binary)
• Classifying images, video, or sounds (most often multiclass, with potentially very many different classes)
• Assigning categories or tags to news articles, web pages, or other content (multiclass)
• Discovering e-mail and web spam, network intrusions, and other malicious behavior (binary or multiclass)
• Detecting failure situations, for example, in computer systems or networks
• Ranking customers or users in order of probability that they might purchase a product or use a service
• Predicting customers or users who might stop using a product, service, or provider (called churn)

These are just a few possible use cases. In fact, it is probably safe to say that classification is one of the most widely used machine learning and statistical techniques in modern businesses, especially, online businesses.

In this chapter, we will do the following:

• Discuss the types of classification models available in ML library
• Use Spark to extract appropriate features from raw input data
• Train a number of classification models using ML library
• Make predictions with our classification models
• Apply a number of standard evaluation techniques to assess the predictive performance of our models
• Illustrate how to improve model performance using some of the feature-extraction approaches fromChapter 3,Obtaining, Processing, and Preparing Data with Spark
• Explore the impact of parameter tuning on model performance, and learn how to use cross-validation to select the most optimal model parameters

Types of classification models

We will explore three common classification models available in Spark: linear models, decision trees, and naïve Bayes models. Linear models, while less complex, are relatively easier to scale to very large datasets. Decision tree is a powerful non-linear technique, which can be a little more difficult to scale up (fortunately, ML library takes care of this for us!) and more computationally intensive to train, but delivers leading performance in many situations. The naïve Bayes models are more simple, but are easy to train efficiently and parallelize (in fact, they require only one pass over the dataset). They can also give reasonable performance in many cases where appropriate feature engineering is used. A naïve Bayes model also provides a good baseline model against which we can measure the performance of other models.

Currently, Spark's ML library supports binary classification for linear models, decision trees, and naïve Bayes models, and multiclass classification for decision trees and naïve Bayes models. In this book, for simplicity in illustrating the examples, we will focus on the binary case.

Linear models

The core idea of linear models (or generalized linear models) is that we model the predicted outcome of interest (often called the target ordependent variable) as a function of a simple linear predictor applied to the input variables (also referred to as features or independent variables).

y = f(wTx)

Here, y is the target variable,w is the vector of parameters (known as theweight vector), andx is the vector of input features.

wTx is the linear predictor (or vector dot product) of the weight vectorw and feature vectorx. To this linear predictor, we applied a functionf (called thelink function).

Linear models can, in fact, be used for both classification and regression, simply by changing the link function. Standard linear regression (covered in the next chapter) uses an identity link (that is, y = wTx directly), while binary classification uses alternative link functions as discussed here.

Let's take a look at the example of online advertising. In this case, the target variable would be 0 (often assigned the class label of -1 in mathematical treatments) if no click was observed for a given advert displayed on a web page (called an impression). The target variable would be 1 if a click occurred. The feature vector for each impression would consist of variables related to the impression event (such as features relating to the user, web page, advert and advertiser, and various other factors relating to the context of the event, such as the type of device used, time of the day, and geolocation).

Thus, we would like to find a model that maps a given input feature vector (advert impression) to a predicted outcome (click or not). To make a prediction for a new data point, we will take the new feature vector (which is unseen, and hence, we do not know what the target variable is), and compute the dot product with our weight vector. We will then apply the relevant link function, and the result is our predicted outcome (after applying a threshold to the prediction, in the case of some models).

Given a set of input data in the form of feature vectors and target variables, we would like to find the weight vector that is the best fit for the data, in the sense that we minimize some error between what our model predicts and the actual outcomes observed. This process is called model fitting, training, or optimization.

More formally, we seek to find the weight vector that minimizes the sum, over all the training examples, of the loss (or error) computed from some loss function. The loss function takes the weight vector, feature vector, and the actual outcome for a given training example as input and outputs the loss. In fact, the loss function itself is effectively specified by the link function; hence, for a given type of classification or regression (that is, a given link function), there is a corresponding loss function.

For further details on linear models and loss functions, see the linear methods section related to binary classification in the Spark Programming Guide athttp://spark.apache.org/docs/latest/mllib-linear-methods.html#binary-classification andhttp://spark.apache.org/docs/latest/ml-classification-regression.html#linear-methods.

Also, see the Wikipedia entry for generalized linear models at http://en.wikipedia.org/wiki/Generalized_linear_model.

While a detailed treatment of linear models and loss functions is beyond the scope of this book, Spark ML provides two loss functions suitable to binary classification (you can learn more about them from the Spark documentation). The first one is logistic loss, which equates to a model known as logistic regression, while the second one is the hinge loss, which is equivalent to a linearSupport Vector Machine (SVM). Note that the SVM does not strictly fall into the statistical framework of generalized linear models...

Table of contents

• Title Page
• Copyright
• Credits
• About the Authors
• About the Reviewers
• www.PacktPub.com
• Customer Feedback
• Preface
• Getting Started - Setting Up an Environment
• Network Information Gathering
• Network Vulnerability Assessment
• Network Exploitation
• Web Application Information Gathering
• Building a Classification Model with Spark *
• Web Application Vulnerability Assessment
• Web Application Exploitation
• System and Password Exploitation
• Privilege Escalation and Exploitation
• Wireless Exploitation
• Pen Testing 101 Basics