Data Analysis and Applications 1

Clustering and Regression, Modeling-estimating, Forecasting and Data Mining

Christos H. Skiadas, James R. Bozeman, Christos H. Skiadas, James R. Bozeman

• Publisher: Wiley-ISTE
• Year: 2019
• ISBN:
  9781119597681
• Edition: 1
• Topic: Mathematics
• Subtopic: Probability & Statistics
• Index: Mathematics

This series of books collects a diverse array of work that provides the reader with theoretical and applied information on data analysis methods, models, and techniques, along with appropriate applications. Volume 1 begins with an introductory chapter by Gilbert Saporta, a leading expert in the field, who summarizes the developments in data analysis over the last 50 years. The book is then divided into three parts: Part 1 presents clustering and regression cases; Part 2 examines grouping and decomposition, GARCH and threshold models, structural equations, and SME modeling; and Part 3 presents symbolic data analysis, time series and multiple choice models, modeling in demography, and data mining.

PART 1
Clustering and Regression

1
Cluster Validation by Measurement of Clustering Characteristics Relevant to the User

There are many cluster analysis methods that can produce quite different clusterings on the same data set. Cluster validation is about the evaluation of the quality of a clustering; “relative cluster validation” is about using such criteria to compare clusterings. This can be used to select one of a set of clusterings from different methods, or from the same method ran with different parameters such as different numbers of clusters.

There are many cluster validation indexes in the literature. Most of them attempt to measure the overall quality of a clustering by a single number, but this can be inappropriate. There are various different characteristics of a clustering that can be relevant in practice, depending on the aim of clustering, such as low within-cluster distances and high between-cluster separation.

In this chapter, a number of validation criteria will be introduced that refer to different desirable characteristics of a clustering, and that characterize a clustering in a multidimensional way. In specific applications the user may be interested in some of these criteria rather than others. A focus of the chapter is on methodology to standardize the different characteristics so that users can aggregate them in a suitable way specifying weights for the various criteria that are relevant in the clustering application at hand.

1.1. Introduction

The aim of this chapter is to present a range of cluster validation indexes that provide a multivariate assessment covering different complementary aspects of cluster validity. Here, I focus on “internal” validation criteria that measure the quality of a clustering without reference to external information such as a known “true” clustering. Furthermore, I am mostly interested in comparing different clusterings on the same data, which is often referred to as “relative” cluster validation. This can be used to select one of a set of clusterings from different methods, or from the same method ran with different parameters such as different numbers of clusters.

In the literature (for an overview, see Halkidi et al. 2016), many cluster validation indexes are proposed. Usually, these are advertised as measures of ...

Table of contents

• Cover
• Table of Contents
• Preface
• Introduction: 50 Years of Data Analysis: From Exploratory Data Analysis to Predictive Modeling and Machine Learning
• PART 1: Clustering and Regression
• PART 2: Models and Modeling
• PART 3: Estimators, Forecasting and Data Mining
• List of Authors
• Index
• End User License Agreement