Verification, Validation, and Testing of Engineered Systems

Avner Engel

• Publisher: Wiley
• Year: 2010
• ISBN:
  9781118029312
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Electrical Engineering & Telecommunications
• Index: Technology & Engineering

Systems' Verification Validation and Testing (VVT) are carried out throughout systems' lifetimes. Notably, quality-cost expended on performing VVT activities and correcting system defects consumes about half of the overall engineering cost. Verification, Validation and Testing of Engineered Systems provides a comprehensive compendium of VVT activities and corresponding VVT methods for implementation throughout the entire lifecycle of an engineered system. In addition, the book strives to alleviate the fundamental testing conundrum, namely: What should be tested? How should one test? When should one test? And, when should one stop testing? In other words, how should one select a VVT strategy and how it be optimized?

The book is organized in three parts: The first part provides introductory material about systems and VVT concepts. This part presents a comprehensive explanation of the role of VVT in the process of engineered systems (Chapter-1). The second part describes 40 systems' development VVT activities (Chapter-2) and 27 systems' post-development activities (Chapter-3). Corresponding to these activities, this part also describes 17 non-testing systems' VVT methods (Chapter-4) and 33 testing systems' methods (Chapter-5). The third part of the book describes ways to model systems' quality cost, time and risk (Chapter-6), as well as ways to acquire quality data and optimize the VVT strategy in the face of funding, time and other resource limitations as well as different business objectives (Chapter-7). Finally, this part describes the methodology used to validate the quality model along with a case study describing a system's quality improvements (Chapter-8).

Fundamentally, this book is written with two categories of audience in mind. The first category is composed of VVT practitioners, including Systems, Test, Production and Maintenance engineers as well as first and second line managers. The second category is composed of students and faculties of Systems, Electrical, Aerospace, Mechanical and Industrial Engineering schools. This book may be fully covered in two to three graduate level semesters; although parts of the book may be covered in one semester. University instructors will most likely use the book to provide engineering students with knowledge about VVT, as well as to give students an introduction to formal modeling and optimization of VVT strategy.

Part I

Introduction

Chapter 1

Introduction

1.1 OPENING

This chapter serves as motivation for learning about systems Verification, Validation and Testing (VVT) as well as a map for using the book as a reference source on this complex and multifaceted process. We emphasize here the multitude of reasons for applying VVT. It sets the tone for the subject matter we hope to cover. It gives the reader insight into the attitudes of the author and the care with which the book was prepared. A clear statement is made of the purpose for which the book has been written.

The book is a compendium of facts about systems VVT. In fact, we think little has yet been published that is as comprehensive on this subject. By listing the potential audience for the book, we hope to encourage its wide distribution and to increase among engineers, managers, academicians and students an appreciation of the benefits of rigorously applying VVT to almost every endeavor involving a product or service, be it for purposes commercial, private or public. This chapter contains the following elements:

Opening. This part provides a background, purpose and the intended audience of the book. In addition, it describes its structure and contents as well as the scope of application and some terminology descriptions.

VVT systems and process. This part introduces VVT systems and processes as components of engineered systems. In addition, it describes basic VVT definitions and elaborates on the fundamental VVT dilemmas. Also, this part describes modeling of systems and VVT lifecycle as well as modeling of VVT processes and risks as cost and time drivers.

Canonical systems VVT paradigm. This part introduces the concept of canonical systems VVT paradigm which includes phases of systems’ lifecycle, views of systems and VVT aspects of systems.

Methodology application. This part introduces methodology application including VVT methodology overview, VVT tailoring and typical VVT documentation.

1.1.1 Background

The manufacturing industry used to be concerned with the design, development, production and maintenance of stand-alone products, whether simple or complex. Today, however, manufacturing has broadened its scope to include products, services or solutions that include a variety of components, integrate a large mix of technologies and involve both people and machines. It is this broad range of complex entities that we address in this book. The basic term we use for these complex entities is engineered systems. However, throughout this book, when appropriate, we will freely use terms such as products or services. The term engineered systems is distinguished from systems in the sense that the former is created by engineers who apply science and mathematics to find suitable solutions to problems.

Traditional and high-technology manufacturing industries are responding to the challenge to satisfy consumer needs and ensure competitive and sustainable growth by reducing time to market and customizing products (or expanding product ranges) while producing the required goods in the quantities demanded with the appropriate quality at reduced costs. For instance, in the automobile sector, the lead time for manufacturing a car at the beginning of the 1990s was five to six years, whereas today it is about two to three years and is estimated to be only 18 months in the near future. Therefore, controlling schedules, costs and quality in product deve­lopment, manufacturing and maintenance remains a major challenge for today’s industries. Increases in complexity, decreases in development budgets and shortened time to market for new products, services and solutions are leading developers to search for new ways of improving the quality of what they deliver by improving their technologies, processes, methodologies and tools.

The overall development process is only as strong as its weakest link. A critical and largely ignored link in this process is system VVT, which comprise vital activities and involve processes. A tool of systems engineering, VVT focuses on ensuring that engineered systems are delivered as error free as possible, are functionally sound and meet or exceed the user’s needs. Often VVT is carried out as merely a vehicle for finding and eliminating errors. It can do much more than that. Today, many system developers perform VVT only in the test phase of the project, a late and highly constrained period in the product development cycle. As a result, increases in overall development time and costs associated with product rework often exceed 20% of expanded engineering efforts (Capers, 1996). Admittedly, balancing testing cost and schedule with quality is difficult. However, quality problems discovered later by the user can necessitate expensive repairs and are likely to damage the reputation of the system or, worse, damage the reputation of the system’s developer.

Given the fundamental role of VVT in achieving product quality and reducing waste, this book aims at rectifying two critical current VVT problems, namely, lack of comprehensive system VVT methodology and lack of a practical, quantitative VVT process model for selecting a VVT strategy to optimize testing cost, schedule and economic risk. This book, which to a large measure is based on the European Commission–supported SysTest project, was written in order to rectify these problems.

1.1.2 Purpose

One of the central objectives of this book is the creation of generic VVT methodology. This VVT methodology consists of a selection of VVT activities and methods which can be applied throughout the system lifecycle in different industrial application fields and can be tailored according to the individual project needs.

The VVT methodology delivers generic means for comprehensive cost-effective VVT in the industry. In addition, the objectives of this methodology are as follows:

• To cover the entire product lifecycles from the definition to the disposal of the system
• To supply tailoring rules for different industry domains (e. g. electronics/avionics, control systems, automobile, food packaging systems, steel production), development cycles and project types
• To specify activities and methods for VVT on the system level together with their interrelationship
• To define VVT strategies that can be used in a broad variety of industrial applications

1.1.3 Intended Audience

The VVT methodology described in this book is applicable to all regional and industrial sectors. Although system VVT is performed throughout industry, it has not become a topic for research within the international community either in industry or in academia. Therefore, the definition of a generic VVT methodology will provide comprehensive knowledge for many students and practitioners. This book was written for the reader who has a background knowledge of project management, systems engineering and quality assurance. Those who participate in system development will benefit from the material covered in this book. These include:

1. Project Managers and VVT Managers. This book can guide project and VVT managers in the methods they select, adapt and tailor for planning, control and tracking of projects.

2. Quality Assurance (QA)/Quality Control (QC) Staff. For QA and, QC staff, this book offers an overview of the system QA activities and methods available and their principal advantages and disadvantages. Quality assurance staff can apply the VVT methodology guidelines for the selection of VVT procedures and the estimation of process and product risks.

3. Members of a VVT Team. This book serves as an aid for test teams by providing them with an overview of useful procedures for conducting a VVT process within the context of system development projects and beyond. Thus, the VVT methodology guidelines of this book become a useful tool for categorizing VVT activities within the system lifecycle overall context and by referencing further information.

4. System Developers and Maintainers. This book is relevant for system developers in that they deliver insight into the measures of error avoidance and error detection. Developers can draw important conclusions about the functional domains of the system developed that are critical where VVT are concerned.

5. Mechanical, Electronics and Software Designers. Other specialists need this book in order to take VVT aspects into account when they determine structures and select the technologies for system development, production and maintenance. This book can be an important basis for this, as it shows not only the possibilities but also the limitations of VVT procedures.

6. Component and Subsystem Suppliers. A clear definition and a specification with respect to VVT measures are essential, especially for system development projects that involve supplier companies. This book forms a convenient basis for those projects since it provides a mutual definition, nomenclature and techniques as well as a body of VVT methods.

7. Auditors. To evaluate the maturity of a development project, auditors and auditing agencies can also apply the VVT methodology. Adherence to standards, deployment of established procedures, as well as the maturity of the processes’ implementation can be evaluated in this way.

8. Regulatory and Standardization Agencies. Material presented in this book may be helpful in forming and updating national or international standards and regulations of standardization committees in which certain pro...

Table of contents

• Cover
• Series page
• Title page
• Copyright page
• Dedication
• Preface
• Part I: Introduction
• Part II: VVT Activities and Methods
• Part III: Modeling and Optimizing VVT Process
• Appendix A: SysTest Project
• Appendix B: Proposed Guide: System Verification, Validation and Testing Master Plan
• Appendix C: List of Acronyms
• Index
• End User License Agreement