Infrastructure Computer Vision

Ioannis Brilakis, Carl Thomas Michael Haas

• Publisher: Butterworth-Heinemann
• Year: 2019
• ISBN:
  9780128172582
• Topic: Technology & Engineering
• Subtopic: Construction & Architectural Engineering
• Index: Technology & Engineering

Infrastructure Computer Vision delves into this field of computer science that works on enabling computers to see, identify, process images and provide appropriate output in the same way that human vision does. However, implementing these advanced information and sensing technologies is difficult for many engineers. This book provides civil engineers with the technical detail of this advanced technology and how to apply it to their individual projects.

• Explains how to best capture raw geometrical and visual data from infrastructure scenes and assess their quality
• Offers valuable insights on how to convert the raw data into actionable information and knowledge stored in Digital Twins
• Bridges the gap between the theoretical aspects and real-life applications of computer vision

Chapter 1

Introduction

Ioannis Brilakis ¹ , and Carl Haas ^{2 1} Department of Engineering, University of Cambridge, Cambridge, United Kingdom ² Department of Civil and Environmental Engineering, University of Waterloo, Ontario, Canada

Keywords

infrastructure; computer vision; infrastructure computer vision; construction automation; image analysis

1. 1.1 Executive summary
2. 1.2 The current business environment for ICV
3. 1.3 Documented inefficiencies of traditional approaches
4. 1.4 ICV: new tools and new processes
   1. 1.4.1 Level of automation
   2. 1.4.2 Application scope or “use case”
   3. 1.4.3 Capabilities
   4. 1.4.4 Delivery model
   5. 1.4.5 Dominant data source
   6. 1.4.6 Learning curve
   7. 1.4.7 Intended users
5. 1.5 Synergies of ICV and other technologies
6. 1.6 What is not ICV technology?
7. 1.7 What challenges can be expected?

1.1. Executive summary

Infrastructure computer vision (ICV) has become a thriving commercial ecosystem within the engineering, construction and infrastructure business, and government sectors. Within this ecosystem, technical advances in sensors, deployment platforms, software as a service (SaaS) models, algorithms, development environments, and cloud-based delivery systems are all integrated into multidisciplinary services that involve highly trained computer vision specialists, field engineers, software developers, and business entrepreneurs.

Separating this ecosystem from those such as the 3D design environments (e.g., Building Information Modeling [BIM]), emerging 3D printing technologies, automated prefabrication systems, asset management systems, and the emerging digital twin environments is impossible. They are symbiotic and synergistic ecosystems. Understanding ICV therefore is more than just a technical challenge. It is a challenge in understanding complex systems spanning a number of domains. If this is true, where should we place the boundaries of the ICV body of knowledge, within which this book will delve?

Fig. 1.1 is a roadmap to help answer that question, describe the ICV body of knowledge, and explain the structure of this book. It describes ICV as consisting of layers of technologies, services, processes, and participants. At the bottom layer are the devices and sensors that are combined and used in multiple ways to acquire the raw sensor data used to generate images and point clouds; altitude, location, and spectral data; and raw models (as in simultaneous localization and modeling—SLAM—systems). Those data are used in the data acquisition layer by integrated hardware and software systems to generate image data in structured forms that are nominally standardized so that they are useful at the data management layer. At this layer, data are structured, further defined by meta-data, organized into databases, fused with 3D models' data, resampled, and converted into representations more useful to the next layer. At the next layer, experts use mathematical, statistical, or artificial intelligence (AI) tools to convert structured image data to useful models such as BIMs and digital twins, derive information (such as the existence, location and orientation of objects of interest), and add semantic content such as material properties and defects of objects and surfaces scanned. This information begins to generate business value at the services layer. This layer serves to detect incidents, such as heavy equipment dangerously close to workers; measure flow of people, materials, and equipment, including queuing assessment and activity analysis (e.g., direct work rate); assess conditions such as the pavement serviceability index from multiple sources of information; apply dimensional quality control such as floor flatness, steel alignment, and pipe fitting; conduct quantity and earned value tracking for construction projects; detect and track materials for management applications; and control fabrication processes through fit-up analysis, relative position, and orientation assessment (e.g., robotic brick laying) and volumetric analysis (e.g., automated welding and earth moving). All this information is used at the next layer in business and government applications that add value, such as asset management, project control, and design. At the top layer is the pull from owners, operators, fabricators, constructors, and architects who match market needs to ICV business and service offerings. In practice, researchers and businesses typically span some of these layers and their elements.

Figure 1.1 ICV body of knowledge covered in this book.

For example, consulting companies combine AI, computer vision, and manual activities to process images from unmanned aerial vehicle (UAV)-mounted and fixed-location cameras to (1) monitor safety conditions such as hardhat-wearing compliance, (2) conduct activity analysis that classifies workforce direct work, travel, planning, idle, and other states over time, (3) maintain security over materials stores and laydown yards, (4) conduct automated earned value tracking for volumetric activities such as concrete, steel, and services placement, and (5) support remote presence for senior management and engineers. Other companies collect road surface condition data and process it using manual and computer vision–assisted approaches to deliver pavement condition assessments to transportation asset managers and owners. Other companies make the laser scanners and the software that makes using the laser scanners easy and streamline the process of acquiring merged point clouds and feeding them into the information generation layer (Fig. 1.1). Research groups at universities typically focus on one or two of the layers in Fig. 1.1.

In this book, the following chapters generally progress through the layers in Fig. 1.1 from the bottom up. However, examples that help describe the application of a basic concept described in chapter 2 on data acquisition might focus on an asset management application implemented by a transportation system owner-operator. Chapter 3 necessarily spans the data acquisition, management, and information generation layers, while reaching up and down the layers to develop and explain specific examples. The following chapters from the owner's, engineer's, architect's, contractor’s, subcontractor’s, and vendor's perspectives typically span the services and business layers.

In the following sections, the scope, nature, and context of ICV are further explored in order to help the reader better understand and apply the knowledge transferred in this book.

1.2. The current business environment for ICV

It is not necessarily obvious to early career readers with little work experience what is the industry model within which ICV exists. It varies in its nature across different regions of the world, and acronyms abound that have different meanings depending on context and world region. At its simplest, the ICV business environment boundary for this book can be described as infrastructure and construction engineering and management. Typically, a business environment can be further defined by its stakeholders, sectors, drivers, trends, sources of change (external and internal), and key uncertainties.

Infrastructure and construction engineering and management stakeholders and their roles with respect to ICV include the following:

1. • Owners—Infrastructure and business capital asset owners are responsible for maintaining and operating existing assets as well as planning, financing, and managing the delivery (building) of new assets. Their interest in ICV is typically related to its potential for efficiently delivering objective, detailed, and quantitative asset condition information such as cracks and corrosion in building elements. They also have an interest in using ICV to generate as-built models and digital twins of facilities where their records are poor, which unfortunately is most often the case. For example, a power plant owner may have out-of-date two-dimensional drawings from decades earlier when the plant was first constructed, and they need information about precise dimensions, locations, and tie-in points for modifications and additions to the plant that can be acquired using ICV.
2. • Asset managers—Many large asset management firms exist who provide the service to owners (such as health care providers) of managing their distributed physical assets, because the owners recognize that their core business and capabilities may not directly relate to the assets from which they are delivered, such as health care delivered from hospitals. Asset managers are interested in condition assessment, for example, as well as the modeling, visualization, and communications that interest owners. Typically, they might use applications of ICV such as building heat loss analysis.
3. • Engineers and architects—Engineers and architects design and analyze for owners and asset managers. They may focus on as-built models derived from ICV for design, visualization, and interference checking. They are also interested in highly specialized applications of ICV such as scan-to-FEA (3D scans conversion and structuring for finite element analysis), for example, for turbine blade condition assessment and reverse engineering of physical asset elements.
4. • Contractors—Contractors are called constructors in some countries and builders in others. They are responsible for marshaling the resources required and managing them to make what the architects and engineers have designed. ICV appeals to them as a tool for project cost, schedule, and quality control (for example, ICV helps with automated quantity tracking), for safety improvement (ICV may be used to detect dangerous conditions), and for risk mitigation (ICV can be used, for example, to document as-installed conditions to avoid or quickly resolve costly conflicts that occur when such information is disputed after the fact).
5. • Subcontractors—Distinguished by their higher level of specialization and narrower scope...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• Dedication
• Contributors
• Foreword
• Acknowledgments
• Chapter 1. Introduction
• Chapter 2. Surveying, Geomatics, and 3D Reconstruction
• Chapter 3. Scene understanding and model generation
• Chapter 4. Use Cases for Owners and Maintainers
• Chapter 5. Use Cases for Architects and Engineers
• Chapter 6. Use Cases for Contractors
• Chapter 7. Use Cases for Subcontractors and Fabricators
• Chapter 8. The Future
• Author Index
• Subject Index