Evaluation of Rail Technology

A Practical Human Factors Guide

Anjum Naweed, Jillian Dorrian, Janette Rose, Chris Bearman

• Publisher: CRC Press
• Year: 2017
• ISBN:
  9781317138693
• Edition: 1
• Topic: Technik & Maschinenbau
• Subtopic: Gesundheit & Sicherheit in der Industrie

Rapid advancements in train control and in-cab technologies provide significant opportunities for rail operators to improve efficiency and enhance their operations. New technologies often provide elegant solutions to existing problems or new capabilities for the operator. However, new technologies may also represent a significant form of risk. Thus, it is important to balance the potential for significant improvement with justifiable concern about how the technology may unpredictably change the nature of the work. If a technology is designed and implemented without considering the substantive human factors concerns, that technology may lead to unintended consequences that can introduce safety issues and disrupt network performance. It is important to note that even a well-designed and beneficial technology may be rejected by the users who see it as a threat to their jobs, status or working conditions. This book discusses the issues surrounding rail technology and introduces a 'toolkit' of human factors evaluation methods. The toolkit provides a practical and operationally focused set of methods that can be used by managers considering investing in technology, staff charged with implementing a technology, and consultants engaged to assist with the design and evaluation process. This toolkit can help to ensure that new rail technologies are thoughtfully designed, effectively implemented, and well received by users so that the significant investment associated with developing rail technologies is not wasted.

Chapter 1
Introduction: The Promise and Perils of New Technology

Drew Dawson
Central Queensland University, Appleton Institute, Adelaide, Australia

Chris Bearman
Central Queensland University, Appleton Institute, Adelaide, Australia

Anjum Naweed
Central Queensland University, Appleton Institute, Adelaide, Australia

Gareth Hughes
Railcorp NSW, Sydney, Australia

In the last decade the rail industry has witnessed an unprecedented flow of new technologies predicated on improving the safety and efficiency of modern rail operations. Over the next decade we will see an even greater penetration of these twenty-first-century technologies into an industry where lower levels of technology are still common. The introduction of new technology in any industry or workplace is a complex socio-technical phenomenon and it carries the potential for significant improvement with a justifiable concern about how the technology might also change the nature of work in unpredictable ways. This is especially the case in the rail industry where the historically dominant technologies have been mechanical rather than digital and the dominant organisational cultures are often relatively resistant to change.

This book is directed toward those in the rail industry who are charged with introducing new technology and ensuring that we do not fall foul of the law of inadvertent consequence. To that end, it is directed toward three types of readers. The first group is rail managers contemplating the introduction of new rail technologies. In deciding whether to introduce a new technology, this group typically needs to determine its cost-benefit analysis and ensure that the technology is implemented in a way that delivers the promised benefits. These readers will require high-level conceptual advice on the kinds of issues that need to be considered when contemplating the introduction of new technology. They will also need to be able to define the key issues associated with the design, implementation and evaluation of new technologies, but given their high-level vantage point, they will be less concerned with the details of this process.

The second group of readers is those charged with the implementation of a new technology into a specific rail operations setting. These readers will need to have a more detailed understanding of the specific technology being proposed and will be intimately involved in its design, implementation and evaluation. They will need specific advice on how to undertake each of these phases of the process and gain access to best-practice approaches. They will also need to provide operational answers to the questions raised by rail managers and ensure that the introduction of any new technology follows a sound process. This group of readers is, in effect, responsible for ensuring that an organisation has exercised due diligence in the selection and introduction of a new technology.

The third group of readers is human factors professionals who may be asked to provide independent advice or work on the introduction of new technologies but lack specific experience in a particular technology or in the rail industry in general. This book provides that group of readers with a number of new tools, some specific case studies of evaluating rail technology, and examples of how general theoretical principles and practice in Human Factors have been operationalised in a rail setting.

Each of these reader groups have different needs and, as a consequence, may well read this book in different ways. For the manager deciding on whether to investigate the feasibility of a new technology, the more generalist chapters around technology introduction (such as chapters 2, 3 and 7) will be of the most benefit initially. These will present the reader with key technology-related human factors issues for introducing technology, describe how the associated tasks may be evaluated, and demonstrate how technology may also be resisted. Similarly, the case studies that are presented in the specified chapters provide both exemplars and cautionary tales around ensuring that new technologies deliver on their technical promise. The two train simulation chapters (chapters 8 and 9) illustrate the seductive nature of technology and demonstrate how the purpose of technology may become easily disconnected from its goals during the early specification stages. For those from a primarily technical background (possibly with limited experience of the social and psychological issues that can arise around the introduction of new technologies), the case studies and chapters 2 and 3 will be particularly instructive.

For those in the second group of readers, the methodological chapters (chapters 4 to 10) will be of primary relevance. In these chapters, a sound understanding of the methodological challenges around participatory design and how to consult effectively is presented. Readers can learn about mock-ups and their surprising utility (chapter 4), the ways in which technology can be evaluated by structured qualitative research methods (chapter 5) and how a consideration of a technology’s past, present and future can reveal striking insights. Understanding the methodological rigour required for effective consultation, participatory design, monitoring and evaluation of new technologies will be critical for those charged with ensuring that an organisation effectively fulfils its legal obligation to exercise due diligence around the introduction of a new technology. In many cases, the introduction of a new technology in organisations characterised by ‘high engineering’ cultures can be frustrated by the failure to understand the social, psychological or even industrial consequences of a new technology. The ‘Luddites’ of the nineteenth century still persist within many organisations and there is a clear need to sell the case for change to those required to use, adapt to or be displaced by a new technology. These issues are discussed in the chapter on resistance to technology (chapter 3).

The third group of readers, the human factors professionals looking to acquire specific knowledge and/or expertise in an operational rail setting, will likely find both the theoretical and methodological aspects of the chapters of considerable interest. Chapters 5, 8, 9 and 10 in particular provide discussions about theoretical issues concerning their subject matter. There are an increasing number of human factors professionals entering the rail industry to meet demand created by an increased focus on safety and a growing awareness of the need for human factors expertise. As a consequence, human factors professionals entering this industry may want to understand the peculiarities of the rail domain, and the ways in which general principles and practices have been specifically contextualised and/or operationalised by the rail industry. Therefore, the chapter that describes task analysis (7), the two chapters (8, 9) exploring train simulation, and the last chapter (10), which investigates how situation awareness may be measured in rail, will be of particular interest to this group of readers.

Given the diversity of the readership, the book is, by definition, the type of publication into which one can dip into in many ways. It may even be the case that different chapters will be more or less relevant at different points in the process of introducing new technologies to an organisation. On the one hand, this book is a tool-kit, and different readers may use the perspectives contained in these chapters on a very much ‘as needs’ basis, but on the other hand, it contains a number of integrating themes that will reward those who read it all. If we look at the introduction of new technologies in any workplace, a number of key themes will emerge. First amongst these themes is the nature of technological innovation. In many cases, the rail industry has not yet experienced the relentless technological innovation that has characterised other industries (such as aviation) over the last 50 years. If a freight locomotive driver¹ were to go into a cab today compared with 50 years ago, the differences would probably be significantly less than for an aviator going into a cockpit. However, the rate of change in the rail industry in the next few decades is likely to be significantly greater compared to that of previous years. As such, the issues that may arise from rapid technological change and the potential for unintended or perverse outcomes might well increase exponentially.

It seems reasonable to predict that changes in the rail industry will be similar to those experienced by industries that have already undergone significant innovation, such as aviation. This provides some opportunity to learn from others’ experience. If we look at aviation over the last few decades, the major technological changes (from a human factors perspective) have focused on improving safety and efficiency through automation and, where this is not possible, augmented cognition. It seems likely that similar changes will also occur in the rail industry in the near future. The case for increasing automation in rail operations is based on the idea that a very significant proportion of accidents and injuries result from human error (e.g. Reason 1990). Similarly, the efficiency demands in an operating environment where fuel costs are critical mean that even small improvements in driving performance can carry significant cost consequences at the organisational level.

For organisations dominated by engineering cultures, de-looping the human from these systems or providing automated decision support carries an intuitive appeal. It is argued that if humans are the source of the error, eliminating them or reducing our reliance on their judgment will make the system more efficient and/or safer. This leads to the design of new technologies that seek to eliminate the ‘weaknesses’ of humans in the system so that the system will, to paraphrase the NASA imperative, operate faster, safer, cheaper. However, the literature on introducing new technology suggests that while some benefits will undoubtedly occur, it is generally the case that new and different problems emerge to replace the old problems (McLeod, Walker and Mills, 2005; Sarter, Woods and Billings, 1997). For example, in highly automated systems which are designed to reduce human error from slips and lapses, it can be more challenging for the operator to maintain a good understanding of how the technology is functioning, which can lead to inappropriate control inputs and operational errors (Sarter et al., 1997). Another issue that has been highlighted with automated systems is the potential for the operator to place too much trust in the technology and disengage with the task (Parasuraman, Molloy and Singh, 1993). As we move to an increasingly automated rail industry, it is important to fully consider how the introduction of new technology changes rather than reduces errors, and how systems need to adapt in order to ensure that errors do not translate into accidents. This is an important theme of the book and one that each chapter is designed to address.

One of the focal themes of this book is the question of why the unintended occurs; what do we know about it already in other domains and how might one minimise the likelihood of it in the rail environment? In simple terms, the authors of the chapters in this book see the unintended as the consequences of overly narrow disciplinary focus on a problem. For example, the introduction of computer screens into the cab may provide additional information to support decision-making around the better use of momentum in achieving fuel efficiencies. However, it also means that operators are ‘heads down’ more often and less able to detect unpredictable changes in the external environment that are still dependent on being ‘heads up’, after all, railways are very much an open system. In many cases, the providers of highly technical solutions may lack experience or familiarity with the total task or the ways in which the task is conceptualised and executed by the operator as distinct from an engineer. In some cases it may reflect the different value systems that underlie the different disciplinary perspectives.

One of the key forces shaping the introduction of new technologies in rail is the sheer size or complexity of the rail network in many countries, particularly in freight operations. The need for greater energy efficiency, shorter separations between trains and greater volumes will mean that the complexity of optimising tasks safely may well exceed the capacity of a human decision-making system – especially at 5 o’clock in the morning! Increasingly we are providing workers with more cognitively dense decision spaces. As technology proliferates the operators will need to quickly process multiple sources of complex information in order to make decisions. In many cases we provide information on the basis of ‘more is better’ or because we do not want to leave it out in case it might have been useful and we are held accountable for not providing available information. However, there is a point at which additional information may be counterproductive (Roy, Breton and Rousseau, 2007).

The first problem is one of perception. As the amount of information increases, people struggle to see the critical dimensions of task performance due to low signal to noise ratios associated with high levels of distracting information. That is, important information relevant to a task can get lost in the mass of information that is presented to a user. The second problem is one of comprehension as the large amount of information makes it difficult for people to build and maintain an adequate mental model of the ongoing situation. Important questions that need to be asked about the provision of information to human operators are: What is the optimal information density for the decision space associated with specific tasks in the rail industry? How best can we display this on a monitor? At what point does additional information carry significant risk of distracting from, rather than enhancing, task performance? How do we determine this? How does the information provided by the technology support the development and maintenance of the operator’s mental model? The answers to such questions shape the way we respond to the challenge of developing technologies that allow the operator to develop a good understanding of their operational world, given the increasing complexity of information provided by multiple interacting systems in the context of possibly degraded human functioning. Chapters 2, 7 and 10 address some of the theoretical issues that need to be considered in order to answer such questions along with some clear, simple techniques for determining this information at the task level in a specific organisation.

As the trend continues towards maximising network capacities, and the space separating trains is becoming less, drivers and train controllers must develop more sophisticated and time-sensitive mental models in order to execute the driving task as safely and effectively as possible. As a consequence, traditional ‘foot plate’ training programs may well not train staff sufficiently quickly or enable skill or route knowledge currency to be maintained.

To manage this we have seen a significant shift toward simulator-based training in recent years. According to the advocates of these systems, simulated environments can be used to accelerate training and to ensure currency where real-world currency is difficult to maintain. Experience tells us that these systems have significant benefits but are not necessarily interchangeable with a real-world training environment. What are the effects of significantly altering the relative frequency of situations in a simulated operational environment? Does frequent exposure to danger in a simulator lead to complacency and/or a false sense of security? Does route knowledge developed in a simulated environment translate to the real world? How does the lack of consequence in simulated environments influence risk-taking in the real world? Chapters 8 and 9 deal with these issues in detail.

While a technology may be conceived in isolation, the introduction of a technology into a train cab or railway control room occurs in a broader social, psychological and industrial/political context. The consideration of such factors, which may be outside the scope or experience of a technology solution provider, may determine the ultimate success of a new technology. There are many cases where new technologies can de-skill a task from those whose identity and seniority is defined by their ability to execute it themselves.

For example, when the FuelMiser technology for optimising driving strategies was introduced in the late 1990s in Australia, some train drivers felt devalued by the idea that years of accumulated expertise could readily be replaced by a computer algorithm developed by a physicist who had never driven a train. The algorithm, the screen technology and the way it was initially operationalised in the cab rendered the drivers redundant, and in their terms, they were now nothing more than ‘horizontal elevator operators’. Moreover, the potential to automate the entire operational process using a remote operations approach was not lost on them. Not surprisingly, the inappropriate social and industrial contextualisation of the technology subverted its introduction and the vendors struggled to gain drivers’ and unions’ acceptance of the technology.

Perversely, when the same information was provided as an electronic equivalent of the old ‘coasting boards’ and it was suggested that the new technology merely provided a competitive benchmark against which the drivers could compete to demonstrate who was better in the field – amongst the drivers or against the computer – acceptance was much better. As the drivers built trust in the new technology, the technology was increasingly relied upon when drivers were in difficult, undulating territory or aware of the potential effects of fatigue on route planning and fuel economy. This shows the critical importance of non-technical aspects of new technology, and how a very clear understanding of how best to shape the way in which a technology will be socially constructed is critical to gaining support for and ensuring that the potential benefits of new technologies are realised. Chapter 3 discusses the issue of acceptance and resistance to technology.

Perhaps one of the biggest challe...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Contents
• List of Figures
• List of Tables
• About the Editors
• List of Contributors
• Foreword
• Acknowledgements
• 1 Introduction: The Promise and Perils of New Technology
• 2 Key Technology-Related Human Factors Issues
• 3 Resistance to Technology
• 4 Effective User Feedback: The Practical Value of Mock-ups
• 5 Qualitative Research Rules: Using Qualitative and Ethnographic Methods to Access the Human Dimensions of Technology
• 6 Future Inquiry: A Participatory Ergonomics Approach to Evaluating New Technology
• 7 Using Task Analysis to Inform the Development and Evaluation of New Technologies
• 8 Evaluating Your Train Simulator Part I: The Physical Environment
• 9 Evaluating Your Train Simulator Part II: The Task Environment
• 10 Applying the Theories and Measures of Situation Awareness to the Rail Industry
• Index