Arts evaluation is a term that is both innovative and daunting. The inception of any arts organization, program , performance, exhibition, or collection comes with the need for assessing the value of these experiences and understanding how people appreciate and interpret different art forms. When we attend the opening of a new gallery, we bring expectations for the visual art we will examine and analyze. When we attend the opening of a new concert, play, dance , or musical, we prepare to experience the performing arts in new and exciting ways. When we spend our weekends at the local movie theater , we expect to be entertained and transported by film . When students prepare for a visit from a teaching artist , they anticipate learning about the arts, engaging and participating, in addition to diving into their daily curriculum .
In that sense, as participants, audience members, and patrons of various arts forms, we are constantly evaluating what the arts mean in our lives (Rajan, 2015). These informal assessments help to shape our own understanding of the arts, help us to learn about the arts, and learn from the arts, even if we are not formally doing so. As such, the idea of evaluating the arts should seem simple and obvious, right?
Probably not. The term evaluation itself has been known to challenge even the most experienced artist and educator . But why? Evaluation is a term simply defined as ājudging the merit, worth, or value of somethingā (Scriven, 1967). Isnāt that a component of what we are already doing with the arts? Why then, is evaluation, and evaluating the arts, such a difficult concept ? And, one that is often relegated to the idea that the impact of the arts canāt be measured?
This dichotomy is not necessarily new and certainly not a product of evaluation as a burgeoning field of study and practice . The arts themselves have been challenged in our schools, curriculum , and community (Chand OāNeal & Runco, 2016; Rajan, 2012a, 2012b). Our culture has to value the arts before our educational system can. It is only in recent years that the experience of music, theater , dance , visual, and multimedia arts has been given credibility and value in our school system (Chand OāNeal, 2014; Rajan, 2015). And, that change has been coming. The evidence is found in the development of an educational outreach program in nearly every arts organization across the United Statesāfrom national champions of the arts like the Lincoln Center and the John F. Kennedy Center for the Performing Arts, to smaller, regional organizations found in each state and city. Each of these development or outreach programs brings the necessity of evaluating and assessing the strength, challenges, benefits , and viability of each program.
The purpose of this chapter is twofold. First, we hope to present an overview of evaluation theories, models, and applications to readers who are unfamiliar with this field. Additionally, we hope that those who are reading this book with a strong understanding of evaluation practices will be challenged further to examine how evaluation is applied and aligned to meet the unique requirements inherent to the various artistic disciplines outlined in each chapter. We begin then with a brief description of program evaluation, both as a supplementary component to grants and as its own field of study, before applying these ideas to the arts.
Throughout this book, we encourage you to think about the following questions that are woven throughout each contributed chapter: How has program evaluation changed in the past decade? What are the challenges arts evaluators face that other disciplines avoid (and why)? How can we strengthen our own understanding of the arts, and capacity to experience the arts, through evaluation? How do our findings support, augment, or impede arts policy and practice ?
So, What Is Evaluation?
Program evaluation as a field of study is relatively new. A theory-based practice , āthe evaluation process identifies relevant values or standards that apply to what is being evaluated, performs empirical investigation using techniques from the social sciences, and then integrates conclusions with the standards into an overall evaluation or set of evaluationsā (Scriven, 1991). Guskey (2000) added to this definition by stating that evaluation is a systematic process used to determine the merit or worth of a specific program, curriculum , or strategy in a specific context. One perspective views evaluation as originating as a counterpart to grant writing (Rajan & Tomal, 2015). Much of the challenge with defining evaluation is that the term is often used interchangeably with research , and they are not the same. While research is traditionally a hypothesis-based, systematic collection of data for analysis and generalizability, evaluation is the theory-driven, systematic collection of data focused on assessing a programās goals and viability. Though research and evaluation are both grounded in scientific inquiry , evaluation is often used to give focused information on a specific program. Evaluation is about finding ways to sustain an organization, maintain connections to the community , and utilize innovative methods for documenting the individual strengths and benefits of a program. An evaluator has to have the ability to ānot only state preferences, but render judgments ā (Eisner, 2007, p. 425). This is even more important when considering not just what we are evaluating, but how our evaluations can make an impact.
Program managers are now finding great value in the systematic documentation of organizationsā goals and the development of measurable outcomes . Evaluation reports are proudly displayed as part of an organizationās mission and outreach. Program managers want us to know that they are investing in evaluation. This, in turn, has also made evaluation more accessible for practicing professionals. Consider how there are now several universities that offer degrees, courses, and certificates in program evaluation. There are graduate programs dedicated to understanding evaluation theory , logic models, and methodology . And, ironically, these programs are often housed in a department or division of research . As the field of evaluation continues to grow (and gain validation) so too does the need for individuals to have an educational background that demonstrates their understanding of evaluation theory.
But, does that really matter? Do we need a degree in evaluation to be an evaluator? Sure, we wouldnāt want a physician to treat us without a recognized medical degree, or teachers working with our children who didnāt have the licensing to support their knowledge base. What makes evaluation different? And, how do we define this field (especially in the arts)āboth for its programmatic value and practical application?
Evaluating the Arts
It is no secret that arts programs typically receive less funding than other disciplines (Rajan, 2015). Some of this is attributed to how the arts (and their value) need to be continuously justified (Cohen, 2009). But, the argument is deeper than how the arts support student achievement in the form of grades and standardized testing, or whether they generate more ticket sales. The larger issue is how are we assessing the impact and value of the arts? Is it documenting the quantity or quality ? Consider how āa majority of funders require that quantitative data be included in the final report , something that is not always accounted for when planning an artistic performanceā (Rajan, 2015, p. 32). It is important to capture not only what is happening and how it is being done in a program, but to maintain the integrity of each art form in the process.
Early efforts to address the relationship between student outcomes and arts programming began in 1982, the year the Presidentās Committee on the Arts and the Humanities (PCAH) was established by Ronald Reaganās Presidential Executive Order. The dual purpose of this Committee was to embolden private sector support, and to increase public appreciation, understanding , and value of the a...