How Not to Chase a Chimera
One of the foundational texts of detective fiction, and a classic that continues to be revered, is Edgar Allan Poe’s “The Purloined Letter.” The plot of this short story is deceptively simple. An epistle is stolen from the French Queen containing information that could be extremely harmful to her if revealed to the third parties. The Police know that the perpetrator is Minister D., who uses the letter to blackmail the Queen, and that he keeps it in the hotel apartment where he lives. For various reasons, they cannot arrest or even interrogate him, and the only available course of action is to try to retrieve the letter secretly from that hotel. So, whenever the minister leaves the hotel, which he does frequently, a team of police agents meticulously scours the apartment and the rest of the hotel.
This team of agents searches all of the places it seems the letter could be hidden. They examine “the furniture of each apartment” and open “every possible drawer.” They “scrutinize each individual square inch throughout the [hotel] including the two houses immediately adjoining, with the microscope.” They examine “the moss between the bricks,” open “every package and parcel” and “every book.” They remove “every carpet,” examine the cellars, look to “the mirrors, between the boards and the plates,” and “the bed-clothes”. They look behind “the curtains and carpets,” too (Poe 1994, 342–44).
This work continues for months but without success. And just as the inspector in charge begins to despair (and the reader with him), the solution is provided by an amateur detective, C. Auguste Dupin, in what has become one of the most famous unveilings in all detective fiction. We learn that the entire time the police had been searching the hotel, the epistle was there, right before their eyes, placed in a card-rack “that hung dangling by a dirty blue ribbon” in one of the rooms that had been examined so methodically (Poe 1994, 353). In other words, it was not hidden at all, at least not in the sense which the policemen assumed.
Like any other classic, “The Purloined Letter” can be profitably read in a number of ways, and this potential has been exploited by numerous commentators, including such prominent figures as Jacques Lacan, Jacques Derrida, and Barbara Johnson (Muller and Richardson 1987; Felski 2015, 85). But the message we find most interesting given the purpose of our book is the deceptively simple one that if you look for something, then whether you are a detective or a scholar or anybody else, you have to begin by clearly explaining to yourself what it is exactly that you are looking for. Unless you do, then no matter how methodically you search, you may end up empty-handed.
The policemen in the story were apparently looking for a letter belonging to the Queen, but they also assumed (quite naturally) that they were looking for something hidden. This is why they literally could not recognize the letter that was placed in plain sight as the one they were there for. This is also why they owe the failure of their search to themselves: they were looking for a hidden letter, but no such letter existed. They were after a figment of their own imagination, a ghost, a chimera. All their critical intelligence, skills, equipment, and experience could not help them, and all because of the simple mistake they made in the very beginning.
The continuing relevance of Poe’s story lies in part in the universal character of the mechanism it describes. For some reason, humans have always been prone to make the kind of mistake the police detectives committed: not only in criminal investigations, but in virtually any area of life, including science. The history of any scholarly field is replete with stories of huge projects collapsing precisely because researchers uncritically assumed something about their subject that turned out to be entirely wrong, or that their assumptions about it were too vague to lead to any concrete results (Becker 2014; Loeb 2014). This is why we called the methodological message of the short story deceptively simple. None of us thinks we need to be reminded of such a triviality (“Of course I know what I am looking for!”), but somehow we forget about it every single day. This is why practically all Psychology 101 students are dutifully reminded by their professors to first precisely define the phenomenon they want to study before they embark on gathering data. And this is why professors need to remind themselves about it too.
In the Introduction, we used a couple of different formulations in order to describe the subject of our study. We said that we are interested in whether stories can raise our concern for the well-being of animals, shape the way we think about it, and can have an impact on our attitudes toward it. All these phrases seem to refer to one phenomenon, but the contours thereof seem to be blurred. As we did not want to end up chasing a chimera, we began our project by defining what it is that we are after, and we eventually agreed that we should focus on measuring the impact of literary stories on attitudes toward the well-being of animals.
Admittedly, such an influence is not everything that scholars and animal advocates refer to when they talk about the moral or social impact of animal stories. They also talk about plenty of other things such as an increase in empathy, the expansion of imagination, the raising of awareness, the changing of behavior, and the like. We agree with them that all these things are real phenomena, and that our focus merely on attitudes will present only a partial picture of what the moral impact of animal stories consists in. But still, that part of the picture is important, and moreover, it can be defined precisely enough to yield concrete results when studied.
The main reason it can be defined with such a precision is that attitudes belong to one of the most thoroughly researched social psychological phenomena. In fact, there was a time, back in the 1920s, when “such was the importance of work on attitude measurement that that social psychology was often defined as the study of attitudes” (Maio and Haddock 2012, 5). While it cannot accurately be defined in this way now, there is no doubt that as far as attitudes are concerned we can take advantage of a massive body of work produced over more than a century of research (Eagly and Chaiken 1993; Eagly and Chaiken 1998; Crano and Prislin 2008). What this means is that researchers have had enough time to produce sound conceptualizations of attitudes and develop sophisticated techniques for measuring them, not to mention commit numerous errors from which we can learn today.
Attending to Attitudes
The definition of “attitude” that we assume in this work is the standard psychological one, which states that an attitude is “a psychological tendency that is expressed by evaluating a given entity with some degree of favor or disfavor” (Eagly and Chaiken 1993, 1). If an individual is prone to conceive of X as possessing a given value (positive, negative, or anything in between), then he or she can be said to have an attitude toward X. All of us harbor attitudes toward numerous things – be they tomato soup, rats, or the philosophy of Kant – and attitudes constitute a fundamental part of our identity. Just think about how often we define ourselves in terms of whether we value this or that, and how often we judge others on the basis of whether they attach an equal value to those things. Why is it you think detective novels are trash? How can Andrew enjoy hunting? Why is Beverly in favor of euthanasia? How can anyone savor foie gras? If you consider this, and if you additionally consider how much social conflict is due to differences in attitudes and the great extent to which social cooperation is dependent on managing attitudes, then it becomes clear why social psychology is so concerned with attitudinal change and with how to measure it.
Just as in our project we adopt a textbook understanding of attitudes, we also adopt a textbook, tried-and-tested, way to measure them. The method relies on so-called scales: sets of questionnaire items which allow only for fixed, quantifiable responses, where the total score achieved on all the items indicates the extent to which a person harbors a given attitude (Maio and Haddock 2012, 10–12). A typical item in a scale would be a statement of a particular belief, accompanied by a range of answers describing different levels of agreement with that statement. For instance, the statement might be something like “The slaughter of whales and dolphins should be immediately stopped even if it means that some people will be put out of work,” plus a selection of answers that indicate how strongly or otherwise one believes the statement, ranging from “Strongly agree,” through “Agree,” “Undecided,” and “Disagree” to “Strongly Disagree,” where these answers are ascribed points from five to one, respectively. That particular example is in fact taken from an actual scale which was developed by Harold Herzog and colleagues in order to capture whether one’s attitude is more or less sympathetic toward animal welfare (Herzog, Betchart, and Pittman 1991). The higher the score achieved on all the items in that scale (called the Animal Attitudes Scale, or AAS), the more pro-animal one can be taken to be. There are 20 items altogether in this scale which means that the total score can range anything from 20 (the least pro-animal welfare) to 100 (the most pro-animal).
But note that not all items in the AAS are like the one about dolphins and whales, where a higher score obviously represents a more pro-animal attitude than a lower one would. Consider, for instance, the following item: “Basically, humans have the right to use animals as we see fit.” Obviously, if one strongly agreed with this one, and scored more points on it, then we would be entitled to suspect that the person in question is less pro-animal welfare than a person who strongly disagreed with it. Such items are “reverse scored” because in calculating the summary score achieved on the scale, one has to translate the score achieved on these items by reversing it. For instance, if somebody replied “Strongly disagree” to the item on using animals as we see fit, then they would receive five points, not one; if they replied “Disagree,” they would receive four, not two, points, and so on.
Such items are to be found not only in Herzog’s scale, but in practically all scales used by psychologists. And if one wonders why they are there, then the answer is not that psychologists like to make life difficult for themselves, but rather that their subjects often make life difficult for them. It so happens, terribile dictu, that not all people who volunteer to participate in psychological studies value the scientific truth enough to fill the questionnaires the way they are supposed to be filled, that is, by reading carefully all the items and giving thought out and sincere answers to them. Some do not feel like making that kind of effort and answer randomly or by applying a kind of system that allows them to answer the questions automatically by giving the same, or roughly the same answer to each item. Moreover, one cannot rule out that among one’s participants there will be people incapable for this or that reason of fully grasping the meaning of the items. By putting reversed items in their questionnaires, psychologists are able to detect such respondents and reject their questionnaires as invalid (Maio and Haddock 2012, 12). For instance, if somebody replied “Strongly agree” to all the items on Herzog’s scale (including the one on dolphins and the one on using animals as we see fit), then that would mean that he or she was either cheating or did not understand the items. Either way, no researcher should take such a questionnaire into account.
But in order to be trustworthy, scales need not only to contain reversed items but, most importantly, have to meet the criteria of validity and internal consistency. “Internal consistency” concerns the correlation between replies to particular items within the scale. In other words, if a scale measures the same phenomenon, (or construct, as psychologists like to say), then the scores for particular items should correlate when tested on a larger group of people (Maio and Haddock 2012, 20–22). For instance, the subjects’ scores on reversed items should be lower than those on the non-reversed items, and they also should be similar within each category. If they are not, then we are entitled to judge that the scale which includes them is inconsistent and therefore useless as a psychometric tool.
Validity, in turn, means that a scale picks out exclusively, and at the same time comprehensively, the phenomenon it aims to study (Maio and Haddock 2012, 20–22). Thus understood, validity may seem to be so obvious a criterion that it does not even merit stating, but the fact that in practice it is very often violated (including by experienced researchers) indicates that the question does demand special attention. As to how easy it is to err in this regard, consider the hypothetical item, “I enjoy watching wild animals in their natural habitat,” which might prima facie seem a good candidate for inclusion in a scale measuring attitudes toward animal welfare (at least insofar as it apparently constitutes the opposite of the item “I sometimes feel upset when I see animals in cages at zoos,” which does appear in the AAS). However, “enjoy” is an ambiguous term that denotes all kinds enjoyment (moral, aesthetic, cognitive, etc.), and for this reason we cannot be sure that the respondents will understand our item in a way that could reveal the level of their concern for animal welfare.
For instance, if “I enjoy watching wild animals in their natural habitat” is understood in purely aesthetic terms, it will have nothing to do with concern for the welfare of these animals because taking pleasure in the aesthetic qualities possessed by a living being does not necessitate caring for its welfare. And conversely, if someone “enjoys” watching wild animals in their natural habitat in the sense that he or she is happy that these animals can fully exercise their natural capacities, then this may have nothing to do with appreciating aesthetically what he or she sees. Thus, both a committed animal activist and a recreational hunter may reply to that particular question in the very same way, which may negatively affect the validity of the scale as a whole. One lesson that flows from this example is to avoid ambiguous statements since their content may be filled out by the participants in many different ways.
The AAS, which we have been referencing throughout this sub-chapter, does a fine job at avoiding ambiguity, and it is also possessed of a very high level of internal consistency (H. A. Herzog and Mathews 1997, 171). This is one of the reasons why it has been used by various scholars across different fields (Flynn 2003; Rothgerber 2014; Lee et al. 2015). However, despite its being a prima facie natural choice for our project,1 we could not use it for two reasons. The first one is its cultural specificity. It was developed in the USA and bears the cultural marks of that country to such an extent that some...