Adapting Psychological Tests and Measurement Instruments for Cross-Cultural Research

An Introduction

Vladimir Hedrih

• Publisher: Routledge
• Year: 2019
• ISBN:
  9780429559808
• Edition: 1
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Adapting Psychological Tests and Measurement Instruments for Cross-Cultural Research provides an easy-to-read overview of the methodological issues and best practices for cross-cultural adaptation of psychological instruments.

Although the development of cross-cultural test adaption methodology has advanced in recent years, the discussion is often pitched at an expert level and requires an advanced knowledge of statistics, psychometrics and scientific methodology. This book, however, introduces the history and concepts of cross-cultural psychometrics in a pedagogic and simple manner. It evaluates key ethical, cultural, methodological and legal issues in cross-cultural psychometrics and provides a guide to test adaptation, data analysis and interpretation.

Written in an accessible manner, this book builds an understanding of the methodological, ethical and legal complexities of cross-cultural test adaptation and presents methods for test adaptation, including the basic statistical procedures for evaluating the equivalence of test versions. It would be the ideal companion for undergraduate students and those new to psychometrics.

1
Culture

Culture as a concept

For issues related to cross-cultural adaptation of psychological measurement instruments, culture is a central concept. Culture can be considered a frame that gives meaning to behaviors, gestures, words and relationships between people. It represents a general context in which all of these happen. For example, if we see two people on a street encompassing each other with their hands, culture will determine whether we will perceive these two persons as two people in romantic love embracing each other, or as close friends who have not seen each other for a long time greeting each other. Culture will also determine whether we will see this gesture as an expression of friendship, or of domination, or whether we will perceive this as an ongoing fight and the two persons fighting each other. Geert Hofstede (Hofstede, 2011, p. 3) defines culture as “the collective programming of the mind that distinguishes the members of one group or category of people from others”, although there are many other definitions. Straub et al. (2003) divide definitions of culture into several categories: 1) definitions based on common values, 2) definitions based on problem solving and 3) general, all-encompassing definitions. These first two categories comprise the central part of what people typically understand as culture.

Hofstede et al. represent manifestations of culture as a group of concentric circles:

• In the innermost, central circle are common values defined as “wide, nonspecific feelings for good and bad, beautiful and ugly, normal and abnormal, rational and irrational”. They state that these values create feelings that are often unconscious and that are rarely subject to discussion, but which still manifest in behavior.
• The second circle are rituals – collective actions that are practically superfluous, but are essential from the social standpoint, and are therefore performed for their own sake.
• The third circle of manifestations of culture represents heroes – “persons, alive or dead, real or imaginary who possess characteristics highly prized in the cultures and who thus serve as models for behavior”.
• The fourth and the widest circle represents symbols – “words, gestures, pictures and objects that carry a certain meaning within a culture”.
• These authors consider symbols, heroes and rituals to be examples of “practices” or common behaviors because these three types of manifestations of culture are visible to an external observer, “although their cultural meaning lies in the way they are perceived by insiders” (Hofstede, Neuijen, Ohayv, & Sanders, 1990, p. 291).

From the description of what culture is, it is clear that culture is a collective phenomenon first. Common values require a community for which these values would be common. But how large does a community need to be so that it can be justifiably considered to possess a culture of its own? We know that individual persons are not all the same, but that they differ in many things, including values, and surely in all these other constructs that comprise manifestations of culture. And also, when we observe any larger natural group of people, how do we know if all members of that group belong to the same culture?

In common speech, culture is primarily tied to ethnic groups, nations or sometimes to groups of people speaking the same language. However, aside from such uses of the term culture, there is also the concept of “organizational culture” and the concept of subculture. The concept of subculture refers to a smaller group of people that are part of some bigger, usually national, culture, but who have some specific cultural characteristics of their own. The concept of “professional culture” is also being used ever more and this is a concept based on the data of ever-increasing body of research showing that, in many aspects, people working in same professions from different countries may be more similar than even people from the same country working in different professions.

Considering such a wide scope of the concept of culture, it should be noted, as was noted by Straub et al (Straub et al., 2003), that there are individual differences between people within each group, and that they do not all accept the same values literally or to the same extent. These authors also state that the same person may accept an array of different cultural patterns, i.e., that influences of different cultures may manifest themselves in the same person. In accordance with this, they suggest that each individual be considered a combination of cultures or subcultures it belongs to. Aside from the national culture, these cultures should include cultural patterns of different collective identities the person accepts, such as gender, profession, sports club and of other smaller social groups cultural norms of which the person accepts. These authors believe that, within this approach, which they consider to be based on the social identity theory, culture should be assessed on the individual level, by examining the individual. In this way, culture would be studied as an individual phenomenon, and conclusions about the culture of the entire group could then be based on the aggregation of individual data.

A question that than arises is which definition or what scope of culture should be taken into account and applied in the practice of psychological testing? Taking into account only cultures of large social groups, such as nations, would potentially lead to psychological testing practices providing inadequate results for many individuals whose culturally determined psychological characteristics differ from those typical for the majority of their compatriots. On the other hand, adopting an approach that would take into account cultural differences on the individual level would make the process of psychological testing so complicated that psychological testing would probably be impossible without the use of complex software, if even then. It is quite probable that such a practice would also compromise one of the key requirements of psychological testing – the requirement that psychological tests be administered, scored and interpreted in the same way for all test-takers.

A solution that is the most common in practice is that the criterion for the maximum size of the social group is the language. In the maximum variant, a test is, without any additional adaptations, used on test-takers who speak the same first language. If test-takers do not share the same first language,¹ most psychologists would now agree that assessing them with the same test would be problematic at least, and that the test should be adapted to the first language of test-takers. And while it is up to debate whether it is justified to create special adaptations of a test for smaller social groups, the need to create different language versions of a test for people who speak different languages is an issue about which there is more or less a general consensus.

Culture, language and psychological testing

Why is culture important for the practice of psychological testing? For a psychological test to function as intended, it is necessary that it be administered, scored and interpreted in the same way for all participants. For test scores to be valid, it is necessary that responses of the test-taker to stimuli in the test (test items) be influenced or incited by the same psychological trait or construct – the trait or construct the test was designed to measure in the test-taker. If it so happened that the same item produced responses influenced by one psychological trait in one group of test-takers and responses caused by some other, completely different, psychological trait in another group of test-takers, that would completely compromise the integrity of the testing procedure. In the same way, if differences in familiarity with test contents, which of themselves are not the construct that the test proposes to measure, caused certain items to be harder or easier for test-takers with the same level of the measured trait from one population then for test-takers from another population, this would represent a source of variability of test scores that would seriously compromise the validity of conclusions drawn from the test. For example, if some general information test contained an item asking the test-taker to name the US state in which Salt Lake City is located, such an item would be much easier for test-takers living in Utah, USA, then for test-takers living for example in England, UK with the same level of the measured trait or construct.

Culture, as a framework that gives meaning to actions, words and objects, critically influences ways in which a person will interpret the meaning of various elements of the psychological test as well as the meaning of the test as a whole. Cultural differences cause or may cause two different persons to attach different meaning to the same elements of a psychological test, and in that way cause the psychological test to function differently for these two persons.

From a practical standpoint, cultural differences create problems for the practice of psychological testing by causing the same test to sometimes function differently when used on test-takers belonging to different cultures. For these reasons, modern standards for psychological testing (International Test Comission, 2017) proscribe that the equivalent functioning of a test in two cultures or in two different populations may not be presumed in advance, but must be empirically verified. Aside from that, differences between cultures, as well as properties of each culture are not static, but tend to change over time. For this reason, the equivalence of functioning of the same test in different cultures must be periodically reexamined.

When considering the relationship between culture and language, it should be noted that language need not represent a border of a culture. Although language and culture are often equated in everyday life, in the sense that members of the same culture speak the same language, this need not be always the case. It may be possible that speakers of the same or of very similar languages belong to cultures that are so different that the validity of a test that works fine in one group would be completely compromised in the other group without adaptation. In the same way, it might be possible to find groups that speak different languages, but whose cultures are similar enough for psychological tests that are valid in one group to function adequately in the other with only a simple translation to the other language.

Related to this issue, one very important factor that needs to be taken into account is globalization. Globalization is typically defined as an increased interaction between people through growth of international flow of money, people and ideas (https://en.wikipedia.org/wiki/Globalization). The start of globalization is usually placed in modern times and is especially related to the expansion of internet, but there are authors who believe that we should look for the first moments of globalization in the European “Age of Discovery”, particularly in the time period when European sailors discovered the Americas and set forth exploring and conquering the world. Although the concept of globalization seems to primarily refer to the process of economic integration and strengthening of international exchange, it also has important social and cultural aspects. Through increased communication, travel and exchange between cultures, globalization, on one side, increases differences between inhabitants of one territory, i.e., inside national groups, and on the other hand, reduces differences between cultures throughout the planet.

Increases in differences between inhabitants of a certain territory happens because, through communications and exchange of cultural contents, individuals obtain an opportunity to adopt cultural norms and values that are dominant in some other, often geographically distant, social groups. Aside from that, moving of people through emigration and immigration leads to a situation in which a single territory that was once ethnically, culturally and linguistically relatively homogenous, now hosts members of different cultures who bring with them their values and other aspects of their culture. Reduction of differences between cultures happens through multiple mechanisms:

• Members of various cultures throughout the world are now exposed to same cultural products or contents (movies, music, media contents) thanks to the availability of international exchange of cultural products, thus producing an opportunity to change the properties of the domestic culture by adopting cultural elements contained in these cultural products.
• People learn foreign languages (currently, mostly English), in order to be able to understand people who do not speak their language. Through this activity, they adopt and become aware of concepts contained in the foreign language, which might not even exist in their own language. They also become aware of connotative meanings of words and expressions in the foreign language.
• People more often meet people belonging to cultures different from their own and have more opportunities to communicate either directly or through communication devices. Communication and exchange allows people to be acquainted with properties of other cultures, and, through time, this creates opportunities for synchronization of values and other elements that comprise culture.
• The synchronization of characteristics happens through intentional creation of similar or compatible national institutions with the goal of making international flow of people, ideas and capital easier – this process can be observed in various areas from the organization of government administrations, through laws and their contents, to the synchronization of educational systems and systems of professional qualifications. For example, in many European countries, one of the requirements for a university study program to be accredited is that its contents must be similar enough to contents of programs that educate people for the same professions in foreign countries (one of the components of the Bologna process – https://ec.europa.eu/education/policies/higher-education/bologna-process-and-european-higher-education-area_en). The national laws of most countries are often required to be in line with various international treaties, conventions or norms of various international organizations, and this causes them to be similar to laws regulating the same area in other countries.

In this way, there are less and less large differences between societies of various countries, and through this, between cultures. This trend is visible in some areas even when psychological constructs, i.e., functioning of psychological test, is in question (Hedrih, Stošić, Simić, & Ilieva, 2016). For example, in the area of vocational interests assessed through the scope of Holland’s theory, during the second half of the 20th century, researchers often obtained results showing inadequacy of this theory in various countries. In contrast, in the first two decades of the 21st century, such results seem to be much rarer. Even studies in some countries where negative results were previously obtained, for example in China, now produce results that confirm the validity of both this theory, and of tests based upon it (Long, Adams, & Tracey, 2005).

We should also be aware that effects of globalization do not seem to reach all parts of a society equally. While there are parts of society, i.e., groups of people who are intensively involved in the process of international or intercultural communication and exchange, there are also parts of the society these processes reach much more slowly or not at all. In less developed, poorer strata of a society, among nonintegrated, isolated or semi-isolated social groups, as well as among the older or less-educated people, we can expect these effects to be much less pronounced than in, for example, groups of young people, educated in the scope of the official school system and who grow up in places and in conditions that provide them ample opportunities to come into contact with foreigners and foreign cultural contents.

We can conclude from everything previously listed that in a large number of practical situations, the decision if two persons should be treated as belonging to a single culture or as members of different cultures depends on multiple factors. However, one factor that surely represents a clear border when psychological tests and psychological testing are in question is the language the person speaks. It is probably self-evident that there is no point in administering a psychological test to a test-taker if the said test is in a language the test-taker does not understand. For this reason, from a psychometrics point of view, language represents a hard border, marking a line at which test adaptation is obligatory. But creating a version of a test in another language is far from being an issue that can always be solved by a simple translation.

Unlike most other materials, where the goal of the translation process is to produce a translation that is “as accurate as possible”, with psychological tests, accuracy of the translation is not as important as obtaining a version of a test that is “psychologically” identical to the original. Each psychological test is composed of a series of stimuli, i.e., items, each of which is carefully selected so that, when administered to a test-taker, it produces a response caused by the very psychological trait or construct the test proposes to measure. If translated stimuli (items) in the new language version of the test no longer produce responses caused by the trait or construct the test proposes to measure, such language version of the test is of no practical use, even though it might be very accurately translated. This is the reason why the process of creating a new language version of a test is termed adaptation and not translation. From a ps...

Table of contents

• Cover
• Half Title
• Title
• Copyright
• Contents
• Preface
• 1 Culture
• 2 Copyright and author’s rights
• 3 Test adaptation
• 4 Assessing equivalence of different language versions of a test
• 5 Interpretation of individual results
• 6 Rights of test-takers, legal and ethical issues of psychological testing
• Index