Experiments

10 Key excerpts on "Experiments"

• 

  eBook - ePub

  Studying Cities and City Life

  An Introduction to Methods of Research

  • Mark Abrahamson(Author)
  • 2016(Publication Date)
  • Routledge

    (Publisher)

  chapter 3 Experimental design

  Outline

  • Experiments introduced

  ∘ Manipulating the independent variable

  • Contrasting Experiments and ethnographic studies

  • Experimentation in social science

  ∘ Urban Experiments

  ∘ Helping behavior

  ∘ Climate change

  • Causal inference

  ∘ Temporal ordering

  ∘ Association

  ∘ Spuriousness

  ∘ Statistical and experimental control

  ∘ Theoretical congruence

  ∘ Types of causal reasoning

  • Experimental designs

  ∘ One group design

  ∘ The classic experiment

  ■ Pre-test contamination

  ∘ More complex designs

  ∘ Simulating an experimental design

  • Assigning subjects to experimental conditions

  ∘ In a natural setting

  ∘ In a laboratory setting

  ∘ Contamination and blinding

  ∘ In Internet-based Experiments

  • Internal and external validity

  ∘ Internal validity

  ∘ External validity

  • Notes

  • Glossary

  The first thing that comes to many people’s minds when they think of an experiment is a laboratory. They imagine a white-coated scientist surrounded by test tubes. There is relatively little about social life in cities that can be examined in a laboratory, so if that was all there was to experimentation we would not be interested. However, the principles of experimental design can be applied outside of laboratories and utilized very effectively with other research methods.

  What is valuable about the principles of experimental design is their unmatched ability to enable an investigator to reach a conclusion concerning a causal connection between variables. To illustrate, consider the other everyday meaning in which to experiment typically means to change something, and then note its consequences. For example, a student may try re-writing notes after each lecture, rather than waiting until just before an exam, and see whether test scores improve. If the student was really anxious about grades, then the student might also attend class more often, spend more time reading assignments, participate more in classroom discussion, etc. All of that might combine to produce the desired result, a better grade; however, it would also spoil the experiment because it would then be impossible to know how much any one of the changes contributed to the improved result.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Laboratory Experiments in the Social Sciences

  • Murray Webster, Jane Sell(Authors)
  • 2014(Publication Date)
  • Academic Press

    (Publisher)

  Although Experiments are a recognized part of today’s social science research techniques, for many social scientists they are still not well understood. Training in laboratory experimentation is still not part of the graduate training of the majority of social scientists (psychology may be the exception). That is unfortunate for many reasons. Those who might wish to conduct experimental research may not feel confident enough in their skills to approach this method. Social scientists who use other methods, such as survey researchers in sociology, may misunderstand the goals and uses of Experiments. Because every science relies on peer review of research, misunderstandings can slow the accumulation of knowledge; good Experiments may be criticized on inappropriate grounds, and real flaws in an experiment may be overlooked.

  With the continuing growth and development of experimental methods in social science, it will not be long before understanding Experiments is an important part of every social scientist’s professional skills. We and the other authors in this book hope to contribute to that understanding. For new experimenters, we offer suggestions to improve the quality of their work; for those who read and wish to assess experimental research, we describe techniques and offer guidelines. In these ways, we hope to contribute to the growing quality of experimental research in social science. A poorly designed experiment will either produce no results or, worse, will produce results that are not what an experimenter thinks they are. This book brings together “best practices” by several of today’s outstanding experimental researchers. The chapters can be read as “how to” manuals for developing one’s own Experiments or as sources of criteria to judge and improve the quality of experimental research by practitioners and by the professional audience. All of the chapters contain background to their individual topics that explicitly address common and some uncommon points crucial for understanding this method.

  Experimental research is one kind of intellectual activity. A good way to approach Experiments, either those one plans to conduct or those conducted by others, is to ask what they contribute to knowledge. What do we know as a result of an experiment or what do we hope to learn from a contemplated experiment? As will be clear in several of the chapters, the central issue in experimental research, as well as in other kinds of research, is how the research can contribute to knowledge of social processes and social structures.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Social Work Research for Social Justice

  • Beth Humphries, Jo Campling(Authors)
  • 2008(Publication Date)
  • Bloomsbury Academic

    (Publisher)

  In social situations, of course, the opportunities for such laboratorylike conditions are severely limited, and ‘naturalistic’ (or field) Experiments involve people in the context in which they live their lives, not controlled by the researcher. It is this kind of experimental research that is most commonly used in social work and social care. It is of course this very characteristic (the attempt to control the conditions of the study) and the ethical implications flowing from issues of control and of the ethical dimension of administering or refusing ‘treatment’ that raises questions about the validity of Experiments in the social world, of which more later. Sarantakos (2005, p. 181) sets out the strengths of Experiments:

  Replication – Experiments are constructed in a manner that allows replication, assuming that repeating the procedure will not lead to different results.

  Prediction – their structure and process, and their detailed and rigorous design allow reasonable predictions to a higher degree than many other methods.

  Causality – Experiments possess all the methodological parameters required for establishing causality.

  Precision – rigorous planning and checking of the status of variables, and the validity and reliability of methods allow a high degree of precision in all steps of the research process.

  Convenience – the size of the samples, the detailed and accurate preparation of the experimental conditions, and the detailed research design make Experiments a most convenient research method.

  As can be deduced from the above, experimentation involves a fair degree of manipulation of the research environment in order to rule out factors other than those related to the variable introduced to stimulate change. In the everyday life of human beings this is very difficult to achieve unless people are confined, with all other factors closed off. Even then it is near impossible to achieve the laboratory conditions that exclude distortions and allow the experiment to remain intact. Hypotheses relate to situations, and what is true in one situation may be false in others – ‘what works’ cannot necessarily be generalized, or taken from one setting and imposed on another. Moreover, the values and interests of the researcher influence the questions asked, the experimental (or any other) design, and the way data are interpreted. The whole process can be viewed as an overly mechanistic way of viewing what is in fact a dynamic interaction of social factors. Nevertheless experimental designs have been employed extensively in social work and social care for the evaluation of interventions.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Research Design

  Succesful Designs for Social Economics Research

  • Catherine Hakim(Author)
  • 2012(Publication Date)
  • Routledge

    (Publisher)

  9 Experimental social research

  DOI: 10.4324/9780203354971-9

  The sole purpose of experimental research is to study causal links: to assess whether a given factor X has an impact on another factor Y, or whether changes in one variable produce changes in another. The simplest studies seek only to ascertain whether or not there is a direct link between the two factors. More complex studies seek, in addition, to assess the magnitude or importance of the effect produced (the size of the effect) and even the relative importance of the effect, that is, relative to other factors that also produce a similar impact. Experimental social research is thus relatively narrow, or focused, in the type of information it produces, but it can provide more definitive answers to questions about causal links than do other types of study, and is hence essential for the development of soundly based explanations of social events, behaviour and attitudes.

  As noted in Chapter 11 , sample selection bias can be a problem in some studies. Experimental studies should completely eliminate this problem. The two essential elements of the experimental design are randomisation and controlled application of the factor X that is to be studied. Experimental control involves deciding which individuals, groups or organisations will be exposed to the information, experience or event whose impact is to be tested, rather than allowing them to choose (or even allowing someone else to make the choice for them) – on the grounds that those who voluntarily elect to participate are immediately identified as different from those choosing not to, so that the impact of the experimental factor will be confounded with the impact of selection effects (also termed self-selection effects). Randomisation involves allocating the people or other units being studied to the experimental group (which is exposed to the information, experience or event being tested) or to a control group (which is not exposed to the same experience, or is given a ‘placebo’ treatment instead) on an entirely random basis, taking no account of their characteristics or preferences. The logic of random allocation to the experimental and control groups is that the two groups thus formed will be exactly similar in all aspects relevant to the study, except for the single difference of being exposed, or not, to the experimental treatment. The two groups are studied before and after the experimental treatment and at the same points in time. Comparisons of the ‘before’ and ‘after’ information for the two groups then allows conclusions to be drawn about the impact of the treatment, if any. More complex versions of this design are also available, for example using four groups instead of two; using several levels of intensity in the treatment applied to the experimental group; or carrying out repeated follow-ups of the two groups in order to differentiate the immediate short-term impact from the longer-term impact, as these may differ in strength and in character (Stouffer, 1950 ; Campbell, 1957 ; Campbell and Stanley, 1963 ; Cook and Campbell, 1979

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Principles of Methodology

  Research Design in Social Science

  • Perri 6, Christine Bellamy(Authors)
  • 2011(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  Most people have some idea about experimental methods used in the natural sciences from their school days. Many of us recall doing simple laboratory Experiments in chemistry, in which the one thing that is allowed to vary is the presence, absence or perhaps the amount of one compound. Assuming that we kept the apparatus at constant heat, and did not turn it to a different angle or allow the amounts of the other compounds to vary, then, when our substances turned blue, crystalline or whatever, we were able to conclude that the only possible cause was the introduction – or a change in the amount – of the compound we were testing.

  In social science, too, there is a place for experimental research. Psychologists do laboratory Experiments as a normal part of their research. Occasionally, other social scientists do Experiments, too. In sociology, economics and political science, for example, researchers who are interested in how people make decisions conduct laboratory Experiments with volunteers, by asking them to carry out tasks, play decision-making games, or undertake exchanges with other participants. In each run, the researchers vary some variables – the information provided, the age or experience of the participants, the participants’ abilities to communicate with each other, the nature of the tasks or games or exchanges, and so on – to see whether any of these variables affects the nature of decisions or the process of decision-making.

  However, this kind of work represents a small proportion of the research done in political science, sociology, anthropology, development studies, business and management studies or economics.

  Field Experiments

  A common critique of laboratory Experiments is that their highly controlled conditions are not sufficiently like the ones we see in everyday life to warrant inferences from the controlled to the social environment. So a compromise is sometimes sought between control and external validity, by attempting a field experiment. In development studies, for example, studies have randomly assigned villages with similar social, educational, transport and infrastructure conditions, to different ‘treatments’ or programmes of economic development (e.g. Humphreys and Weinstein, 2009).

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Research Methods and Organization Studies

  • Alan Bryman(Author)
  • 2003(Publication Date)
  • Routledge

    (Publisher)

  Chapter 1 by Ganster (1980) and Orpen (1979) allow claims about the effectiveness of job enrichment to be directly evaluated; does enriched work cause greater job satisfaction and individual performance than non-enriched work? The experimental design that was employed by each of these researchers allows the causal hypothesis that underpins this question to be examined. In this chapter, the factors that facilitate the detection of causality from experimental designs will be elucidated.

  Second, because of the facility with which researchers employing experimental designs are able to establish cause and effect, the experiment is frequently perceived as a model research design. This point will be especially apparent when survey research, from which causal findings are less easy to establish, is examined.

  In everyday speech, we frequently employ the term ‘experiment’ very loosely to denote trying something out and observing the consequences of our action. Social scientists and others who write about Experiments see them as involving something more than merely trying something out; in particular, the idea of ‘control’ is essential. The idea of control implies the elimination of alternative explanations of the apparent connection between a putative cause and a particular effect. We may say that we are going to ‘experiment’ with a different route when we drive to work one day; if we find that the time taken is a few minutes more, we should be very careful about saying that the alternative route was the cause of the greater time taken; the volume of traffic may not have been typical, and we may have driven more tentatively because of our unfamiliarity with the route. In other words, we have not controlled, and hence eliminated, these alternative possibilities. This notion is akin to the need to control the potentially contaminating effects of heat, light and humidity in the natural scientist’s laboratory. What we want to be able to demonstrate is that our supposed independent variable, and that variable alone, is the cause of variation in the dependent variable.

  On the other hand, to anticipate slightly some of the issues that will be examined below, conducting research on themes deriving from organization studies runs into some specific problems when the natural scientist’s laboratory is used as a yardstick. Most obviously, experimental organizational research is conducted on people rather than on relatively inert matter. The very fact that people will know that they are objects of study may have an impact on their behaviour. Natural scientists acknowledge that experimentation can have an independent impact on the matter that they study (for example, the Heisenberg principle), but this matter is not capable of the self-knowledge and awareness to which human beings are susceptible. Of course, this problem is not specific to organizational research, but pervades all experimental research in the social sciences. What is particularly conspicuous in organizational research is the special cluster of problems that doing research in organizations as such entails. Much organizational research based on Experiments can be construed as either laboratory or field experimentation. The research studies by Ganster (1980) and Orpen (1979) respectively are examples of each of these two types of study. When research is conducted in real organizations—field Experiments—special difficulties may be anticipated. Researchers are unlikely to have the relatively free hand in setting up experimental arrangements that they can confer upon themselves in the laboratory; those on whom the research is conducted may be suspicious of the true aims of the study (for example, perceiving it to be an instrument of senior management); subjects may have a vested interest in certain outcomes; and so on. The first of these three difficulties field experimentation rarely permitting a carte blanche

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Testament for Social Science (RLE Social Theory)

  An Essay in the Application of Scientific Method to Human Problems

  • Barbara Wootton(Author)
  • 2020(Publication Date)
  • Routledge

    (Publisher)

  In the social sciences this final stage of the scientific process—empirical testing of the hypothesis—in its turn follows, in principle, the same course as in the natural sciences. Just as in the natural sciences this is the point at which controlled experiment is generally conclusive, so in the social sciences it is here that experimental methods are most seriously missed. But as has just been said, no hard and fast line separates controlled experiment from controlled observation. Hypothesis must be tested by the nearest approximation to controlled experiment that is available. And if this is conscientiously done, even in the social sciences hypotheses need no longer waste their fruitful potentialities in rank proliferation into the field of pure imagination.

  In the absence of experiment, we must, however, normally be content with probabilities that fall in varying degrees short of certainty.

  IV

  Two further characteristics of scientific knowledge require notice. The first is its progressive character. It is of the essence of scientificmethod that, by its use, additions to knowledge are cumulative through time. The researches of today illumine the dark places, improve the accuracy and extend the scope of the generalisations of yesterday. Some of the conclusions of earlier investigations are definitely proved wrong by later researches; low probabilities are replaced by higher ones; and year by year new additions are made to the body of knowledge. It follows that study of the work of scientists of an earlier generation is incidental to, rather than an integral part of, scientific learning. One may explore the history of, say, chemistry, because the lives of the great chemists, like those of other great men, have a human interest of their own; or one may hope to learn from the study of the past something of the methods and conditions which are favourable to scientific progress ; or one may think that scientific discoveries which have stood the test of time are more clearly presented by those who first made them than in any later compilation; or one may even find that the mistaken guesses of the past have a certain entertainment value of their own or are suggestive of other and more fruitful ideas in the present. But if one wishes to learn chemistry, it is the chemistry of today that is relevant with the latest discoveries included and the errors of the past excluded.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Quantitative Research for the Qualitative Researcher

  • Laura M. O′Dwyer, James A. Bernauer(Authors)
  • 2013(Publication Date)
  • SAGE Publications, Inc

    (Publisher)

  8 Experimental Research Designs

  Chapter Outline

  • Essential Characteristics of Experimental Research
  • Quasi-Experimental Between-Subjects Research Designs
  • True Experimental Between-Subjects Research Designs
  • Strengths and Limitations of Quasi- and True Experimental Between-Subjects Research Designs
  • Common Threats to External and Internal Validity for Quasi- and True Experimental Between-Subjects Research Designs

  The purpose of this chapter is to introduce readers to experimental research designs as a way for generating data to address important research questions. We begin by reinforcing for our readers the difference between quasi-Experiments and true Experiments and proceed to describe some of the most common experimental research designs used in social science research studies.

  Essential Characteristics of Experimental Research

  If the researcher's aim is to examine the effect of a treatment or intervention on some attribute or outcome (e.g., whether a new teaching or therapeutic method can lead to better outcomes compared with a traditional method), experimental research may be the most appropriate. In the simplest case, experimental research requires the implementation of a treatment with one group (the treatment group) and not with another (the control group). In this way, the independent variable is manipulated by the researcher. Subsequent to applying the intervention to the treatment group, the researcher compares the two groups on the attribute or outcome hypothesized to respond to the treatment. The primary purpose of experimental research is to be able to establish “cause and effect” or, more technically, to make causal inferences. That is, the researcher aims to conclude that the treatment caused the difference that was observed between the groups on the attribute that is being studied. Recall that it is only possible to conclude that attribute A causes attribute B when (a) A occurs before B, (b) changes in A are associated with changes in B, and (c) all other possible explanations for the observed association between A and B

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Repeated Measures Design for Empirical Researchers

  • J. P. Verma(Author)
  • 2015(Publication Date)
  • Wiley

    (Publisher)

  Chapter 1 Foundations of Experimental Design

  Introduction

  Empirical research provides knowledge to the researchers through direct or indirect observations or experiences. Empirical research may either involve correlational or experimental approach. In correlational research one looks to establish relationship between two variables. In such studies a premise is made that two variables may be related in some way and then values of both the variables are obtained under different conditions to test a hypothesis if indeed there is a relationship between the two. The obtained correlation is tested for its significance. The drawback of the correlational study is that it does not establish the cause and effect relationship even if the correlation is found to be statistically significant. For instance, if the observed correlation between the caffeine intake and concentration of mind is significant and positive, it cannot be said that caffeine causes concentration. The increase in the concentration due to the increase in the caffeine intake may be due to age, motivation, gender, other lifestyle parameters.

  On the other hand, experimental research provides cause and effect relationship because in such experiment a treatment is deliberately administered by a researcher on a group of individuals or objects to see its impact under a controlled environment. In other words, if changes are made in the variable A that leads to changes in variable B, one can conclude that A causes B. For example, to see the impact of exercise on muscular strength a researcher may administer different intensity of exercise to different groups of individuals to see its effect. If a particular intensity of exercise improves muscular strength more than others, one may conclude that exercise intensity causes muscular strength. On the other hand, if there is no difference in the average muscular strength among different exercise groups, it may be inferred that the exercise intensity has nothing to do with muscular strength.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Population-Based Survey Experiments

  • Diana C. Mutz(Author)
  • 2011(Publication Date)
  • Princeton University Press

    (Publisher)

  30 Experimental realism is usually greater in the context of a field experiment, but researchers often lack the ability to ensure that the treatments have actually reached subjects, an issue that is far less problematic in population-based Experiments. If the manipulation check suggests that one has effectively brought about the desired change in the independent variable, the most difficult hurdle has already been overcome. If the level of experimental realism is high enough in a given setting for the experimental treatment to produce a significant manipulation check, demonstrating that the treatment brought about the desired change in the independent variable, then it is probably also adequate for producing the outcome of interest in the dependent variable. Both require experimental realism; the main problem is that many researchers neglect to include manipulation checks in their designs.

  If desired, social context and other forms of variation in settings can be explicitly incorporated into the design and analysis of population-based Experiments. For example, many researchers suggest that social context often conditions treatment effects,31 which seems a definite possibility for many social science theories. For example, if there is social support for a given kind of behavior in the environment, it may be far easier to induce than when there is not. When the relevant characteristic of settings cannot be manipulated as part of the design, the solution is to measure and block on the relevant characteristics of social context in order to understand the extent to which they matter and thus improve the efficiency of the design.

  The more serious limitations on the generalizability of settings involve restrictions on the time frame.32 Of course, all studies must be done in a particular time and place, so this is not unique to Experiments. But both laboratory and population-based Experiments tend to be short-term, onetime interventions with one-time measures of the outcome of interest. This problem is not inherent in the method, but it is a common restriction due to limited research budgets. It is always more time-consuming and costly to track subjects down later and/or induce them to return to the lab for a second or third time. But it can be done. Moreover, as discussed in Chapter 6

  Sign up to read

  Learn more about book