Essentials of Research Design and Methodology

Geoffrey R. Marczyk, David DeMatteo, David Festinger

• Publisher: Wiley
• Year: 2010
• ISBN:
  9780470893531
• Edition: 1
• Topic: Psychology
• Subtopic: Research & Methodology in Psychology
• Index: Psychology

Master the essential skills for designing and conducting a successful research project Essentials of Research Design and Methodology contains practical information on how to design and conduct scientific research in the behavioral and social sciences. This accessible guide covers basic to advanced concepts in a clear, concrete, and readable style. The text offers students and practitioners in the behavioral sciences and related disciplines important insights into identifying research topics, variables, and methodological approaches. Data collection and assessment strategies, interpretation methods, and important ethical considerations also receive significant coverage in this user-friendly guide. Essentials of Research Design and Methodology is the only available resource to condense the wide-ranging topics of the field into a concise, accessible format for handy and quick reference. As part of the Essentials of Behavioral Science series, this book offers a thorough review of the most relevant topics in research design and methodology. Each concise chapter features numerous callout boxes highlighting key concepts, bulleted points, and extensive illustrative material, as well as "Test Yourself" questions that help you gauge and reinforce your grasp of the information covered.

Seven

DATA PREPARATION, ANALYSES, AND INTERPRETATION

As we have discussed in previous chapters, in most research studies, the researcher begins by generating a research question, framing it into a testable (i.e., falsifiable) hypothesis, selecting an appropriate research design, choosing a suitable sample of research participants, and selecting valid and reliable methods of measurement. If all of these tasks have been carried out properly, then the process of data analysis should be a fairly straightforward process. Still, a variety of important steps must be taken to ensure the integrity and validity of research findings and their interpretation.

In most types of research studies, the process of data analysis involves the following three steps: (1) preparing the data for analysis, (2) analyzing the data, and (3) interpreting the data (i.e., testing the research hypotheses and drawing valid inferences). Therefore, we will begin this chapter with a brief discussion of data cleaning and organization, followed by a nontechnical overview of the most widely used descriptive and inferential statistics. We will conclude this chapter with a discussion of several important concepts that should be understood when interpreting and drawing inferences from research findings. Because a comprehensive discussion of statistical techniques is well beyond the scope of this book, researchers seeking a more detailed review of statistical analyses should consult one of the statistical textbooks contained in the reference list.

DATA PREPARATION

Virtually all studies, from surveys to randomized experimental trials, require some form of data collection and entry. Data represent the fruit of researchers’ labor because they provide the information that will ultimately allow them to describe phenomena, predict events, identify and quantify differences between conditions, and establish the effectiveness of interventions. Because of their critical nature, data should be treated with the utmost respect and care. In addition to ensuring the confidentiality and security of personal data (as discussed in Chapter 8), the researcher should carefully plan how the data will be logged, entered, transformed (as necessary), and organized into a database that will facilitate accurate and efficient statistical analysis.

Logging and Tracking Data

Any study that involves data collection will require some procedure to log the information as it comes in and track it until it is ready to be analyzed. Research data can come from any number of sources (e.g., personal records, participant interviews, observations, laboratory reports, and pretest and posttest measures). Without a well-established procedure, data can easily become disorganized, uninterpretable, and ultimately unusable.

Although there is no one definitive method for logging and tracking data collection and entry, in this age of computers it might be considered inefficient and impractical not to take advantage of one of the many available computer applications to facilitate the process. Taking the time to set up a recruitment and tracking system on a computer database (e.g., Microsoft Access, Microsoft Excel, Claris FileMaker, SPSS, SAS) will provide researchers with up-to-date information throughout the study, and it will save substantial time and effort when they are ready to analyze their data and report the findings.

One of the key elements of the data tracking system is the recruitment log. The recruitment log is a comprehensive record of all individuals approached about participation in a study. The log can also serve to record the dates and times that potential participants were approached, whether they met eligibility criteria, and whether they agreed and provided informed consent to participate in the study. Importantly, for ethical reasons, no identifying information should be recorded for individuals who do not consent to participate in the research study. The primary purpose of the recruitment log is to keep track of participant enrollment and to determine how representative the resulting cohort of study participants is of the population that the researcher is attempting to examine.

In some study settings, where records are maintained on all potential participants (e.g., treatment programs, schools, organizations), it may be possible for the researcher to obtain aggregate information on eligible individuals who were not recruited into the study, either because they chose not to participate or because they were not approached by the researcher. Importantly, because these individuals did not provide informed consent, these data can only be obtained in aggregate, and they must be void of any identifying information. Given this type of aggregate information, the researcher would be able to determine whether the study sample is representative of the population.

In addition to logging client recruitment, a well-designed tracking system can provide the researcher with up-to-date information on the general status of the study, including client participation, data collection, and data entry.

DON’T FORGET

Record-Keeping Responsibilities

The lead researcher (referred to as principal investigator in grant-funded research) is ultimately responsible for maintaining the validity and quality of all research data, including the proper training of all research staff and developing and enforcing policies for recording, maintaining, and storing data. The researcher should ensure that

• research data are collected and recorded according to policy;

• research data are stored in a way that will ensure security and confidentiality; and

• research data are audited on a regular basis to maintain quality control and identify potential problems as they occur.

Data Screening

Immediately following data collection, but prior to data entry, the researcher should carefully screen all data for accuracy. The promptness of these procedures is very important because research staff may still be able to recontact study participants to address any omissions, errors, or inaccuracies. In some cases, the research staff may inadvertently have failed to record certain information (e.g., assessment date, study site) or perhaps recorded a response illegibly. In such instances, the research staff may be able to correct the data themselves, if too much time has not elapsed. Because data collection and data entry are often done by different research staff, it may be more difficult and time consuming to make such clarifications once the information is passed on to data entry staff.

One way to simplify the data screening process and make it more time efficient is to collect data using computerized assessment instruments. Computerized assessments can be programmed to accept only responses within certain ranges, to check for blank fields or skipped items, and even to conduct cross-checks between certain items to identify potential inconsistencies between responses. Another major benefit of these programs is that the entered data can usually be electronically transferred into a permanent database, thereby automating the data entry procedure. Although this type of computerization may, at first glance, appear to be an impossible budgetary expense, it might be more economical than it seems when one considers the savings in staff time spent on data screening and entry.

Whether it is done manually or electronically, data screening is an essential process in ensuring that data are accurate and complete. Generally, the researcher should plan to screen the data to make certain that (1) responses are legible and understandable, (2) responses are within an acceptable range, (3) responses are complete, and (4) all of the necessary information has been included.

Constructing a Database

Once data are screened and all corrections are made, the data should be entered into a well-structured database. When planning a study, the researcher should carefully consider the structure of the database and how it will be used. In many cases, it may be helpful to think backward and to begin by anticipating how the data will be analyzed. This will help the researcher to figure out exactly which variables need to be entered, how they should be ordered, and how they should be formatted. Moreover, the statistical analysis may also dictate what type of program you choose for your database. For example, certain advanced statistical analyses may require the use of specific statistical programs.

While designing the general structure of the database, the researcher must carefully consider all of the variables that will need to be entered. Forgetting to enter one or more variables, although not as problematic as failing to collect certain data elements, will add substantial effort and expense because the researcher must then go back to the hard data to find the missing data elements.

DON’T FORGET

Retaining Data Records

Researchers should retain study data for a minimum period of 5 years after publication of their data in the event that questions or concerns arise regarding the findings. The advancement of science relies on the scientific community’s overall confidence in disseminated findings, and the existence of the primary data serves to instill such confidence.

The Data Codebook

In addition to developing a well-structured database, researchers should take the time to develop a data codebook. A data codebook is a written or computerized list that provides a clear and comprehensive description of the variables that will be included in the database. A detailed codebook is essential when the researcher begins to analyze the data. Moreover, it serves as a permanent database guide, so that the researcher, when attempting to reanalyze certain data, will not be stuck trying to remember what certain variable names mean or what data were used for a certain analysis. Ultimately, the lack of a well-defined data codebook may render a database uninterpretable and useless. At a bare minimum, a data codebook should contain the following elements for each variable:

• Variable name

• Variable description

• Variable format (number, data, text)

• Instrument or method of collection

• Date collected

• Respondent or group

• Variable location (in database)

• Notes

Data Entry

After the data have been screened for completeness and accuracy, and the researcher has developed a well-structured database and a detailed codebook, data entry should be fairly straightforward. Nevertheless, many errors can occur at this stage. Therefore, it is critical that all data-entry staff are properly trained and maintain the highest level of accuracy when inputting data. One way of ensuring the accuracy of data entry is through double entry. In the double-entry procedure, data are entered into the database twice and then compared to determine whether there are any discrepancies. The researcher or data entry staff can then examine the discrepancies and determine whether they can be resolved and corrected or if they should simply be treated as missing data. Although the double-entry process is a very effective way to identify entry errors, it may be difficult to manage and may not be time or cost effective.

DON’T FORGET

Defining Variables Within a Database

Certain databases, particularly statistical programs (e.g., SPSS) allow the researcher to enter a wide range of descriptive information about each variable, including the variable name, the type of data (e.g., numeric, text, currency, date), label (how it will be referred to in data printouts), how missing data are coded or treated, and measurement scale (e.g., nominal, ordinal, interval, or ratio). Although these databases are extremely helpful and should be used whenever possible, they do not substitute for a comprehensive codebook, which includes separate information about the different databases themselves (e.g., which databases were used for each set of analyses).

As an alternative to double entry, the researcher may desig...

Table of contents

• Essentials of Behavioral Science Series
• Title Page
• Copyright Page
• Dedication
• SERIES PREFACE
• Acknowledgements
• Essentials of Research Design and Methodology
• One - INTRODUCTION AND OVERVIEW
• Two - PLANNING AND DESIGNING A RESEARCH STUDY
• Three - GENERAL APPROACHES FOR CONTROLLING ARTIFACT AND BIAS
• Four - DATA COLLECTION, ASSESSMENT METHODS, AND MEASUREMENT STRATEGIES
• Five - GENERAL TYPES OF RESEARCH DESIGNS AND APPROACHES
• Six - VALIDITY
• Seven - DATA PREPARATION, ANALYSES, AND INTERPRETATION
• Eight - ETHICAL CONSIDERATIONS IN RESEARCH
• Nine - DISSEMINATING RESEARCH RESULTS AND DISTILLING PRINCIPLES OF RESEARCH ...
• References
• Index