Practical Research Methods in Education

An Early Researcher's Critical Guide

Mike Lambert, Mike Lambert

• Publisher: Routledge
• Year: 2019
• ISBN:
  9781351188371
• Edition: 1
• Topic: Education
• Subtopic: Education General

Practical Research Methods in Education is a hands-on guide which critically explores and scrutinizes research methods used in educational enquiry. Drawing on the research, practical experience and reflections of active researchers, each chapter offers explanations, examples, tasks for students to undertake and suggestions for further reading, all of which are designed to strengthen understanding of practical methods of data collection in educational and social-science research.

This insightful book offers:

• 
  
  

• Detailed illustration of a range of data-collection methods and approaches used in educational research

• 
  
  

• Chapters written by active researchers, experienced in addressing challenges of carrying out practical research in education

• 
  
  

• Examples, study tasks and suggestions for further reading in each chapter

• 
  
  

• An exploration of critical reflection and decision-making in relation to research methodology in education

• 
  
  

• Close attention to research ethics

Exploring practical methods of data-collection for educational and social-science research, Practical Research Methods in Education is a unique and valuable resource for any students interested and engaged in the planning and completion of their own investigations.

Chapter 1

Using questionnaires

Brendan Bartram

Introduction

This chapter explores the online and offline use of questionnaires as a research tool in education and offers a critical discussion of their benefits and limitations in relation to design and data analysis. The discussion draws on examples from my own experiences of using questionnaire surveys in quantitative and qualitative research projects. Special consideration is additionally afforded to a relatively under-utilized but very helpful form of questionnaire, based on a word-association technique. This innovative method arguably has advantages for novice and experienced researchers alike and its use and potential are again illustrated by examples derived from my own practical experiences of investigating educational issues.

Merits and limitations

When we think about investigating educational phenomena, questionnaires are perhaps one of the first research tools that spring to mind. Their use in the social sciences as a whole is very well established and students of education soon become particularly familiar with studies, reports and papers whose findings are based on data collected in this way. There are very good reasons for their ubiquity (Munn and Drever, 2004), as will be discussed below, though like any other research technique, they come with their own challenges and are not necessarily suitable for all types of enquiry. Nonetheless, their merits are many, with issues of time and scale chief among them: questionnaires often allow researchers to collect large amounts of data from sizeable groups of respondents in a relatively short amount of time. The large datasets which can be generated from standardized questions give researchers the option of using statistical means of analysis and representation, and where the sample size is both large and diverse, they offer the additional possibility of analysing and comparing particular sub-groups which may be of interest, defined, for example, by gender, age, professional roles, location or nationality. Whether or not statistical means of analysis are applied, it is nearly always possible to analyse and present findings succinctly and clearly using figures and percentages, for instance: ‘Eighty per cent of respondents agreed that university students should sit exams’. Questionnaires can also be particularly useful when the aim of the research is to capture a surface impression of the extent to which groups of people agree or disagree on an issue, or to establish the range of thoughts and views in relation to certain topics, for example: ‘Fifteen out of 20 respondents felt that exams were stressful, while only five suggested they were an enjoyable challenge’.

As such, questionnaires are an ideal tool when we are confident of a high return rate, and they often work particularly well in combination with interviews, which allow questionnaire data to be fleshed out with greater depth and detail. One factor often considered to facilitate a higher number of returns is related to an important ethical consideration: the anonymity and confidentiality that questionnaires offer. Well-designed versions usually highlight this feature in an attempt to encourage completion based on honest and frank responses, with respondents confident that their personal views will not be identifiable.

These factors are some of the central advantages of questionnaires, but, as with all data-collection instruments, there are certain limitations which researchers need to bear in mind. Some of these are simply the reverse side of their strengths. For example, just as questionnaires are well suited to collecting large amounts of data, they are often designed only at a descriptive, surface level (Yes/No selection, tick-box items, etc.). Although more open questions can be included, they are not always an effective means of gathering in-depth and detailed responses and this can sometimes limit the usefulness of the data collected and what can be claimed these data reveal. Given that questionnaires are predominantly a written means of data gathering, this is also an important consideration where the target population may face challenges or concerns in relation to literacy skills and confidence. And while questionnaires can sometimes produce very high numbers of returns, they are also notorious for low rates of completion. This can be for a range of reasons, in addition to the aforementioned literacy concerns. For instance, the very anonymity of the method, combined with the communication distance often involved, especially with online versions, allows disinclined respondents to opt out and hide very easily (ethically important but unfortunate for the eager researcher) and, in an age where all of us are increasingly asked to share our views on what we buy, use and experience, the notion of ‘questionnaire fatigue’ is something to which most of us can relate.

Further limitations relate to the fundamental issues of design and construction. Although assembling a set of questions on a topic may seem a relatively straightforward task, this is often far from the case. First of all, decisions need to be made about the type of questions to include. These can be open, closed, scaled, multi-choice, based on sentence completion, or indeed a mixture of some or all of these. Where researchers decide on a combination of question types, they run the risk of deterring respondents by producing a confusing mixture of formats. They may also end up setting themselves something of a challenge later on, when they have to decide how to analyse, interpret and represent a wide variety of response types.

There is also the very basic issue of question phrasing. This is something notoriously difficult to get right, and many questionnaire designers fall into the trap of failing to avoid leading questions (‘Do you agree that exams should be scrapped?’), double questions (‘Do you think exams are difficult and invalid ways to assess students?’), and overly complex, ambiguous and sometimes even offensive questions. Such potential pitfalls highlight the need for careful piloting to ensure that the planned questionnaire is as clear, valid and effective as possible. Cohen et al. (2018) also offer useful guidance in terms of question sequencing, suggesting that it is generally better to begin with unthreatening, factual questions (collecting demographic or ‘categorical’ details, for example), followed by a set of closed questions, before finishing with the most demanding open ones. Clarity of presentation and instruction are, of course, additional, vital considerations in this respect.

The importance of piloting is just one particular aspect of the logistical challenge posed by questionnaires. Trialing the planned instrument sometimes raises difficult questions with regard to time, place and arrangements for collection and return, let alone issues of negotiating access to a relevant sample, all of which need careful, advance consideration. These apply equally to administration of the ‘live’ questionnaire, and especially when the researcher is unable to be personally present (questionnaires may quite often be distributed by headteachers or school staff, for example). In this respect, electronic questionnaires mitigate some of these challenges, though they present the additional challenge of selectivity – the fact that some potential respondents, receiving the questionnaire link via email, opt not to take part may mean that those that do are not necessarily representative of the broader target population. For example, an electronic questionnaire about the cross-curricular use of information technology in secondary schools may see far more responses from active enthusiasts than from the disgruntled or indifferent (or perhaps vice versa). The potential for such selectivity effects must be carefully borne in mind when considering the validity of the findings generated and what can be claimed on this basis.

Trustworthiness and validity

One of the advantages of using questionnaires is that it allows you to get data from a large number of respondents. To what extent might having many respondents automatically increase the trustworthiness or validity of the data you collect?

Ethics

Clearly too, there are a number of ethical considerations that researchers will need to consider when using questionnaires, in addition to those of anonymity and confidentiality mentioned above. One relates to the issue of informed consent and voluntary participation, which is particularly pertinent when school children or college and university students, for example, are asked to complete a questionnaire. In this kind of ‘captive’ situation, it would be very easy for respondents to feel coerced into completion. This may be because they feel under pressure to comply with what they are asked to do, or even to fit in with what others in the group appear happy to go along with. This is a difficult issue to address completely, but it highlights the researcher’s responsibility to emphasize the voluntary nature of completion, to provide assurances of confidentiality...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Contents
• List of illustrations
• Contributors
• Foreword
• Editor’s introduction
• Chapter 1 Using questionnaires
• Chapter 2 Using interviews
• Chapter 3 Observations
• Chapter 4 Involving children
• Chapter 5 Using video as data
• Chapter 6 Using quantitative data
• Chapter 7 Document analysis
• Chapter 8 ‘Doing text’: Madness in the methods
• Chapter 9 Q-methodology: A science of subjectivity
• Chapter 10 Mosaic: Participatory research in the early years
• Chapter 11 Ethnography
• Chapter 12 Case study
• Chapter 13 Grounded theory
• Closing words
• Index