Measuring Academic Research

How to Undertake a Bibliometric Study

Ana Andres

• Publisher: Chandos Publishing
• Year: 2009
• ISBN:
  9781780630182
• Topic: Languages & Linguistics
• Subtopic: Library & Information Science
• Index: Languages & Linguistics

Measuring Academic Research outlines how to undertake a bibliometric study, a topic of vital importance in academic research today. Scientometrics studies assess scientific productivity and can be applied to all disciplines. Many analyses have been applied in relation to bibliometric studies, but few have shown how to actually carry out the analysis. This book provides a guide on how to develop a bibliometric study, from the first step in which the topic study has to be set, to the analysis and interpretation.

• A practical and easy to read guide on how to carry out a bibliometric study
• Gives a wide and up-to-date view about the most common scientometric indexes
• Analyses are illustrated with multiple and practical examples about their application

1

Introduction

This book aims to illustrate the main analyses applied in bibliometric studies. The objective of bibliometrics is basically to assess scientific literature in a given field, hence its broad applicability to all manner of disciplines. Consequently, readers can choose an analysis from the present book according to the characteristics of their own research field. The step-by-step explanations about how to carry out a bibliometric study, together with the many examples of the different indices and analyses that can be performed, provide a comprehensive guide to the main methods used in bibliometrics.

The book begins by defining bibliometrics and then considers the steps which must be taken before carrying out a bibliometric study. The following chapters focus on the specific analyses used to study the productivity of authors and journals. For example, we will see that it is possible to study collaborations between researchers, while the widespread use of citation analysis to evaluate scientific output will be considered with respect to both researchers and journals separately. The main limitations of the analyses shown will be discussed in order to give an overview of their uses and consequences in bibliometric research. Furthermore, it will be argued that a critical attitude should be adopted when choosing the most appropriate analysis for a bibliometric study.

In sum, the book offers a guide to the many analyses that could be included in a bibliometric analysis, and describes how to calculate and interpret them. Readers should take into consideration those tools which could be of interest for their bibliometric study, and seek to apply and interpret them as accurately as possible.

Bibliometrics: a science of science

The study of scientific literature has a long history dating back to the early decades of the past century. However, despite the amount of research in this area it was not until 1969 that the term bibliometrics first appeared in print (Pritchard, 1969). It was defined as the ‘application of mathematical and statistical methods to books and other media of communication’, and the term was quickly adopted and used, particularly in North America (Wilson, 1999). At almost the same time, Nalimov and Mulchenko (1969) coined the term scientometrics to refer to ‘the application of quantitative methods which are dealing with the analysis of science viewed as an information process’. In contrast, this term was widely used in Europe (Wolfram, 2003). Initially, therefore, scientometrics was restricted to the measurement of science communication, whereas bibliometrics was designed to deal with more general information processes. At present, however, bibliometrics and scientometrics are used as synonyms (Glänzel, 2003).

In the 1990s the term informetrics, previously introduced by Gorkova (1988), was also used to designate a more general sub-field of information science that dealt with the statistical analysis of communication processes in science. Informetrics also deals with electronic media, including analyses carried out in electronic libraries (Glänzel, 2003). Finally, other terms such as webometrics or cybermetrics (Almind and Ingwersen, 1997; Björneborn and Ingwersen, 2001) have also been introduced to designate the study of scientific literature from electronic resources. In the present book the terms bibliometrics and scientometrics will be used without distinction to refer to the study of scientific literature.

The first evidence of bibliometrics dates back to 1873, when de Candolle described changes in the scientific strength of nations according to membership of scientific societies. With this study he aimed to identify factors that might influence the scientific success of a nation (van Raan, 2004). Subsequently, Lotka (1926) analysed the frequency distribution of scientific productivity, and his work led to the development of Lotka’s law. This law, one of the most widely used in bibliometrics, assesses patterns in author productivity. Another pioneering study is that of Gross and Gross (1927) regarding citation analysis. These authors aimed to identify those journals with a high impact in their own research field, chemistry. This work has had enormous consequences, since citation analysis is now one of the main areas in bibliometrics. Another key study is that of Bradford (1934), who considered the frequency distribution of papers across journals. As in the case of Lotka’s work, the resulting Bradford’s law is now widely used in bibliometrics to study journal productivity. Another pioneering author was Zipf (1935, 1949), who studied the frequency of words in a text. His law can be considered as a generalisation of both Lotka’s and Bradford’s laws.

However, the real breakthrough in bibliometrics arrived some years later through the work of Garfield (1955) and Price (1963). Garfield developed a Science Citation Index, i.e. a multidisciplinary database in which authors could find articles from across many fields. This proved to be a visionary tool that greatly facilitated the researcher’s task. The consequences of the indexation system proposed by Garfield will be widely discussed in this book. As van Raan (2004) states, his work has marked the rise of bibliometrics as a powerful field within the study of science. Another seminal work is Price’s Little Science, Big Science, a book first published in 1963. This revolutionary book represented the first systematic approach to the structure of modern science that was applied to science as a whole. It also established the foundation of modern research evaluation techniques (Glänzel, 2003). The main contributions of Price’s book to the development of scientometrics will be discussed below.

As the interest in bibliometric studies began to rise, specific publications started to appear. The first periodical publication in this area was the journal Scientometrics, founded by Tibor Braun in 1978. However, in the 1980s this interest in bibliometric studies came up against the limits facing researchers who wished to carry out this kind of study. The lack of availability of documents, the manual collection of data and the licence fees charged for obtaining documents all hindered progress in the field. The breakthrough came as a result of new technological developments during the 1990s (Glänzel, 2003) and the availability of online data regarding publications meant that the traditional indexation systems which compiled journal information in paper volumes were replaced by online databases.

There are now many specific or multidisciplinary databases providing indexation information for thousands of journals, papers, books and proceedings. Undoubtedly, the technological age has enabled enormous strides to be made in the field of bibliometrics.

Finally, it is worth mentioning the various applications that bibliometrics has at present. Table 1.1 shows the three bibliometric topics which Glänzel (2003) has identified as sub-areas of contemporary bibliometrics.

Table 1.1

Applications of bibliometric sub-areas

Sub-area	Application
Methodology research	These studies focus on the methodology used to carry out bibliometric research and refer to the development or improvement of bibliometric indicators. Researchers specialised in this area will basically be bibliometricians.
Scientific disciplines	These bibliometric studies may be conducted by researchers from any discipline. The aim is to apply bibliometric indicators to a given area of study. Consequently, these studies apply metrics in order to describe science.
Science policy	This is the most important topic in the field. Here, bibliometric studies are used to assess different levels of productivity. This research is conducted by policy-makers with the aim of deciding how to distribute available resources.

The bibliometric analyses discussed in the present book are considered mainly in terms of their practical application to scientific disciplines. Of course, bibliometric indicators can also have implications for science policy, and we will examine the assessment of different levels of productivity.

Little Science, Big Science

The behaviour of scientific productivity has been a traditional topic of study in scientometrics. By considering all the documents published in an area of research it is possible to determine how they are distributed according to different variables. This section discusses the main characteristics of scientific growth.

Derek J. de Solla Price was the first scientist to formulate a specific exponential growth law applied to science. It became his most famous contribution and is now known as Price’s law. The law was presented and discussed in his most well-known publication, the book entitled Little Science, Big Science (Price, 1963). The term ‘big science’ refers to large-scale instruments and facilities, supported by funding from government or international agencies, in which research is conducted by teams or groups of scientists. In fact, this term was previously introduced by Weinberg (1961). Price’s short book has had a huge impact on the formulation of scientific growth, as well as on the foundations of bibliometrics.

In his book Price explains how science has progressed from ‘little science’, which was traditionally carried out by a small group of erudite scholars who then became eminent in their field of study. In comparison, ‘big science’ is characterised by large amounts of money being invested in personnel and infrastructure. ‘Big science’ has now taken precedence over its forerunner, and investment in the advancement of science plays an important role in the economy of developed countries. However, the transition from ‘little’ to ‘big science’ has been gradual and less dramatic than it might seem at first sight. In order to analyse how this change from ‘little’ to ‘big science’ has come about, it is necessary to measure productivity over time. Price stated, on the basis of various numerical indicators taken from many fields and aspects of science, that there is regularity in the growth of its production. This growth pattern fits an exponential function. Consequently, science grows in a multiplicative way over time and, according to this exponential function, the growth rate will be proportional to the population size, i.e. the bigger the population is, the faster it grows.

Price’s law has two main properties. The first is that its validity remains precisely constant across broad periods of time. Consequently, Price states that exponential growth in science has been maintained for two or three centuries. The second main characteristic of this exponential function applied to science is its rapid growth. Accordingly, the author states that the gross size of science in terms of personnel or publications tends to be duplicated in a time period of 10 to 15 years. In general terms, one could consider all the production achieved during this time, without taking its quality into account; as such, both high- and low-quality publications will be included in this count. If the requirement level is increased a little, one could consider that overall scientific production will be duplicated during a time period of 15 years. In this case, we would be more selective, including only good-quality authors and publications. Should we wish to be even more demanding and consider only very high-quality publications, then the necessary time period to duplicate production will increase to 20 years. The conclusion to be drawn from these statements is that science grows very rapidly and, consequently, the number of productions and scientists also grows in a multiplicative way.

Contemporary science

If production and scientists grow in a multiplicative way, then authors will be contemporary. In other words, if scientists are constantly duplicating every 15 years up to the present moment, a large proportion of scientists will be contemporary. Therefore, according to this rule, most science is current. This fact is a consequence of the accelerated growth. If, in 15 years’ time, the number of scientists will have duplicated, then as many new scientists will appear during that time as existed in the whole of the previous period. However, scientists who are contemporaries in a given time period are not only new scientists who have appeared during that period. Consequently, Price (1963) estimated...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright page
• Dedication
• List of figures and tables
• About the author
• 1: Introduction
• 2: Descriptive analyses
• 3: Author production
• 4: Journal productivity
• 5: Scientific collaborations
• 6: Author citation analysis
• 7: Journal citation analysis
• 8: Important considerations
• 9: Final considerations
• References
• Index