Public Policy Evaluation

Making Super-Optimum Decisions

Stuart S. Nagel

• Publisher: Routledge
• Year: 2018
• ISBN:
  9780429831089
• Edition: 1
• Topic: Politica e relazioni internazionali
• Subtopic: Politica

First published in 1998, this volume examines how super-optimum decisions involve finding alternatives to controversies whereby Conservatives, Liberals, or other major groups can all come out ahead of their best initial expectations simultaneously. This book is organised in terms of concepts, methods, causes, process, substance, and the policy studies profession. Concepts clarify that policy evaluation traditionally involves: (1) Goals to be achieved; (2) Alternatives available for achieving them; (3) Relations between goals and alternatives; (4) Drawing a conclusion as to the best alternative in light of the goals, alternatives, and relations; and (5) Analysing how the conclusion would change if there were changes in the goals, alternatives, or relations. Super-optimizing also involves five related steps, but with the following improvements: (1) Goals are designed as conservative, liberal, or neutral; (2) Alternatives get the same designations; (3) Relations are simplified to indicate which alternatives are relatively high or low on each goal; (4) The conclusion involves arriving at an alternative that does better on Goal A than Alternative A, and simultaneously better on Goal B than Alternative B; and (5) The fifth step involves analysing the super-optimum or win-win alternative in terms of its feasibility as to the economic, technological, psychological, political, administrative, and legal matters.

Part I
Concepts, Methods and Causes

1 What is Policy Evaluation? Conceptual Theory

Evaluating Public Policy Evaluation

Public policy studies can be defined as the study of the nature, causes and effects of governmental decisions for dealing with social problems. The field of policy studies is also known as policy science, policy analysis and policy evaluation.

Since about 1970, the policy studies field has developed into a well-organized discipline, sub-discipline, and/or multi-discipline with courses, curricula, schools, organizations, journals, textbooks, conferences, panels, summer institutes, awards, funding sources, research institutes, job opportunities and other indicators of scholarly activity. The origins and growth stem partly from (1) the intense concern for public policy problems in the 1960s and after, (2) government as a supplement to the academic world for research funding and job opportunities, and (3) the development of new methodologies for evaluating alternative public policies.

During the early years of growth, there have been many evaluative comments made by social scientists and others. Some of those evaluations have been conflicting, with policy studies being criticized for having too much of a certain characteristic and also for having too little of the same characteristic. Those characteristics include being (1) a temporary fad or stale material, (2) too practical or too theoretical, (3) too multi-disciplinary or too narrowly focused on political science, (4) too quantitative or too subjective, (5) underutilized or overutilized, and (6) too conservative or too liberal.

The purpose of this chapter is to show briefly that public policy studies is a combination of diverse ideas that enable the field to deal better with the systematic evaluation of alternative public policies. In that sense, the field is based on (1) long-term philosophical principles and new analytical methods/ substantive problems, (2) theory and practice, (3) political science and multiple disciplines, (4) inherent subjectivity and potential objectivity, (5) occasional utilization and frequent non-utilization necessary to generate the important successes, and (6) liberal and conservative uses.

A temporary fad or stale material

Policy studies is not a temporary fad. It reflects a long-term philosophical concern in the social sciences for public policy problems. That concern goes back to the roots of social science in the social philosophies of such classical philosophers as Plato and Aristotle. They indicated their strong concerns for evaluating public policy in The Republic and the Nichomachean Ethics, respectively, even if they did not use the same types of analysis that are used by contemporary policy analysts. Social philosophy was not so much a concern in the Middle Ages, but the Renaissance brought a flood of evaluative political thinking in the form of such philosophers as Macchiavelli, Hobbes, Locke, Montesquieu, Rousseau and Hegel, who evaluated alternative policies for structuring the modern state. As of the nineteenth century, policy evaluation became more class-oriented, as reflected in the evaluations of Marx, Trotsky, Lenin and Laski, who praised socialistic solutions, and the evaluations of Adam Smith, Nietzche and Pareto, who praised capitalistic solutions.

On the heels of political philosophy came the scientific study of politics and government. Philosophy emphasizes a concern for evaluation and causation on a high level of generality. Philosophy tends to use methods of concept development, logical deduction and wandering speculation. Political science emphasizes a concern for describing governmental institutions and explaining variations in political behavior across places, times and people. Political science tends to use methods of legal analysis, inductive reasoning from many specific instances, and the expression of relations in mathematical form. From the beginning of the American Political Science Association in 1903, there has been an increasing concern for statistical and mathematical analysis. That analysis distinguishes political science from political philosophy, political journalism, public law and political history.

Although public policy evaluation has been around since the dawn of social philosophy, new aspects have developed since the 1960s. The main new aspect is the idea of synthesizing the essentially normative philosophy (associated with policy evaluation from Plato to Marx and the anti-Marxists) and the scientific method (associated with such political scientists as Charles Merriam and Harold Lass well). The new policy evaluation studies are especially concerned with making normative judgments concerning political institutions and social policies, but the judgments are based on systematic statistical and mathematical analyses relevant to criteria of effectiveness, efficiency and equity.

Since the 1970s, there have been a number of more specific trends that relate to the analysis of societal goals, alternative policies and methods for relating policies to goals. As for societal goals, there is a trend toward more taking of goals as givens and then attempting to determine what policies will maximize or optimize them, rather than taking policies as givens and then attempting to determine their effects. Policy analysts are also becoming more sensitive to social values, with more questioning of goals in evaluating alternative policies.

As for alternative policies, there is a trend toward showing increased sophistication with regard to the political feasibility of policies being adopted and the feasibility of alternative delivery systems or implementation systems after they are adopted. Policy evaluation is also becoming increasingly interdisciplinary in drawing upon a variety of disciplinary sources as to means or policies for achieving goals. This includes economics, political science, sociology and psychology among basic disciplines, and business administration, planning, public administration, law, social work and education among applied disciplines.

As for methods, policy evaluation research has been building on business analysis, especially with regard to maximizing benefits minus costs, but it has been developing its own methodology, especially in matters of measurement, equity, negative social indicators and administrative psychology. Policy evaluation is also developing increased precision with its methods, but at the same time it is increasingly recognizing that simple methods may be enough for many policy problems. Policy evaluation is also becoming increasingly proactive or pre-adoption rather than reactive or post-adoption, which relates to deductive modeling as contrasted to before-and-after analysis.

Key trends within public policy analysis since 1970 can be seen in the changing emphasis from the left side to the right side of the logo of the Policy Studies Organization. That logo puts public policy in the middle, with causes on the left and effects on the right. Since 1970, there has been a four-stage movement across the logo:

1. As of 1965, the emphasis was on working backwards from a public policy or set of public policies to their causes. That meant answering such questions as: why do some states spend more money per capita on welfare than other states, or give welfare recipients more rights?
2. The next logical stage, as of about 1970, was to synthesize such studies so that one could answer questions like: what is the effect on public policy of class conflict, democracy, industrialization, redistricting, high per capita income and so on?
3. As of 1975, there was a new emphasis on talking about the impact or implementation of public policies, which meant asking such questions as: what processes occur after an unemployment program is instituted?
4. As of about 1980, books began appearing that talked in terms of deciding what effects are desired and then in terms of what policies or decision-making procedures might maximize those effects.

The four stages emphasize policy formation, causation, implementation and optimization, respectively.

Too practical or too theoretical

It is possible for policy evaluation to be too practical in the sense of being too narrowly focused on a policy problem of a specific place at a specific point in time. An example might be a study of whether the city of Urbana, Illinois, as of 1988, should adopt a city manager form of government or an elected mayor form of government. Such a study would have virtually no broad theoretical interest if it were confined to interviewing people in Urbana and gathering relevant Urbana data.

Such a study would have broad interest if it attempted to gather data on a sample of about 100 cities that have city managers and a comparable sample of 100 cities that have mayor systems. That kind of comparative analysis could lead to generalizations concerning the effects of city managers as compared to mayors, including a causal analysis designed to explain the differential effects. That kind of comparative analysis could also be helpful to the city of Urbana and other specific cities in deciding between those two forms of municipal government.

Much of public policy studies is or such a comparative nature, as contrasted with what is sometimes referred to as 'applied political science', which tends to be designed to provide answers to the questions that are raised by heads of government agencies rather than by political science scholars. There is certainly a need for applied political science, but it is a different type of activity from public policy studies. One might say that public policy studies is to applied political science what legislation is to adjudication, or evaluation research to program evaluation. In each pair of concepts, the first is concerned with broad principles that go beyond the immediate problem or parties. The second concept is concerned with an immediate problem, a pair of litigants, or a practical decision that may not have any precedent or value.

The terms 'theoretical' and 'practical' probably should not be used in referring to policy studies versus applied political science. Systematic comparative policy evaluation can be quite practical in comparison to a narrowly focused analysis that has little practicality for anyone beyond those who are immediately concerned. Likewise, a case study may sometimes have considerable theoretical significance beyond the immediate situation, such as the case study by Robert Dahl of who governs in New Haven, the case study of the TVA by Philip Selznick, or studies of the Army Corps of Engineers, the Forest Service, Middletown, USA (that is, Muncie, Indiana) and so on.

Policy studies is not just concerned with systematic comparative analysis in its theoretical concerns. It is also concerned with high-level meta-problems that cut across virtually all the methodological and substantive problems of policy studies, such as the following:

1. Who are or should be the decision makers who decide among the alternative ways of dealing with the substantive and methodological problems?
2. How does one obtain values to use as criteria, and how does one obtain perceptions of relations between alternatives and criteria?
3. What goals are worth achieving, and what is their relative importance?
4. What are the alternatives on a high level of generality?
5. What is good policy analysis?

On the especially important matter of what is good policy analysis, five criteria tend to be discussed:

1. validity which includes (a) internal consistency in drawing a conclusion, (b) external consistency with empirical reality in relating to goals, and (c) accuracy in encompassing the total set of feasible alternatives and the major goals of the relevant policy makers;
2. importance, which includes (a) policy importance as indicated by the size of the societal benefits or societal costs being analyzed, and (b) theoretical importance as indicated by the broadness of the explanatory power of the research propositions;
3. usefulness, which includes (a) at least being referred to, (b) possibly reinforcing ideas and thereby accelerating their adoption, and (c) the unusual occurrence of converting decision makers from negative to positive or vice versa;
4. originality, which refers to the extent to which policy research differs from previous research; and
5. feasibility, which is concerned with how easily the research can be implemented, given the limited resources of the researcher.

Too multi-discip...

Table of contents

• Cover
• Dedication
• Title
• Copyright
• Summary Table of Contents
• Contents
• List of Tables
• List of Figures
• Introduction: four Converging Streams
• PART I CONCEPTS, METHODS AND CAUSES
• PART II PROCESS AND SUBSTANCE
• PART III POLICY STUDIES PROFESSION
• Index