The Official History of the British Civil Service

Reforming the Civil Service, Volume I: The Fulton Years, 1966-81

Rodney Lowe

• Publisher: Routledge
• Year: 2011
• ISBN:
  9781136830136
• Edition: 1
• Topic: History
• Subtopic: British History
• Index: History

This first volume of the Official History of the UK Civil Service covers its evolution from the Northcote-Trevelyan Report of 1854 to the first years of Mrs Thatcher's government in 1981.

Despite current concerns with good governance and policy delivery, little serious attention has been paid to the institution vital to both: the Civil Service. This Official History is designed to remedy this by placing present problems in historical context and by providing a helpful structure in which others, and particularly former officials, may contribute to the debate. Starting with the seminal 1854 Northcote-Trevelyan Report, it covers the 'lost opportunity' of the 1940s when the Service failed to adapt the needs of 'big government' as advocated by Beveridge and Keynes. It then examines, in greater detail, the belated attempts at modernisation in the 1960s, the Service's vilification in the 1970s and the final destruction of the 'old order' during the first years of Mrs Thatcher's government.

Particular light is shed on the origins of such current concerns as

• the role of special advisers
• the need for a Prime Minister's Department
• the evolution of Parliamentary Select Committees to resolve the potential tension between bureaucracy and Parliamentary democracy.

This Official History is based on extensive research into both recently released and unreleased papers as well as interviews with leading participants. It has important lessons to offer all those, both inside and outside the UK, seeking to improve the quality of democratic government.

This book will be of great interest to all students of British history, British government and politics, and of public administration in general.

Part 1
THE LEGACY

1
THE NORTHCOTE-TREVELYAN REPORT AND THE EVOLUTION OF THE CIVIL SERVICE, 1854–1916

1.1 Introduction

The reform of the Civil Service in the last third of the twentieth century was dominated by two reports: the Report of the Fulton Committee, published in 1968, and Improving Management in Government: the Next Steps, published in 1988. A third report, however, casts a long shadow over this period: the Northcote-Trevelyan Report (or more fully the Report on the Organisation of the Permanent Civil Service) published in 1854. The perceived administrative inadequacies of the 1960s were widely accredited to its malign influence. The opening paragraph of the Fulton Report, for example, notoriously asserted:

The Home Civil Service today is still fundamentally the product of the nineteenth-century philosophy of the Northcote-Trevelyan Report. The tasks it faces are those of the second half of the twentieth century. This is what we have found; it is what we want to remedy.¹

In direct contrast, the Report was simultaneously perceived – not least by Lord Simey in his formal reservation to Fulton – to have laid the foundations for the high international standing which the Service had long enjoyed. Just as being ‘one of us’ became the test used by Mrs Thatcher to judge political soundness, so being a ‘Northcote-Trevelyan man’ [sic] was, and continues to be, the measure of administrative integrity and impartiality.

This chapter briefl y looks at the gestation and nature of the Northcote- Trevelyan Report, and its implementation up to 1916. Its popular and academic reputation is mixed. For some it remains ‘one of the great state papers of the nineteenth century’; and its ‘vision in the middle of the nineteenth century of the sort of civil Service’ needed in the twentieth was ‘one of the most fortunate things in the history of British government’. Detailed historical research, however, has injected an element of caution. As has been argued, for instance, the Report

was not a blueprint for reform. The rhetorical reference to it in the Fulton Report . . . proves on closer examination to be wholly misleading (unless we detach the ‘philosophy’ of Northcote and Trevelyan entirely from their actual proposals). The Civil Service in the 1960s was the product of many things, but very little of it can be traced clearly and directly back to the report of 1854. ²

What is myth and what reality? Historically, how far was the Report moulded by the peculiar social and political pressures of its times? How did other countries respond to similar pressures? How were the Report’s recommendations, to the extent that they were implemented, reconciled with issues which it did not explicitly address, such as ministerial accountability to Parliament? More generally, as an initiative internal to the Civil Service (like Next Steps), does the Report provide any insight into how to effect administrative reform? These questions will be addressed by looking at the implementation of the Report’s specific recommendations first in detail, then in their historical and comparative context, and finally in the light of the evolution of the Service as a whole.

1.2 The Northcote-Trevelyan Report and its implementation

The Northcote-Trevelyan Report was brief and blunt. It ran, in its original, to only 23 quarto pages and like Fulton it was gratuitously offensive. Just as officials were caricatured in the 1960s for being wedded to the ‘obsolete cult’ of the ‘amateur’ before belated reference was made to the ‘Service’s very considerable strengths’, so in the 1850s the Civil Service was condemned as a magnet for the ‘unambitious, and the indolent or incapable’ before any acknowledgement was made of ‘numerous honourable exceptions’. The litany of failings would not have appeared wholly out of place in the populist campaign against the Service in the 1970s:

Those whose abilities do not warrant an expectation that they will succeed in the open professions, where they must encounter the competition of their contemporaries, and those whom indolence of temperament, or physical infirmities unfit for active exertions, are placed in the Civil Service, where they obtain an honourable livelihood with little labour, and with no risk; where their success depends on their avoiding any flagrant misconduct, and attending with moderate regularity to routine duties; and in which they are secured against the ordinary consequences of old age, or failing health.³

The object of the Report was to remedy this situation by identifying ‘the best method of providing [the Service] with a supply of good men, and of making the most of them after they had been admitted’. Currently ‘no pains’ were taken to appoint

‘good men’, or to train and motivate them. Indeed, perverse incentives were rife Promotion was determined by seniority and discipline was lax, so officials knew that ‘if they work hard, it will not advance them – if they waste their time in idleness, it will not keep them back’. Hence the Report sought to establish the principle that:

the public Service should be carried on by admission into its lowest ranks of a carefully selected body of young men, who should be employed from the first on work suited to their capabilities and their education, and should be made constantly to feel that their promotion and future prospects depend entirely on the industry and ability with which they discharge their duties, that with average abilities and reasonable application they may look forward confidently to a certain provision for their lives, that with superior powers they may rationally hope to attain to the highest prizes in the Service, while if they prove decidedly incompetent, or incurably indolent, they must expect to be removed from it.⁴

To realise this ideal, three explicit recommendations were advanced: recruitment by a ‘proper system of examination’, promotion by merit and the greater unification of the Service. Implicit in the first recommendation was a fourth: a clear distinction should be made between ‘intellectual’ and ‘mechanical’ labour.

The examination system should recognise this distinction by generally examining candidates for the ‘superior situations’ typically between the ages of 19 and 25, and those for the ‘inferior Offices’ between 17 and 21.

What was remarkable about this analysis was not so much its boldness as its narrowness. Wholly unaddressed, for example, were issues such as the political role of officials in relation to both Ministers and the public – or, in other words, the key constitutional issues of ministerial responsibility and official anonymity These were not just critical for the future. They had already been raised in acute form by, for example, the transformation in 1847 of the Poor Law Commission into the Poor Law Board (to reassert Parliamentary control over policy) and in 1854 by the dismissal of the outspoken Edwin Chadwick from the General Health Board. Equally remarkable for a report widely held to mark an historic watershed in the public administration of Britain, and of the Western world, was its tardy and incomplete implementation. There was, as recommended, no Act of Parliament to implement its proposals and little progress was made until 1870 Even then reform was disjointed and largely surreptitious. It was impelled by undebated Orders in Council, Treasury minutes and a series of public enquiries held not so much to honour the Northcote-Trevelyan ideal but in response to current political pressures: demands for greater economy, discontent within the Service and the relentless growth of government. Indeed, as late as the 1912–14 MacDonnell Royal Commission on the Civil Service, the ideal of ‘unity’ remained as unwelcome a prospect as ever for many senior officials. Even more seriously, the major expansion of government resulting from new welfare legislation had brought into question the very principle of open competition.

The narrowness of the Report will be examined in Chapter 1.3. This section concentrates on the substance and partial implementation of Northcote-Trevelyan’s four principal proposals.

1.2.1 Open competition

The most effective way to end ‘the evils of patronage’ and thus administrative inefficiency was identified by the Report to be competitive literary examinations, overseen by a Central Board and held at fixed times. For entry into ‘superior positions’, which required ‘intellectual’ labour, there was to be a national examination reflecting the highest academic levels. However, it should not necessarily ‘exclude some exercises directly bearing on official duties’. Subjects such as history, jurisprudence, political economy and geography should be included as well as the ‘staples of classics and mathematics’. This would ensure that width and not just depth of knowledge was tested so that ‘the greatest and most varied amount of talent’ would be attracted to the Service. For entry into the ‘lower class of appointment’, which only required ‘mechanical’ labour, there was to be a series of district examinations. Their nature was unspecified.⁵

A central board, in the form of the Civil Service Commission, was almost immediately established. It was not, however, established in the spirit intended. Political opposition thwarted the drafting, let alone the passage, of the proposed Civil Service Act to implement the Report. It also contributed to a change in government. Reform was therefore effectively stalled until further evidence of maladministration during the Crimean war led to the formation of the Administrative Reform Association which demanded inter alia that, in the appointment of officials, patronage should be replaced by a test of practical, not literary, skills. It was to forestall just such a calamity that the Commission was established in 1855.

The Commission was charged, as the Report wished, with the certification of the age, health, moral character and ‘requisite knowledge’ of all entrants to the Service. No-one could be employed without such a certificate. There was to be, however, no distinction between recruitment to ‘intellectual’ and ‘mechanical’ work, no national or district examinations and no open competition. Candidates could still be nominated by senior politicians or officials and their ‘knowledge’ tested by an examination jointly set for the occasion by the Commission and the relevant department. There need only be a single candidate although a limited competition between three or so nominees increasingly became the norm. Even this, however, represented little progress since the favoured candidate was often pitched against two of Hayter’s ‘idiots’ (candidates of somewhat limited ability of whom one chief whip, Sir William Hayter, appeared to have a bottomless supply). The initial impact of the Commission was therefore limited. Of the 9826 recruits to the Service between 1855 and 1868, 70 per cent were appointed after no competition, 2763 after limited competition and only 28 as the result of open competition.⁶

The situation finally changed in 1870 when Gladstone as Prime Minister, and more importantly Robert Lowe as Chancellor of the Exchequer, secured by Order in Council the establishment of a national open competition. It was divided into two ‘schemes’, respectively for graduates and school leavers, and so appeared to consummate the Northcote-Trevelyan ideal. Accordingly 1870 has been conventionally acclaimed as the ‘crucial year for enduring Civil Service reform’.⁷ The first series of examinations was held in 1871–2, when there were 142 candidates for 10 Class ivacancies and 732 candidates for 95 Class Iiposts.⁸ Appearances, however, can be deceptive. Open competition in fact continued to remain so limited that the MacDonnell Commission on the Civil Service found that only one-third of the 60,000 appointments made before 1910, which fell within its remit, had been so recruited.⁹ Why was this?

The majority of exempted posts had no policy implications, although they did reveal the continued existence of widespread patronage. They were either peripheral or very junior posts. Hence until 1885 favoured local MPs were invited to nominate sub-postmasters when a vacancy occurred within the 17,000 strong national network; and until 1912 the Treasury reserved the right to appoint its own nominees as messengers, porters and cleaners within all the revenue departments and national galleries.¹⁰ However, a significant number of appointments to senior posts were also exempted from open competition. This did not necessarily offend Northcote- Trevelyan principles. The Report itself had accepted that senior policy advisers to Ministers (then termed ‘staff appointments’) should be so exempt, as should posts which required ‘special talents and attainments’ (such as factory and school inspectors) – although its implicit hope, particularly in relation to ‘staff appointments’, was that fewer ‘strangers’ would be appointed. Acts of pure political patronage did duly cease in the 1880s – with, ironically, two of the last practitioners being Northcote himself (who secured a post for his son) and Gladstone (who nominated two of his private secretaries as heads of departments).¹¹ Nevertheless an increasing number of senior advisers continued to be recruited under various dispensations which exempted from open competition those whose qualifications were ‘wholly or in part professional or otherwise peculiar and not ordinarily to be acquired in the Civil Service’.¹² The principal reason for this was a further rapid expansion of government after 1880, particularly into new areas of social policy.

The Board of Trade provides a prime example. It acted as a magnet for social reform after 1880 culminating after 1908, initially under the presidency of Winston Churchill, with the introduction inter alia of labour exchanges, unemployment insurance and minimum wages. During this time it established such a tradition of appointing mature experts to senior posts that, of the 13 senior officials advising Churchill on labour policy, none had been recruited by open competition.¹³ This tradition was maintained, and typified by, the recruitment of William Beveridge in 1908. He had an unparalleled academic knowledge of, and practice in running, labour exchanges. He had accordingly given detailed evidence to the Royal Commission on the Poor Law and expert advice to the Board on the establishment of a national system. When such a system became a serious possibility Churchill was advised by Sidney and Beatrice Webb (who had already introduced the two) that ‘if you are going to deal with unemployment you must have the boy Beveridge’. A conference was duly held at the Board and Churchill immediately decided to take the Webbs’ advice. Beveridge was summoned to the Board to name his terms and the appointment made the following day.¹⁴

When the first complement of exchange staff was recruited, open competition based on literary exams was similarly waived – with the result, to the disgust of Civil Service staff associations, that three times as many manual workers as serving officials were recruited as exchange managers. The foremost method was a competitive interview with, as a guard against charges of patronage, the First Civil Service Commissioner overseeing the whole process. Churchill, however, decided to preside himself over the appointment of the twelve most senior executive officials (the divisional Officers). The safeguard against accusations of patronage was now the requirement, devised by Beveridge in two hours, that each candidate draft a reply to an irate employer. The successful candidates included two trade unionists, two soldiers and a former American gold speculator who claimed to have ‘run a labour exchange in Chicago, with a revolver provided as part of the Office equipment’. The star, however, was J. B. Adams, Shackleton’s second in command in the expedition to the South Pole. He provided the winning answer to Beveridge’s test, by inviting the employer to lunch. He also held Churchill in thrall during the interview by tales of his naval exploits. This was adjudged to compensate for his somewhat modest specialist knowledge. When asked his opinion of the Labour Exchange Act, for example, he reputedly replied : ‘couldn’t understand a word, mate’; and when further asked about what had impressed him most about Beveridge’s recently published book on unemployment, he replied ‘the price’.¹⁵ Although largely vindicated by its results, this selection process was hardly more rigorous than those pilloried in the 1850s.

Why was there such resistance to open competition both before and after 1870? There were three principal reasons. The first, as with the overall rejection of the Report, was personal: Trevelyan’s abrasiveness and the anger generated by his caricature of the existing Service.¹⁶ More substantially, patronage had – and was widely seen at the time to have – many virtues. It was an integral part of the social and political system. Open competition, for instance, could reasonably be described to Queen Victoria as ‘republican’ because it threatened the traditional means by which the Crown and the aris...

Table of contents

• COVER PAGE
• TITLE PAGE
• COPYRIGHT PAGE
• ACKNOWLEDGEMENTS
• ABBREVIATIONS
• INTRODUCTION
• PART 1: THE LEGACY
• PART 2: THE REFORM MOMENTUM
• PART 3: THE POLITICS AND PLANNING OF REFORM
• PART 4: WIDER ISSUES, 1966–81
• PART 5: CONCLUSION
• CHRONOLOGY
• NOTES
• REFERENCES
• SOURCES AND SELECT BIBLIOGRAPHY