The Defence of the Realm

Praise for
The Defence of the Realm

‘The most complete history of the agency ever published'
Time

‘Andrew's scholarship is meticulous and extensive. MI5 could not have wanted a better historian than him. He has captured every important detail of the Service, but also its ethos and its place in England as an institution. Buy it'
National Post

‘Illustrates through the story of the security service, the way the values of our society and our politics have changed over 100 years' Jonathan Powell,
New Statesman
, Books of the Year

‘Authoritative history'
The Globe and Mail

‘As complete and thorough as such a history may be and as engrossing as any spy novel' Tim Rutten,
Los Angeles Times

‘Engagingly successful in bringing the spirit and the personality of the service's culture and members through its complete survey. The common thread of keen intellect is evident . . . For those with the slightest interest in the intelligence world of the present and indeed the last 100 years, the book is assuredly essential reading'
Edmonton Journal

‘MI5 is the first major security or intelligence service in the world to give a historian free range of its records . . . it has been well worth the effort.
The Defence of the Realm
throws new light on an important area of the running of the country . . . It will be enthusiatically scrutinised by historians, intelligence buffs and conspiracy theorists' Stella Rimington,
Financial Times

‘Interesting, engaging . . . A fascinating read for lovers of espionage and security issues . . . Canada makes an appearance in Andrew's narrative in the well-known story of Soviet cipher clerk Igor Gouzenko who defected in Ottawa . . . Andrew has a sense of humour and has fun describing the early recruits to the service . . . His descriptions could have been the basis for a Monty Python skit'
Winnipeg Free Press

‘Compelling . . . an important book'
Irish News

PENGUIN CANADA

THE DEFENCE OF THE REALM

CHRISTOPHER ANDREW
is Professor of Modern and Contemporary History and former Chair of the Faculty of History at Cambridge University. He is also Chair of the British Intelligence Study Group, founding Co-Editor of
Intelligence and National Security
, former Visiting Professor at Harvard, Toronto and the Australian National University, and a regular presenter of BBC Radio and TV documentaries. His fifteen previous books include
The Mitrokhin Archive
volumes 1 and 2, and a number of path-breaking studies on the use and abuse of secret intelligence in modern history.

 

MI5's self-image at the end of 1917 on a Christmas/New Year card designed by its deputy head, Eric Holt-Wilson, and drawn by the leading illustrator, Byam Shaw. MI5, in the guise of a masked Britannia, impales the loathsome figure of Subversion with her monogrammed trident before he can stab the British fighting man in the back and prevent him achieving ‘Mankind's Immortal Victory' – MIV (MI5 in pseudo-roman form).

(
opposite
) The Security Service's all-seeing eye with a slightly unorthodox interwar Latin motto intended to mean ‘Security is the reward of unceasing vigilance.'

CHRISTOPHER ANDREW

The Defence of the Realm

The Authorized History of MI5

 

PENGUIN CANADA

Published by the Penguin Group

Penguin Group (Canada), 90 Eglinton Avenue East, Suite 700,

Toronto, Ontario, Canada M4P 2Y3 (a division of Pearson Canada Inc.)

Penguin Group (USA) Inc., 375 Hudson Street, New York, New York 10014, U.S.A.

Penguin Books Ltd, 80 Strand, London WC2R 0RL, England

Penguin Ireland, 25 St Stephen's Green, Dublin 2, Ireland (a division of Penguin Books Ltd)

Penguin Group (Australia), 250 Camberwell Road, Camberwell, Victoria 3124, Australia (a division of Pearson Australia Group Pty Ltd)

Penguin Books India Pvt Ltd, 11 Community Centre, Panchsheel Park, New Delhi – 110 017, India

Penguin Group (NZ), 67 Apollo Drive, Rosedale, North Shore 0632, New Zealand (a division of Pearson New Zealand Ltd)

Penguin Books (South Africa) (Pty) Ltd, 24 Sturdee Avenue, Rosebank, Johannesburg 2196, South Africa

Penguin Books Ltd, Registered Offices: 80 Strand, London WC2R 0RL, England

First published in a Viking Canada hardcover by Penguin Group (Canada),

a division of Pearson Canada Inc., 2009.

Published in this edition with updated material, 2010

1 2 3 4 5 6 7 8 9 10 (WEB)

Copyright © Crown copyright, 2009, 2010

All rights reserved. Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise), without the prior written permission of both the copyright owner and the above publisher of this book.

Set in 10.13/13.8 pt. Sabon

Typeset by TexTech International

Manufactured in Canada.

Library and Archives Canada Cataloguing in Publication data available upon request to the publisher.

ISBN: 978-0-14-317458-5

Except in the United States of America, this book is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, re-sold, hired out, or otherwise circulated without the publisher's prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser.

Visit the Penguin Group (Canada) website at
www.penguin.ca

Special and corporate bulk purchase rates available; please see

www.penguin.ca/corporatesales
or call 1-800-810-3104, ext. 2477 or 2474

Contents

List of Illustrations

Foreword by the Director General of the Security Service

Preface

Acknowledgements

Section A

The German Threat, 1909–1919

Introduction: The Origins of the Secret Service Bureau

1  ‘Spies of the Kaiser': Counter-Espionage before the First World War

2  The First World War: Part 1 – The Failure of German Espionage

3  The First World War: Part 2 – The Rise of Counter-Subversion

Section B

Between the Wars

Introduction: MI5 and its Staff – Survival and Revival

1  The Red Menace in the 1920s

2  The Red Menace in the 1930s

3  British Fascism and the Nazi Threat

Section C

The Second World War

Introduction: The Security Service and its Wartime Staff: ‘From Prison to Palace'

1  Deception

2  Soviet Penetration and the Communist Party

3  Victory

Section D

The Early Cold War

Introduction: The Security Service and its Staff in the Early Cold War

1  Counter-Espionage and Soviet Penetration: Igor Gouzenko and Kim Philby

2  Zionist Extremists and Counter-Terrorism

3  VENONA and the Special Relationships with the United States and Australia

4  Vetting, Atom Spies and Protective Security

5  The Communist Party of Great Britain, the Trade Unions and the Labour Party

6  The Hunt for the ‘Magnificent Five'

7  The End of Empire: Part 1

8  The End of Empire: Part 2

9  The Macmillan Government: Spy Scandals and the Profumo Affair

10  FLUENCY: Paranoid Tendencies

11  The Wilson Government 1964–1970: Security, Subversion and ‘Wiggery-Pokery'

Section E

The Later Cold War

Introduction: The Security Service and its Staff in the Later Cold War

1  Operation FOOT and Counter-Espionage in the 1970s

2  The Heath Government and Subversion

3  Counter-Terrorism and Protective Security in the Early 1970s

4  The ‘Wilson Plot'

5  Counter-Terrorism and Protective Security in the Later 1970s

6  The Callaghan Government and Subversion

7  The Thatcher Government and Subversion

8  Counter-Terrorism and Protective Security in the Early 1980s

9  Counter-Espionage in the Last Decade of the Cold War

10  Counter-Terrorism and Protective Security in the Later 1980s

11  The Origins of the Security Service Act

Section F

After the Cold War

1  The Transformation of the Security Service

2  Holy Terror

3  After 9/11

Conclusion: The First Hundred Years of the Security Service

Appendix 1: Directors and Director Generals, 1909–2009

Appendix 2: Security Service Strength, 1909–2009

Appendix 3: Nomenclature and Responsibilities of Security Service Branches/Divisions, 1914–1994

Notes

Bibliography

Index

List of Illustrations

Plates

1  Vernon Kell (Hulton Deutsch Collection/Orbis)

2  Major (later Brigadier General) James Edmonds (National Army Museum)

3  William Le Queux with his publisher (Frederic G. Hodsoll/National Portrait Gallery, London)

4  William Melville (By kind permission of Andrew Cook)

5  Gustav Steinhauer (in disguise) (Steinhauer,
The Kaiser's Master Spy: The Story as Told by Himself
, John Lane/The Bodley Head Ltd, 1930)

6  Winston Churchill, Sidney Street Siege, 1911 (Hulton Archive/Getty Images)

7  William Hinchley Cooke in German military uniform (Service Archives)