Archive

Large scale document archive

The world of Confidential Filing

How MT's official papers at No.10 have been archived

what are these files & how did the system work?

File cover - note the tag hole at top left

Official papers submitted to the Prime Minister were handled by her Private Secretaries, four officials in their 30s or 40s of high abilities and bright prospects many of whom filled Whitehall's top jobs later in their careers. This Private Office team was headed by the Principal Private Secretary, who handled some of the most sensitive material such as intelligence. Hours were long and the work sometimes very stressful, if fascinating. Generally MT got on very well with these officials, and probably saw more of them than she saw of anyone else.

The Private Office chose what papers to submit to the PM, adding their own take when needed, then gathered and acted upon her annotations, often by letter to the relevant department ("The Prime Minister has read your Secretary of State's paper of 15th May and had the following comments ..."). At the end of the whole process the paperwork was sent off - if thought worth keeping - to “Confidential Filing” a few steps away from the Cabinet Room. Here even the most sensitive and historic state papers had holes punched in their top left corner and were stored in buff-coloured A4 files, held together by “Treasury tags”, bits of coloured string with shiny metal retainers on each end.These are something of a disaster in conservation terms because pages tend, over time, to tear at the weakened corner and fall out of the file, their original position not always possible to determine, if they can be found at all - it is a fair bet that some must have been lost altogether.

If a file had reached the maximum practical size, around 300 pages, a new folder with the same or somewhat similar name would be started, labelled “Part 2” (or 3, 4, or whatever - on the n+1 principle). The most important files would have many parts, perhaps 5 or 6 within a calendar year.

Although new files were opened whenever there was a change of government or Prime Minister, whichever came sooner, generally files often ran over year endings - there was no provision to open a new part on 31 December. Accordingly many files started during a particular year of the Thatcher Government will not be available to the public with papers from that year, because on 31 December they were not yet full enough to justify the expenditure of opening a new part. Folders and Treasury tags cost money after all, so they will have run into the next year (or beyond) and will be declassified with files from the year in which they ended. This approach can produce long delays in release: for example, a single document from 1986 at the conclusion of a file relating otherwise purely to 1979 will not be opened until 1986 documents are processed. Canny officials might have filed a late-dated letter in order to extend the closure of the whole file, but prior to Freedom of Information record-keeping was generally free of such trickery.

Files were organised in blocs according to topic - there were blocs of files on Economic Policy or European Policy, for example - and within those blocs there were a few big, subject files, used year in year out, from one goverment to the next. For example, within Economic Policy there was always a file, in many parts, on public spending. There was another on the annual budget. There was a multi-part file of dismal length on the Northern Ireland situation, categorised - rather unhappily from a Unionist point of view - as part of the bloc on 'Ireland'. Similarly, Confidential Filing opted for a unified Germany, but for some reason sustained the partition of Korea.

File bulk gives one a good indication of the time spent on a topic. In terms of size, the Rhodesia file easily took the prize in 1979, running to eleven parts. The Lancaster House Conference of late 1979 represented the crescendo, of course, the climactic moment that resolved the Rhodesian problem (from the British point of view at least, for the time) and future years saw a sharp fall in the output of paper.

In addition, files were created ad hoc where a topic did not fit into an existing main file. Often these are on trivial topics, some close to bizarre. In 1979 a file was opened on the proposal that the PM send the Civil Service a Christmas Message. This must have taken all of two minutes of her time (the decision was in the negative.) Other ad hoc files were more substantial, but they are almost always secondary in importance to the big, standard files.

A weakness of the system was that, over time, the names of these big files with many parts tended to drift a considerable distance from their original, making it difficult for people to grasp that the file was a single entity, or even to find all the parts. Unfortunately the generally excellent TNA online catalogue faithfully reproduces the fault. To avoid this problem, the file names used on this site have been rationalised, so they will differ from the TNA names on occasion.

Despite this flaw, the No.10 system was simple and easy to use. Compare it with the bewildering complexities of the White House Office of Records Management (lovingly known as WHORM), which generated files on a scale several orders of magnitude greater than No.10 and required a computer to keep track of everything from the early 1970s on, generating a huge database which US archivists aren't even allowed to see. One can be sure in the British system that one has found everything available on any particular topic, a certainty absolutely lacking in systems like the US - indeed, the reverse holds.

what was kept & what not?

Here is some of a mystery. Certainly not every piece of paper circulating in No.10 was archived in the first place. Much was considered trivial and discarded. It is possible that some things were discarded for the opposite reason - a conclusion one might reach on the Blair Government from studying No.10 files submitted to the 2004 Butler Review of Intelligence on Weapons of Mass Destruction, to give a near contemporary instance.

Some files preserved at the time were later deemed of no lasting value and destroyed. These include many things historians would have liked to see, such as the notebooks kept by No.10 Private Secretaries, which among other things record who said what in ministerial meetings. Minutes of such meetings were written up and filed separately of course, but often in a form that anonymised the discussion. A document destroyed is a document that cannot, and will not need to be, declassified, so all things being equal destruction reduces the burden of work on officials.

Some documents were de-accessioned and given to the outgoing PM. Through the good offices of the Garden Room, c.1994, key administrative files from No.10 during MT's tenure were given to her, largely relating to engagements, greatly enhancing our knowledge of what you might call the back office functions of the place. Those files are stored at the Churchill Archive Centre in Cambridge. There has since been a further generous gift of speech files from the Cabinet Office, which include drafts and covering notes. Bernard Ingham kept a mass of Press Office files, including lobby briefing notes. All of this kind of material has generally been destroyed for previous administrations, so the record from 1979-90 is significantly richer.

A few years prior to the beginning of declassification, an academic employed by the Cabinet Office weeded the Thatcher files, reducing their bulk by discarding documents reckoned of no long-term value. The basis for such judgments is obscure and the amount discarded uncertain. Most administrative files tended to be destroyed at some point: we find nothing at TNA, for example, on the running costs of No.10, on personnel or the vexed question of Prime Ministerial cars. Finally, during declassification itself, selected papers within surviving files were destroyed and their destruction logged there, mainly duplicates of cabinet papers and minutes which were discarded on the ground that they are available elsewhere in the official archive. Their survival in situ would have been a great convenience to future readers, but since the PM rarely if ever annotated such documents, there is no loss otherwise. Unfortunately the PM's copies of briefing for summit meetings have often also been destroyed, on the same principle, so that annotated versions have been discarded in preference to the beautifully-bound but crisply clean, unannotated copies kept in the Cabinet Office's own archive. Thankfully some have been saved for TNA, notably the briefing she took with her to the Dublin European Council in November 1979 where she asked for our money back - listed last in the series of 1979 releases, out of sequence, and probably therefore an accidental (or last minute) survivor of the cull. A number of the other briefings have been donated to the Churchill Archive Centre, which holds some also in MT's purely private papers.

closed & missing files

Some files have been closed for longer periods than 30 years. Oddly, in some cases the file covers have made it to TNA, so that an empty file can be viewed with an explanatory slip inside, in others there has been a complete withholding.

The most important of these held-back files relate to defence and intelligence.

1979

Perhaps the most significant missing item from 1979 is part 1 of the file on the renewal of Britain's strategic nuclear deterrent, which is listed as "temporarily retained by department" - effectively an indefinite closure (PREM19/14). (Several files on the same topic from the 1974-79 Labour government were retained on that basis, but have recently found their way to Kew.) Another such is the file on the case of Anthony Blunt, juicily titled "The Security of the Secret Service" (PREM19/120). Two files on the Royal Family have been held back, as they always are, neither likely to have been of any great importance (PREM19/85 & 118). One part of the Northern Ireland situation file has been closed, covering the assassination of Lord Mountbatten and the bombing at Warrenpoint (PREM19/81). Again, all are "temporarily retained" rather than subject to particular terms of extended closure. Finally, a single file has been closed under Section 3.4 of the Public Records Act, 1958, relating to Zambia (PREM19/132). No term has been set for its release.

Unnoticed by the press files on sterling were wholly missing from the 1979 releases. In fact files exist on the abolition of exchange control and EMS, but both ran over the year end, and it seems that no main sterling file was created for MT, probably because the trigger for such files in the past had been sterling crises (i.e., sudden falls in the currency). The steep upward movement of the pound during 1979-80 caused much difficulty and angst, but no file.

1980

Temporarily retained items for 1980 include two important defence files, PREM19/159 & 160: part 2 of the strategic nuclear deterrent file and part 1 of the "UK-US understandings on the use of nuclear weapons & bases". Three much less significant files are also held back on the same basis, all relating to the Royal Family. One is a file on proposed amendment of the 1701 Act of Settlement (PREM19/287), doubtless relating to the perennial issue of the bar the Act places on a Catholic ever ascending the throne, an issue raised in Parliament on 7 July by Ian Paisley - the day before the file was started. The second concerns BBC interviews with Lord Mountbatten, which correspondence in the Hailsham MSS suggests may relate to the Suez crisis (PREM19/294). Finally there is a file on allegations that secret agents had stolen the Duke of Windsor's papers (PREM19/355).

Closed outright till 2034 is an Energy file on the Offshore Supplies Interest Relief Grants Scheme, which includes a "report on irregular payments" (PREM19/209).

Leo Pliatzky's report on culling quangos is in current use in Whitehall, perhaps to good effect (PREM19/245).

A file of "Intelligence on Irish terrorists in the Republic [of Ireland]" is closed indefinitely under Section 3.4 (PREM19/284), as are a number of files from the Security bloc: one on Intelligence Liaison (PREM19/359), a second on the Lamble case (PREM19/361), a third and fourth on Policy on interception (Parts 1 and 2) (PREM19/363-364). Finally there is a Section 3.4 closure on Soviet propaganda use of the Moscow Olympics (PREM19/372).

link to the list

Below is a link to the downloadable files as catalogued by TNA, organised in the subject blocs used by Confidential Filing.

Some of these downloads are very big - as large as 75MB. Most are in the range 1-10MB. To keep sizes down, images have been compressed, though legibility should never be a problem.

Where documents within a file have been selected for the site database, they can easily be viewed by clicking the link after the file name ("key documents from this file"), as well as via conventional searches from the main search screen. These selected items are the most important documents in the file. Over time most files will have had documents selected in this way, but it will take time and care is being taken not to allow the site database to become so large as to be unmanageable.

These selected documents include every minute, letter, conversation note ('memcon', to use the American term) by or involving the Prime Minister, as well as all minutes and letters sent her by the key ministers such as the Chancellor of the Exchequer and Foreign Secretary. If you read nothing else, read these. These selected items are stored in a higher quality format than the whole files. High quality copies can also be supplied, where a need is established, on application to the site editor.

Download PREM19 files