The world of Confidential Filing

What are these files & how did the system work?

Official papers submitted to the Prime Minister were handled by her Private Secretaries, four officials in their 30s or 40s of high abilities and bright prospects many of whom filled Whitehall's top jobs later in their careers. This Private Office team was headed by the Principal Private Secretary, who handled some of the most sensitive material such as intelligence. Hours were long and the work sometimes very stressful, if fascinating. Generally MT got on very well with these officials, and probably saw more of them than she saw of anyone else.

The Private Office chose what papers to submit to the PM, adding their own take when needed, then gathered and acted upon her annotations, often by letter to the relevant department ("The Prime Minister has read your Secretary of State's paper of 15th May and had the following comments ..."). At the end of the whole process the paperwork was sent off - if thought worth keeping - to “Confidential Filing” a few steps away from the Cabinet Room. Here even the most sensitive and historic state papers had holes punched in their top left corner and were stored in buff-coloured A4 files, held together by “Treasury tags”, bits of coloured string with shiny metal retainers on each end.These are something of a disaster in conservation terms because pages tend, over time, to tear at the weakened corner and fall out of the file, their original position not always possible to determine, if they can be found at all - it is a fair bet that some must have been lost altogether.

If a file had reached the maximum practical size, around 300 pages, a new folder with the same or somewhat similar name would be started, labelled “Part 2” (or 3, 4, or whatever - on the n+1 principle). The most important files would have many parts, perhaps 5 or 6 within a calendar year.

Although new files were opened whenever there was a change of government or Prime Minister, whichever came sooner, generally files often ran over year endings - there was no provision to open a new part on 31 December. Accordingly many files started during a particular year of the Thatcher Government will not be available to the public with papers from that year, because on 31 December they were not yet full enough to justify the expenditure of opening a new part. Folders and Treasury tags cost money after all, so they will have run into the next year (or beyond) and will be declassified with files from the year in which they ended. This approach can produce long delays in release: for example, a single document from 1986 at the conclusion of a file relating otherwise purely to 1979 will not be opened until 1986 documents are processed. Canny officials might have filed a late-dated letter in order to extend the closure of the whole file, but prior to Freedom of Information record-keeping was generally free of such trickery.

Files were organised in blocs according to topic - there were blocs of files on Economic Policy or European Policy, for example - and within those blocs there were a few big, subject files, used year in year out, from one goverment to the next. For example, within Economic Policy there was always a file, in many parts, on public spending. There was another on the annual budget. There was a multi-part file of dismal length on the Northern Ireland situation, categorised - rather unhappily from a Unionist point of view - as part of the bloc on 'Ireland'. Similarly, Confidential Filing opted for a unified Germany, but for some reason sustained the partition of Korea.

File bulk gives one a good indication of the time spent on a topic. In terms of size, the Rhodesia file easily took the prize in 1979, running to eleven parts. The Lancaster House Conference of late 1979 represented the crescendo, of course, the climactic moment that resolved the Rhodesian problem (from the British point of view at least, for the time) and future years saw a sharp fall in the output of paper.

In addition, files were created ad hoc where a topic did not fit into an existing main file. Often these are on trivial topics, some close to bizarre. In 1979 a file was opened on the proposal that the PM send the Civil Service a Christmas Message. This must have taken all of two minutes of her time (the decision was in the negative.) Other ad hoc files were more substantial, but they are almost always secondary in importance to the big, standard files.

A weakness of the system was that, over time, the names of these big files with many parts tended to drift a considerable distance from their original, making it difficult for people to grasp that the file was a single entity, or even to find all the parts. Unfortunately the generally excellent TNA online catalogue faithfully reproduces the fault. To avoid this problem, the file names used on this site have been rationalised, so they will differ from the TNA names on occasion.

Despite this flaw, the No.10 system was simple and easy to use. Compare it with the bewildering complexities of the White House Office of Records Management (lovingly known as WHORM), which generated files on a scale several orders of magnitude greater than No.10 and required a computer to keep track of everything from the early 1970s on, generating a huge database which US archivists aren't even allowed to see. One can be sure in the British system that one has found everything available on any particular topic, a certainty absolutely lacking in systems like the US - indeed, the reverse holds.

what was kept & what not?

Here is some of a mystery. Certainly not every piece of paper circulating in No.10 was archived in the first place. Much was considered trivial and discarded. It is possible that some things were discarded for the opposite reason - a conclusion one might reach on the Blair Government from studying No.10 files submitted to the 2004 Butler Review of Intelligence on Weapons of Mass Destruction, to give a near contemporary instance.

Some files preserved at the time were later deemed of no lasting value and destroyed. These include many things historians would have liked to see, such as the notebooks kept by No.10 Private Secretaries, which among other things record who said what in ministerial meetings. Minutes of such meetings were written up and filed separately of course, but often in a form that anonymised the discussion. A document destroyed is a document that cannot, and will not need to be, declassified, so all things being equal destruction reduces the burden of work on officials.

Some documents were de-accessioned and given to the outgoing PM. Through the good offices of the Garden Room, c.1994, key administrative files from No.10 during MT's tenure were given to her, largely relating to engagements, greatly enhancing our knowledge of what you might call the back office functions of the place. Those files are stored at the Churchill Archive Centre in Cambridge. There has since been a further generous gift of speech files from the Cabinet Office, which include drafts and covering notes. Bernard Ingham kept a mass of Press Office files, including lobby briefing notes. All of this kind of material has generally been destroyed for previous administrations, so the record from 1979-90 is significantly richer.

A few years prior to the beginning of declassification, an academic employed by the Cabinet Office weeded the Thatcher files, reducing their bulk by discarding documents reckoned of no long-term value. The basis for such judgments is obscure and the amount discarded uncertain. Most administrative files tended to be destroyed at some point: we find nothing at TNA, for example, on the running costs of No.10, on personnel or intelligence reports seen by the Prime Minister. Finally, during declassification itself, selected papers within surviving files were destroyed and their destruction logged there, mainly duplicates of cabinet papers and minutes which were discarded on the ground that they are available elsewhere in the official archive. Their survival in situ would have been a great convenience to future readers, but since the PM rarely if ever annotated such documents, there is no loss otherwise. Unfortunately the PM's copies of briefing for summit meetings have often also been destroyed, on the same principle, so that annotated versions have been discarded in preference to the beautifully-bound but crisply clean, unannotated copies kept in the Cabinet Office's own archive. Thankfully some have been saved for TNA, notably the briefing she took with her to the Dublin European Council in November 1979 where she asked for our money back - listed last in the series of 1979 releases, out of sequence, and probably therefore an accidental (or last minute) survivor of the cull. A number of the other briefings have been donated to the Churchill Archive Centre, which holds some also in MT's purely private papers.

Link to the list

Below is a link to the downloadable files as catalogued by TNA, organised in the subject blocs used by Confidential Filing.

Some of these downloads are very big - as large as 75MB. Most are in the range 1-10MB. To keep sizes down, images have been compressed, though legibility should never be a problem.

Where documents within a file have been selected for the site database, they can easily be viewed by clicking the link after the file name ("key documents from this file"), as well as via conventional searches from the main search screen. These selected items are the most important documents in the file. Over time most files will have had documents selected in this way, but it will take time and care is being taken not to allow the site database to become so large as to be unmanageable.

These selected documents include every minute, letter, conversation note ('memcon', to use the American term) by or involving the Prime Minister, as well as all minutes and letters sent her by the key ministers such as the Chancellor of the Exchequer and Foreign Secretary. If you read nothing else, read these. These selected items are stored in a higher quality format than the whole files. High quality copies can also be supplied, where a need is established, on application to the site editor.