Appraisal—the state of the art

Michael Piggott

Paper delivered at a professional development workshop presented by
ASA South Australia Branch
26 March 2001

This paper has two parts.

The first attempts to identify and explain one or two of the principal appraisal ideas that have emerged in Western archival discourse over the past 20 years. This is risky – not simply because of the amount of published literature to condense (though embarrassingly little of it is Australian), but also because there is comparatively little writing on non-government archives appraisal.(1) I will do my best to summarise for ALL archivists—echoing a favorite politician’s promise to govern for ALL Australians—be they life cyclers, recordkeepers, manuscript curators, or librarians with part-time responsibility for the ‘old stuff’.

The second part of the paper sets the scene for some discussion (firstly in groups, then all together) of case studies of controversial and contentious appraisals.

As a lead in to part one, however, I thought I’d try to list what have become almost universal appraisal truisms: some might prefer to call them the motherhood clichés. Here’s my list. See if you agree.

1. appraisal is not a science, not a mechanistic or formula driven process that somehow magically produces a definitive, correct, infallible answer;
2. appraisal is not free of bias and subjectivity; its results reflect the cultural and other values of the time;
3. appraisal is the primary archival function on which all other functions depend – you can’t provide access or do ‘a & d’ on material which doesn’t exist; our very definition of what archives are is directly tied to appraisal, in both the traditional and newer meanings of appraisal; some even say consigning records to the furnace is our professional ‘rite of passage’;
4. the great trinity mystery:
   • apart from some exceptional cases, it is beyond our resources and power to keep all records; which is a pity, because
   • beyond their original use, all records conceivable have their uses; we’ve come to expect unexpected uses and yet
   • it is almost impossible to accurately predict future use, and when we try, the passage of time can cause serious havoc with appraisal judgments (examples?)
5. destruction to protect privacy is not appraisal, though sometimes very attractive to politicians, privacy extremists, community groups(2), and those with a self appointed role to keep the family legends intact;
6. disposal doesn’t just mean perpetual preservation or eventual destruction; it includes alternative custody e.g. material which lacks enough significance to be state archives offered to local historical societies or ‘places of deposit’;
7. it is very easy to justify retaining or accepting something;(we’re preservers by nature), but intrinsically difficult to justify destruction or refuse an offer;
8. we’ve always been reluctant to copy then destroy archival records,(3) even though we nominate very few items in our holdings as having ‘intrinsic’ value;
9. the older records are, and the fewer there are to choose from, the more inclined we are to keep them.

I tried without success to think of a tenth; if you want to add to or comment on the list, hold the thought for later discussion. For the moment, let’s leave the basic list and survey the appraisal scene of the past 20 years. Again you may have a different set of highlights; but my top hits include:

• a rediscovery of the importance of consulting the user, particularly by government archivists for political and accountability reasons, a value now enshrined (regrettably without guidance) in the Australian Standard for Records Management, AS 4390;
• also especially prevalent in the government sector has been an emphasis on accountability both as a strong reason for proper records creation (remember Ros Kelly’s ‘whiteboard’ affaire), as one of the classic appraisal criteria, and as part of proper disposal procedure; note too some writers speak of ‘historical accountability’;
• ‘risk management’ an overall description of what is happening in appraisal, again enshrined in AS 4390;
• the addition of ‘intrinsic’ value and ‘symbolic’ value to the list of orthodox appraisal criteria;
• important new thinking on how to assess case files; and
• finally, there’s been a renewed interest in de-accessioning (or collection review or re-appraisal) [hardly need to mention that to this audience].

But the two stand-out developments?; ‘macro-appraisal’ and a new definition of appraisal.

Macro-appraisal (4)

Macro-appraisal has been called ‘the third way’, i.e. an alternative to Jenkinson’s emphasis on the record creator’s business criteria and Schellenberg’s emphasis on the record user’s informational and research criteria; - an alternative which combines them by focusing on the values of society at large.

Some of the more theoretical writing about macro-appraisal is very challenging. But when one finally grasps the essence of its practical application, it’s difficult to fathom what all the fuss is about. It is nothing more than deciding [what records to create and] how long they should be kept or deciding what archival records to collect, by first mapping the territory; by first identifying and analyzing the theoretical documentary universe. This overview-analyzing focuses on the institutions and people (individually or in combination), whose activities gave rise to records. The analysis is usually centred on functions and activities, but not all macro-appraisal is functional appraisal.

When looking at combinations of institutions or people, the tendency has been to focus on broad societal functions, but also sometimes on projects, communities, localities or events. (This is why I don’t regard macro-appraisal and functional appraisal as synonymous). When the focus is on a single institution or person, they are known typically as ‘business functions’ (from AS 4390) and for individuals, personal roles or activities.

Only after that intensive research based examination of societal or business functions, says macro-appraisal, should you do ‘micro-appraisal’. In this second stage you go looking for the key records themselves. This so-called ‘top-down, mind-over-matter’ approach can save time, helps better cope with large volumes of records, and arguably, results in better archives. All of the traditional appraisal values now come into play plus the secondary factors such as existence of control records, cost, and preservation.

Government archivists have been the most prevalent users of macro-appraisal encompassing combinations of institutions or individuals performing broad societal functions, particularly the Canadians but there is a strong following in the main Australian government archives as well. Their study of the entire machinery of government results in rankings of important common and routine functions, and thus assists priority setting and avoids duplication of effort. Ranking aside, some government archivists come close to actually evaluating functions, with a predictable preference for central core policy functions (and consequent records), an approach expressed most directly by the Dutch National Archives.(5) Or, to quote Terry Cook’s Melbourne conference paper:

In the office of a headmaster, records may be created for the implementation of a major revision of the curriculum and for the ordering of pencils, but clearly one function generates records of long-term importance and the other does not. That in a nutshell is the reason for functional macro-appraisal: determining which functions, work processes, activities, or individuals are likely to produce records of enduring value and which are not…(6)

It is not always as simple as that of course: there is a great deal of research required, which is one of the criticisms of the approach. But be in no doubt that once significant or core functions can be separated from non-significant, large quantities of records can be processed and discarded.

The business sector has also been analysed from the top down, e.g. by Danish business archivists using the ‘extended sector’ method, and most famously by the Minnesota Historical Society in an attempt to rationalize its collecting of modern business records, by prioritizing Minnesota businesses for targeting and depth of collecting using criteria such as local economic impact, uniqueness to Minnesota and existing strengths and gaps.(7) In addition to ‘government’ and ‘business’ within a particular state, one might cite from the North American archival literature macro approaches to other societal functions domains or documentation categories such as scientific and technological research, health care, the moon shot and localities such as the western region of NY.

The best known examples of macro-appraisal covering individual studies again all seem to be of US or Canadian origin, though the Dutch ‘PIVOT” project has also been well documented. They include functional investigations of individual government departments, but also a generic high technology company and a generic university, the latter championed by Helen Samuel based on her experience as archivist at MIT and written up in a famous book called Varsity Letters.(8) With single focus macro-appraisal, functions are not always ranked or themselves appraised; in the generic studies such as Varsity Letters all functions are treated equally and used as a template to assess what records are currently being generated and functions which don’t usually generate extensive documentation which might need intervention. Samuels has illustrated this in relation to the teaching function, which hardly appears in the administrative record, and which prompted the preservation of a web site, e-mails, photos, videotapes and oral history interviews with teachers and students.(9)

Samuels was confident however that the generic functions model could be adapted to hospitals, museums, banks, courts, churches and businesses. Terry Cook, who has articulated an enhanced macro-appraisal model particularly as applied to government, argues likewise. His citizen-state axis has its parallel in universities and students, churches and parishioners, businesses and clients, and unions and members. Locally, Jan Riley also supported the relevance of Samuel’s model to independent schools.(10)

There is no reason why functional analysis shouldn’t also help with individuals, generic or specific, though we have yet to anyone‘s experience reported in the professional literature. There is a little on scientists and technologists, whose lives were classified into typical personal, professional and scientific activities with each further subdivided (11); there were some tantalizing references to Wellington as soldier, statesman and musician made by Chris Hurley in his ‘ambient function’ Archivaria article (12); and there was a strong assertion by Sue McKemmish that the functional roles society assigns us arising from relationships provide an individual-centred appraisal framework:

Spouse, lover, long-time companion, taxpayer, parent, citizen, subscriber, member of a club, professional society or church, student, ratepayer, flat mate, customer, ancestor, descendant, …me, all these words place individuals in relation to others and in society…. What records of the activities associated with these various roles do individuals want or need to capture, and in what documentary forms? Why do they need to capture them? How long do they need to keep them and why…?(13)

These are good questions, but 5 years since they were asked, I see nothing in the literature to indicate we’re any closer to knowing how to answer them. Saying exactly how appraisal judgments fall out of these kinds of categorizations needs much more work—assuming that individuals generate records attributable to roles [‘functions’] in the first place.

New definition of appraisal

According to the Australian Standard for Records Management, ‘appraisal’ is

The process of evaluating business activities in determining which records need to be captured and how long records need to be kept, to meet business needs, the requirements of organizational accountability and community expectations. [AS 4390.1, para 4.3 My italics]

The spectre of electronic records more than any other single factor led to this broader concept of appraisal (and in part the new ‘recordkeeping- records continuum’ approach too). Chris Hurley wrote what in my opinion is the best succinct statement of why this happened and had to happen in a list serv posting in early 1997 and reproduced in Archives and Manuscripts in the ‘In the Agora’ section of the May 1997 issue. Essentially, the argument is that if we don’t get involved at the ‘front end’ of recordkeeping systems design, there won’t be accessible authentic reliable records around later on to appraise anyway, and besides, the techniques used to appraise are the same which should be used to create records on the first place. We’re referring of course to functional analysis of business processes. There is ample advice on how this can be done, and Catherine Robinson has written up the story from a NSW State Archives’ perspective in Archives and Manuscripts.(14)

In fact, the release of AS 4390 endorsed an idea which had been around for as long as disposal schedules, i.e. if you were matching disposal action to records classes which were in effect types of activities or transactions, one might as well classify records from the very beginning aligned with those disposal classes. This can be especially effective for activities common to many agencies, whether housekeeping activities like staff recruitment or more complex areas such as policy development. To take a simple example, if you’ve decided that dog licensing records can be managed by retaining ‘applications granted’ for 20 years and applications denied’ for 12 months, it makes sense to have two categories of files in the first place, rather than a simple license applications file.

Even so, in the days of paper records, the life cycle and absence of a demarcation dispute between records managers, archivists and manuscript librarians, appraisal as commonly understood assumed the records prior existence. With electronic records, it is hardly an option. We have to get involved in records creation; there won’t be any to appraise down the track otherwise.

This goes for collecting archives too!(15) How collecting archives do front end appraisal, particularly in relation to private organisations’ and individuals’ papers with their impossible numbers and unregulated jurisdictions, is far less obvious. Where is the relevant DIRKS manual for example? There are certainly benefits if we do, something which also applies in the paper world and which manuscript curators have always known. The sooner one establishes a relationship with a prospective depositor, the more likely important material is identified and preserved and properly ordered. How one spots early enough a prospective Bob Hawke, Cathie Freeman, Michael Dransfield…. or Martin Bryant takes experience, resources and luck, and in a country with a high degree of immigration additional complications too. How one influences the design of individuals’ electronic recordkeeping “systems”, complete with built-in functionally aligned disposal classes is also not straightforward, but there are the beginnings of solutions being reported. The National Archives’ experiences coping with ministers’ papers is relevant and potentially adaptable to the CEO’s offices of large business and non profit organisations. The National Library is also systematically surveying its manuscript collections for obsolete discs, while acknowledging the importance of having had donors explain the original systems context!(16) And now we have an excellent report in the latest ‘A&M’ from Lucie Paquet, an archivist specializing in personal electronic records with the Manuscripts Division of the National Archives of Canada.(17)

Case studies (lead in to small group discussions of Heiner, NAA review of records, census reversal and a collecting archives case study)

One trend that is definitely not evident from our review of the recent appraisal scene is a strong interest in performance measures. Few if anyone would seem to be asking,(18) following appraisal, did we get it right? Were one’s decisions to keep indefinitely/collect, keep for a time, and cull within x years/don’t collect’, vindicated and validated? Do we (‘we’ including researchers, society, etc…) regret or stand by our predecessors’ decisions; and more to the point, who’s checking? And no one seems to be asking, even when we think we have failed, how serious were the consequences, even though we must have applied risk management thinking.

The problem is that it is so difficult to pin down the basis on which we would say ‘yes we got it right’, or ‘no, that was a blunder’. The most obvious measure, high or low use, has all sorts of complications(19); another, prevailing community values, is equally fraught as they can only be applied backwards (no one cared less about the census records destruction until the 1950s). Such questions also apply to collecting archives; ‘failures’ to collect are usually only vaguely consciously taken decisions, or ‘generic’ decisions (….“oh, we’ve never collected the papers of self-publishing authors, etc etc”) especially if collecting is reactive.

So not knowing how to decide whether we ‘got it right’ or not, we resort to approximations:

We assert, usually defensively, that yes, we must have got it right because

• we reduced quantities of records and can now fit into the new repository
• lots of people seem to be in the reading room
• we followed correct procedure as set out in the manual/the standard/…, and/or
• no one protested.

Or we concede with a shrug that, no, we can’t have got it completely right

• because the government or the archives board of management over-ruled our recommendations
• because there was a public outcry
• because our processes were not followed or the sentencing contractors were not properly trained
• because an auditor criticized an agency’s failure to create records
• because a ‘disposal freeze’ was prompted by a court awarding damages to someone, or
• because a historian or author bemoaned the absence of records or their shredding.

There is a third option. We might succumb to complete irresponsibility (and relativism) and argue that since there is never an infallible definitive right answer, and risk management is not an exact science, and anyone can be wise after the event, the entire question of performance measures is irrelevant.

Personally, I believe we owe it to current and future researchers, taxpayers and those who employ us for our professional expertise to account for our judgments and their consequences. There is no doubt that we need to do further work on appraisal performance measures. We also need more case studies of presumed appraisal success and failures written up in the professional and user literature and list servs. There are precious few examples to emulate(20), and this final session is one attempt to help remedy that gap.

(1) Reflection on the relative poverty of Australian appraisal literature warrants a paper in itself. North American references admittedly predominate here; for a wider survey, see for example the papers from the 1994 Oslo Symposium on Appraisal, in Janus, 1997.2.

(2) Special Branch police dossiers and census records would be two well known examples attracting conflicting community interests. The NSW State Records now seems to agree with the privacy and civil liberties argument (Accountabilities and Expectations, July 2000, p 4). When files containing sensitive personal information are destroyed, however, it is surely not because appraisal investigations have found them of no further use or importance, but to guarantee no future inappropriate use will occur.

(3) The practice of the National Archives of Singapore is noteworthy here: it preserves little more than 5-10% of its paper archives in the original (to meet intrinsic values or legal requirements); the remainder is filmed and destroyed. See Pitt Kuan Wah, ‘Preservation microfilming programme at the National Archives of Singapore’, ARCHIVUM vol XLIV (1999), pp 189-198.

(4) I am aware my use of the term is considerably broader than some writers, who essentially equate macro-appraisal with functional appraisal or functional analysis (e.g. Mark Greene and Todd Daniels-Howell in James O’Toole, ed., Records of American Business, SAA 1997, p 168). Interestingly, Brian Beaven calls the Minnesota Historical Society’s approach to the business sector ‘a macro-functional appraisal framework adapted to private institutional records collections’. Note 73, p 197.

(5) ‘The presumption is that if an organization’s functions do not contribute directly to an action which is considered significant enough to be documented for posterity (that means an activity which is one of government’s principle outputs to society) the records produced by it do not merit permanent retention.’ As for transactional records, they don’t ‘contribute to a reconstruction of the principal government activities in relation to the society within which it operates’ and thus can be destroyed. See Peter Horsman, ‘Appraisal on Wooden shoes. The Netherlands PIVOT project’, Janus, 1997.2, pp 35-41.

(6) ‘Beyond the Screen: The Records Continuum and Archival Cultural Heritage’, p 6. Available at http://www.archivists.org.au/whatsnew.html

(7) Mark Greene and Todd Daniels-Howell, ‘Documentation with an Attitude: A Pragmatist’s Guide to the Selection and Acquisition of Modern Business Records’, chapter 7 of James M. O’Toole, ed., The Records of American Business, SAA, 1997. Since presenting this paper, Bruce Smith has drawn my attention to Joanna King and Frank Rankin’s ‘From Minnesota to South Lanarkshire: Approaches to Business Archives in a Scottish Local Authority’, Business Archives, No 75 May 1998, pp 45-58. I hope by early 2002 to be able to report on an Australian adaptation of the Minnesota method by the University of Melbourne Archives, being undertaken with Associate Professor David Merrett of the Faculty of Economics and Commerce. On the ‘extended sector’ method, see H. Fode and J. Fink, ‘The Business Records of a Nation: The Case of Denmark’, American Archivist, Winter 1997, pp 72-87.

(8) Varsity Letters; Documenting Modern Colleges, SAA & Scarecrow Press, 1992.

(9) See Helen Samuels, ‘Drinking from the Fire Hose: Documenting Education at MIT’, Archives and Manuscripts, May 1997, pp 36-49.

(10) See her ‘Integrating Archival Programs into core business of the Independent School’, Archives and Manuscripts, May 1997, p 59.

(11) J. Haas, H. Samuels and B. Simmons, Appraising the Records of Modern Science and Technology: A Guide, MIT 1985.

(12) ‘Ambient Functions—Abandoned Children to Zoos’, Archivaria, No 40 Fall 1995, pp 21-39.

(13) Sue McKemmish, ‘Evidence of me…’, Archives and Manuscripts, May 1996, p 30.

(14) ‘Records Control and Disposal Using Functional Analysis’, Archives and Manuscripts, November 1997, pp 288-303.

(15) Adrian Cunningham was one of the earliest to stress this, back in 1994! For a later summary of his argument, see ‘From here to eternity: collecting archives and the need for a national documentation strategy’, LASIE, March 1998, pp 32 – 45, esp. ‘Strategies for Getting’, pp 35-37.

(16) See Deborah Woodyard, ‘Data Recovery and Providing Access to Digital manuscripts’, 2001 On Line On Disc paper: http://www.nla.gov.au/nla/staffpaper/2001/woodyard3.html.

(17) See her ‘Appraisal, Acquisition and Control of Personal Electronic Records: From Myth to Reality’, Archives and Manuscripts, November 2000, pp 71-91. On ministers, see Paul Dalgleish, ‘The appraisal of personal records of members of parliament in theory and practice’, Archives and Manuscripts, May 1996, pp 86-101.

(18) One exception is Helen Samuels; ‘In the end one has to ask: Is this an adequate record? Did we create and gather the RIGHT STUFF?’. See her ‘Drinking from the fire hose..’, p 44.

(19) E.g. how many decades (or centuries) would we give the records to ‘prove’ their worth; use can only test saved records, not—perhaps inappropriately—destroyed records; it is very difficult to isolate factors influencing use (e.g. finding aids and publicity; self-perpetuating use) separate from informational value assessed through appraisal; issue data cannot distinguish between requested records on the one hand and degrees of actual usefulness on the other, etc. See also Greene and Daniel-Howell, Records of American Business, p 169.

(20) To see ultimately what might be possible, we could look to Great Information Disasters edited by F.W. Horton and Denis Lewis (Aslib, 1991). The authors collected 12 case studies ‘of how information mismanagement led to human misery, political misfortune and business failure’. Few archives, collecting or government, seem to be as open in admitting errors as the US National Archives, however. See for example ‘Inquiry into the Disposal of Records of the Naval Research Laboratory Stored at the Washington National Records Center in Suitland, Maryland,’ NARA Bulletin 99-03 (Washington, D.C.: National Archives and Records Administration, 24 April 1998), available at http://www.nara.gov/records/nrlrpt.htm