Interpreting annual report totals

Created: 10 June 2016

I’ve extracted access examination statistics from NAA Annual Reports. The question now is how do these stats compare to the data I’ve harvested about closed files from RecordSearch.

In theory, the ‘Withheld from public access’ number in the annual reports should be greater than or equal to the number of records in my database with access decisions made within the financial year. There may be fewer records in my database because (hopefully) some files have been opened since the original decision was made, but there shouldn’t be more.

This is complicated by the fact that it’s not clear how the totals in the Annual Reports were arrived at. The 2011-2012 Annual Report notes that:

For 2011–12 all data was sourced from the reference and access module of RecordSearch. In previous years a combination of different electronic databases was used to collate the statistics.

Let’s have a look…

Financial year Annual Report ClosedAccess
1996-1997 165 273
1997-1998 103 209
1998-1999 227 302
1999-2000 172 101
2000-2001 159 105
2001-2002 123 114
2002-2003 739 616
2003-2004 345 215
2004-2005 171 112
2005-2006 245 204
2006-2007 299 125
2007-2008 448 290
2008-2009 267 109
2009-2010 250 134
2010-2011 289 381
2011-2012 285 1233
2012-2013 239 384
2013-2014 659 1742
2014-2015 1,026 1572

Ok, so there’s odd things going on at either end of this time period – the number of files ‘withheld from public access’ according to the annual reports is less than the number of closed files (on 1 January 2016) that have an access decision within the financial year. I suspect that there’s two separate causes for this discrepancy.

If you look at the access examination statistics for the years 1996-7 to 2003-4 you’ll notice they had an extra category called either ‘items containing no open period material’ or ‘other (eg closed period)’. If you look at my harvested data you’ll see that it includes ‘Closed Period’ files for these years. So it seems reasonable to add the ‘withheld from public access’ figures to the ‘other’ category. If we do that we see:

Yikes! The early years are now totally out of kilter with everything else. This seems to indicate that the ‘other’ category doesn’t just include closed period files.

At the other end of the period, I suspect that the big differences between what’s reported and what I’ve harvested are due to the non-reporting of records in the ‘withheld pending advice’ category. If I exclude these from my data we see:

With the exception of 2013-2014, this seems to produce a reasonable-looking comparison. I think this is good evidence that the statistics in annual reports don’t include records that are ‘withheld pending advice’. Is this reasonable? What does this mean for the transparency of the access examination process? I think I need to explore this further elsewhere.

What was going on in 2013-2014? If you look at the other reasons records were closed in that period you’ll see there were lots of ‘Parliament Class A’ records (1034 of them!). Are these included in the annual report stats? I think some more poking around in 2002-2003 and 2013-2014 is required.

What have I learnt? – that there’s no straightforward way of reading the annual report statistics.

Related projects