A flood of data to create a data-literate citizenry

In David Cameron’s first podcast as British Prime Minister he outlined plans to make Government more transparent and allow people to hold ministers and public services to account.

One of the central themes of the podcast was that his government would be one that “gives power away to people instead it taking it from them.” He explained how a big part of this was providing the public with more information about government and especially how and where it spends its money (1:50 – 2:16):

It’s your money, your government, you should know what’s going on.

So we’re going to rip off that cloak of secrecy and extend transparency as far and as wide as possible. By bringing information out into the open, you’ll be able to hold government and public services to account. You’ll be able to see how your taxes are being spent. Judge standards in your local schools and hospitals. Find out just how effective the police are at fighting crime in your community.

Public Spending information

Cameron went on to explain how transparency could help “re-build trust in our politics”, through making politicians more accountable for their spending decisions. Analogous to this he previewed the release of details of “public spending over the past 12 months, information about hospital infections, and some of the salaries of senior officials in government.”

He warned, however, that the information would not be perfect, not always in the most convenient format or free from mistakes. Nevertheless, his view was:

I don’t want to hang around making sure everything is perfect – I want to get on with it, to make a start on this transparency revolution that we’re planning.

In time, I want our government to be one of the most open and transparent in the world.

Following on from the podcast, the Prime Minister sent a stark letter to all British government departments. It began with a stated commitment to hold public bodies to account, and ensure value for money in public spending:

Greater transparency across Government is at the heart of our shared commitment to enable the public to hold politicians and public bodies to account; to reduce the deficit and deliver better value for money in public spending; and to realise significant economic benefits by enabling businesses and non-profit organisations to build innovative applications and websites using public data.

The Government must set new standards for transparency, and our Coalition Programme for Government sets out a number of specific commitments. The Government’s initial transparency commitments are set out below, alongside deadlines for publication. Limited exemptions on national security and personal privacy grounds will be permitted.

The commitments include:

• Historic Combined Online Information System (COINS) spending data to be published online in June 2010. – (Released on 4th June)
• All new central government ICT contracts to be published online from July 2010.
• All new central government lender documents for contracts over £10,000 to be published on a single website from September 2010, with this information to be made available to the public free of charge.
• New items of central government spending over £25,000 to be published online from November 2010.
• All new central government contracts to be published in full from January 2011.
• Full information on all DFID international development projects over £500 to be published online from January 2011, including financial information and project documentation.

Along with this he announced how this spending transparency would relate to local government:

• New items of local government spending over £500 to be published on a council-by-council basis from January 2011 – (however, not forced by law).
• New local government contracts and tender documents for expenditure over £500 to be published in full from January 2011.

The priority attached to this was highlighted with the request to all departments to take “immediate action to meet this deadline for data transparency”:

Given the importance of this agenda, the Deputy Prime Minister and I would be grateful if departments would take immediate action to meet this timetable for data transparency, and to ensure that any data published is made available in an open format so that it can be re-used by third parties. From July 2010, government departments and agencies should ensure that any information published includes the underlying data in an open standardised format.

Of course, the release of the datasets specified in the Coalition Programme is just the beginning of the transparency process. In advance of introducing any necessary legislation to effect our Right to Data proposals, public requests to departments for the release of government datasets should be handled in line with the principles underpinning those proposals: a presumption in favour of transparency, with all published data licensed for free reuse.

Some journalists have likened this spirit of online transparency and the “Right to Data proposals” for government-held datasets, as akin to the US Public Online Information Act (POIA) where “public means online” is set as an operating principle for government.

Lifting the Government Spending “Cloak of secrecy”

Late last week, the government enacted Cameron’s pledge to release COINS spending data, and published the entire contents of the Treasury spending database. This documented where public money comes from, what it is spent on for every financial year from 2005/06 to 2009/10.

It’s a complicated set of data which the government has admitted needs ‘some degree of technical competence’ to make use of. In this vein they’ve asked the Open Knowledge Foundation to help make it ‘more accessible,’ and have also promised ‘more accessible formats’ and user-friendly subsets by August.

The datasets can be downloaded from data.gov.uk, or analysed through the Guardian’s data-explorer.

Making sense of the data

Charles Arthur, the Guardian’s technology editor explained how we now need people to make sense of this data and to explore its trends and intricacies:

Now what is needed is people who can make it make sense for the rest of us; we have the transparency but need lenses to bring out the detail.

It’s crucial for citizens to find ways to examine and interpret the data; otherwise it may as well be  – as David Cameron says – “locked away in a vault marked sort of private for the eyes of ministers and officials only”.

We need data.gov.uk/dataset/coins to pass the ‘Mumsy’ test, so citizens can do their own investigations on subjects such as general government spending on consultants, or specified towards a particular agency.

Open data activist David Eaves sums up the challenge going forward:

We need a data-literate citizenry, not just a small elite of hackers and policy wonks. And the best way to cultivate that broad-based literacy is not to release in small or measured quantities, but to flood us with data. To provide thousands of niches that will interest people in learning, playing and working with open data. But more than this we also need to think about cultivating communities where citizens can exchange ideas as well as involve educators to help provide support and increase people’s ability to move up the learning curve.

His call for a data-literate citizenry is one of the reasons we need a massive release of open data, and also an encouragement and incentives for coders to share ideas and skills on how to use and engage with government data (my emphasis):

It is worth remembering: We didn’t build libraries for an already literate citizenry. We built libraries to help citizens become literate. Today we build open data portals not because we have a data or public policy literate citizenry, we build them so that citizens may become literate in data, visualization, coding and public policy.

[…] But smart governments should not only rely on small groups of developers to make use of open data. Forward-looking governments – those that want an engaged citizenry, a 21st-century workforce and a creative, knowledge-based economy in their jurisdiction – will reach out to universities, colleges and schools and encourage them to get their students using, visualizing, writing about and generally engaging with open data. Not only to help others understand its significance, but to foster a sense of empowerment and sense of opportunity among a generation that could create the public policy hacks that will save lives, make public resources more efficient and effective and make communities more livable and fun.

When we think of libraries, we often just think of a building with books.  But 19th century mattered not only because they had books, but because they offered literacy programs, books clubs, and other resources to help citizens become literate and thus, more engaged and productive. Open data catalogs need to learn the same lesson. While they won’t require the same centralized and costly approach as the 19th century, governments that help foster communities around open data, that encourage their school system to use it as a basis for teaching, and then support their citizens\’ efforts to write and suggest their own public policy ideas will, I suspect, benefit from happier and more engaged citizens, along with better services and stronger economies.

The release of such large amounts of government data represents the beginning of the journey, not the end. The presumption of openness in relation to spending data, represents a sea change in the government’s relationship with the public, and how it wants to structure the debate on government expenditure.

These are small steps, but they’ll only make a big difference if the government utilities these resources as a catalyst towards ensuring citizens can become literate in data, visualization and coding. Ensuring the data is released in open standardised formats (as COINs data has been), allows newspapers and other organisations to create user friendly interfaces to interrogate the data, and will allow for the creation of new apps (check Alpine Interactive’s great visualisation app, and Dharmafly’s Gov expenditure app) and a more data-literate citizenry to emerge.

David Cameron noted how “People will be the masters. Politicians the servants. And that’s the way it should be”. I for one agree, and the release of such data enforces this mantra.

For more check:

• Guardian live-blog on the release of COINs data
• Guardian COINS data-explorer
• Charles Arthur on making sense of COINs data
• COINS data release: The 10 things we found out
• Tech Weekly Podcast with comment from Tom Watson MP on the release of COINs data (25:29 – 34:34)
  
• Francis Maude – Minister for the Cabinet Office and Paymaster General – on Opening up government to scrutiny
• COINs analysis appplication (rosslynanalytics)
• COINs on data.gov.uk
• Programme for Goverment (Transparency)