What is open data?

Open data is data that can be freely used, re-used and redistributed. Open data is subject only, at most, to the requirement to attribute and sharealike.

The full Open Definition gives precise details as to what this means:

• Availability and Access: the data must be available as a whole and at no more than a reasonable reproduction cost, preferably by downloading over the internet. The data must also be available in a convenient and modifiable form.
• Re-use and Redistribution: the data must be provided under terms that permit re-use and redistribution including the intermixing with other datasets.
• Universal Participation: everyone must be able to use, re-use and redistribute – there should be no discrimination against fields of endeavour or against persons or groups. For example, ‘non-commercial’ restrictions that would prevent ‘commercial’ use, or restrictions of use for certain purposes (e.g. only in education), are not allowed.

Further reading: What is Open Data? (opendatahandbook.org)

Why open data?

Open data, especially open government data, is a tremendous resource that is as yet largely untapped.

From the perspective of local authorities in Wales, we are very data rich but currently have very little accessible or machine readable data.

All government data is public data by law, and therefore could be made open and made available for others to use.

This untapped potential can be unleashed if we turn public government data into open data.

This will only happen, however, if it is really open, i.e. if there are no restrictions (legal, financial or technological) to its re-use by others.

Applying a license for open data

Applying an open license is the first step towards making your data available and, well, open.

To do this, you must first ensure that the data on your website is applicable for open data or if there is any data that has intellectual property rights.

You can then apply a suitable ‘open’ license that licenses all of these rights and supports the definition of openness discussed in the section above on ‘What Is Open Data?’.

To look at open licenses, view the Open Government Licence (nationalarchives.gov.uk).

A short 1-page instruction guide to applying an open data license can be found on the Open Data Commons site: Open Data Commons: legal tools

Current LAs with open license

I went to each of the local authority websites across Wales to see if I could find their open data and a link. to their open data license.

These are the LAs I easily found links and data:

The five stages of open data

Open data needs to be technically open as well as legally open. Specifically, the data needs to be available in bulk in a machine-readable format.

The five stages:

1. Publish and make data available on the web under an open licence.
2. Publish data in in the form of structured data (e.g. in a spreadsheet).
3. Published in a non-proprietary file format such as CSV.
4. Use URIs* as identifiers in the data. You can the use RDF as a model for encoding semantic relationships between items of data so that these relationships can be interpreted computationally.
5. Level 5 is to publish data as Linked Data which includes links to other data sources (see linked data).

*URI ‘URI’ stands for

• Uniform (there’s (partial) semantics and a standard generic syntax (specified by IETF RFC 3986))
• Identifier (URIs are identifiers, that is, names (not addresses)).
• Resource (What it is they identify — anything at all)

Publishing data as CSV

Formatting a spreadsheet

1. Create a new document in Excel.
2. Add a column heading for each piece of information you want to record (for example: age range, county, internet access, income bracket, occupation type), and then type the information in the appropriate columns.
3. When you’re done, your Excel file should look something like this.
   

Exporting as a .csv

1. Select File > Save As.
2. Use the drop-down box to select CSV (Comma delimited) (*.csv), give your file a name, and then select Save.

Making data accessible

If you have managed to publish your data in an open data format, it is then the final stage to make sure people can find it.

Some tips:

• Provide unique web pages for each piece of content.
• Avoid making people download content unnecessarily. If you require people to download the content and for it to open outside of a web browser, then fewer people will read it. Search engines are less likely to index the content. People are less likely to click to download.
• Provide narrative and context around the data. A few sentences about what the data is and what it may be used for will help people to find the data they are looking for.

Further reading: So I’ve Opened Up Some Data, Now What? (opendatahandbook.org)

Sharing local authority data with government

Central government, local authorities and public bodies can publish on data.gov.uk.

Data.gov.uk does not host datasets so, once you have published the dataset (or supporting document) on your local authority’s website, you can then add the dataset to data.gov.uk, which will display a description and a link to the dataset.

How to publish to data.gov.uk:

• Publish and manage data – Data.gov.uk
• Publishers – data.gov.uk

Four and five stars

Very few organisations make it past 3 stars – it is hard!

For stage 4, you will need to learn some coding, it can become very technical (take it from someone who has completed RDF and Turtle training and still struggles!).

If you are interested thought, here are some useful links:

• Turtle – Terse RDF Triple Language (w3.org)
• Understanding Linked Data Formats | by Angus Addlesee | Wallscope | Medium
• RDF Nuts & Bolts (cambridgesemantics.com)
• Exercise: try out RDF! (futurelearn.com)

Links to good data reports

• Manufacturing sector performance, UK – Office for National Statistics (ons.gov.uk)
• Total GERD by sector of Performance: Government – Office for National Statistics (ons.gov.uk)
• Ethnicity facts and figures
• Updating the GOV.UK Performance Dashboard
• GOV.UK Performance Dashboards
• Magna Charta repo
• Magna Charta demo
• UK Export Finance: annual report and accounts published

Further reading

• 5-star Open Data (5stardata.info)
• Linked Data – Design Issues (w3.org)
• Open Data Essentials |(worldbank.org)
• Open data – Ofcom
• Making analytical publications accessible