Can big data solve the world’s problems, including corruption?

Click on image for a hi-res version.

Governments, academics and tech gurus over the last couple of years have been transfixed with the prospect that better and bigger data can help us solve the world’s more entrenched problems – including corruption and poverty.

The United Nations has also caught on to this trend and yesterday released a report, compiled by experts, which endorsed the position that increased, more accessible and better quality data will lead to better policy decisions and greater accountability.

The report is supposed to set the scene for how data will be used and integrated into new global sustainable development commitments, which will replace the Millennium Development Goals when they end in 2015. Despite its ambitions, the report still falls short of setting off a real revolution.

For this to happen, transparency, accountability and the participation of people must be at the centre.

Data, even lots of it, does not equal information. Putting numbers out there that no one can understand and use, let alone find, is not transparency. Just having a data trove does not offer any guarantees that people will be able to hold their leaders to account for their actions, or shape the policies affecting their lives. First it is important to understand what data people want and not just swamp them with information. To do this you need to consult with the people. This is something that the UN data report failed to do, even at the drafting stage.

The final report also falls short in outlining the steps needed to ensure that data collection is transparent, accountable and participatory, from the start.

Need for open data standards

Today when people cannot find the information to monitor budgets or inform policy decisions, it is often because there is no established data sharing framework – legal or voluntary – not because the data doesn’t exist.

More open data standards are needed, not just to make data available, but also comparable, accessible and understandable.

We are beginning to see what can be done when common data standards are used for development aid and public procurement. When there is a common standard, it makes it possible to move beyond simply answering the question of “how much money was spent” to understanding questions such as “how” and “why” it was spent. It can also help flag instances of abuse, misuse and corruption.

We also need to go beyond the UN report’s over emphasis on national statistical offices and official statistics. Citizen-produced data, which is often done in real time, can also give us important snap shots of what is happening on the ground. It can show whether policies and planned spending are working. Putting resources into standardising this information and using common frameworks is what is needed, not just more funding for government statistical offices.

Working with citizen-driven data is what Transparency International is trying to do across its movement of 100 chapters world-wide to better prevent, detect and address corruption. What Transparency International has learned from its work is that transparency is the means to an end, not the end point. This is how there will be a real revolution and that is where big data, properly managed, can really help.

 Carousel image: Flickr, Infocux Technologies