Can big data solve the world’s problems, including corruption?

Click on image for a hi-res version.

Governments, academics and tech gurus over the last couple of years have been transfixed with the prospect that better and bigger data can help us solve the world’s more entrenched problems – including corruption and poverty.

The United Nations has also caught on to this trend and yesterday released a report, compiled by experts, which endorsed the position that increased, more accessible and better quality data will lead to better policy decisions and greater accountability.

The report is supposed to set the scene for how data will be used and integrated into new global sustainable development commitments, which will replace the Millennium Development Goals when they end in 2015. Despite its ambitions, the report still falls short of setting off a real revolution.

For this to happen, transparency, accountability and the participation of people must be at the centre.

Data, even lots of it, does not equal information. Putting numbers out there that no one can understand and use, let alone find, is not transparency. Just having a data trove does not offer any guarantees that people will be able to hold their leaders to account for their actions, or shape the policies affecting their lives. First it is important to understand what data people want and not just swamp them with information. To do this you need to consult with the people. This is something that the UN data report failed to do, even at the drafting stage.

The final report also falls short in outlining the steps needed to ensure that data collection is transparent, accountable and participatory, from the start.

Need for open data standards

Today when people cannot find the information to monitor budgets or inform policy decisions, it is often because there is no established data sharing framework – legal or voluntary – not because the data doesn’t exist.

More open data standards are needed, not just to make data available, but also comparable, accessible and understandable.

We are beginning to see what can be done when common data standards are used for development aid and public procurement. When there is a common standard, it makes it possible to move beyond simply answering the question of “how much money was spent” to understanding questions such as “how” and “why” it was spent. It can also help flag instances of abuse, misuse and corruption.

We also need to go beyond the UN report’s over emphasis on national statistical offices and official statistics. Citizen-produced data, which is often done in real time, can also give us important snap shots of what is happening on the ground. It can show whether policies and planned spending are working. Putting resources into standardising this information and using common frameworks is what is needed, not just more funding for government statistical offices.

Working with citizen-driven data is what Transparency International is trying to do across its movement of 100 chapters world-wide to better prevent, detect and address corruption. What Transparency International has learned from its work is that transparency is the means to an end, not the end point. This is how there will be a real revolution and that is where big data, properly managed, can really help.

 Carousel image: Flickr, Infocux Technologies

Share and enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Twitter
  • Facebook
  • Digg
  • StumbleUpon
  • Reddit
  • MisterWong
  • Google Bookmarks
  • Technorati
  • LinkedIn
  • NewsVine
  • YahooBuzz
  • Print
  • email
Craig Fagan

About Craig Fagan

Craig Fagan is Senior Policy Coordinator at Transparency International.

, , , ,

3 Responses to Can big data solve the world’s problems, including corruption?

  1. Bill Anderson 10 November 2014 at 9:46 am #

    Craig i think we need to be careful about throwing the baby out with the bathwater. I agree that national budget data is already available and needs opening up in many countries. I agree that the consultation process for the new UN report was, too put it politely, ill conceived.

    But there is a lot of data about the governance and impact of development that doesn’t currently exist in a sustainable format. There is an urgent need for countries to develop sustainable and inclusive national statistical systems: where government, civil society, the private sector and academia work together to collect and share data and provide usable information to both decision makers and those who monitor them.

    Pitting civil society against government and citizen data against statistics is not going to help anybody.

  2. Bill Anderson 12 November 2014 at 4:25 pm #

    I submitted a comment 48 hours ago.



  1. 大きなデータの破損など、世界の問題を解決できますか?|Can big data solve the world’s problems, including corruption? | Open Data and Programming - 8 November 2014

    […] 大きなデータの破損など、世界の問題を解決できますか?|Can big data solve th… […]