Giving people a voice: lessons from measuring corruption

As the world embarks on the ambitious global development agenda known as the Sustainable Development Goals (SDGs), it is important to have accurate measures of progress, or even lack of progress. Otherwise the goals themselves are at risk of becoming illegitimate. To track the goals, we need to have meaningful indicators that reflect what is actually happening to people on the ground.

Unfortunately, there is a significant risk that political arguments are getting in the way of deciding the best way to track progress. That’s why when the expert group tasked to choose the SDG indicators meets this week in Mexico, it must endorse and recognize the right kind of data. For us this means a significant commitment to including surveys about what people think and have experienced. What is the point of promising to promote the rule of law (target 16.3), if people still feel there is no justice in the country?

At Transparency International we have been measuring people’s perceptions of and experiences with corruption for more than a decade. What we have learned is that the best way to develop policies that are rooted in reality is to use good public survey data. The SDG indicators must reflect this.

Goal 16 on peace, justice and strong institutions, includes target 16.5 to reduce “substantially corruption and bribery”. The data to feed into the indicators for this goal, exist. We’ve been collecting them for more than a decade and refining the process, but some governments are reluctant to use data sources that they do not control. If the data is good, they should not be afraid to use it.

Take the case of data on bribery.  Getting good survey data outstrips all other data sources if you want to accurately understand and measure the phenomenon.

Even a cursory look at the data that police and other law enforcement agencies collect on reported bribery will immediately tell you that it is extremely patchy. It can’t – and will likely never – come close to the “true number” of bribery incidents. Due to the nature of the crime, there is severe under-reporting. It is much better to conduct a survey and measure people’s experiences with having to pay a bribe.

Many general social surveys – particularly in the field of democracy and governance (see here and here) – and Transparency International’s Global Corruption Barometer (here) rely on asking individuals questions about people’s experiences with corruption.  Survey researchers have learned over time from all this data collection what constitutes a good survey on bribery – and one that is likely to produce accurate results.

This is a list of do’s and don’ts for running population surveys on experiences with bribery.

• Do always first ascertain whether respondents actually had a risk of paying a bribe. Did s/he have contact with the respective service? People who do not have contact cannot be bribe-payers.
• Do make sure that the public service situation is relevant in the specific country context. If not, replace with an appropriate “functional equivalent”. For example, looking at people’s experience with bribery when accessing utilities is not relevant in a country where most people are off the electric grid.
• Do use professionally, properly trained interviewers, who will put the survey respondents mind at ease that all their answers will be kept private and confidential.
• Do make sure the survey is nationally representative and uses the best survey methodology for the country context. That means ensuring that the sample is robust and covers people from across the entire country including those who are harder to reach such as women, those living in rural areas and poor people.
• Don’t ask about bribery in the abstract. Always identify bribery in specific public service (e.g. in a public hospital, school or with the police).
• Don’t just use the word “bribe” (which is open to interpretations) but make sure to add relevant, country-specific descriptions of situations in which a member of the public interacts with a public officials who, for example, asks for an unwarranted favour (“vignettes”).

Results from those surveys that adhere to these standards have high levels of consistency and what researchers call external validity.

Transparency International is currently implementing a question module based on these standards in cooperation with a number of regional governance and corruption survey networks in 110 countries as part of our Global Corruption Barometer. The Africa edition of the Global Corruption Barometer is available here. Other regional editions are scheduled for release from May to September 2016.

For now the proposed SDG indicator for the corruption and bribery target is currently based on what people report. But the indicators are not final and governments still have the opportunity to object to using the indicator and data that exists, including those provided by third parties.

We strongly believe that good quality indicators and data will be key if the world truly wants to achieve not only Goal 16 but all the SDGs. That is why the conference in Mexico is so important and why opinion and experience based data, which is collected in a professional manner that adheres to clear standards, must be included in the process.