Measuring What Matters

Measure what you treasure…or you might just treasure what you measure.

Imagine for a moment that you are the chief statistician of your country. You regularly generate data on everything your nation asks for, along with the statistical analyses that make the data useful. Together with the chief statisticians of other countries, you make up the UN Statistical Commission (UNSC).

Suddenly people not sophisticated in statistics come along with non-traditional ways of thinking. They want measurement of, say, the nexus of transport systems, agriculture, and water use. Also, the data collected are to be disaggregated rather than clumped in traditional ways. Gender, age, socio-economic class, ethnicity, location (that is, urban or rural), disability status, and other categories are now perceived as meaningful distinctions, and your government wants to compare the different groups. There is demand, too, for data about processes, not just outcomes. A new understanding of complexity underlies this need for non-traditional data, and you may well have to devise new measuring tools.

GOALS, TARGETS, AND INDICATORS

Last year the UN General Assembly (GA) agreed on a list of 17 Sustainable Development Goals (SDGs). These goals will apply to all nations, will be in effect from 2015 to 2030, and are intended to provide transformational guidelines for both the elimination of human poverty and protection of the planetary biosphere.

Within each of the 17 SDGs, multiple targets have been identified. Measured changes in these targets will indicate progress toward the goals, or the lack of same. The GA has agreed on 169 targets, many of which will require new statistical procedures, called “indicators.” The UNSC is charged with providing these indicators. It’s a whopper of a challenge that the UNSC doesn’t welcome.

DISGRUNTLED STATISTICIANS AND CONFLICTING NATIONS

The UNSC will meet for the first time in May, but back in January its members received the list of 169 targets. One observer remarked, “it was very clear that this was new to them.” In responding, the UNSC asserted that “just 29% (of the targets) are well defined and based on the latest scientific evidence, while 54% need more work and 17% are weak or non-essential.” The UNSC would like their task changed to something more manageable.

What will the systemic transformation promised by the SDGs entail? Targets and meaningful indicators are very important. Powerful nations like the United States and Canada are pushing to drop some targets and combine others. They say further that such early “improvements” will create a precedent for making further changes as time goes on. This is a threat to weaker countries which hold that loss of crucial information from targets would make it easy to ignore difficult systemic problems and to continue “business as usual.” They insist the decision reached last year was a hard-won political compromise that must be respected, and that statisticians are not the right body to shape international policy …even if they are being asked to stretch their competence to include the complexity of current understandings.

CIVIL SOCIETY TO THE RESCUE?

I attended a March meeting of the UN General Assembly (GA) on targets and indicators for the SDGs. Every morning we in civil society had a meeting of our own. This was the best hour of the day. We brought each other up to date on what had happened the day before and looked ahead at the day to come. Exchanges were frank and open, and people got to know each other.

Although the UNSC had previously stated that they have no interest in collaborating with civil society organizations (CSOs), there was great energy at these meetings to get CSOs involved in the nuts and bolts of working on indicators. There are statistical models being used by civil society that the UNSC may not know about. CSOs could offer examples, even put together a coordinated “best practices” demonstration of what CSOs are doing. A platform where these demonstrations are posted could be useful, but interaction at the beginning of the UNSC process would be best.

Ambassador Kamau of Kenya, a co-chair of the meeting, has been a champion of increasing civil society participation for the last two years. Before the end of the week, Kamau said he would ask the UNSC to consider “space for civil society input.” Our fingers are crossed.

In May, the whole thing will be opened up again, this time with the statisticians in attendance. I plan to be there. If the world is going to understand the consequences of policy actions and make better decisions, we need good data. I want to know whether the UN will be able to measure what matters.

RESOURCES

• The UN Statistical Commission is comprised of the chief statisticians of all UN member states. It is not itself a branch of the UN, but it oversees the UN Statistical Department, which is a UN branch.

• For an awesome example of disaggregated data, go to https://www.worldwewant2015.org/. To learn more about the criteria for the Sustainable Development Goals (SDGs), go to https://sustainabledevelopment.un.org/topics/sustainabledevelopmentgoals.