Here I repost my article published in the Business Day:
How does South Africa’s data ecosystem stack up internationally?
Data matter. Reliable and timely data are crucial for decision making. Data drives innovation and is an important source of value. In the public sector, data helps drive better development outcomes through evidence-based policies, and improved transparency and accountability. Estimates from the Organization for Economic Cooperation and Development (OECD) suggest that the social and economic benefits produced by data are material, ranging between 1%-2.5% of GDP.
How does South Africa compare in terms of the volume of data available? We scraped the World Bank data portal to see how many economic, financial, and socio-economic series are available for South Africa compared to other countries. We found that South Africa compares favourably to wealthy countries and other emerging markets in terms of the volume of indicators available, although our data are available on a less timely basis than for many major economies. So, while we do not lag on data volumes, policymakers and firms in South Africa are left to make decisions with less up-to-date information than in peer economies.
To assess how our data ecosystem stacks up internationally, it is worth summarising a useful benchmark for assessing data sharing and governance – called the ‘FAIR principles’. The ‘F’ stands for ‘findability’ – can data be discovered and read by humans and machines? The ‘A’ stands for ‘accessibility’ – are there open protocols for retrieving data and descriptions of the data? The ‘I’ stands for ‘interoperability’ – can data be integrated with other data and used by different applications for storage, processing, and analysis? And lastly, the ‘R’ stands for ‘reusability’ – can data be used in different settings, and do they meet required data standards? As I will argue, a lot of South African data do not meet these requirements.
But let us start with the areas where South Africa’s data ecosystem compares favourably with international best practice. The Global Data Barometer is a benchmarking study of over a hundred countries’ capabilities in terms of data infrastructure, data quality and the regulations that bear on data. The study shows that South Africa performs well with respect to the availability of vital statistics, particularly the availability of budget and public finance data. South Africa also has data protection governance frameworks in place, such as the Protection of Personal Information Act (POPIA) that protects the rights of data citizens. The high ranking for South Africa’s budget transparency is backed up by the Open Budget Survey, which finds that our National Treasury provides comprehensive public finance information to the public that allows budgets and government performance to be assessed in fine detail at different levels of government.
However, South Africa is falling behind in the implementation of data frameworks for use and sharing. The Barometer shows South Africa is far behind with respect to open government data rules and initiatives, frameworks that support data sharing or the availability of procurement data. South Africa lags countries like Rwanda, Uganda and Nigeria in many areas related to the openness of public data and the existence of frameworks for promoting data use. International best practice is ‘open by default’ rules in public institutions to make data available unless not doing so can be justified on privacy or national security grounds. Most South African public institutions have poor data management so that there are no comprehensive data catalogues that summarise what data are available. Classifications of the sensitivity of data are usually not readily available. Low data management maturity leads to a reluctance to share data because most data are treated as sensitive.
Why are we falling behind? One possibility might be that South Africa is not investing enough in data. If we compare national spending on official statistics in several countries around the world, South Africa spends a similar proportion of GDP on official statistics as major emerging market economies, such as Mexico or Chile. So, it is not clear that we are not prioritising official statistics enough. In fact, there has been an increase in the number of statistical releases by Statistics South Africa over the last two decades and South Africa has met 59 out of 63 of the G20 data gaps initiative targets.
Instead, South Africa’s poor relative performance on data availability reflects lack of progress on making published data easy to use and share. For example, World Bank estimates show that South Africa ranks poorly on machine-readability of data. This means that it is very difficult to access time series of data or automate processes that use data.
South Africa also stands out for limited progress in developing standard-setting institutions that create an enabling environment for responsible data use and sharing. Best practice is for information regulators to provide things like data sharing agreement templates, promote common technical standards, and for government institutions to enforce explicit open data rules. If you go to our Information Regulator’s website, the only templates available are for compliance with data protection regulations. Standardised protocols for the various legal, information technology or governance aspects of data use are comparatively rare in South Africa. While we have frameworks for safeguarding privacy in place, these must balance protection of data citizens and the rights of intellectual property holders against ensuring an enabling environment for data use and sharing for public policy and commercial purposes.
The good news is that advances in technology have made it easier and cheaper to implement best practice approaches to responsible data governance and sharing. The COVID-19 pandemic also encouraged many countries’ governments to prioritise modernising data collection, experiment with using ‘alternative’ data sources to understand high frequency developments in different regions and apply big data approaches for evidence-based policymaking.
Yet South Africa’s progress in this respect has been slow compared to countries with fewer resources. What do we need to do to catch up? The Global Data Barometer suggests that an enabling regulatory environment that promotes the use and reuse of public data is a key stumbling block to be removed. An independent stock-take of the extent of our data access challenges would help identify their scale and what needs to be prioritised. A peer review through the OECD Council on Good Statistical Practice would be a good start.
The reason less developed African countries lead South Africa in several aspects of data maturity is that South African government institutions tend to focus on institution-wide, long-term projects instead of empowering small teams to work on well-defined problems. Large committees struggle to solve complex problems. Its much better to have small, high capability teams work on achieving ‘quick wins’ to demonstrate the value that can be realised from data projects. How do you eat an elephant? Piece by piece.
Government and state-owned enterprises must also begin releasing datasets in machine-readable format. If South Africa is to benefit from the promise of the fourth industrial revolution, government and businesses need to understand what is happening in the economy on a day-to-day basis and be able to identify challenges that need to be dealt with. These are also preconditions for South Africa’s fintech industry to mature and ultimately for South Africa’s companies to remain competitive internationally. Universities could also act as ‘honest data brokers’ to ensure responsible data sharing between researchers, and the public and private sectors.
It is time for government to unleash the potential of the rich data available in South Africa. We have all the resources we need and a private sector that is eager to help. It is time for government to let us.
Dr Daan Steenkamp
Daan is CEO of Codera Analytics and a research fellow with the Economics Department at Stellenbosch University.
