Blog

The term ‘Big Data’ refers to the troves of data that the world’s population creates thanks to its ever growing use of electronic devices such as computers and mobile phones. These devices record transactions, movements and communications, among many other forms of information. These may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions.

In particular, increased use of mobile devices in low to middle income countries (LMICs) has created unprecedented opportunities to delve into people’s needs and behaviours. For the development sector, this has created a wealth of new opportunities to change people’s lives for the better. This includes coordinating emergency responses to disasters and mapping migration patterns to estimate the spread of malaria.

Yet with new opportunities come new threats. First, there are issues of privacy and security. Second, unequal access to computer infrastructure, software and skills risk exacerbating existing socioeconomic inequalities and leaving vulnerable citizens even further behind. Policy makers need to ensure that an increasing digital divide does not outweigh the benefits of new technologies. NGOs and the private sector have important roles to play in facilitating this.

The data science revolution

It is now possible to access data from the personal electronic devices of billions of people. This has created data sets of sizes never previously seen in standard statistical analyses. These developments have encouraged commentators to predict the end of the scientific method; who needs to create and test theories and models when we can literally track and measure almost everything people do? Whether or not you agree with such sentiments, it is difficult to deny that Big data and analytics are changing the world as we know it. It is revolutionising the work of almost every sector, from banking and communications to healthcare and government.

New horizons, viewed from Silicon Valley

So what can Big Data bring to the field of international development? Examples of its uses include measuring rainfall in areas without ground-based sensors, using attenuation from radio signals caused by raindrops between cellular towers, as well as those mentioned earlier. Such information can be highly useful to farmers and water resource managers. Meanwhile, initiatives such as Global Viral analyse data gathered from the Internet (including social media) to identify the locations and causes of disease outbreaks. They claim to be able to successfully predict outbreaks weeks before bodies such as the World Health Organisation. This is because these organisations still rely on more traditional methods.

There are too many examples of innovative uses of Big Data to explore them all here. The UN Global Pulse suggests that on a more general level, it can be used to provide:

• Early warnings in times of disasters to enable faster emergency responses
• Real-time information to guide the design and targeting of programmes and policies
• Real-time monitoring of programmes and policies while they are still underway to allow changes

Big Data can also help achieve the Sustainable Development Goals. One example is studying spending patterns on mobile phone services to measure income and poverty levels. This is because mobile phone usage is so essential to many people’s day-to-day lives. Therefore changes in expenditure on mobile devices could reflect changes in income.

Privacy issues

Big Data is undoubtedly going to have positive effects on the development sector. Yet its use also presents numerous issues that should not be taken lightly. There are some privacy issues surrounding the use of large data sets. For example, people who use electronic devices create masses of data and may be unaware of their future uses.

Furthermore, recent instances of cybercrime highlight how easily online datasets can fall into the wrong hands. Indeed, unknown bodies stole data such as names, address, and incomes from almost every Bulgarian adult just a few weeks ago. The potential for such data to be used for crime such as fraud is very high. This will only increase as more and more data comes online.

An increasing digital divide?

Big Data use also has the potential to deepen existing inequalities by increasing the digital divide between countries. As Martin Hilbert explains in his paper, ‘Big Data for Development: A Review of Promises and Challenges’, there are three main areas which contribute to this divide.

The first is access to ICT infrastructure, as the majority of Big Data hardware capacity is located in high income countries. Similarly, while more people have smartphones than ever before, it is estimated that a third of the world’s population still lacks access. This brings up issues of data representativeness; do datasets only represent wealthier subsections of societies? Secondly, LMICs may lack the financial and human resources to use large datasets effectively. This is because much fewer people work in software and computer services. Finally, data specialists in lower income countries may not have the training and skills to effectively use Big Data. Hilbert suggests that the latter is one of the main reasons why Big Data applications have failed in development.

There is also the risk that when lower income countries do employ Big Data, existing elites may be more effective at utilising it. All in all, these factors suggest that these technologies could further reinforce existing socioeconomic structures, rather than challenge them.

The need for incentives and regulation

Governments and policy makers should design public policy and private strategies to maximise Big Data’s benefits while minimising threats posed. Hilbert suggests two possible approaches: incentives and regulation. The former includes financial incentives and subsidies that may encourage the adoption of Big Data technology. This is because they may make it cheaper for LMICs to build the required hardware and skills capacity. Furthermore, countries should foster open data initiatives. These advocate making data freely available to the public. This may help more people to use Big Data effectively.

Meanwhile, regulation can be utilised to control how companies and public bodies use and gather data. This could help address concerns around privacy and security. Governments can also enact transparency laws to encourage the provision of open government data. It is important to realise that these different options are not mutually exclusive; a certain combination will be needed to make sure Big Data technology truly realises its positive potential. Finding the optimum mix is one of the main challenges that still lies ahead.

Sebastian Wells is currently studying for an MSc in Violence, Conflict and Development at SOAS University of London. His main area of interest is the interaction between security, technology and development.