FROM VOLUMES TO VALUES – WHY WE NEED A NEW SYSTEM FOR MEASURING DIGITAL INCLUSION

July 15, 2015

Share this story


Written by Douglas White, Head of Advocacy, Carnegie UK Trust

Over the past 8 months the Carnegie UK Trust has been working with the Digital Inclusion Research Working Group, established by Government Digital Service, to find a new, more effective way of tracking and evaluating digital inclusion across the UK. This week we’ve launched the first products from this major piece work. Douglas White, our Head of Advocacy, explains all:

 Why data matters

“In God we trust, all others must bring data”

W. Edwards Deming

The question of how best to measure social progress is not a new one for public policy.  There has long been a desire to find the ‘right’ data to allow us to assess the effectiveness of interventions and deploy resources accordingly. The process of identifying useful, relevant data is not a static one. Rarely, if ever, is an optimum state of measurement reached. It is a continually evolving process, as policy makers respond to new theories and evidence about the best approach and adapt their priorities accordingly as the purpose of any given activity changes over time.

The challenge

In the field of Digital Inclusion we are in still in the relatively early stages of the debate about what we should measure. This is both exciting and challenging. The explosion of digital technology over the past 20 years has produced fantastic benefits for those able to access and use the new technology – in education, employment, health, entertainment, knowledge, communication, finance, retail and participation.

But for those who remain outside of the digital world – for reasons of cost, motivation, skills and confidence – these benefits are elusive. This matters – because data shows that those demographic groups who are most likely to be digitally excluded are also those most likely to be disadvantaged according to many other social and economic measures.

Excellent, inspiring work is being undertaken by many organisations across the UK to tackle this problem. This includes expert charities such Citizens Online, Tinder Foundation and Go On UK. Those working with certain groups who are much more likely to be excluded, such as older people or social housing tenants. Public service providers, including local authorities, health services and libraries. Local voluntary organisations and community groups. And private companies including BT, Lloyds and Barclays.

Meanwhile, across the UK a wealth of high quality research on digital inclusion is taking place, providing vital insights into this critical issue. Regular UK-level surveys are carried out by the Office for National Statistics, the Oxford Internet Institute and Ofcom; while in depth studies on specific aspects of digital inclusion are conducted by universities, charities, businesses and public bodies.

However, despite the fantastic on-the-ground activity and robust research base, there has never previously been a single, agreed set of measures setting out what, as a society, we hope to achieve as a result of all of our work to help people get digitally included.

There has also never previously been a common evaluation framework available for organisations tackling digital exclusion, which has led to many different approaches being developed, making consistent assessment difficult.

All of this presents significant challenges when attempting to assess our progress in tackling digital exclusion, identify ‘what works’ and generate further investment to help many more people become digitally included in the future.

The solution – a Digital Inclusion Outcomes Framework

As part of the new Digital Inclusion Research Working Group – established last year by the Government Digital Service – the Carnegie UK Trust has been working with colleagues from academia, government, the private sector and charities to come up with a solution to these challenges.

The result of our work is a new Digital Inclusion Outcomes Framework for the UK – and accompanying evaluation toolkit – which has this week been published on the Go On UK Digital Skills platform.

The Framework is designed to address the existing limitations in the UK’s digital inclusion measurement by providing a single, joined up model that supports both ongoing tracking of progress at a national leveland more consistent evaluation at a local level.

The Framework’s starting point is that getting people online and engaged in digital technology can help to improve people’s quality of life. To that end, it’s comprised of a set of Outcomes which group into three categories:

Digital outcomes, which are ‘stepping stones’ required to achieve wider benefits. These measure people’s access to digital technology, their use of it and their motivation and confidence to do so.

Economic outcomes, which look at how being online can support employment, education, managing money, and creativity or entrepreneurialism.

Health and social outcomes, which look at how being online can support improved health, communicating and connecting, access to leisure and entertainment, use of public services, and democratic and civic participation.

A final, cross-cutting category of ‘inequality outcomes’ is designed to ensure that the model focuses properly on reaching those demographic groups who are most in need.

Progress towards each of the outcomes in the Framework is tracked via a ‘basket’ of multiple, relevantIndicators, which are specific and measurable. Indicators are essentially ‘proxies’ for the outcomes. No one indicator is itself likely to determine whether an outcome is being achieved, but tracking changes across each of them gives a good assessment of whether progress is being made against the outcomes.

To allow ongoing assessment of progress at UK level, it is essential that indicators can be tracked on a regular basis. Indicators for the Framework were therefore selected on the basis that they correlate with reliable Data that is collected regularly for the whole population.

What are the benefits – why use the Framework? 

The Framework offers a number of potential benefits for those who use it:

1. Nationally, the Framework enables ongoing tracking of a broad, coherent set of digital inclusion outcomes at a UK level, with updates provided to the Framework data each time one of the key national surveys is carried out. This means that for the first time, we’ll be to track comprehensively the progress we are making as a society in achieving digital inclusion.

2. The Framework sets out the varied, wider benefits of digital inclusion. This means at a local level it can be used as a menu, with projects and organisations choosing from it selectively from according to their particular priorities. As such the Framework is applicable across diverse initiatives, settings, target group needs, and funder requirements, while at the same time improving comparability and consistency across projects.

3. This improved comparability will give us better evidence about what works. Having better evidence makes it easier to share best practice and design activities accordingly – improving our use of resources and increasing the chances of success.

4. Testing out new approaches and being able to assess them consistently really matters, as we are now tackling the hard to reach final 20% of the population who remain offline or lacking basic digital skills – and we are still working out the best ways to support this group.

5. If we can robustly evidence our success we are more likely to attract further resources and investment.

Working and measuring together, we can really improve our understanding of digital inclusion across the UK and ultimately, help more people to get online.

Click here to read more about the Carnegie UK Trust’s work on digital participation.