Measuring the digital impact of Australian governments.
BLOG: How Australian governments can lead the way with impact measurement.
Do governments have the answer to measuring digital impact?
We believe that the effective use of digital experiences can increase impact in meaningful and measurable ways. By focusing on impact, our work in the digital space is centred on helping organisations and their staff use digital assets to make a real difference.
Fundamentally, ‘digital’ should be about making people’s lives easier. That’s why any digital tool or service should put people first – whether they’re clients, customers, staff or stakeholders. Most organisations struggle to articulate and measure their digital impact and government, for the most part, is no different.
A review of the state Budget Papers (2016-2017, 2017-2018) to identify what government is measuring around digital engagement shows that the skew of measurement relates to website visitation. Organisations like Transport Safety and Taxi and Hire Vehicle Services measure customer satisfaction while Drug Services and Consumer Affairs monitor enquiries and information provision (although the latter does not make distinction between how information is provided) but a majority of the focus is on volume.
A budget report indicates that they are discontinuing an existing metric of “links from Tourism Victoria consumer sites”, because “The 2016 -17 performance measure ‘Visitors to Visit Victoria consumer websites’ is a more accurate indicator of the effectiveness of government expenditure”. Anyone who has worked in marketing or advertising will tell you that reach is not always a good indicator of spend nor effectiveness.
This is similarly reflected in the GovCMS approach of pricing per page views. Understandably, that relates to server costs, but it is a very one-dimensional view of what a digital asset should be.
As shown with the recent myhealthrecord debacle and the previous online census mess, Australian citizens have a dim (at best) view of the ability of government to correctly use and deploy digital tools.
For this perception to change, how government approaches digital impact needs to change. Visitation is not important. What is important is the ability of citizens to understand and engage with digital assets. Governments can leverage newer technologies like chat bots, translation services and where appropriate, augmented and virtual reality in its quest to deliver timely, relevant and actionable information and services to citizens, no matter their life stage, circumstance or literacy levels.
But these choices need to be made intelligently. Just because you can, doesn’t mean you should. There have never been so many options available to increase a digital footprint, however too much of a good thing can be a terrible thing. It’s better to be in the right place rather than in every place to ensure your digital presence is impactful.
How we compare globally
By comparison, the UK measures cost per transaction, user satisfaction, digital take up and successful completion rates. These are mandatory KPIs. They even host a resource dedicated to best practice: “How to set performance metrics for your service” which shows an improved approach to digital impact:
- Your plan to lower cost per transaction or equivalent, for non-transactional user journeys
- Your plan to improve user satisfaction
- Your plan to increase completion rate or equivalent, for non-transactional user journeys
- Your plan to increase digital take-up and reduce reliance on assisted digital
In the manner Ellis Jones thinks of impact, what good is visitation if people are not provided information they need? What good is providing information if it isn’t actionable? How do you know something is actionable if it isn’t measurable? Accessibility (collectively, how someone can access, understand and action information) is critical for the work government does, particularly related to lower socio-economic populations and those from Aboriginal and Torres Straight Island and English as a Second Language communities.
Take an even more progressive government, like Estonia. Their measurements include the percentage of the population who uses the internet regularly, the number of e-services for citizens, the time saved using e-gov services, the amount saved through digital signatures and a plethora of other metrics that truly define digital impact.
What Estonia does successfully is to be much more specific in defining metrics: the difference between “amount saved through digital” (broad) and “amount saved through digital signatures”.
The detail becomes important because multiple metrics of impact are important to properly and eventually make more informed improvements. This is what Australia should strive for.
Our Australian Digital Transformation Agency is starting to move into the space, defining (at minimum) four KPIs:
- User satisfaction – to help continually improve the user experience of your service
- Digital take-up – to show how many people are using the service and to help encourage users to choose the digital service
- Completion rate – to show which parts of the service you need to fix
- Cost per transaction – to make your service more cost efficient
The language feels very familiar.
The Victorian government echoes this, however the only standard that must be met is the inclusion of Google analytics tracking codes. That is a disturbing gap between intention and execution. It’s an unfortunate reality of trying to drag a legacy-laden environment kicking and screaming into the last decade (heaven forbid we aim for the future).
What needs to change?
A smart way to make these changes is to begin with impact measurements. To define what each digital asset should be achieving and how it will be measured. From there, to design and implement solutions that are simple, elegant and most importantly can be deployed quickly and improved over time, based on real world user feedback and usage.
Specificity will unlock government innovation and move Australia from playing catch up to leading the way. There should be multiple metrics, assessed concurrently, and appropriate to the audience, task and outcome. The opportunity exists to own our unique challenges and infrastructure and push the thinking of how to address these. We have enough talent and desire to overcome previous mistakes. But it requires dedication and boldness to think beyond playing tag along. And to achieve outcomes a million visits could only dream of.