Why Saving Time on Intranets Remains a Bad Metric
Fourteen years ago, I wrote a piece titled “25 reasons why saving time on your intranet is a bad metric.” Over the years I’ve written thousands of articles (and several books), but this blog post remains a favorite for more than a few practitioners.
The argument goes like this: we conducted usability testing, and it takes a user 2.5 minutes on average to find the leave policy. We made improvements to the navigation and structure of the intranet, and re-testing, it now only takes 30 seconds. That’s a 2-minute saving, multiplied by the number of users, and the number of times they’re expected to do it in a year, and the result is 150,000 minutes saved!
At the time, a leading usability expert took things a step further, not only claiming that firms with average intranet usability could save $2.4 million per year by making improvements to UX, but also that "the world economy would save $311,294,070,513 per year." There are still many organizations that focus primarily — or exclusively — on using intranet time savings to build business cases or demonstrate benefits.
I had a big problem with these approaches, and outlined 25 reasons why they were fatally flawed. Over a decade later, I’m returning to the fray.
Real vs. ‘Fake’ Time Savings
Let me start by acknowledging that in the age of sudden labor shortages, productivity benefits are incredibly important for organizations. If time can be saved in a key step when manufacturing a car, then more cars can roll off the production line. These types of simplifications and automations can deliver real, and often substantial, benefits.
Similar benefits can be delivered in digital ways. In 2020, Step Two’s Intranet & Digital Workplace Awards gave a gold trophy to SP Group (Singapore’s power company) for automating frontline processes using the Power Platform in Microsoft 365.
SP Group conducted ‘value stream mapping’ to identify tangible time savings that would allow the field workforce to be more productive:
In comparison to the real benefits SP Group identified, ‘fake’ benefits focus just on the time saved looking for things on the intranet ie usability benefits in the narrower sense.
It’s these less tangible benefits that can get us all into trouble, and are what I want to call out. Let’s revisit the issues I outlined 14 years ago, to see if they’re still relevant.
Related Article: Intranet Platforms 2022: The State of the Industry
The Big Problems With ‘Time Saved’ Intranet Metrics
These remain my key concerns about measuring time saved:
1. We’re not measuring end-to-end task completion. The 'time saved' metric typically focuses solely on the time needed to find information on the intranet, not the task or activity itself.
2. Can we realize the value? In financial jargon, financial benefits must be “realized” before they count. In other words, if we ‘save $1.7 million’ when the CEO turns up at our desk, can we provide this in cash or equivalent?
3. Time saving and productivity is complex. We’ve saved each staff member two minutes a day, so do they now spend all that time in productive ways? Or do they spend it reading the newspaper or chatting on Facebook?
4. We are multiplying a lot of assumptions. How valid is a figure when we have to make a dozen different assumptions, and then multiply them all together?
5. Many financial people won’t accept it. In many organizations, CFOs and their staff simply won’t accept these types of metrics, for the intranet or any other system. Better check first!
To this I added a laundry list of other issues, including:
6. It’s only about usability. Measuring time spent on the intranet is easy, but not very impactful.
7. How many staff are using the intranet? We comfortably multiply out the time saved by the number of staff in the organization, but this seems somewhat optimistic. How many staff are actually using the intranet?
8. Increasing usage now costs us money. We’ve made the argument that reducing time spent on the intranet will save the organization money, yet simultaneously we are aiming to increase intranet adoption. By this argument, the greater they number of intranet users, the more money we lose. We can’t have it both ways.
9. The intranet is considered in isolation. The intranet sits alongside — and competes with — every other way of completing a task or getting an answer. To make our calculations valid, we have to assume the intranet is the only source of information.
Learning Opportunities
10. We don’t consider experience. Experienced staff just know how things work, as well as the answers to common questions and issues, so they don’t need to look up information on the intranet.
11. What about learnability? No matter how bad an intranet is, given enough time, staff will often learn how to use what they need (or will give up entirely).
12. How often are these tasks done? Savings are generally calculated based on the before-and-after testing of a number of “common” tasks on the intranet, such as looking up a phone number, or applying for leave. How often does an individual staff member actually do these tasks? In the case of applying for leave, it may only be once or twice per year, certainly not daily.
13. Are we testing the right tasks? In order to be relevant across a wide range of staff, tests tend to focus on general intranet activities. These generic tasks often relate to the staff directory, HR, finance, news, etc which aren’t directly related to the day-to-day business of staff.
14. Can we generalize out to the whole intranet? We’ve tested a few “representative” tasks and have found before-and-after time savings. But this is just for six to 12 tasks. Can this be generalized out to time savings across the whole intranet?
15. What are we actually testing? A typical task may be to find out how to apply for leave. Should that be a quick or slow task? Do we want people to carefully read the policy (perhaps), or are we just measuring the time it takes to find it?
16. We oversell the value of these metrics. As indicative findings, they have great value. But many organizations go way beyond this, claiming millions of dollars of savings, which is dangerous territory.
17. We are treating our managers as fools. The many numbers involved in justifying time savings smacks of “blinding with science.” In taking this approach, we risk treating our managers and stakeholders as fools, selling them vague figures as concrete business cases.
18. Negative business cases are dangerous. Once we start down the road of negative business cases based on money saved, we can get ourselves in a lot of trouble Wouldn’t the quickest and easiest way to save money be to sack the intranet team and fully devolve intranet management?
Related Article: Why Collaboration Analytics Is Crucial to the Employee Experience
Measuring Real-World Benefits
Now's the time for me to own up to using these time saved metrics in our consultancy work at Step Two .... Used appropriately, they do meaningfully demonstrate that we’ve made the intranet’s information architecture better, but that’s where we leave it.
For intranet teams, the real opportunity is to make the intranet a “place for doing things,” not just a “place for reading things.” It then becomes much easier to demonstrate real benefits, including quantifiable dollar savings.
Teams should focus on demonstrating improvements to end-to-end tasks, and on delivering direct solutions to business needs. The closer the intranet gets to supporting the day-to-day operational needs of a business (often relating to frontline workers), the easier it becomes to demonstrate tangible outcomes.
Where do you stand on this argument? Share your thoughts, let's continue the discussion.
Learn how you can join our contributor community.
About the Author
James Robertson is the originator of the global movement towards digital employee experience (DEX). Twenty years in this space, he’s one of the leading thinkers on intranets and digital workplaces. He’s the author of the books “Essential Intranets: Inspiring Sites that Deliver Business Value” and “Designing Intranets: Creating Sites that Work.”
Connect with James Robertson:
Featured Research
Buyer's Guide
Read Now
Research Report
Read Now
eBook
Read Now
Research Report
Read Now
White Paper
Read Now
Research Report
Read Now
Guide
Read Now
Guide
Read Now
Related Stories
Entries for the 2023 Reworked IMPACT Awards are now Open
