Documentation Metrics: are your users happy with your docs?

Was this page helpful? Yes / No

You can often see this question at the bottom of some public-facing documentation websites, followed by a feedback box.

But is it enough to measure your docs site performance?

Documentation metrics are vital for companies. Time and resources are needed to drive success for user documentation, and without proper tracking in place, you might be in the dark.

Producing a piece of content that is helping the user takes time, and it means you understand what docs need to be created, which ones need to be updated, so a data-driven approach helps with prioritization.

When product documentation is considered an asset that can generate a growth loop, it needs to be measured by specific KPIs, and proper tracking tools must be used.

Before we talk about documentation metrics, let's look at what type of tools are used typically.

Tools to measure documentation performance

If you can't measure it, you probably shouldn't be doing it. Having the right tools to measure documentation performance is something you should consider when you decide to create a docs website. It's never too late to get started so consider the following tools to track documentation metrics.

Google Analytics

There are great alternatives out there to measure web traffic, but Google Analytics is the most common. Rather than covering the features, I'll link to some resources about setting it up, managing it, and starting to track the traffic.

• Setup Google Analytics
• Beginners Guide to Google Analytics
• Google Analytics Reports Created by Experts

Search Console

Search console can help improve the performance of your documentation website by understanding how search engines like Google see your pages. Crawling, indexing, and usability, are the main features, but there is a little trick to find out what keywords people use to find your docs website. Here is the getting started guide.

Built-in Search Analytics

Search analytics is strongly related to the platform you are using to build the documentation website. Having a feature that gathers the queries users write in the search bar can be essential to understanding what they want to read about. That's the way to know and not guess what to write.

There are ways to set up search analytics and get the user searches, but ideally, you would have them aggregated with the total number of searches and the number of documents.

Archbee has a custom algorithm for search where you can see what your customers or team are looking for.

Clarity

Clarity is a product from Microsoft for heatmaps and session recordings. Compared with other solutions (usually HotJar), Clarity is free and built on open-source.

The power for such tools is in discovering where users get frustrated while visiting your pages. You can segment the session by rage clicks, excessive scrolling, dead click, or quick backs. These are hints for errors or situations that you might not learn from just looking at web traffic data.

And the best part is that you get the session recording to see what the user visited.

Chat / Survey pop-up

Having a chat on your documentation page is a double-edged sword. You might find yourself answering requests from users, but at the same time, you can pick some powerful insights from people that need to talk with a human. You don't have to keep it all the time, but consider it every time you have a big update, or just target the chat to specific pages that need human interaction.

The alternative is to use pop-up surveys asking the user for feedback.

Think of the product documentation website like a product on its own. It needs to help the business bottom line, so having the right metrics can make you look good in front of your peers. There are two types of metrics that you need to look into: quantitative and qualitative.

Let's break it down.

Quantitative documentation metrics

Consumption metrics

Pageviews are a starting point for measuring the health of a docs website. Use it as a metric to understand what topics attract the audience the most.

Users. This metric shows the individuals that visited the website during a period. Usually, this is an aggregate number, so you might want to count for the unique users.

Average time on page. It's essential to understand how this one is measured since it can mislead you. Usually, the time on the page shows you if visitors are reading the content or just scanning it. Compare this with the total read for a documentation page.

Pages per session. A session is a group of page views, so this metric will tell if the user engages with the content. Like any other metric, this needs to be put in context, maybe each page solves a problem, so they don't need to read multiple pages. This lets you ensure that interlinking is done correctly and people can navigate to the next subject.

User flow. This special report in Google Analytics will tell you how a user is navigating through the documentation website. Always default to this report to add context to the other consumption metrics.

SEO performance metrics

For public-facing doc sites, looking into the SEO metrics can help you learn if your content is the one answering questions clients might search for in search engines. For this, you need to set up Search Console to get the following metrics: 

• Impressions: How often someone saw a link to your site on Google.
• Clicks: How often someone clicked a link from Google to your site.
• Average Position: What position in the SERP do your pages have? Are you securing first place? 
• Click-through rate: the percentage of impressions that resulted in a click. Are your titles and meta descriptions getting the click?

What you will be looking for is to secure the first place instead of users asking questions on other forums?

Qualitative engagement metrics:

Publishing and reporting on the quantitative documentation metrics is just half of the context. Understanding website engagement will answer your questions when the numbers don't make sense.

Visitor feedback - ask your visitor what they would like to read about. It's as simple as that. Run a chat or pop-up survey and ask a simple question: What would you like to learn more about?

Session recordings - using Clarity will help debug situations where you actually need to see what the visitor is doing on the documentation website.

Scroll depth and heatmaps - when using long-form documentation pages, it's good to know if people are reading the whole page, or is there any breakpoint? Using scroll depth metrics can help you decide if there is a better way to split the content into multiple pages.

Readability - there are multiple ways to improve writing software documentation, but readability needs to be adapted to your audience. The general rule of thumb is to keep the content at a simple reading level. That's where Hemingway App can help make those edits for readability.

Content Quality - Text is fine, but are you using images or videos? There are no metrics per se but think of people that prefer video instead of text. Adding media to your documentation improves the content quality.

Design - for product documentation websites, the design needs to put focus on the content. You still need excellent navigation and layout, but don't go overboard with brand design requirements.

What to avoid when measuring documentation performance

• The number of docs published and the number of words is a pitfall that even larger companies like Microsoft have fallen in. No matter how comprehensive your documentation is, users will most of the time have the same questions that they need an answer for fast. So make sure you understand what they need before your add volume.
• Bounce rates are typically measured when a person lands o a page but doesn't visit a second one. Using this to measure performance might be good and bad at the same time. If any landing pages answer a user query and leave the page, it doesn't necessarily mean they didn't engage.
• Not tracking anything. Sure, having all the tools in place is ideal, but missing data is a lost opportunity because you didn't think of metrics from the start.

Conclusion

If you think of the documentation website as a product, you will need to ask yourself some questions related to business results. Before you define any metrics, take some time to answer some of these questions:

• In what ways do the public-facing docs support our strategic goals? - define metrics that demonstrate the alignment.
• Who is the intended audience of the documentation? 
• How will they evaluate the document's usefulness?
• What level of reading should it be for?
• Do you think it needs to be assessed against a set of measurable criteria?
• Does the documentation cover all the features of the product? Just use a simple spreadsheet that lists all the implemented features and the relevant documentation page.