Technical documentation is no longer an optional feature you can add to embellish your offer—the modern customer expects to be guided through the product.
However, accurate technical documentation doesn’t equal helpful technical documentation, which is why you should test how user-friendly your instructions are. You can check this list of best practices for technical documentation!
With usability testing, you can determine how successful your documents are in educating and assisting your users.
That way, you’ll be able to implement all necessary improvements before publishing the docs.
We know that usability testing, the term originating from the software industry, may not sound exactly compatible with technical documentation.
So, let’s see why and how you can test the usability of your documents and provide your readers with an excellent user experience.
What Is Documentation Usability Testing?
Document usability is a term that refers to how easily and effectively a person can use a document to achieve a purpose.
Therefore, we can define documentation usability testing as the process of assessing how helpful your documentation is for people reading it to accomplish something with your product.
If that sounds too abstract, let’s try a more specific approach.
Say you’ve joined Twitter and want to tag a friend in a tweet. Searching the internet for the query “how to tag a friend” would lead you to Twitter’s help center.
What you see in the image above is the answer to your question, available in Twitter’s product documentation.
However, the purpose of usability testing isn’t to determine if your documentation answers the users’ questions—it’s to see if they can find the answers and how easily they can do that.
Jakob Nielsen, a usability expert, defines five qualities of usable design. The qualities apply to everything from the user interface to product documentation.
Returning to the example of Twitter, we could say that that specific piece of documentation does great in terms of learnability.
Still, considering that the user has to scroll to the bottom of the document to find the answer, there’s room for improvement regarding efficiency.
We’ll learn how you can perform usability testing on your documentation in a bit, but let’s first see what benefits you get from testing.
Why Test Your Technical Documents?
Testing the usability of your technical documents gives you a preview of how well your documentation will perform once published. Here's a quick example of technical documentation examples we love.
Based on the preview, you can adjust the docs where necessary and provide the actual users with a more enjoyable experience.
The most evident reason for testing your technical docs before publishing is the financial one.
If the user can’t solve an issue using self-service resources such as guides and tutorials, they’ll contact your support center, denting your budget by $15,56 on average with each support ticket.
You can find the factors contributing to such a high price in the following image provided by HDI, the company that calculated the average cost of customer support.
Wouldn’t it be more cost-effective to ensure that the documentation provides users with solutions before they resort to contacting support?
So, if you’re aiming to keep your product financially viable, you shouldn’t skip testing the usability of your technical documentation.
Doing so can help you build user empathy and increase their satisfaction with the product.
While you could wait until publishing to see if your documentation solves users’ problems, it’s better to take a proactive approach and test the usability of the docs in advance.
According to Tom Johnson, a technical writer at Google, a big portion of documentation ends up unusable because nobody tests the documents, which allows imprecise instructions to go unnoticed.
By asking people who aren’t directly involved in the production to test the documentation, you’ll be able to get an impartial assessment of whether the docs are clear, logical, and informative enough to assist your real customers. Here's a list of critical characteristics of technical writing.
That way, you’ll detect potential weaknesses with how instructions are structured and address them in time to present the users with improved documentation.
What to Test in Technical Documentation?
While the overall goal of usability testing is to see whether the documentation is truly helpful, you still need to identify specific qualities you’ll test.
We’ll review four crucial areas where your documentation has to excel to pass the usability test.
1. Readability
The first item on your testing agenda should be readability—the ease with which a reader can understand the text.
If you let the lead developers write documentation, you’d end up with highly precise information.
Unfortunately, most users wouldn’t understand such technical language, so it’s a good idea to ask technical writers to ensure high readability levels. Let’s see how and why SaaS startups need technical writers.
According to Google’s official technical writing style guide, you can achieve greater levels of clarity in technical documentation by using the active voice and breaking sentences into smaller chunks.
Now, that doesn’t mean you’re doomed if you’ve already written some parts of your documentation differently.
There are technical writing tools that tech writers frequently rely on to evaluate and improve readability. One such tool is Grammarly.
When we inserted Google’s recommended version of user instructions in Grammarly, the software deemed the readability appropriate.
But look what happened when we analyzed the longer version written in the passive voice.
Grammarly marked the clarity as poor and provided suggestions for improvement.
In addition to sentence voice and length, you should also test whether your vocabulary matches the target audience and determine the users of your API documentation.
You can ask your non-technical coworkers to read the documentation and let you know if the overuse industry jargon makes the instructions hard to understand.
2. Visual Appeal
Since you’ve put so much effort into writing excellent technical documentation, it would be a shame if all the helpful information stayed inaccessible due to issues with document design.
This is why you have to test your documentation’s visual appeal.
If you visited the DITA 1.2 specification to learn more about content inclusion, chances are that you wouldn’t be able to find the relevant information, even if it was clearly written.
One look at DITA’s technical documentation will reveal why.
As you can see, the specification is painfully dense with information and has no visual elements to differentiate between the hierarchy of the material.
Now, as a counterexample, take a look at this portion of technical documentation designed by ChartHop, an HR solution provider.
The actions that users need to take to share a view are clearly described and listed in a numbered sequence of steps.
To make things even clearer, the authors have marked the names of the buttons users need to click with bolded text, and there’s even a screenshot illustrating the procedure of technical documentation.
So, if you want your docs to look more like ChartHop’s and less like DITA’s, you should evaluate the visual appeal of your technical documentation.
Documentation testing is an opportunity to find where you can change the font size, color scheme, and layout of the document to make your docs not only accurate, but also helpful.
3. Content Structure
In addition to checking the visual appeal of your documents, you should also test how navigable the docs are.
If you want to assist the users in finding the info they need faster, you have to organize the content structure in a way that lets them see what section contains the information they need.
Datree, a CLI tool, has an excellent content structure for its product documentation.
Let’s say you were looking for a way to set up a Git hook to connect Datree with the Git repository.
In that case, you could look at the left side of the screen, find the docs about integrations, and use an expandable table of contents to locate the instructions on Git hooks.
Another table of contents on the right side would give you an overview of the steps listed in that particular document.
That way, you wouldn’t have to spend time scrutinizing the docs for a relevant piece of information.
Similarly, you should bear in mind that users want to access information as quickly as possible.
That’s why your docs should have a powerful search option, like the one available in Archbee, our product documentation platform.
So, when testing the content structure, don’t forget to check how effectively the users can locate the information, be it manually, by using the table of contents for navigation, or by speeding up the process with a search tool.
4. Content Quality
In the words of Janice “Ginny” Redish, a UX and usability expert with more than 40 years of experience,
“The reason the person comes to your content is almost never ‘because I want something to read.’”
So, if you want to enable the users to achieve what they had intended when they opened your docs, you have to ensure that the content quality is of the highest level.
Content quality is an umbrella term that covers several aspects that make the content practical. Here’s a list of crucial qualities to test in technical documentation.
- Sequence of actions
- Amount of information
- Accuracy
- Completeness
- Relevance
While accuracy is a guiding principle when it comes to presenting any kind of instructions, you have to keep in mind that users will likely abandon documentation if the information is presented in a non-intuitive manner.
With that in mind, you should design your usability testing process so that it shows whether users can find and carry out instructions without difficulty.
3 Ways to Test the Usability of Technical Documents
Now’s the time to test the usability of your technical documents. There are three ways you can do that, each suitable for a different purpose.
Paraphrase Testing
Paraphrase testing is a testing method that allows you to see if the users understand the message of your text.
This method evaluates how successful technical writers have been with the choice of vocabulary with regard to the target audience.
It also tests if the sentences are easy to understand.
Let’s see what paraphrase testing looks like in practice, using the example from our own documentation.
In the following image, you can see a paragraph describing Archbee’s feature that clones documents. Our clients use it for organized versioning.
In paraphrase testing, you ask a person to read a fragment of the text and report on the meaning of what they’ve just read.
Here’s how a non-technical staff member paraphrased the paragraph:
“The feature allows the doc owner to have multiple versions of the same document available for sharing with end-users, which is sometimes needed for API documentation, as some documentation users might still use version 1, and some might already be on version 2.”
And they were correct, as were most of the test participants.
So, if you’re aiming to test if the message of the text is clear, paraphrase testing is a way to go. However, remember to take the target audience into account.
You should expect different responses from people with varying levels of familiarity with the industry, so you have to determine which ones resemble your actual users the most.
Task-Based Testing
As we’ve already mentioned, users rarely read technical documentation for the fun of it. Instead, they turn to technical documentation when they want to solve a problem or find an answer.
With task-based testing, you can observe how users navigate the documents to find the necessary information, what parts of the document they read, and if they can successfully follow the instructions to accomplish the goal.
This testing method is the most suitable for user manuals and instructions. Let’s visit Slack’s help center as an example here.
If you had a similar knowledge base, you could ask the participants to solve a particular problem, such as enabling two-factor authentication.
Then, you’d measure how long it took them to find the relevant document and record any detours along the path.
Task-based testing would also allow you to identify potentially redundant parts of the documentation that you could remove for increased clarity of the docs.
Lastly, examining if the participants were able to solve the problem only by using the instructions provided in your documentation will tell you how informative the content is overall.
Plus-Minus Testing
Plus-minus testing requires a more holistic approach to usability testing.
Rather than examining an isolated portion of the document, this method evaluates how participants react to a document as a whole.
Here’s how the authors of the method define it:
“The method involves asking participants to read a text from start to finish and to mark their positive and negative reading experiences with pluses and minuses, respectively, in the margin.”
So, rather than testing the accuracy of the information, plus-minus testing targets the emotional impact of the document.
This method is especially beneficial if you’re aiming to adjust your technical documentation to reflect your brand.
In such cases, comments such as “+, the instructions have a friendly tone” can tell you if you’re on the right track.
Remember, the way the facts are presented matters as much as their accuracy.
Because of that, you shouldn’t consider the documentation usability testing done without performing the plus-minus method.
Conclusion
When we write technical documentation, we become so familiar with the content that we can’t objectively tell if the documentation could help a less-informed audience.
That’s why testing the usability of technical documentation is a vital part of the document creation process.
So, after you finish writing the accurate descriptions and instructions, you should test the usability of your technical documents to ensure that they can assist the users once published.
