What Machines Can’t Detect: Manual Checks For Digital Accessibility  

Accessibility Matters at UC - Automated Testing, with an image of Brian McNeilly, Web Accessibility Specialist

By Brian McNeilly, Web Accessibility Specialist

Whenever the discussion of digital accessibility comes up, whether it’s about PDF accessibility trends or reviewing vendor documentation, the same advice keeps coming up: don’t rely just on automated testing. For people new to accessibility, this is easier said than done! In this blog post, we’ll talk about ways you can level up your accessibility reviews by checking for things automated tools fail to find. From checking for semantic markup, to reading order and reflow, a few simple checks can take you beyond what any automated tool can do. 

Automated tests are incredibly helpful tools for accessibility. These tests can quickly determine errors that would be incredibly time-consuming (and error-prone) for a human to spot. For example, they can check whether every link, button, and input field on a page has a name assistive technologies can detect. At the same time, there are many tests that computers will have no way to determine, such as whether the names in our previous example are correct. As those two examples illustrate, many digital accessibility issues that need manual review relate to the inherently human aspect of digital accessibility. The crux of digital accessibility is ‘did a person get access to information?’ and only a human is going to be able to accurately answer that question reliably. 

In this post, we’ll introduce three of the most common types of issues that require human attention in accessibility and share ways you can test them yourself. 

Appearance matching structure

Many issues automated checks miss fall under the umbrella of something visually looking one way but being structured differently. For those interested in the nitty-gritty details of digital accessibility, these issues fall under the WCAG Success Criterion 1.3.1 Info & Relationships. A classic example of this is a document broken into sections that lack properly defined headings. Visually, the content might have headings, but within the structure of the document there is no difference between the “heading” and any other piece of text. If you’ve ever heard the term semantic markup within the context of HTML, this is what it’s referring to. The HTML language has tags that mean something (these are the semantics). To use our heading example, the following tag semantically means that “Hello, World” is a heading level 1: <h1>Hello, World</h1>. Assistive technologies can then convey these semantics, and issues arise when the semantics differ from what is visually presented. 

Other examples you may encounter as you investigate accessibility include lists created just by typing out numbers instead of using the built-in tools to semantically define them as a list. Similarly, especially in PDFs, lists may be visually understood as one list but be broken into multiple lists for assistive technologies. This often happens when a list spans across two pages and is automatically tagged as two lists, rather than one. 

Any of the issues that fall under this banner will be challenging for computers to find automatically because machines have a really difficult time looking at content holistically. But to verify this as a human, all you have to do is double-check that the authoring tool’s built-in features are being used. To go a level deeper, you can also use a screen reader to verify the content is announced correctly or review the HTML yourself. 

Reading order 

For a human, reading order is a straightforward concept: if you are reading something and there’s a sudden gap or jump in a sentence that doesn’t make sense, something has gone wrong. Perhaps a paragraph or page is missing or something failed to load accurately. Accessibility testing for reading order does exactly this: it verifies that the order in which content is being given to assistive technologies is the same as what the author intended. These issues fall under the aptly named 1.3.2 Meaningful Sequence Success Criterion in WCAG.  

While this task is almost intuitive for a human, computers tend to get overwhelmed by anything other than the simplest layouts. Testing algorithms can typically guess that a column or two of English text will be read top to bottom and left to right. But once you add sidebars, callouts, or anything else, the results will quickly become unpredictable. 

In web pages, the “correct” reading order of content is typically straightforward: there is a header, one column of main content, a footer, and perhaps there’s a sidebar that you need to consider. Testing is also fairly straightforward: you can select content with your cursor, and the order it is selected in will be the order it will be exposed to in assistive technologies (for the tech-minded, this is also the order the content appears in the DOM). While web pages tend to be easy, PDF layouts, especially magazine-style ones, can be much more challenging. Determining the “correct” reading order for a callout or sidebar is almost an authorial question, and therefore, it relies on understanding the material and where a break from the main content makes sense. In a PDF, reading order is determined by the order content appears in the Tags tree, so simply reviewing that order should be enough to determine if your document has any issues. 

These questions of intent and the use of quotes around “correct” in the previous paragraph highlight why this task is still a challenge for computers. Determining a document’s cadence and how content should be read is still a challenge for computers.

Resizing text and reflow 

In our previous examples, we discussed content that automated tests can sometimes assist with, even if they can’t reliably test it. This contrasts sharply with issues around text resizing and page reflow. As a brief primer, many users magnify their screens or text size to increase legibility. WCAG success criterion 1.4.4 Resize Text requires users to be able to zoom content up to 200% without content breaking in any way. When users magnify text beyond 200%, many pieces of content require left-to-right scrolling to read a full line or paragraph of text, in addition to scrolling up and down. If you’ve ever done this even briefly (e.g., while trying to read a PDF on your phone), you likely know how frustrating it can be. This is why there is a separate requirement, 1.4.10 Reflow, which requires that as users magnify their content further (effectively to 400% zoom), they no longer have to do this. 

Although it’s incredibly easy for a human to test and verify this success criterion by zooming in and seeing the changes, I haven’t seen any automated testing that checks it. This likely comes down to the more nuanced decisions that need to be made about what is correct versus an error, compared to our previous examples. 

Wrapping up 

While these are only a few examples, hopefully they give you a few actionable ways to help verify your content’s accessibility. Many of these checks take seconds but would otherwise go unverified by an automated test. 

Author 

Brian McNeilly, Web Accessibility Specialist

Brian McNeilly 
Web Accessibility Specialist 
UC Office of the President 

About the Accessibility Matters at UC series

We heard you! When we reviewed our content survey results, one message came through clearly: there’s a growing need for more focus on accessibility. And you’re right—accessibility affects everyone! Follow along as we share stories that highlight why accessibility truly matters at UC. Do you have a story to contribute to the series? Submit an intake form! Submit a UC Tech News blog intake form found here.