Testing, Release 1.0, Dec 2018 and Jan 2019

Context

Between December 2018 and January 2019 we tested Release 1.0 with five people. All the participants work as professional developers, most of them routinely writing frontend code (except for one who is an Android developer). Two of the participants were born and also lived in Germany, two were German and another one lives in Switzerland now but originally comes from the US.

For more information, contact Andreas Moesch.

Feedback

Content

Prior knowledge assumptions and framing

• It doesn’t give any context or explanation for the term “accessibility”. (e.g. there is this common notion that accessibility is “just for people with disabilities” but as we know there is a lot to differentiate there)
  • —> Decide what prior knowledge we expect: Either we decided that we expect prior knowledge and ignore this or we should explicitly frame the notion of accessibility and makes a case for why this is important.
• It doesn’t give much context for the Screen Reader testing results under the widget examples. For people with very little knowledge this is hard to understand
  • —> Provide more explicit labels for the SR testing results table as well as context information (why did we test with specifically those?)

Credibility/Reliability

• It’s not obvious why one should trust this source. In a Swiss context the logos of Access for All or the other agencies might add credibility but outside of Switzerland they mean nothing.
• The relationship to other standard bodies, especially W3G and WCAG isn’t clear (One tester said “you don’t want it to become another HTML reference”). It might help to link to them or at least specify that we’re not a counter proposal. (this has already been discussed)
  • —> Clarify relationship to WCAG (already WIP, see issue 161)
  • —> Link to WCAG success criteria where appropriate (already WIP, see issue 176)
  • —> Mention laws and how we stand to those
• The relationship to laws in a particular country or region isn’t clear either. Of course we can’t cover everything there.
• Beyond logos and laws another big factor for building trust is the professional community: If you see/hear/read someone who you respect mention the ADG this is worth a lot. Can be through Social Media, Talks, Workshops, internal code reviews, open source contributions, etc.
• In Switzerland there is also the special case that A4a creates reports when they test a tool. Those reports can and should link to the ADG to provide more in-depth information.

General

—> Strengthen the places for “meta information”

• Rename/reframe the “Introduction” section. A lot of the information is there but under “Introduction” people might not expect it. Make it more inviting for people who want to know more about the Guide itself.
• Better use the space on the start page
• Make the “about” section more visible.

Macro Information architecture:

How should this guide be used? At the moment it’s this hybrid of a book and a reference.

• Reference: It’s not a very good reference:
  • Things are hard to find. Topical information is distributed (e.g. headings, screen readers) with little to no cross-references. People might find one thing and conclude that it’s everything there is.
  • Written in a conversational tone, which is hard to parse and too verbose.
  • Search isn’t seen
  • Code examples are applied inconsistently. Sometimes there is one, sometimes it’s not. There is no system.
  • Terminology in the navigation and in titles is vague or misleading: What does «Knowledge» mean? What does «relevant combos» mean?
• Book
  • As book it might work better but that is quite a barrier. What’s the assumption here? That people will actually read this? Or just a chapter? What’s the “smallest unit of information” we expect?
  • People should be able to relate the chapters and titles. They at least need to have an idea what could be meant by it – especially in a topic that is so out of most people’s comfort zone. The content itself is written in a pretty accessible language. But the IA is not
• Setting expectations: The way the navigation is structured is setting expectations. It looks more like a reference. E.g. “Setup” = Manual, spec, “Windows” = this is the Windows portion, where is the macOS part?

—> Investigate whether there could be a hybrid of a book part and a documentation part that build on each other:

• Knowledge more like a book: Talks about things in generic terms, about principles and context. Consolidate all the knowledge in there and link to examples.
• Examples more like a reference: Demonstrates things in specifics. Pick examples deliberately and think about what it should demonstrate. Then only talk about that example in the text and link to “knowledge”. Optimise this section for hands-on work: How can people take this concrete example and understand its parts?

—> Investigate whether we can set a standardised content structure for different sections (would pull it more into the direction of a reference). Maybe for part of it.

—> Make it more visible how the guide should be used. The information is there but under “introduction” people might not look for it.

Micro Information architecture

—> Improve the chapter overview pages. They are currently hard to parse. This is partially also due to the fact that the blurbs of the different sub pages are written independently from each other. When they are pulled together on the overview page they don’t really fit together. First, improve content, later also the design. They should give a real high-level overview and shouldn’t just be verbose links.

—> Improve the titles and the labels in the navigation.

• Remove ambivalent or coloquial terms
• Navigation links to a certain page (Nav labels, prev/next links at the bottom of the page) and the respective page titles should be very similar, ideally the same.
• Make the text sizes in the navigation slightly smaller so that you can have more than just two words.

—> Investigate possibility to create a standard content structure at least for some parts (e.g. examples). This might also make it easier to contribute

Layout

• The full-width default leads to oversized images, especially if they have a portrait format. Tears apart the text, requires a lot of scrolling, etc.
  • —> Avoid oversized images. e.g. by defining max-width and max-height.
• On the example pages the information is spread a lot. If people want to relate a technical comment to a code example they have to scroll up and down.
  • —> Make the example parts more practical to work with, e.g. by increasing information density, maybe adapting the layout or at least providing anchor links.
• Navigation is really difficult on small viewports. Menu and breadcrumbs are at the top, you have to scroll all the way and then jump into another content. Everything is “navigationally sandboxes.”
  • —> Improve Navigation layout on small viewports e.g. by providing a fixed access to the menu.

Search

• The search functionality is there but really hard to spot. People didn’t see it. They might find it when they look for it but as it is now the UI doesn’t present search as a “first-class” research tool
  • —> Make search more visible.
• The auto-suggest feature works but needs a moment to display results. Until then people might already have hit “Enter”, assuming that the featured doesn’t exist.
  • —> Add loading indicator while search suggestions are loading. Needs immediate feedback
• Whether a word is written in British or American English has an impact on search results. E.g. “colour” returns all the matches in the text while “color” returns all the matches in CSS code.
  • —> Make British and US spelling interchangeable
• While the matching words are highlighted in the auto-suggestions they aren’t in the list of actual search results.
  • —> Highlight key term in search result, not just in auto-suggest