NOTE: This is my opinion (David MacDonald) and not necessarily the consensus opinion of the Accessibility Guidelines Working Group (AGWG).

In order to write Success Criteria, I think we have consider the intersection of (at least) 3 stakeholders.

1. ACCESSIBILITY: what will make a significant difference to our stakeholders with disabilities.
2. VIABILITY: what is reasonable to expect of businesses stakeholders.
3. FEASIBILITY: what is mature enough to technically require of authoring stakeholders.

 

 

WCAG is all about Success Criteria. The Principles and Guidelines are just "buckets" for the Success Criteria, to organize them logically. The Conformance Criteria just provide some context of how to assess the Success Criteria on a page.

This article started with my opinion on the characteristics of Success Criteria, with significant input from Loretta Guarino Reid, Gregg Vanderheiden, and Jason White, previous editors of WCAG 2.0. Since then, the WCAG Working Group (now called the Accessibility Guidelines Working Group) chaired by Andrew Kirkpatrick and Alastair Campbell, created a WCAG Success Requirements document, which has general concensus. I've updated this article for uniformity with those requirements. Here I hope to provide helpful advice for those trying to write Success Criteria, or understand them.

Characteristics of a Success Criterion

Success Criteria are hard to write. Many hundreds of millions of dollars have been spent making content meet the WCAG Success Criteria around the world. There is a lot of pressure to get our Success Criteria right when we are creating them. The needs of the disability communities have to be balanced with the needs of the authors, and sometimes massive organizations that own web sites in 50 or more countries. Here are the things we are looking for in Success Criteria.

1. Success Criteria address a situation where a user with a disability will be disproportionately disadvantaged (as compared to a user without a disability) if the criteria is not met.
2. They must be testable through automated or manual processes.
3. They describe the specific condition required to meet the criteria, not the method to address the criteria.
4. They use the existing WCAG 2.0 A/AA/AAA structure.
5. All new SCs should be backwards compatible and not infringe on any WCAG 2 SC.
6. Success Criteria apply to ALL content unless there are specific exceptions
7. Success Criteria apply across technologies to the greatest extent possible. (Technology-specific issues should usually be addressed in Techniques.)
8. Avoid creating a requirement for something that is already required by an existing Success Criterion.
9. They have Techniques which demonstrate that each Success Criterion is implementable, using readily-available formats, user agents, and assistive technologies.

1) Success Criteria address a situation where a user with a disability will be disproportionately disadvantaged (as compared to a user without a disability) if the criteria is not met.

So usability issues that affect all users equally are out of scope, and are better handled in various usability standards.

2) Make testable statements

Every Success Criterion needs to be testable. A web accessibility professional should be able to determine whether the content passes or not with a "high degree of confidence". This testability can be automated, manual, or both. For instance, an automated checker can determine whether there is a text alternative for an image but not whether it provides the "Equivalent Purpose". A human who is knowledgeable on the subject needs to do that.

3) Describe the affirmative condition of the passing content

The author doesn't know who the end user is going to be. So a Success Criterion can't describe the user's experience, because that may differ among end users. Instead, it is important to tease out the underlying properties that will permit (but not guarantee) that the user will have a good experience. Success criteria describe the state of the content, not the process for creating it. So it describes properties that must be true about the page when the Success Criteria is satisfied. If the condition is true then the content passes. A simple is example is

"1.1.1. All non-text content that is presented to the user has a text alternative that serves the equivalent purpose ..."

Success Criteria don't have negative action words telling the author what to do, like "don't", "should not", "Avoid" etc. nor do they have brackets or quotes in the testable statements. However, in some Success Criteria, the elements of the content "do not" have certain characteristics. (See 1.3.3, 2.3.1, 2.3.2, 4.1.1)

4) Success Criteria in 2.1, use the existing WCAG 2.0 A/AA/AAA structure.

This is fairly self evident. As a dot release WCAG 2.1 needs to have the same look and feel of WCAG 2.0.

5) Success Criteria in WCAG 2.1 should be backwards compatible with 2.0

Success Criteria for the version 2.1 of WCAG should not reduce the requirements of WCAG 2.0. In other words they can introduce new requirements but not rescind current WCAG 2.0 requirements. If you pass 2.1, you will automatically pass WCAG 2.0 also.

6) Success Criteria apply to ALL content unless there are specific exceptions

Success Criteria must apply to all web content, unless explicitly excluded. Some content has constraints that make it more difficult to work with and has to have exceptions (e.g., changing the appearance may conflict with the purpose of the content). But if exceptions are put on a Success Criterion, consider the accessibility implications, because the excluded content will present accessibility barriers to some people. However, there can be legal reasons for some constraints, or practical economic reasons. The other option is to move Success Criteria that don't apply to all content to Level AAA where there is more room for specialized requirements.

7) Success Criteria are technology neutral

It is especially challenging to write technology-neutral Success Criteria. We tend to start from problems encountered in a particular technology, and it takes careful thought to identify the underlying, abstract issue, then to describe it in abstract terms that are still testable. This process is one of the things that has made WCAG2 so challenging to write, and challenging for people to understand; it leads to abstract, unfamiliar language.

8) Avoid creating a requirement for something that is already required by an existing Success Criterion.

In particular, the proposed Success Criterion should not be redundant, as in a case where satisfying one success criterion is always sufficient to satisfy another. Success criteria in WCAG next are for *new* requirements, not to clarify or better articulate old ones. Fundamentally, if satisfying an Success Criterion makes a second Success Criterion pass automatically, then one of them is redundant and should not be included. Explaining the nuances on an Success Criterion is the purpose of the Understanding documents, techniques, etc., so we don’t add success criteria just because the application of an existing requirement might not be obvious in certain cases. Instead we clarify it in the non-normative documents.

9) Have Success Techniques which demonstrate that each Success Criterion is implementable, using readily-available formats, user agents, and assistive technologies.

If we are going to require authors to do something, we have to be able to demonstrate that it can be done and it works. So we need to provide ways this is possible today.

Other tips when writing SCs

1. Define new terms carefully
2. Use existing Success Criteria for examples of how to say things
3. Sometimes it helps to split up a Success Criterion
4. Not all proposals can become Success Criteria
5. No set of Success Criteria can meet the requirements of ALL users
6. Try to address needs encountered by different disability groups in a single Success Criterion
7. Use WCAG Glossary terms instead of new terms wherever possible

1) Define new terms carefully

When making a document that will be used in legislation, policy, etc., we have to be very careful about the words we use. If we use a term in a specific context that may not be understood, we define what it means in our standard.

2) Use existing Success Criteria for examples of how to say things

Much of the heavy lifting has been done for us in WCAG 2 over the course of years of voting, discussion and consensus building. Thousands of developers have learned those terms of reference used in the WCAG. We can leverage current WCAG 2 phrases and expressions when writing new Success Criteria.

3) Sometimes it helps to split up a Success Criterion

It is sometimes easier to write different Success Criteria that focus on different aspects of a problem. This makes it easier to isolate different properties. But this also seems to confuse people when more than one Success Criteria fails due to related problems in the content.

4) Not all proposals can become Success Criteria

We can't assume just because a lot of time has been invested into a proposal that is has the characteristics necessary to become an Success Criteria. Don't be afraid to rip up a proposal and start again.

Success Criteria are like salmon swimming upstream or baby turtles trying to make it from the sand to the ocean. Many proposals do not successfully meet the necessary conditions of a Success Criteria. Some of them can become Best Practices.

5) No set of Success Criteria can meet the requirements of ALL users

No set of Success Criteria will ever be complete and capture all accessibility requirements. They provide a floor (although there is a danger that they will be considered a ceiling!) It is valuable to find ways to describe best practices that will make a better experience, and encourage authors to follow those practices when possible. We hope to place more emphasis on Best Practices in the next version, so good advice is not lost even though they did not have the characteristics of Success Criteria.

6) Try to address needs encountered by different disability groups in a single Success Criterion

Also, try to address related needs encountered by people with very different disabilities in a single success criterion, where feasible. For instance, criteria that improve support for orientation and navigation assists people with widely differing access needs. Be sure to identify clearly who benefits in designing the SC, and document it for the explanation and rationale.

7) Use WCAG Glossary terms instead of new terms wherever possible

Use terms which are already defined in the glossary where applicable and appropriate to your needs. Read the definition of any term you use carefully to be sure that it says what you mean to express in your SC. The defined terms in WCAG have been carefully honed over time, so taking advantage of this work where it makes sense is good practice.

Three tips on how to work in the group successfully

1) Listen to every voice, the most useful suggestions sometimes come from those who are not as active

The group conscience is extremely useful, and every voice counts. Many times when we have been stuck, a voice that we don't often hear mentions something tentatively, which solves the problem, or kick starts the new direction to solve the problem. I've learned to trust the group conscience more than my own opinions.

2) Try to compromise as much as possible without losing the objective, in order to gain consensus and unity

There are many stakeholders for the WCAG. They include:

• Diverse communities of people with many types of dis-Abilities.
• Accessibility professionals who have dedicated their lives to making the world more accessible
• Corporations meeting the WCAG, senior managers, project managers, developers, content writers, designers, back end architects etc.
• Governments, policy makers and lawmakers who enact the WCAG
• And many more...
  

We need to listen to all opinions, which sometimes conflict, and find the way to gain consensus and come up with a language that everyone can live with. The 2006 draft of WCAG 2 received 1200 comments. The 2008 final draft had not one formal objection from all these stakeholders. This was because we worked hard on consensus.

3) Listen to every objection and discern whether it is fundamental to the Success Criterion, or if it can be managed with exceptions, qualifications etc.

Don't give up on a proposed Success Criteria because a stakeholder raises an objection. Try instead to understand the objection and discern whether it is fundamental to the backbone of the proposal or if it can be managed with rewording or exceptions. If it is fundamental to the Success Criterion, then the entire group needs to look at the objection and compare it to the benefits of the Success Criterion, and the consequences of proceeding. Also, see if it can be written in such a way that all in the group "can live with it".

There are occasions when the accessibility needs for different users can conflict. An example is whether moving keyboard focus to a control should automatically activate it, which is something that is valuable for some motion impaired users but harmful for users who must navigate through controls using the keyboard. It can be difficult to recognize when such a conflict exists; they are often quite subtle. When they do exist, crafting Success Criteria that supports both sets of user needs can be challenging. 

Conformance considerations

One page serves all users

For WCAG2, we assume that the same page can serve all users, that is, we do not require that the author provide different versions for different users. The user agent may provide a variety of ways of rendering the content that can be personalized by the user, but we assume it is possible to create one version of content that can be rendered in different ways. (It is not absolutely necessary to take this approach, but life becomes much more complicated and difficult for everyone - author, user, evaluator - if it is not true.) 

Alternate versions can be reached accessibly

Sometimes it is easier to let authors provide an alternate version (or buttons to provide an accessible view) which conforms to WCAG, so they can have that new cutting edge component on the main page. The path to triggering that customization must be accessible. For instance, if the user needs to change a setting to enable keyboard access, then the path to enabling keyboard access must already be keyboard accessible. If the user must customize the colors used, the path to performing that customization must be usable before it happens.

A few other tips

Gregg Vanderheiden, describes one approach to writing Success Criteria as "3 Level Deep Thinking". When someone presents an accessibility issue. There are levels to get to a good solution, each level goes deeper than the previous.

1. What is the answer to the question?
2. Why did they ask the question?
3. What are all the possible consequences of all the possible ways to answer this?

In WCAG 2 it plays out like this:

1. What is the objective - what is the requirement we want to have be true
2. Why do we want this - who does it help - will this actually help them - how do we word this to help them
3. What are all the possible ways this can be a problem:

 

Write and re-write until the Success Criteria is as simple as possible without unnecessary words, but don't omit any necessary words.

These statements will be translated to other languages, so avoid jargon and terms that don't translate well. Proposed language should be reviewed by people who speak languages that the WCAG may be translated to.

Conclusion

In short, Success Criteria are hard to write. I've provided some of the tips we used to write them. I'd be glad to add to this article if others have suggestions.

The rewards of well written WCAG Success Criteria are huge. They can make the web more accessible to millions of users with disabilities, and also improve usability for all.

Feel free to comment on Twitter @davidmacd

Author information:

David MacDonald is a veteran WCAG member, co-editor of Using WAI ARIA in HTML5 and HTML5 Accessibility Task Force Member. Opinions are my own.

The WCAG 2.1 contains Normative and Non-Normative documents. The non-normative documents are about 400 pages of help (Techniques, Failures, How to Meet, etc.). The main document is the Normative WCAG 2.1 standard which about 40 pages printed. It contains:

• 4 Principles
• 13 Guidelines
• 78 Success Criteria (50 at AA)
• 5 Conformance Criteria