Media Alt Technologies - HTML Accessibility Task Force Wiki

Contents

• 1 Document for Discussion
• 2 Introduction
• 3 Scenarios where users benefit from text alternatives for media
• 4 Requirements on text alternatives
• 5 Useful text alternatives / use cases
• 6 Potential attributes to use
• 7 Recommended solution
  • 7.1 1: Use @aria-label for the text description of player
  • 7.2 2: Use @aria-describedby for the longer text alternative
  • 7.3 3: Use @transcript for providing a link to a file with the full transcription
• 8 Requirements check
• 9 Example uses
  • 9.1 Example 1: Video with a movie poster as the @poster image (A Clockwork Orange)
  • 9.2 Example 2: video with text on placeholder image
  • 9.3 Example 3: video without placeholder image
• 10 Terminology

Document for Discussion

Introduction

At the moment all discussion around text and media alternatives for the <audio> and <video> elements focuses on time-aligned alternatives such as captions, video descriptions, and sign language. While these provide accessibility solutions for the situation where a video is autoplaying, we don't have a solution for when a video is not autoplaying and just has a placeholder frame presented on screen.

Traditional alt technologies as used for images are focused on what the vision-impaired user is being told when reaching the element on the page, or what is being displayed if images are turned off or a voice browser is used to examine the page.

In the same vein, we will need text alternatives for video in the situation where a vision-impaired user tabs onto the element, it is not autoplaying, and the user wants to gain information about the element, such as whether it is worthwhile starting to play the resource. Similarly, we need text alternatives for audio and video where imagery content cannot be displayed in general such as in a text-only browser.

At the moment, there is no recommendation for authors and accessibility APIs about what attributes to use and interpret.

There is a related bug in the HTML WG: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12228

This document tries to clarify this situation with examples and more details. It makes a proposal for a consistent and succinct means of providing media text alternatives.

Scenarios where users benefit from text alternatives for media

• They have a very slow connection or want to save on download cost. [Note: is there a setting for turning off video in browsers similar to turning off images?]
• They have a vision impairment and use text to speech software to find out about the media, e.g. to decide whether to play it back.
• They have a cognitive impairment and use text to speech software.
• They are using a text-only browser.
• They are listening to the page being read out by a voice Web browser.

Requirements on text alternatives

1. need to support graphical and text-only browsers

2. need to support HTML5 and legacy browsers (i.e. <video> element support)

3. need to allow describing the default and custom video player layout

4. need to allow for short and long text alternative for placeholder image

5. need to allow for short and long text alternative for video

6. need to work when text alternative is on page / not on page

Useful text alternatives / use cases

• short and potentially structured alternative text: this text is a short-ish piece of text that can potentially contain structured content such as a table and describes the content of the audio or video resource as well as the placeholder image in more detail. Often, this information is available in a different location on the Web page already and just needs programmatic association. For video it is often created from metadata, such as the title, the producer, a description, etc. For the placeholder image it makes sense to include text that serves the same purpose as the image, namely to encourage users to press the "play" button.

• full transcription: this text is a non-timed full text alternative to the audio or video and is useful when no timed presentation of the media elements is possible. It should contain all details about the resource, including a description of the placeholder frame, a transcription of everything that is being heard (the captions), and a transcription of everything that is being seen (the video description). It is best provided as a separate Web page, since it can be very lengthy.

• text description of player design: this text is a very short, text-only piece of text to be used when the element is reached on the page, e.g. by a screen reader, a text-only browser, or a voice Web browser. The purpose is to inform the user of any special visual "labelling" that has been applied to the video or audio player, i.e. as a branded player for a site. It is given in the language in which the Web page is provided. It needs to be short - like somebody having a cursory look at the player - in order not to stop other information about the element from being presented. It should be used sparingly.

Potential attributes to use

• title
• name
• alt
• aria-label
• aria-labelledby
• aria-describedby
• longdesc
• figcaption
• new attribute

Also see

• http://lists.w3.org/Archives/Public/public-html-a11y/2010Nov/0246.html
• http://lists.w3.org/Archives/Public/public-html/2009Sep/0592.html
• http://webaim.org/techniques/css/invisiblecontent/

Recommended solution

1: Use @aria-label for the text description of player

This label is read out by a screenreader as soon as a user tabs onto the video.

It provides a short string for description of the expected interaction and possibly the audio or video player design if required.

It can also be read out by a screenreader on a paused video.

Only use this for a really really short styling mentions - the use of @aria-describedby is generally more important to non-sighted users (see 2 below).

Reasons for choosing @aria-label:

• @title is used for different purposes (use for tooltips)
• @name is used for different purposes (use for naming the area)
• @figcaption and @figure are not typically used to publish audio or video

2: Use @aria-describedby for the longer text alternative

This description is read out by a screen reader right after @aria-label.

It can also be read out by a screenreader on a paused video.

Reasons for choosing @aria-describedby:

• @aria-describedby allows linking to structured content elsewhere on the page which is a typical way in which more information is provided about audio and video content
• @aria-describedby allows marking up sections with language tags thus enabling the use of language-specific speech synthesis models
• @aria-labelledby is used for a different purpose (explaining readable labels for an input box)

3: Use @transcript for providing a link to a file with the full transcription

The new attribute contains a URI to an off-page full transcription.

This is meant for external resources that transcribe the video or audio resource and can thus follow along more easily into embedding-type scenarios through iframes. If the transcript content is available on the same page, @aria-describedby should be used.

The @transcript link should be displayed in browsers without a visual presentation channel in place of the video or audio element in addition to the @aria-label text.

Screen readers should announce the existence of a transcription.

The link should also be provided in a context menu in browsers to allow all users access to the full transcription.

The transcription document itself should also link back to the video page thus making it a more useful web resource.

Reasons for proposing @transcripton:

• a full transcript is typically only useful in the few cases where somebody is not going to actually watch the video or listen to the audio, such as a blind-deaf user or a user in a text-only browser, a voice browser, or on a low-bandwidth link
• if the author does not want the transcription to be displayed on the page (for space reasons or because it's in an embed), this is the way to offer that alternative representation
• for users that don't need the transcription, an external file saves them bandwidth
• @longdesc is too image-specific and not as concrete as @transcript
• @aria-describedby is read out immediately, while this should just offer an option to follow the link

Requirements check

This type of markup satisfies the above requirements as follows:

1. need to support graphical and text-only browsers:

Graphical browsers display the video with the @poster frame as the placeholder image (or the first frame if no @poster attribute is given). Text-only browsers can display similar text to what is now inside the <video>.

2. need to support HTML5 and legacy browsers (i.e. <video> element support):

HTML5 browsers display the video with @poster as the placeholder image (or the first frame if no @poster attribute is given). Legacy browsers display the paragraphs from inside the video element.

3. need to allow describing the default and custom video player layout:

Default layout would have a controls attribute and no text in aria-label bar maybe an encouragement of how to interact with the player. Custom layout announced through the aria-label attribute.

4. need to allow for short and long text alternative for placeholder image:

Short text alternative for the placeholder image is provided through aria-describedby. Long text alternative for the placeholder image is provided through the link inside the paragraph for the short text alternative for the image.

5. need to allow for short and long text alternative for video:

Short text alternative for the video is provided through aria-describedby. Long text alternative for the video is provided through the link inside the paragraph for the short text alternative for the video, and also as an additional attribute on the video element so that a context menu can bring up a link to the transcript for sighted users.

6. need to work when text alternative is on page / not on page:

aria-describedby links to on-page content. A link inside a paragraph that @aria-describedby links to can contain an even longer text alternative. transcript links to off-page content.

Example uses

Example 1: Video with a movie poster as the @poster image (A Clockwork Orange)

In this example, a movie poster is used as the placeholder image for the video.

Error creating thumbnail: Unable to save thumbnail to destination

Originals from: http://www.archive.org/details/ClockworkOrangetrailer1971 and http://popculturefan.com/a-clockwork-orange-poster.jpg and http://pro.imdb.com/title/tt0066921/

Example code 1: text description of player

An @aria-label attribute is added with a short recommendation for the user interaction.

Screenreaders and voice browsers would upon tabbing onto the video element read out the aria-label text.

<video poster="media/ClockworkOrangetrailer.jpg" controls aria-label="hit space to toggle play/pause"> <source src="media/ClockworkOrangetrailer.mp4"> <source src="media/ClockworkOrangetrailer.webm"> <source src="media/ClockworkOrangetrailer.ogv"> </video>

Example code 2: short text alternative

An @aria-label attribute is added with a aria-label. Additionally, there is a longer description on the page with structured content, which @aria-describedby links to.

Screen readers and voice browsers would upon tabbing onto this video element read out the aria-label and then read out the text in the segments marked 'summary', 'more' and 'posteralt'.

<video poster="media/ClockworkOrangetrailer.jpg" controls aria-describedby="videosummary more posteralt" aria-label="hit space to toggle play/pause"> <source src="media/ClockworkOrangetrailer.mp4"> <source src="media/ClockworkOrangetrailer.webm"> <source src="media/ClockworkOrangetrailer.ogv"> </video> <div> <p id="videosummary"> In future Britain, charismatic delinquent Alex DeLarge is jailed and volunteers for an experimental aversion therapy developed by the government in an effort to solve society's crime problem... but not all goes to plan. </p> <ul> <li>Director: Stanley Kubrick</li> <li>Writers: Stanley Kubrick (screenplay), Anthony Burgess (novel)</li> <li>Stars: Malcolm McDowell, Patrick Magee and Warren Clarke</li> <li id="more"><a href="http://www.imdb.com/title/tt0066921/">Details on IMDB</a></li> </ul> </div> <h4>Poster description</h4> <div id="posteralt" style="font-style: italic;"> <p> The video poster frame shows a movie poster with the film's protagonist, Alex (played by Malcolm McDowell) brandishing a knife while peering through a cutout of a stylized "A" or inverted "V". An eyeball appears floating at his wrist. </p> <p> The poster also reads "Being the adventures of a young man whose principle interests are rape, ultra-violence and Beethoven", as well as bold psychedelic type below the image which reads "Stanley Kubrick's Clockwork Orange". </p> <p> The footer text reads "A Stanley Kubrick Production - A Clockwork Orange starring Malcolm McDowell, Patrick Magee, Adrienne Corri and Miriam Karlin. Screenplay by Stanley Kubrick. Based on the novel by Anthony Burgess. Produced and Directed by Stanley Kubrick" </p> </div>

Example code 3: text label, short text alternative and transcription

An @aria-label attribute is added. The poster description is hidden from those users that see the video. Additionally a link is provided in @transcript that leads to the transcription off page.

The browser makes the @transcript link available in a context menu for the element.

Screen readers and voice browsers would, upon tabbing onto this video element, announce the availability of the link through a specific sound, and mention the full link after reading out aria-label.

<video poster="media/ClockworkOrangetrailer.jpg" controls transcript="transcription.html" aria-label="hit space to toggle pause/play" aria-describedby="posteralt"> <source src="media/ClockworkOrangetrailer.mp4"> <source src="media/ClockworkOrangetrailer.webm"> <source src="media/ClockworkOrangetrailer.ogv"> <div id="posteralt"> The placeholder image shows a movie poster with the text "Stanley Kubrick's A Clockwork Orange" with a man peering through the letter 'A' holding a knife. It also reads 'Being the adventures of a young man whose principle interests are rape, ultra-violence and Beethoven'. </div> </video>

Transcription file: transcription.html (extract)

<h1><a href="http://www.imdb.com/title/tt0066921/synopsis">A Clockwork Orange: Synopsis</a></h1> <p> Transcription for <a href="media/ClockworkOrangetrailer.html">the Clockwork Orange Trailer video</a> </p> <h2>"The old ultraviolence."</h2> <p> London, England, the foreseeable future, c. 1995. "Our humble narrator" Alexander DeLarge (Malcolm McDowell) and his droogs, Georgie (James Marcus), Dim (Warren Clarke), and Pete (Michael Tarn), are seated in the Korova Milk Bar stoned on milk laced with narcotics. </p> <p> The gang leaves the Korova for a night of fun. They encounter a Irish wino (Paul Farrell) in a concrete subway tunnel under a local motorway. They beat him with their truncheons. Later, they arrive at a derelict theater. On the stage, another gang, led by a schoolmate named Billy Boy, is preparing to rape a voluptuous girl. Instead, the two gangs have a battle in which Alex and his droogs are victorious. </p> <p>...</p>

Example 2: video with text on placeholder image

In this example, a image with text is used as the placeholder image for the video.

Some text describing the video should be hidden from sighted users if they see the video.

Error creating thumbnail: Unable to save thumbnail to destination

Original from: http://acessodigital.net/video-html5/video-acessibilidade-en.html

Example code 1: short text alternative

An @aria-label attribute is added. Additionally, there are text descriptions on the page with a poster description and structured content, which @aria-describedby links to.

Screen readers and voice browsers would upon tabbing onto this video element read out the aria-label and then read out the text in the segments marked posteralt and shortcuts. Notice the change of language in the posteralt div, which leads to the use of a different speech synthesis model.

<video poster="media/acessodigital.png" controls aria-describedby="posteralt shortcuts" aria-label="hit space to toggle play/pause"> <source src="media/acessodigital_en.mp4"> <source src="media/acessodigital_en.webm"> <source src="media/acessodigital_en.ogv"> <div id="posteralt"> The placeholder image shows fingers on a keyboard titled 'Web accessibility: cost or benefit', <span lang="es">'Accessibilidade Web: custo ou beneficio?'</span>, <span lang="pt">'Accessibilidad Web: costo o beneficio?'</span>. </div> </video> <div id="shortcuts"> <h2>Keyboard shortcuts</h2> <p> Note: On focused video, in browsers that support HTML 5, use the following shortcuts: </p> <ul> <li>To play and pause press space.</li> <li>Increase volume ↑up arrow, decrease volume ↓(down arrow).</li> <li>Switch between available subtitles: Left CTRL + v (english, portuguese or spanish).</li> </ul> </div>

Example code 2: text alternative and transcription

A hidden text alternative is provided in @aria-describedby through a link to a video-internal div element.

A link is provided in @transcript that leads to the transcription off page.

The browser makes the @transcript link available in a context menu for the element.

Screen readers and voice browsers would, upon tabbing onto this video element, announce the availability of the link e.g. through a specific sound, and mention the full link after reading out aria-describedby.

<video poster="acessodigital.png" controls transcript="transcription_en.html" aria-describedby="posteralt"> <source src="acessodigital_en.mp4"> <source src="acessodigital_en.webm"> <source src="acessodigital_en.ogv"> <div id="posteralt"> The placeholder image shows fingers on a keyboard titled 'Web accessibility: cost or benefit', <span lang="es">'Accessibilidade Web: custo ou beneficio?'</span>, <span lang="pt">'Accessibilidad Web: costo o beneficio?'</span>. </div> </video>

Transcription file: transcription_en.html (extract)

<h1><a href="acessodigital_en.html">Web Accessibility: Cost or Benefit?</a></h1> <p> [Jaws:] Web Accessibility: Cost or Benefit? [TAB][INTERNET][ENTER] </p> <p> [Leda:] I'm Leda Lucia and I love the Internet. Because I can communicate with people, I can read newspapers, magazines, and get a lot of things done without leaving the house. I do bank transactions, make payments, shop... </p> <p> I'm a psychologist. I use the Internet to communicate with other professionals; I take part in group discussion lists, and to research about various subjects. </p> <p>...</p>

Example 3: video without placeholder image

In this example, we compare the video when used directly without a placeholder image to the use of a placeholder, using both a short and long text alternative.

Original from: http://www.archive.org/details/Trust_1231

Example code 1: text alternatives with a poster

Error creating thumbnail: Unable to save thumbnail to destination

There are longer descriptions on the page with a poster description and content, which @aria-describedby links to.

Screen readers and voice browsers would upon tabbing onto this video element read out the text in the segments marked posteralt and shortcuts.

<video poster="media/trust.png" controls aria-describedby="posteralt details"> <source src="media/trust.mp4"> <source src="media/trust.webm"> <source src="media/trust.ogv"> <div id="posteralt"> The placeholder image shows a blue steamtrain and titles Trust by Domenic Marbaniang - 'It is better to take refuge in the Lord than to trust in man', Psalm 118:8. </div> </video> <div id="details"> <p>Hindi Gospel song by Domenic Marbaniang, 2007</p> <p>Keywords: Hindi; Gospel; Christian; Pop; Prayer; Worship; Song; Surrender; Trust<p> </div>

Example code 2: text alternatives without a poster

Error creating thumbnail: Unable to save thumbnail to destination

For comparison here we show a video without a poster attribute.

There are longer descriptions on the page with a poster description and content, which @aria-describedby links to.

Screen readers and voice browsers would upon tabbing onto this video element read out the aria-label and then read out the text in the segments marked posteralt and shortcuts.

<video controls aria-describedby="posteralt details" aria-label="Trust by Domenic Marbaniang"> <source src="media/trust.ogv"> <source src="media/trust.mp4"> <source src="media/trust.webm"> <div id="posteralt"> The placeholder image shows a blue steamtrain and steam around its wheels </div> </video> <div id="details"> <p>Hindi Gospel song by Domenic Marbaniang, 2007</p> <p>Keywords: Hindi; Gospel; Christian; Pop; Prayer; Worship; Song; Surrender; Trust<p> </div>

Terminology

Transcript/Transcription = full-text description of the content of a video or audio file including everything that is being heard and seen

Timed Transcript = literal transcription of what is being said in the video displayed in a time-aligned manner underneath the video as full text with changing highlighting of the current phrase spoken in the video

Interactive Transcript = a timed transcript that also allows clicking on the individual phrases to navigate the video playback position to that phrase