HTML5 Video: Fragments, Captions, And Dynamic Thumbnails

Web and application developers who want to do more with online video may find that three little-known, or at least less often discussed, HTML5 video features may open many new and creative techniques to integrate video in new ways.

In this article I’ll describe: media fragments, the track element, and HTML5 video’s ability to integrate easily with other elements.

Key Takeaways

• HTML5 video features such as media fragments, the track element, and integration with other elements enable developers to creatively integrate video in new ways. Media fragments allow video playback to start or end at a specific time, creating a sort of video sprite for HTML game development.
• HTML5 video includes a built-in means of presenting on-screen text timed to match the video content. This can be used for video captioning, translated transcripts, descriptions of what is happening, or even presenting chapter or section titles. This feature uses a track element to describe what kind of text is being added, and provide a source for the text.
• HTML5 video can interact with other HTML elements in ways that third-party plugins cannot. For example, it can be used with the canvas element to create dynamic thumbnails for a video. The screen image is captured every five seconds and displayed on the screen, creating a dynamic thumbnail.

Media Fragments

Media fragments or media fragment URIs are a W3C recommendation created to enable some aspects of native video handling in web browsers.

At present, this feature can be used to start or end video playback at a particular instant in time. One could imagine this feature enabling a sort of video sprite that allowed an HTML game developer, as an example, to load a single video file, but easily play different sections in response to some player action.

In its simplest form, the media fragment start time is added to the video source URL. Notice in the following example, the “#t=20” after the source URL where the “t” represents a temporal media fragment.

<video controls> <source src="102614-video-sample.mp4#t=20"> </video>

In the code above the video would begin playback at 00:20 (assuming mm:ss). Let’s look at another example:

<video controls> <source src="102614-video-sample.mp4#t=6,20"> </video>

The example above would start playing at 0:06 and continue to play until 0:20.

The time values in the src URI may also be specified in hour-minute-second format (hh:mm:ss):

<video controls> <source src="102614-video-sample.mp4#t=00:00:20"> </video>

To demonstrate media fragments, I have a 27-second snorkeling video that has three fairly obvious transitions. The first section starts at the beginning of the video (00:00:00), the next section begins at approximately 00:00:06, and the third transition occurs at about 00:00:17.

In the demo, there is a button representing each of the video segments. I have also included two separate source files to ensure the video will play in most browsers.

Below you’ll find the video code along with the navigation:

<video id="frag1" controls preload="metadata" width="720px" height="540px"> <source src="102614-video-sample.mp4" type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"' data-src="102614-video-sample.mp4"> <source src="102614-video-sample.webm" type='video/webm;codecs="vp8, vorbis"' data-src="102614-video-sample-webmhd.webm"> </video> <div class="nav"> <button data-start="0">Section One</button> <button data-start="6">Section Two</button> <button data-start="17">Section Three</button> </div>

Data attributes have been added to the source elements and buttons to make it easier to insert the time-based media fragments with JavaScript. Effectively, the script loads a new source with a time-based media fragment when the button is clicked.

function mediaFragOne() { var video, sources, nav, buttons; video = document.querySelector('video#frag1'); sources = video.getElementsByTagName('source'); nav = document.querySelector('video#frag1+nav'); buttons = nav.getElementsByTagName('button'); for (var i = buttons.length - 1; i >= 0; i--) { buttons[i].addEventListener('click', function() { for (var i = sources.length - 1; i >= 0; i--) { sources[i].setAttribute( 'src', (sources[i].getAttribute('data-src') .concat('#t=' + this.getAttribute('data-start')))); video.load(); video.play(); }; }); }; } mediaFragOne();

Here’s the demo:

(Please note that because the video files are hosted externally, there will be some delay as an individual file loads, so give the demo some time to display the video.)

See the Pen c376d7feb0826d02d244046ed0e7bd77 by SitePoint (@SitePoint) on CodePen.

Note: If the above demo isn’t working, try this external demo.

Adding Captions or Subtitles

HTML5 video includes a built-in means of presenting on-screen text timed perfectly to match the video content. This can be used to add video captioning for better accessibility, offer a translated transcript (a subtitle), provide a description of what is happening, or even present chapter or section titles.

This feature uses a track element to describe what kind of text is being added, and provide a source for the text.

In this example, a video, which includes spoken English, has a Spanish subtitle track that is displayed by default.

<video id="Subtitle" controls preload="metadata"> <source src="102614-maui-with-words.mp4" type="video/mp4"> <source src="102614-maui-with-words.webm" type="video/webm"> <track src="102614-maui-es.vtt" label="Español Subtítulos" kind="subtitles" srclang="es" default> </track> </video>

Notice that the track element is placed inside of the video element, and that it has several attributes, include src, label, kind, srclang, and default.

• src provides the URL for the timed text file. It is, for obvious reasons, required.
• label is the track’s title. It may be presented to the user.
• kind must have a value of either “subtitles”, “captions”, “descriptions”, “chapters”, or “metadata”.
• srclang indicates the track text’s language, and is required when kind is set to “subtitles”.
• default is a Boolean attribute telling the browser that this text track should load initially.

The text track file linked in the src is in Web Video Text Tracks Format (WebVTT). At its most basic, a WebVTT file needs to declare what it is and provide a series of cues with blank lines in between. Here’s an example:

WEBVTT FILE 1 00:00:03.000 --> 00:00:04.500 Este material de buceo 2 00:00:04.600 --> 00:00:07.900 fue filmada en el cráter Molokini 3 00:00:08.000 --> 00:00:09.500 Maui, Hawaii

Each cue in the WebVTT file may have a number or a name. The interval in which the text should be displayed on the screen is described in hour, minute, second, and millisecond format.

Finally, I should also note that in some browsers, including subtitles will add a closed-caption button to the video controls.

You can view the demo at this location for a working version in Chrome, or view the CodePen example below for one that works in Firefox.

See the Pen HTML5 Video with Subtitles by SitePoint (@SitePoint) on CodePen.

For a more comprehensive look at these features, check out Ankul Jain’s article covering HTML5’s track element.

Dynamic Thumbnails with Canvas

A significant advantage for using HTML5 video is that it can interact with other HTML elements in ways that third-party plugins cannot.

As an example, in 2010, Pete LePage, who works in developer relations for Google, described how to use HTML5 video and canvas together.

In LePage’s example, a video is added to the HTML document, a canvas element is created, and then the screen image is captured every five seconds and displayed on the screen. Here’s the relevant part of the HTML:

<video id="thumb" controls preload="metadata" width="750px" height="540px"> <source src="102614-video-sample.mp4" type='video/mp4;codecs="avc1.42E01E, mp4a.40.2"'> <source src="102614-video-sample-webmhd.webm" type='video/webm;codecs="vp8, vorbis"'> </video> <canvas id="canvas" width="750px" height="540px" style="display:block;"> </canvas> <div id="screenShots"></div>

The JavaScript from LePage’s demonstration includes several event listeners, variables, and functions:

var video = document.getElementById("thumb"); video.addEventListener("loadedmetadata", initScreenshot); video.addEventListener("playing", startScreenshot); video.addEventListener("pause", stopScreenshot); video.addEventListener("ended", stopScreenshot); var canvas = document.getElementById("canvas"); var ctx = canvas.getContext("2d"); var ssContainer = document.getElementById("screenShots"); var videoHeight, videoWidth; var drawTimer = null; function initScreenshot() { videoHeight = video.videoHeight; videoWidth = video.videoWidth; canvas.width = videoWidth; canvas.height = videoHeight; } function startScreenshot() { if (drawTimer == null) { drawTimer = setInterval(grabScreenshot, 1000); } } function stopScreenshot() { if (drawTimer) { clearInterval(drawTimer); drawTimer = null; } } function grabScreenshot() { ctx.drawImage(video, 0, 0, videoWidth, videoHeight); var img = new Image(); img.src = canvas.toDataURL("image/png"); img.width = 120; ssContainer.appendChild(img); }

In the demo, the canvas element is set to display: none, which means we only see the resized thumbnails, not the original canvas image. The demonstration can take a moment to load, but it does show how relatively simple it can be to get HTML5 video to work with other HTML elements.

View the dynamic thumbnails demo here

Conclusion

So that’s a summary of 3 HTML5 video features maybe you haven’t used yet. If you know of any other interesting and little-known tips on HTML5 video, we’d love to hear about them in the comments.

Credits: Music used in the example videos is Thaiz Itch’s “Etude No.5 – 5. SA-GA-MA-PA-NI-SA”. Video is from the my recent trip to Maui, Hawaii.

Frequently Asked Questions (FAQs) about HTML5 Video Fragments, Captions, and Dynamic Thumbnails

How can I set a thumbnail image for my HTML5 video?

To set a thumbnail image for your HTML5 video, you can use the ‘poster’ attribute in the ‘video’ tag. The ‘poster’ attribute specifies an image to be shown while the video is downloading, or until the user hits the play button. If this attribute isn’t included, the first frame of the video will be used as the thumbnail. Here’s an example of how to use it:<video width="320" height="240" poster="thumbnail.jpg"> <source src="movie.mp4" type="video/mp4"></video>In this example, ‘thumbnail.jpg’ is the image that will be used as the thumbnail for the video.

How can I make my HTML5 video start only when clicking on a thumbnail image?

To make your HTML5 video start only when clicking on a thumbnail image, you can use JavaScript to control the play and pause functions of the video. First, you need to hide the video and display only the thumbnail image. When the thumbnail image is clicked, the image is hidden, and the video is displayed and played. Here’s an example of how to do it:<img id="thumbnail" src="thumbnail.jpg" onclick="playVideo()"><video id="myVideo" width="320" height="240" style="display:none;"> <source src="movie.mp4" type="video/mp4"></video><script>function playVideo() { var thumbnail = document.getElementById("thumbnail"); var video = document.getElementById("myVideo"); thumbnail.style.display = "none"; video.style.display = "block"; video.play();}</script>In this example, when the ‘thumbnail.jpg’ image is clicked, the ‘playVideo()’ JavaScript function is called. This function hides the thumbnail image, displays the video, and starts playing the video.

How can I add captions to my HTML5 video?

To add captions to your HTML5 video, you can use the ‘track’ element in the ‘video’ tag. The ‘track’ element is used to specify subtitles, caption files or other files containing text, that should be visible when the video is playing. Here’s an example of how to use it:<video width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"> <track src="captions.vtt" kind="captions" srclang="en" label="English"></video>In this example, ‘captions.vtt’ is the file that contains the captions for the video. The ‘kind’ attribute specifies the kind of text track, ‘srclang’ specifies the language of the text track, and ‘label’ provides a title for the track list.

How can I create dynamic thumbnails for my HTML5 video?

Creating dynamic thumbnails for your HTML5 video involves capturing frames from the video at specific intervals and displaying them as thumbnails. This can be achieved using JavaScript and the ‘canvas’ element. The ‘canvas’ element is used to draw graphics on a web page. Here’s an example of how to do it:<video id="myVideo" width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"></video><canvas id="canvas" width="320" height="240"></canvas><script>var video = document.getElementById("myVideo");var canvas = document.getElementById("canvas");var context = canvas.getContext("2d");video.addEventListener("timeupdate", function() { context.drawImage(video, 0, 0, canvas.width, canvas.height);});</script>In this example, every time the ‘timeupdate’ event is fired, a frame from the video is drawn on the canvas, creating a dynamic thumbnail.

How can I add multiple caption tracks to my HTML5 video?

To add multiple caption tracks to your HTML5 video, you can use multiple ‘track’ elements in the ‘video’ tag. Each ‘track’ element specifies a different caption file. The user can select which caption track to display from the video player’s interface. Here’s an example of how to do it:<video width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"> <track src="captions_en.vtt" kind="captions" srclang="en" label="English"> <track src="captions_fr.vtt" kind="captions" srclang="fr" label="French"></video>In this example, ‘captions_en.vtt’ and ‘captions_fr.vtt’ are the files that contain the English and French captions, respectively.

How can I control the playback rate of my HTML5 video?

The playback rate of an HTML5 video can be controlled using the ‘playbackRate’ property of the ‘video’ element. This property sets the speed of the video playback. For example, a playback rate of 1.0 plays the video at normal speed, 0.5 plays the video at half speed, and 2.0 plays the video at double speed. Here’s an example of how to use it:<video id="myVideo" width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"></video><button onclick="slowDown()">Slow Down</button><button onclick="speedUp()">Speed Up</button><script>var video = document.getElementById("myVideo");function slowDown() { video.playbackRate -= 0.1;}function speedUp() { video.playbackRate += 0.1;}</script>In this example, clicking the ‘Slow Down’ button decreases the playback rate by 0.1, and clicking the ‘Speed Up’ button increases the playback rate by 0.1.

How can I loop my HTML5 video?

To loop your HTML5 video, you can use the ‘loop’ attribute in the ‘video’ tag. The ‘loop’ attribute is a boolean attribute. When present, it specifies that the video will start over again, every time it is finished. Here’s an example of how to use it:<video width="320" height="240" controls loop> <source src="movie.mp4" type="video/mp4"></video>In this example, the video will loop indefinitely.

How can I mute my HTML5 video?

To mute your HTML5 video, you can use the ‘muted’ attribute in the ‘video’ tag. The ‘muted’ attribute is a boolean attribute. When present, it specifies that the audio output of the video should be muted. Here’s an example of how to use it:<video width="320" height="240" controls muted> <source src="movie.mp4" type="video/mp4"></video>In this example, the video will start playing with the audio muted.

How can I control the volume of my HTML5 video?

The volume of an HTML5 video can be controlled using the ‘volume’ property of the ‘video’ element. This property sets the volume of the audio playback. The value must be a number between 0.0 and 1.0, where 0.0 is silent and 1.0 is the loudest. Here’s an example of how to use it:<video id="myVideo" width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"></video><button onclick="turnDown()">Turn Down</button><button onclick="turnUp()">Turn Up</button><script>var video = document.getElementById("myVideo");function turnDown() { if (video.volume > 0.1) { video.volume -= 0.1; }}function turnUp() { if (video.volume < 1.0) { video.volume += 0.1; }}</script>In this example, clicking the ‘Turn Down’ button decreases the volume by 0.1, and clicking the ‘Turn Up’ button increases the volume by 0.1.

How can I display controls for my HTML5 video?

To display controls for your HTML5 video, you can use the ‘controls’ attribute in the ‘video’ tag. The ‘controls’ attribute is a boolean attribute. When present, it specifies that video controls should be displayed. These controls include play/pause, volume, and fullscreen. Here’s an example of how to use it:<video width="320" height="240" controls> <source src="movie.mp4" type="video/mp4"></video>In this example, the video will be displayed with controls.