Skip to content

v3.21.0
Compare
Choose a tag to compare
@grenault73 grenault73 released this 17 Jun 16:18
v3.21.0
5a5c242
This commit was signed with the committer’s verified signature. The key has expired.
peaBerberian Paul Berberian
GPG key ID: 2EB79512E0216466
Expired
Learn about vigilant mode.

Release v3.21.0 (2020-06-17)

Overview

The v3.21.0 brings multiple features amongst which:

  • the RxPlayer now handle DASH EventStream's Event elements to emit timed events defined in a DASH MPD
  • it is now possible to apply track preferences to the content(s) / Period(s) that are already playing.
  • WebVTT "settings" are now considered when parsing subtitles in the html textTrackMode, thanks to @mattiaspalmgren for the contribution! Before, we only relied on WebVTT's CSS extensions to allow stylization of subtitles in this mode.
  • the RxPlayer will now emit "warning" events when it detects minor errors in a DASH MPD, such as optional data in the wrong format.
  • and a lot of other bug fixes and improvements - you can look at the changelog for more information.

Support of DASH EventStream's Event elements

What stream events are for

What we call "stream events" here is metadata synchronized with the content being played. There are several reasons why you might want to include those.

At CANAL+ for example, we started to study the implementation of ad-insertion and tracking solutions for live contents. Ad-insertion can work on the server-side by generating an MPD which will replace media segments by ads, for example by adding Period elements.

On the client-side (the media player), we then may want to know where an ad can be encountered and when the final user begins to see it (or when he/she skip it) for tracking purposes.

Here, DASH EventStream's <Event> elements can be used.
Such elements define timed events associated with the current content. When the user begins to play at a position within the time boundaries of that event, the corresponding Event element will be considered.
This element can contain specific user-defined information, such as the URL of an ad-tracking server.

But DASH <Event> elements are not restricted to advertising use cases, you could use one anytime you want to be notified when an user is at a particular position in the stream.

How the RxPlayer handle DASH EventStream elements

DASH's <EventStream> are elements of the MPD, inside the corresponding <Period> element.
Each EventStream can carry one or more <Event> elements, whose format and inner data depends on an attribute of the EventStream element called the schemeIdUri:

    <Period id="1">
      <EventStream schemeIdUri="urn:uuid:XYZY" timescale="1000" value="call">
        <Event presentationTime="20000" duration="10000" id="1">
          Some data
        </Event>
        <Event presentationTime="40000" duration="10000" id="2">
          Some other data
        </Event>
        <Event presentationTime="60000" duration="10000" id="3" />
      </EventStream>
      <!-- ... -->
    </Period>

Each Event element generally has an assiociated "presentation time" (the initial time the event should be triggered at), some specific data and, sometimes, a duration.

The RxPlayer acts like a passthrough with the Event's data.
Its only role here is to send "streamEvent" events when a user goes into the time boundaries of such events. By catching that event, an application can also be notified when the user "exits" the same event's time boundaries:

// Do something when the user enters a time area linked to an Event element
rxPlayer.addEventListener("streamEvent", (evt) => {
  console.log("Beginning of an event:", evt);

  // Catch the moment when we are exiting the event.
  // Note that only Event elements with a duration (and thus an end) can
  // have an "exiting" notion.
  if (evt.end !== undefined) {
    evt.onExit = () => {
      console.log("End of the event:", evt);
    };
  }
});

The user could also "skip" an event, like when seeking after it. As an application, you may want to know when that happens.

For this reason, the RxPlayer also reports when an event has been "skipped" through a new "streamEventSkip" event:

rxPlayer.addEventListener("streamEventSkip", (evt) => {
   console.log("An event has been skipped.", evt);
});

Other types of events

For now, only DASH EventStream are handled by the RxPlayer but several types of events exist.
DASH also has in-band events that are in the media containers themselves like for example in "emsg" ISOBMFF boxes.
Other streaming protocol have their own event format(s) as well.

We want to support more of them in future versions, the available APIs here were also written with those in mind.
The management of "emsg" boxes, for example, is something we are today considering to add in our next versions.

More information on those APIs

Because the concept and APIs could seem complex we have added both those APIs to our API documentation and added a new tutorial about it.

We advise people wanting to integrate stream event management to first read the tutorial and to then come back to the API documentation for checking what are the arguments awaited.

The tutorial is here.

And here is the documentation of the new APIs:

We also export two new types (which can be imported via "rx-player/types"):

  • IStreamEvent: the object sent with a streamEvent or streamEventSkip event
  • IStreamEventData: the value of the data attribute of an IStreamEvent.

Those are documented here.

Applying track preferences to the current content

The two APIs for track selection

When it comes to track selection in the RxPlayer, we have two set of APIs:

  • the "classic" track selection APIs - like getAudioTrack, setVideoTrack, getAvailableTextTracks - will tell you which tracks are available and allow you to choose a specific one from that list

  • the track preferences APIs - like setPreferredAudioTrack - allow to define a general wanted choice. For example, we could indicate that the RxPlayer should choose by default a french audio track and/or english subtitles if those are available.

The APIs in the first set list the available tracks, so the application can then choose a precize one.

The APIs in the second set let the RxPlayer do the choice itself based on criterias. They are also used in some internal RxPlayer optimizations, like when pre-loading a future content - where the right track to download is for now "guessed" by the RxPlayer.

The problem with track preferences

The track preferences APIs have thus many advantages, but they have a limit: they only apply preferences for future contents and periods.

The content which is currently played and the Periods (for DASH) or sub-contents (for MetaPlaylist) that we played previously will still remain with their last set track configuration.

This is by design, we didn't want to change the current tracks after a setPreferred... call because this could seem unexpected.
So we are left in what we could refer as a "hole" in our API:

  • We can define the track for the current Period (or sub-content) but this won't affect any other Periods
  • We can define a preference for new Periods but this won't affect the current one or the one that have already been played.

What if we just want to set a new global preference, that we want to apply retroactively (to the current and already-played Periods)?

We could call both APIs, but we would still be left with the previous Periods in their previous track configurations.

This problem can be seen for example when you want to enable an "audio-only" mode (by disabling the video track) for multi-period contents:

// The old way

// disable the current track
rxPlayer.disableVideoTrack();

// Update the preferences for the next content / periods
rxPlayer.setPreferredVideoTracks([null]);

But here seeking back to a previous period is going to display the video track again. To work-around that, we have to call disableVideoTrack() again when seeking back to them.

A solution

We did many drafts on how we could improve our track selection APIs so that all possible use-cases are handled.

Among those, we thought of giving "more power" to the classic track selection APIs, so they could also change the track for other Periods.
However, we found that that approach would necessitate a very complex API. For features as simple as applying a global preference, we thought that the API should stay simple.

In the end, we decided to add an optional boolean as a second argument to those track preferences methods:

When set to true that preference will be applied retroactively.

For example, disabling the video track globally now can simply be done this way:

rxPlayer.setPreferredVideoTracks([null], true);

A new tutorial on track selection

As you can see, the RxPlayer allows applications to have a complex track selection management. However, this can come at the cost of a complex API.

We are conscious of that and as such, we decided to add a tutorial page specifically on the RxPlayer's track selection APIs.
This tutorial can be found here.

WebVTT settings in the "html" textTrackMode

What is WebVTT

Together with TTML, WebVTT is a very popular standard and format used to display subtitles on a screen.
They indicate what text should appear in which part of the screen, at what time.

Those two formats can indicate a "rich" stylization for the subtitles they contain, by changing their color, placement, orientation and multiple other settings.

Until today: a poor WebVTT style handling

In the RxPlayer, this stylization is available when enabling the "html" textTrackMode which uses regular HTML elements to display subtitles (as opposed to the <track> element used in the "native" textTrackMode).

Unfortunately, the style settings of the WebVTT format, which are appropriately called settings, were ignored in that mode. It was still possible to have a complex style through the usage of the CSS extensions - which we support partially - but this is not the main way to add a style to WebVTT files.

This is because until now, we mostly concentrated our efforts on TTML for richer text track styles.

What we've done here

Thanks to an external contribution by @mattiaspalmgren, the RxPlayer should now handle most webvtt style configurations.

We didn't dive very far into some configurations yet, such as right-to-left or top-to-bottom text. Though this can be possible to implement for a relatively small effort, do not hesitate to open an issue and contribute for this if those are features you want.

"warning" events when encountering a malformed DASH MPD

An "MPD" is the manifest format used when playing DASH contents. It's a document indicating the different track, qualities and segments available and multiple other metadata related to a content.

One of the main job of the RxPlayer is to interpret this document's content, by "parsing" it.
Because it is in itself a regular XML document, it can happen that a perfectly "parse-able" MPD contains values in unexpected format. For example, we could obtain an empty string where we expected a date or we could have a number where we expected a boolean value.

When this happens, the RxPlayer emitted until now just a warning through its logging system (configurable through the RxPlayer.LogLevel property) and then ignored that property.

A simple log - which is hidden by default - will usually be invisible to applications. As this issue might still be an important problem you might want check, we thought that we weren't doing enough to alert applications there.

So we decided to now emit those errors through "warning" events.
Those warnings will have the already-existing PIPELINE_PARSE_ERROR code and a descriptive message on what the problem was.

Many other fixes and improvements

This release note begins to be very long but there's still a lot of other improvements on that release.

We worked on persistent licenses, and brought a lot of related improvements: re-adding the possibility to serialize the data given to a licenseStorage, setting a limit to the maximum number of stored persistent sessions and other fixes and improvements.

We also worked on the "local-manifest" feature, which allows to play "locally-stored" contents (most of the time in our tests, those are downloaded DASH/Smooth contents). Some bugs made download and watching a content at the same time not very practical.

And many other things. You can look at the changelog for all sensible improvements made in this version.

Changelog

Features

  • api/events: add "streamEvent" event for when a DASH EventStream's event is reached
  • api/events: add "streamEventSkip" event for when a DASH EventStream's event is "skipped"
  • types/events: add IStreamEvent and IStreamEventData - which define the payload emitted by both a "streamEvent" and "streamEventSkip" events to the exported types
  • api/tracks: add second argument to setPreferredAudioTracks, setPreferredTextTracks and setPreferredVideoTracks to be able to also apply them to the currently loaded Periods / content
  • text/webvtt: parse settings attributes of WebVTT subtitles when in HTML mode
  • api/tracks: add codec information to getAvailableAudioTracks and getAudioTrack

Bug fixes

  • dash: do not reduce the minimum position when using the manifestUpdateUrl transportOptions in loadVideo
  • local-manifest: consider language property from a "local" Manifest
  • local-manifest: refresh the Manifest even if we dont have a Manifest URL, as is often the case when playing locally-stored contents
  • local-manifest: allow the "expired" property of a local-manifest to be updated
  • compat/eme/fairplay: for fairplay contents, better format the initialization data given to the CDM. The previous behavior could lead to invalid license requests
  • eme: re-allow serialization into a JSON string of the persisted session data, as presented in the DRM tutorial
  • compat/low-latency: fix compilation of async/await when playing low-latency contents with the default bundled builds
  • eme: ensure that the previous MediaKeySystemAccess used had persistentState to "required" when a new content needs it to be
  • eme: fix closeSessionsOnStop keySystems option actually not removing any MediaKeySession when stopping a content (v3.20.1 regression).

Other improvements

  • dash: emit minor errors arising when parsing the DASH MPD through warning events (whose payload will be an error with the PIPELINE_PARSE_ERROR code)
  • dash: consider AdaptationSet@selectionPriority in our initial track choice if the user preferences lead to multiple compatible tracks
  • misc: do not download video segments when playing on an "audio" element.
  • eme: replace the MediaKeySession's cache entry based on the least recently used instead of on the least recently created to improve cache effectiveness.
  • eme/persistent sessions: Limit the maximum of stored persistent MediaKeySessions to 1000 to avoid the storage to grow indefinitely (higher than that, the least-recently used will be evicted)
Assets 4
Loading