Sideloaded VTT shows double duplicate captions strings on chromecast - chromecast

On my custom cast receiver app, in my mediaManager.onEditTracksInfo callback, I enable/disable sideloaded captions as below:
mediaManager.onEditTracksInfo = function (event) {
if (!event.data || !event.data.activeTrackIds) {
return;
}
// if sideloaded tracks are available, show/hide those
if (sideloadedTracksAvailable) {
updateSideloadedTracksVisibility(event);
} else {
updateEmbeddedTracksVisibility(event);
}
onEditTracksInfoOrig(event);
};
updateSideloadedTracksVisibility = function (data) {
var mediaInformation = mediaManager.getMediaInformation() || {};
// disable currently enabled sideloaded TTML or VTT, if any
mediaPlayer.enableCaptions(false, cast.player.api.CaptionsType.TTML);
mediaPlayer.enableCaptions(false, cast.player.api.CaptionsType.WEBVTT);
enableActiveTracks(data.activeTrackIds, mediaInformation.tracks || []);
}
enableActiveTracks = function (activeTrackIds, tracks) {
// loops over tracks and if requested to be enabled calls
// mediaPlayer.enableCaptions(true, trackType, tracks[i].trackContentId);
}
Below is is what the <video> element looks like in receiver app DOM. As soon as sideloaded tracks are loaded into mediaManager in onLoad(), a track element is added to DOM with src set to url of first track in the tracks-array. Then whenever I call editTracksInfo with activeId of a track, a new track element gets appended to <video> eleement and gets removed whenever I turn captions off.
<video id="videoEl" src="blob:http%3A//blob-url-here" cast-captions-879553="true" crossorigin="anonymous">
<track src="http://vtt/url/here/file.vtt" id="1" label="undefined" srclang="en-US" kind="subtitles"></track>
<track src kind="captions"> <!-- this gets added upon enabling a track, and removed when disabling tracks -->
</video>
Captions are working fine but I notice that whenever I switch to a sideloaded VTT trackId, chromecast shows two caption streams with same content. Is this a known issue or is my code doing something wrong? It only happens with sideloaded vtt tracks. Sideloaded TTML and embedded vtt look fine. I checked the contents of sideloaded vtt file, and it doesnt have duplicate captions strings.

So I finally figured this out. Apparently Media Player Library handles sideloaded VTT tracks on its own. When using MPL, in the overridden onEditTracksInfo event handler, we only have to take care of enabling/disabling embedded TTML/VTT and sideloaded TTML. Sideloaded VTT is handled automatically by MPL upon calling default event handler (onEditTracksInfoOrig). This also explains why a <track> element with vtt url was automatically appended to <video> element. This element was created by MPL. This caused double captions to show because I and MPL both were enabling the VTT track. I changed my code as below:
enableActiveTracks = function (activeTrackIds, tracks) {
// loops over tracks and if enable TTML track if requested
if (trackType === 'ttml') {
mediaPlayer.enableCaptions(true, trackType, tracks[i].trackContentId);
}
}
Unfortunately, this is not documented in MPL documentation or custom receiver documentation. The documentation needs to be updated to make this information easy to find.

Related

Cypress click on html5 video buttons (play/pause, mute, scroll video, fullscreen)

I am trying to click on the options like play/pause, mute, fullscreen etc on an html5 video element. But the problem is these are not available in the DOM for me to select the element. I tried to do click using coordinates but it is not working. Also cypress do not support keyboard actions else I could have used Tab and spacebar to perform the actions. Can someone please suggest a solution to perform the actions on an html5 video element?
Sample html5 video
https://www.w3schools.com/html/html5_video.asp
For mute and play you can use this:
it('html5 video', () => {
cy.visit("https://www.w3schools.com/html/html5_video.asp");
cy.get('#accept-choices').click();
cy.get('#video1').then((video) => {
const element = video.get(0);
element.muted = true;
element.play();
return video;
})
});
The return video it's optional.

Kaltura Flavor Switching

I am currently using Kaltura HTML5 Player ver. 2.26. The documentation suggests that you can switch between video flavors via the "doSwitch" notification, like so:
kdp.sendNotification("doSwitch", { flavorIndex: 3 });
The video I am using has 6 different flavors according to kdp.evaluate("{mediaProxy.kalturaMediaFlavorArray}"), but running this with various different indices has no discernible effect. I would expect to see the kdp fire a switchingChangeStarted event, as happens when using the Source Selector plugin UI, but there's just silence.
Searching through the github repo for doSwitch, I don't actually see it implemented anywhere. Is this some method a lost relic? If not, how do I get the doSwitch notification to work?
KDP is the Kaltura flash player which has this notification for switching bitrates.
The notification is still being used internally when the chromeless flash player is loaded and the source selector button is clicked. But it doesn't look like a V2 player notification.
You could extend the player by adding a new plugin that will expose such a notification that will switch the source similar to how the source selector is doing it (sourceSelector.js::208):
_this.getPlayer().switchSrc( source )
Note that the list of sources could have sources that are not playable on desktop, you should not use those for switching.
For the sake of posterity, here is what I ended up doing, following Roman's suggestion. The plugin below is pretty much the bare minimum, but this does precisely what I want it to do.
In the embed, we need to declare the custom plugin:
kWidget.embed({
...
"flashvars": {
...
"sourceExposure": {
"plugin": true,
"iframeHTML5Js": "js/sourceExposure.js"
}
}
}
In js/sourceExposure.js, we need to declare a plugin that provides a response to a custom event (here, "customDoSwitch"):
(function(mw,$) {
mw.kalturaPluginWrapper(function() {
mw.PluginManager.add( 'sourceExposure', mw.KBaseComponent.extend({
setup: function() {
var _this = this;
this.bind('customDoSwitch', function(evt, flavorIndex) {
var sources = _this.getSources().slice(0)
if (flavorIndex >= sources.length) {
_this.log("Flavor Index too large.");
return;
}
_this.getPlayer.switchSrc(_this.getSources()[flavorIndex]);
})
},
getSources: function() {
return this.getPlayer().getSources()
}
}));
});
})(window.mw, window.jQuery)
When we want to switch to a different flavor, we can now use the custom event and pass the flavor index:
kdp.sendNotification("customDoSwitch", 2) //switches to flavor index 2

Loading Profile Images on VK.com in ActionScript

I have an iframe application in vk.com. I can use their API everything looks fine but when I want to load profile images I get Security Sandbox Error. When I print the result and errors I get This: (I am using Greensock ImageLoader)
MYURL : 'MY Image URL on cs408919 subdomain of vk'
Loading CrossDomain on cs408919
ScriptAccessDenied : Error #2048
SecurityError : Error #2048
Error : Error #2048
ScriptAccessDenied : Error #2123 Security SandBox Violation, No Policy Files Granted Access
It seems to me crossdomain.xml issue but I couldn't find right one. Thanks...
In additional to fsbmain's answer I want say you have to add following code:
Security.allowDomain("*");
Yes, it's crossdomain issue, vk sub-domains for images doesn't provide crossdomain.xml for user avatars, but you still able to load (and add to display list as well) them. What you can't do is to access the loaded content (and set smooth bitmap flag for example, or draw the hole stage with vk images on it).
If you need the access the content you can use this "policy-hack", but it's hack, so it can be fixed in any FP update (I guess even this answer may bring closer this moment:) ):
The idea is to listen the ADDED event if image Loader:
protected var _prepareloaderBitmap:Bitmap;
_prepareloader.addEventListener(Event.ADDED, onPrepareLoader);
_prepareloader.contentLoaderInfo.addEventListener(Event.COMPLETE, onPrepareLoader);
And the listener:
protected function onPrepareLoader(event:Event):void
{
//event ADDED fired only for Bitmap (not for SWFs)
if(event.type == Event.ADDED)
{
_prepareloaderBitmap = event.target as Bitmap;
}
else if (event.type == Event.COMPLETE)
{
if(_prepareloaderBitmap)
{
trace("loaded image size:", _prepareloaderBitmap.width, "x", _prepareloaderBitmap.height);
}
}
}
Having the reference to the loaded Bitmap you can now add it instead of crossdomain issued loader.

Firefox doesn't fire error event on <audio> tag or display fallback text

I'm using the <audio> tag to play audio files across a number of browsers.
var audioTag = document.createElement("audio"),
sourceTag = document.createElement("source"),
sorryTag = document.createElement("div");
sorryTag.innerHTML = "This filetype not supported";
audioTag.onerror = function() {
//some error handling code
}
sourceTag.onerror = function() {
/some error handling code
}
sourceTag.src = "myfile.mp3";
audioTag.appendChild(sourceTag);
audioTag.appendChild(sorryTag);
//add audioTag to DOM
This leads to
<audio>
<source src='myfile.mp3' />
<div>This filetype not supported</div>
</audio>
Firefox can't play MP3 files, and I'm OK with that. Mozilla also promises that an error event will be dispatched if the <audio> or <video> tag can't play the media. And also it will go through the tags nested inside the media tag one by one (<source> or others, the last presumably being an error message) till it finds one it can work with. None of these seem to work for me; the error event is never fired on the elements nor is the error message displayed. What am I doing wrong?
The workaround I found was:
var audioTag = document.createElement("audio"),
sourceTag = document.createElement("source");
//Add error event listeners for browsers other than FF
audioTag.onerror = function() {
console.log("file can't be played. error from audio tag");
}
sourceTag.onerror = function() {
console.log("file can't be played. error from source tag");
}
//The only way to tell that file failed to play on FF
//Timeout is because audioTag.networkState === audioTag.NETWORK_NO_SOURCE
//on IE till it starts downloading the file
setTimeout(function() {
if(audioTag.networkState === audioTag.NETWORK_NO_SOURCE) {
console.log("this hack is only for <audio> on FF.");
console.log("Not for <video> and on no other browsers");
}
}, 3000);
sourceTag.src = "<file_url>";
audioTag.appendChild(sourceTag);
Basically, create the media and source tags, add error handlers, then append the source tag to the media tag and if the error event fires, then you know the file is unplayable.
On FF, the error event doesn't fire and you have to rely on the networkState flag of the <audio> element, comparing it to NETWORK_NO_SOURCE. You can't inspect it immediately after setting the src attribute of the <source> element because on IE networkState === NETWORK_NO_SOURCE till the browser actually starts downloading the file. For this reason, set a timeout of about 3 seconds (it's not an exact science) before checking the flag value and there's a good chance that you will have given IE enough time to determine if it's capable of playing the file.
UPDATE
Wrote a test case for this: http://jogjayr.github.com/FF-Audio-Test-Case/ but the error event fires OK there. Guess I was wrong; that or it was broken on FF14 (which I was using at the time), because the error event fires OK in my application too. Thanks #BorisZbarsky

mediaelement.js multi video gallery flash fallback

Does anyone have a URL of a successful multivideo gallery using mediaelement.js where the same instance of MediaElementPlayer is reused and have it working with the Flash fallback for IE8 & 7?
I have partial success changing the setSrc as a function after the new MediaElementPlayer is created for the first time. This is robust for the HTML5 component but failing for the flash fallback. setSrc is consoling as not available in IE8 & 7. It fails to recognise the object.
Moving the setSrc to the "success" part of the new MediaElementPlayer does load the Flash fallback and HTML5 video as expected. On attempting to change the source of the player I have attempted to "destroy" and recreate the MediaElementPlayer object on the fly without success. I have not declared player using var=player so reasonably have expected to delete it but without success:
player = false;
delete player;
//make a new instance of the mediaelement video player
player = new MediaElementPlayer('#videoPlayer', {
pluginPath: ''+basePath+'_Includes/JS/',
success: function (player, node) {
//set the size (for flash otherwise no video just sound!)
if($("#rg-gallery.smallGallery").length > 0){
player.setVideoSize(400, 225);
} else{
player.setVideoSize(640, 360);
}
player.setSrc([
{ src: mp4, type: 'video/mp4' },
{ src: webm, type: 'video/webm' }
]);
player.load();
player.pause();
//if the video plays stop the slideshow
player.addEventListener('play', function () {
videoPlaying.push('playing');
stopSlideshow();
}, false);
}
});
Research both here and on the web shows that others are attempting to try this type of dynamic gallery but I am unable to find an example that shows it as technically viable.
Any example URLs where someone's cracked it would be lovely. Thanks :)
Solved.
Created a JS variable of video code:
var playerView = '<video width="640" height="360" id="videoPlayer" class="video-js vjs-default-skin" poster="" controls preload="auto"><source id="mp4" type="video/mp4" src="" /><source id="webm" type="video/webm" src="" /></video>';
On initialization removed any DOM rendering of any existing player, set it to false and deleted it:
//remove any existing video player
$(".mejs-container").remove();
player = false;
delete player;
Appended new video view to DOM:
//add a new one
$(".rg-video").append(playerView);
Created new instance of player and set src and load as normal:
//make a new instance of the mediaelement video player
player = new MediaElementPlayer('#videoPlayer', { ...
HTML5 video and Flash video fallback for IE8 & 7 now working as part of dynamic mixed media gallery.
Off for tea and medals.
:)

Resources