I have a big problem when making a screen share demo between chrome browser depends on WebRTC.
Environment：MacBookPro macOS 10.13.3, chrome: 64.0.3282.119
In order to get screen media sourceId, I also make a chrome extention, the background-script.js code as follows:
var screenOptions = ['screen', 'window'];
......
chrome.desktopCapture.chooseDesktopMedia(screenOptions, port.sender.tab, onAccessApproved);
When get media sourceId in my html javascript, a request screen media as follows:
var constraints = {
video: {
mandatory: {
chromeMediaSource: 'desktop',
maxWidth: 1280,
maxHeight: 720,
maxFrameRate: 30,
minAspectRatio: 1.77,
chromeMediaSourceId: sourceId
}
}
};
navigator.mediaDevices.getUserMedia(constraints).then(gotLocalStream).catch(function (e) {
alert('getUserMedia() error: ' + e.name);
});
The problem comes: when I inspect WebRTC stats in chrome://webrtc-internals, I found the googFrameRateInput was very low (<=10) when I play a video in main display, while googFrameRateInput can reach 30 when a do nothing in main display.
googFrameRateInput
I also try this in windows chrome, googFrameRateInput can reach 24~30 even when play a video.
Why googFrameRateInput is too low in mac chrome?
Thank a lot.
Updating electron 6.0.2 solved the issue.
Related
I am trying to capture a screen image from a video, which src is on a different server. So far I got to blank image, which as far as I understand is expectable, because of the tainted canvas or CORS policy error if I give the video tag 'anonymous' crossorgin property. I am using html2canvas, but also tried and with simple canvas context like this:
let ctx = canvas.getContext('2d');
ctx.drawImage( video, 0, 0, canvas.width, canvas.height );
let image = canvas.toDataURL('image/jpeg');
and with html2canvas, trying multiple combinations of options with Vue proxy configurated to the server of the video:
let el = this.$refs.videoPlayer
el.setAttribute('crossOrigin', 'anonymous')
this.output = (
await html2canvas(el, {
allowTaint: true,
useCORS: true,
proxy: '/two',
async: true,
foreignObjectRendering: true,
})
).toDataURL()
Is there any workaround around this? Because I have it working with a local video and now this is quite unfortunate. Is it possible to first download the video on the user system and then play it in order to take the screenshot?
Edit:
The only working workaround I found is to download the video as Blob via axios and then use it, which works perfectly, but uses the user cache for the video.
We have an app published in the Teams App Store which uses task module and tabs.
On Task/Tab pages we use Teams API lib to work with media (camera/gallery). Camera and gallery API works fine if we limit users to only one image to pick.
On Android everything works great (doesn't matter how many media files you pick).
Strange things happen when we let IOS users to pick/make more than one image at a time.
There are two possible scenarios on IOS devices:
Teams App crashes.
Teams API returns attachments.length !== blobs.length.
I did some tests on different IOS Devices. Both have issues. I've updated the app to the latest and it got even worse (started to crash more often):
iPhone Xs (IOS 15.1).
iPhone 11 Pro Max (IOS 15.1).
Steps to reproduce blob issue:
Create an HTML page and load it into Task module.
Trigger microsoftTeams.media.selectMedia with maxMediaCount: 10.
Pick up to 10 small size images.
Save received attachments.length.
Request a blob for each attachment.
Calculate how many blobs and attachments you've got.
Expected: blobs.length === attachments.length.
Result: blobs.length !== attachments.length.
The issue described above appears randomly, sometimes you get expected values, sometimes you don't, sometimes it crashes even with small images.
How to crash IOS Teams App, Steps to reproduce:
Create an HTML page and load it into Task module.
Trigger microsoftTeams.media.selectMedia with maxMediaCount: 10.
Pick up to 10 big size images.
Expected: "Error 10000, Selected file is too big".
Result: Teams App crashes.
My code:
const MEDIA_MAX_FILES = 10;
const imageProp: microsoftTeams.media.ImageProps = {
sources: [microsoftTeams.media.Source.Gallery],
startMode: microsoftTeams.media.CameraStartMode.Photo,
ink: false,
cameraSwitcher: true,
textSticker: true,
enableFilter: true,
};
const mediaInput: microsoftTeams.media.MediaInputs = {
mediaType: microsoftTeams.media.MediaType.Image,
maxMediaCount: MEDIA_MAX_FILES,
imageProps: imageProp
};
const attachementsList = [];
const blobs = [];
microsoftTeams.media.selectMedia(mediaInput,(error: microsoftTeams.SdkError,
attachments: microsoftTeams.media.Media[]) => {
if (error) { /* block is truncated */ return; }
if (attachments) {
attachementsList.push(...attachments);
for (const attachment of attachments) {
attachment.getMedia((attachmentError: SdkError,
blob: Blob) => {
if (blob) {
blobs.push(blob);
}
});
}
}
});
Teams App build versions. Dev Preview is turned on on both devices:
iPhone Xs: 3.20.77.2021202701/1121
iPhone 11 Max Pro: 3.20.77.2021202701/1121
Is this a bug ? Or am I doing something wrong ? Thanks in advance.
I am using OpenTok v2.14.0.0, for screen share. Streaming etc are working seamlessly. The only problem is, on the subscriber side, I am not able to see the complete Publisher's screen. On resizing the subscriber window (in which the streaming is happening), the video inside stretches maintaining the aspect ratio.
I have tried changing my WPF UI element control to Grid and UniformGrid, but did not work. Hence, I believe this has something to do with my subscriber's configurations. On js side, there is a property fitMode, I am looking for a similar setting on windows side.
Here is the video where one can see that the publisher (he is publishing his chrome window), gets stretched outside the bounds.
xaml control:
<Grid x:Name="SubscriberGrid"/>
when I receive the stream:
var uiElement = ((UIElement)subscriber.VideoRenderer);
SubscriberGrid.Children.Add(uiElement);
I had this issue and for me it was because I had set the fitMode property to 'cover' instead of 'contain'. Changing fitMode to 'contain' fixed the issue:
var subscriber = session.subscribe(event.stream, 'subscribers', {
insertMode: 'append',
width: "100%",
height: "100%",
fitMode: "contain"
}, function (error) {
if (error) {
console.error('Failed to subscribe', error);
}
});
I am making an app where an photo is taken and sent to the server, but the files that get sent are of low quality. It seems that when I take a photo using iOS native camera app it come out as a very sharp picture, but when using PhoneGap it reduces the quality massively.
I have set the image quality to 100 etc:
function capturePhoto() {
// Take picture using device camera and retrieve image as base64-encoded string
navigator.camera.getPicture(onPhotoURISuccess, onPhotoURIFail, {
quality: 100,
destinationType: Camera.DestinationType.FILE_URI,
//destinationType: Camera.DestinationType.DATA_URL,
//destinationType: Camera.DestinationType.NATIVE_URI,
sourceType : Camera.PictureSourceType.CAMERA,
correctOrientation: true,
saveToPhotoAlbum: true,
encodingType: Camera.EncodingType.PNG
});
}
But still seems to give a real blurry image.
Has anyone else had this problem? I have also tried using the DATA_URL to get the base64 encoded image, but this makes the app go so slow that I couldn't get to a stage to send to the server.
Is there a way to use the original LOSSLESS file?
Thanks
function capturePhoto() {
navigator.camera.getPicture(onPhotoURISuccess, onPhotoURIFail, {
quality: 50,
destinationType: Camera.DestinationType.FILE_URI,
sourceType : Camera.PictureSourceType.CAMERA,
correctOrientation: false,
encodingType: Camera.EncodingType.PNG,
targetWidth: 1500,
targetHeight: 1500
});
}
Im not sure why mine wouldn't work before, but it may have been because I was trying it on my iPad? I can't quite remember if I had the same problem with my iPhone also at that time.
Also the way the imageURI is retrieved may have made a difference, not sure about that either, but here is my code to get the image URI:
function onPhotoURISuccess(imageURI){
$('#imagePreview img.smallImage').attr("src", imageURI);
}
And then just get the URI from the image when you want to send. It prety much exactly like all the examples but for some reason didn't work before.
Make sure you have your plugins using the latest version too. That's about it!
Not sure what would be causing this, but when I upload some images to my remote server via FileTransfer(), the images sometimes show up either sideways or upside down. However, when I view the images locally on the iPhone, they are positioned in the correct way.
For example, when I select an image like this to upload: http://sharefa.st/view/WBe2QNSK8r8z
It will turn out like this: http://sharefa.st/view/EWdW1Z4G8r8z
I am using the local path to transfer the file, so I don't understand why the image would rotate "randomly".
Here is my upload function:
function uploadPhoto() {
var options = new FileUploadOptions();
options.fileKey = 'file';
options.fileName = imgURI.substr(imgURI.lastIndexOf('/')+1);
options.mimeType = 'image/jpeg';
var params = new Object();
if(logged_in == true) {
params.unique_id = app_unique_id;
params.secret_key = user_secret_key;
}
options.params = params;
loadingStart();
var ft = new FileTransfer();
ft.upload(imgURI, 'http://' + remote_server + '/API/upload', uploadDetails, fail, options);
}
imgURI value looks like this:
file://localhost/var/mobile/Applications/<snip>/tmp/photo_015.jpg
Any insight is appreciated.
Thanks to Humanoidism pointing out that the issue was in fact with the iPhone, and the way it stored images, I was able to figure out a solutuion.
To upload photos in the correct orientation you must add the correctOrientation setting to the options array in getPicture(), and set it to true.
Here are two examples:
function capturePhoto() {
navigator.camera.getPicture(onPhotoDataSuccess, onFail, { quality: 30, correctOrientation: true });
}
function getPhoto(source) {
navigator.camera.getPicture(onPhotoURISuccess, onFail, { quality: 30,
destinationType: destinationType.FILE_URI,
sourceType: source,
correctOrientation: true });
}
The problem is not PhoneGap but iPhone. The iPhone was designed to be used as a wide lens camera. Turn the phone sideways when taking pictures or capturing video if you intend to view them on desktop. Your phone will display them correctly, because it "knows" how you took them, but the computer that you're viewing it on dosen't.
What you could do to prevent this is to rotate the picture before the upload. This is not a recommended fix but at least people on desktop computers will be able to see it. Though when viewing them on iPhone they'll be rotated - maybe a check for mobile devices wether or not to rotate the image could come in handy - but still again, not recommended.