How to convert SoftwareBitmap from Bgra8 to JPEG in Windows UWP. GetPreviewFrameAsync function is used to get videoFrame data in Bgra8. What is going wrong in the following code?. I am getting jpeg size 0.
auto previewProperties = static_cast<MediaProperties::VideoEncodingProperties^>
(mediaCapture->VideoDeviceController->GetMediaStreamProperties(Capture::MediaStreamType::VideoPreview));
unsigned int videoFrameWidth = previewProperties->Width;
unsigned int videoFrameHeight = previewProperties->Height;
FN_TRACE("%s videoFrameWidth %d videoFrameHeight %d\n",
__func__, videoFrameWidth, videoFrameHeight);
// Create the video frame to request a SoftwareBitmap preview frame
auto videoFrame = ref new VideoFrame(BitmapPixelFormat::Bgra8, videoFrameWidth, videoFrameHeight);
// Capture the preview frames
return create_task(mediaCapture->GetPreviewFrameAsync(videoFrame))
.then([this](VideoFrame^ currentFrame)
{
// Collect the resulting frame
auto previewFrame = currentFrame->SoftwareBitmap;
auto inputStream = ref new Streams::InMemoryRandomAccessStream();
create_task(BitmapEncoder::CreateAsync(BitmapEncoder::JpegEncoderId, inputStream))
.then([this, previewFrame, inputStream](BitmapEncoder^ encoder)
{
encoder->SetSoftwareBitmap(previewFrame);
encoder->FlushAsync();
FN_TRACE("jpeg size %d\n", inputStream->Size);
Streams::Buffer^ data = ref new Streams::Buffer(inputStream->Size);
create_task(inputStream->ReadAsync(data, (unsigned int)inputStream->Size, InputStreamOptions::None));
});
});
BitmapEncoder.FlushAsync() method is a asynchronous method. We should consume it like the following:
// Capture the preview frames
return create_task(mediaCapture->GetPreviewFrameAsync(videoFrame))
.then([this](VideoFrame^ currentFrame)
{
// Collect the resulting frame
auto previewFrame = currentFrame->SoftwareBitmap;
auto inputStream = ref new Streams::InMemoryRandomAccessStream();
return create_task(BitmapEncoder::CreateAsync(BitmapEncoder::JpegEncoderId, inputStream))
.then([this, previewFrame](BitmapEncoder^ encoder)
{
encoder->SetSoftwareBitmap(previewFrame);
return encoder->FlushAsync();
}).then([this, inputStream]()
{
FN_TRACE("jpeg size %d\n", inputStream->Size);
//TODO
});
});
Then you should be able to get the right size. For more info, please see Asynchronous programming in C++.
Related
I am using the following sample to resize the uploaded images with Blazor WebAssembly
https://www.prowaretech.com/Computer/Blazor/Examples/WebApi/UploadImages .
Still I need the original file too to be converted to base64 too and I don't know how can I access it...
I tried to find the file's original width and height to pass its to RequestImageFileAsync function but no success...
I need to store both files : the original one and the resized one.
Can you help me, please ?
Thank You Very Much !
The InputFile control emits an IBrowserFile type. RequestImageFileAsync is a convenience method on IBrowserFile to resize the image and convert the type. The result is still an IBrowserFile.
One way to do what you are asking is with SixLabors.ImageSharp. Based on the ProWareTech example, something like this...
async Task OnChange(InputFileChangeEventArgs e)
{
var files = e.GetMultipleFiles(); // get the files selected by the users
foreach(var file in files)
{
//Original-sized file
var buf1 = new byte[file.Size];
using (var stream = file.OpenReadStream())
{
await stream.ReadAsync(buf1); // copy the stream to the buffer
}
origFilesBase64.Add(new ImageFile { base64data = Convert.ToBase64String(buf1), contentType = file.ContentType, fileName = file.Name }); // convert to a base64 string!!
//Resized File
var resizedFile = await file.RequestImageFileAsync(file.ContentType, 640, 480); // resize the image file
var buf = new byte[resizedFile.Size]; // allocate a buffer to fill with the file's data
using (var stream = resizedFile.OpenReadStream())
{
await stream.ReadAsync(buf); // copy the stream to the buffer
}
filesBase64.Add(new ImageFile { base64data = Convert.ToBase64String(buf), contentType = file.ContentType, fileName = file.Name }); // convert to a base64 string!!
}
//To get the image Sizes for first image
ImageSharp.Image origImage = Image.Load<*imagetype*>(origFilesBase64[0])
int origImgHeight = origImage.Height;
int origImgWidth = origImage.Width;
ImageSharp.Image resizedImage = Image.Load<*imagetype*>(filesBase64[0])
int resizedImgHeight = resizedImage.Height;
int resizedImgWidth = resizedImage.Width;
}
In my Xamarin Forms app, I have an image under androidProject/Resources/drawable/myImage.png. To load these from Xamarin, you can simply do
Image myImage = new Image() { Source = ImageSource.FromFile("myImage.png") };
However, there is no way to draw an Image using NGraphics. Instead, NGraphics DrawImage(IImage) requires an IImage. As far as I can tell, there's no way to turn a Xamarin.Forms.Image into an NGraphics.IImage. In fact, the only way I could find to load IImage is
IImage myImage = Platform.LoadImage("myImage.png");
However, this doesn't work because under the hood this uses BitmapFactory.decodeFile(), which requires the absolute file path. And I couldn't find any way to get the absolute file path of a resource (if it even exists?)
So, how do I actually load and display an image using NGraphics?
NGraphics does not provide any helpers to load images from your Platforms Resource files.
You could do something as follows. However, it will add some overhead converting back and forth between bitmap -> stream -> bitmap.
Android:
Stream GetDrawableStream(Context context, int resourceId)
{
var drawable = ResourcesCompat.GetDrawable(context.Resources, resourceId, context.Theme);
if (drawable is BitmapDrawable bitmapDrawable)
{
var stream = new MemoryStream();
var bitmap = bitmapDrawable.Bitmap;
bitmap.Compress(Bitmap.CompressFormat.Png, 80, stream);
bitmap.Recycle();
return stream;
}
return null;
}
iOS:
Stream GetImageStream(string fileName)
{
using (var image = UIImage.FromFile(fileName))
using (var imageData = image.AsPNG())
{
var byteArray = new byte[imageData.Length];
System.Runtime.InteropServices.Marshal.Copy(imageData.Bytes, byteArray, 0, Convert.ToInt32(imageData.Length));
var stream = new MemoryStream(byteArray);
return stream;
}
return null;
}
However, you could go directly from Bitmap to BitmapImage on Android instead like:
BitmapImage GetBitmapFromDrawable(Context context, int resourceId)
{
var drawable = ResourcesCompat.GetDrawable(context.Resources, resourceId, context.Theme);
if (drawable is BitmapDrawable bitmapDrawable)
{
var bitmap = bitmapDrawable.Bitmap;
return new BitmapImage(bitmap);
}
return null;
}
And on iOS:
CGImageImage GetImageStream(string fileName)
{
var iOSimage = UIImage.FromFile(fileName);
var cgImage = new CGImageImage(iOSImage.CGImage, iOSImage.Scale);
return cgImage;
}
BitmapImage and CGImageImage implement IImage in NGraphics.
I am reading color frame from Kinect V2 sensor using Microsoft Kinect SDK v2. I am copying the frame data in a byte array, which is later on converted into EmguCV Image. Below is the snippet from the code-
// A pixel buffer to hold image data from the incoming color frame
private byte[] pixels = null;
private KinectSensor kinectSensor = null;
private ColorFrameReader colorFrameReader = null;
public KinectForm()
{
this.kinectSensor = KinectSensor.GetDefault();
this.colorFrameReader = this.kinectSensor.ColorFrameSource.OpenReader();
this.colorFrameReader.FrameArrived += this.Reader_ColorFrameArrived;
// create the colorFrameDescription from the ColorFrameSource using Bgra format
FrameDescription colorFrameDescription = this.kinectSensor.ColorFrameSource.CreateFrameDescription(ColorImageFormat.Bgra);
// Create a pixel buffer to hold the frame's image data as a byte array
this.pixels = new byte[colorFrameDescription.Width * colorFrameDescription.Height * colorFrameDescription.BytesPerPixel];
// open the sensor
this.kinectSensor.Open();
InitializeComponent();
}
private void Reader_ColorFrameArrived(object sender, ColorFrameArrivedEventArgs e)
{
using (ColorFrame colorFrame = e.FrameReference.AcquireFrame())
{
if (colorFrame != null)
{
FrameDescription colorFrameDescription = colorFrame.FrameDescription;
if (colorFrame.RawColorImageFormat == ColorImageFormat.Bgra)
colorFrame.CopyRawFrameDataToArray(pixels);
else
colorFrame.CopyConvertedFrameDataToArray(this.pixels, ColorImageFormat.Bgra);
//Initialize Emgu CV image then assign byte array of pixels to it
Image<Bgr, byte> img = new Image<Bgr, byte>(colorFrameDescription.Width, colorFrameDescription.Height);
img.Bytes = pixels;
imgBox.Image = img;//Show image in Emgu.CV.UI.ImageBox
}
}
}
The converted image is corrupted after zooming more than 25%. Please see below screenshots-
50% Zoom -
25% Zoom -
12.5% Zoom -
The < TColor> should be Bgra, you can confirm from the "pixels" bytes array.
edit:
As the intensity of the Kinect's color frame is 0, the image won't be visible, so I added this code to solve the issue, I hope there is a better (faster) way to solve it.
private void FixIntensity(byte[] p)
{
int i = 0;
while (i < p.Length)
{
p[i+3] = 255;
i += 4;
}
}
I'm receiving an image on a Metro app through network socket every 1 second, loading it in an array of bytes, then convert it to a BitmapImage and display it later. All of this work fine.
The image is changing constantly on the other side. For some reason, it throws an OutOfMemory exceptions from now and then(like 1 in 10) . I fixed it by clearing the array of bytes every time the image is received. Now it works like charm.
See below for my main issue:
public static BitmapImage imag;
public static byte[] save = new byte[1];
if(recieved)
{
await reader.LoadAsync(4);
var sz = reader.ReadUInt32(); //read size
await reader.LoadAsync(sz); //read content
save = new byte[sz];
reader.ReadBytes(save);
await ImgSrcFromBytes(save)
Array.Clear(save, 0, save.Length); //issue here !!
}
public async Task<ImageSource> ImgSrcFromBytes(byte[] a)
{
imag = new BitmapImage();
var stream = new Windows.Storage.Streams.InMemoryRandomAccessStream();
await stream.WriteAsync(a.AsBuffer());
stream.Seek(0);
imag.SetSource(stream);
return imag;
}
Now, i'm implementing a new function to save the image as a file if requested by the user with the code below, however, if i clear the array of bytes above, i get an unreadable image, but if i don't clear the array, i get a perfect image.
Note that no exceptions are thrown and both images have the same size.
FileSavePicker picker = new FileSavePicker();
picker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
picker.SuggestedFileName = "capture.png";
picker.FileTypeChoices.Add("Png File", new List<string>() { ".png" });
StorageFile file = await picker.PickSaveFileAsync();
if (file != null)
{
CachedFileManager.DeferUpdates(file);
await FileIO.WriteBytesAsync(file, save);
await CachedFileManager.CompleteUpdatesAsync(file);
await new Windows.UI.Popups.MessageDialog("Image Saved Successfully !").ShowAsync();
}
I hope i'm clear. It's a trade-off, if i clear the array, i will get no exceptions while receiving streams over sockets, but i won't be able to get a readable image when saving. and vice versa.
I need to record different sounds in a file. the file may be .mp3, .wav etc. how it is possible in windows phone 7?
There is a simple way to do this in Windows Phone. You are basically using the Microphone class provided by the framework. For a great article on the topic go to http://msdn.microsoft.com/en-us/magazine/gg598930.aspx
void OnRecordButtonClick(object sender, RoutedEventArgs args)
{
if (microphone.State == MicrophoneState.Stopped)
{
// Clear the collection for storing buffers
memoBufferCollection.Clear();
// Stop any playback in progress (not really necessary, but polite I guess)
playback.Stop();
// Start recording
microphone.Start();
}
else
{
StopRecording();
}
// Update the record button
bool isRecording = microphone.State == MicrophoneState.Started;
UpdateRecordButton(isRecording);
}
void StopRecording()
{
// Get the last partial buffer
int sampleSize = microphone.GetSampleSizeInBytes(microphone.BufferDuration);
byte[] extraBuffer = new byte[sampleSize];
int extraBytes = microphone.GetData(extraBuffer);
// Stop recording
microphone.Stop();
// Create MemoInfo object and add at top of collection
int totalSize = memoBufferCollection.Count * sampleSize + extraBytes;
TimeSpan duration = microphone.GetSampleDuration(totalSize);
MemoInfo memoInfo = new MemoInfo(DateTime.UtcNow, totalSize, duration);
memoFiles.Insert(0, memoInfo);
// Save data in isolated storage
using (IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream stream = storage.CreateFile(memoInfo.FileName))
{
// Write buffers from collection
foreach (byte[] buffer in memoBufferCollection)
stream.Write(buffer, 0, buffer.Length);
// Write partial buffer
stream.Write(extraBuffer, 0, extraBytes);
}
}
// Scroll to show new MemoInfo item
memosListBox.UpdateLayout();
memosListBox.ScrollIntoView(memoInfo);
}