Xamarin Forms UWP Capture Screenshot Include Signature Pad - xamarin

I have a Xamarin Forms page using Signature Pad (https://github.com/xamarin/SignaturePad). I'm attempting to capture a screenshot of the entire view. It should include the signature as well.
However, using the following code I'm noticing the signature does not show up.
What is the best way to capture the full Page including the signature? (not just the signature)
public class ScreenshotService : IScreenshotService
{
public async Task<byte[]> CaptureAsync()
{
var rtb = new RenderTargetBitmap();
await rtb.RenderAsync(Window.Current.Content);
var pixelBuffer = await rtb.GetPixelsAsync();
var pixels = pixelBuffer.ToArray();
// Useful for rendering in the correct DPI
var displayInformation = DisplayInformation.GetForCurrentView();
var stream = new InMemoryRandomAccessStream();
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)rtb.PixelWidth,
(uint)rtb.PixelHeight,
displayInformation.RawDpiX,
displayInformation.RawDpiY,
pixels);
await encoder.FlushAsync();
stream.Seek(0);
var readStram = stream.AsStreamForRead();
var bytes = new byte[readStram.Length];
readStram.Read(bytes, 0, bytes.Length);
return bytes;
}
}

According to the "XAML visuals and RenderTargetBitmap capture capabilities" of RenderTargetBitmap class:
Content that can't be captured will appear as blank in the captured image, but other content in the same visual tree can still be captured and will render (the presence of content that can't be captured won't invalidate the entire capture of that XAML composition).
So it could be that the content of InkCanvas is not captureable. However, you can use Win2D. For more you could refer the following code.
public async Task<Stream> CaptureAsync(Stream Tem)
{
var rtb = new RenderTargetBitmap();
await rtb.RenderAsync(Window.Current.Content);
var pixelBuffer = await rtb.GetPixelsAsync();
var pixels = pixelBuffer.ToArray();
var displayInformation = DisplayInformation.GetForCurrentView();
var stream = new InMemoryRandomAccessStream();
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)rtb.PixelWidth,
(uint)rtb.PixelHeight,
displayInformation.RawDpiX,
displayInformation.RawDpiY,
pixels);
await encoder.FlushAsync();
stream.Seek(0);
var readStram = stream.AsStreamForRead();
var pagebitmap = await GetSoftwareBitmap(readStram);
var softwareBitmap = await GetSoftwareBitmap(Tem);
CanvasDevice device = CanvasDevice.GetSharedDevice();
CanvasRenderTarget renderTarget = new CanvasRenderTarget(device, rtb.PixelWidth, rtb.PixelHeight, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
ds.Clear(Colors.White);
var page = CanvasBitmap.CreateFromSoftwareBitmap(device, pagebitmap);
var image = CanvasBitmap.CreateFromSoftwareBitmap(device, softwareBitmap);
ds.DrawImage(page);
ds.DrawImage(image);
}
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
await renderTarget.SaveAsync(randomAccessStream, CanvasBitmapFileFormat.Jpeg, 1f);
return randomAccessStream.AsStream();
}
private async Task<SoftwareBitmap> GetSoftwareBitmap(Stream data)
{
BitmapDecoder pagedecoder = await BitmapDecoder.CreateAsync(data.AsRandomAccessStream());
return await pagedecoder.GetSoftwareBitmapAsync(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
IScreenshotServicecs interface
public interface IScreenshotServicecs
{
Task<Stream> CaptureAsync(Stream stream);
}
Usage
var stream = await SignatureView.GetImageStreamAsync(SignaturePad.Forms.SignatureImageFormat.Png);
var data = await DependencyService.Get<IScreenshotServicecs>().CaptureAsync(stream);
MyImage.Source = ImageSource.FromStream(() => data);

Here is my final implementation including converting to byte array.
public async Task<byte[]> CaptureAsync(Stream signatureStream)
{
var rtb = new RenderTargetBitmap();
await rtb.RenderAsync(Window.Current.Content);
var pixelBuffer = await rtb.GetPixelsAsync();
var pixels = pixelBuffer.ToArray();
var displayInformation = DisplayInformation.GetForCurrentView();
var stream = new InMemoryRandomAccessStream();
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetPixelData(BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)rtb.PixelWidth,
(uint)rtb.PixelHeight,
displayInformation.RawDpiX,
displayInformation.RawDpiY,
pixels);
await encoder.FlushAsync();
stream.Seek(0);
var readStram = stream.AsStreamForRead();
var pagebitmap = await GetSoftwareBitmap(readStram);
var softwareBitmap = await GetSoftwareBitmap(signatureStream);
CanvasDevice device = CanvasDevice.GetSharedDevice();
CanvasRenderTarget renderTarget = new CanvasRenderTarget(device, rtb.PixelWidth, rtb.PixelHeight, 96);
using (var ds = renderTarget.CreateDrawingSession())
{
ds.Clear(Colors.White);
var page = CanvasBitmap.CreateFromSoftwareBitmap(device, pagebitmap);
var image = CanvasBitmap.CreateFromSoftwareBitmap(device, softwareBitmap);
ds.DrawImage(page);
ds.DrawImage(image, 50, 55);
}
InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();
await renderTarget.SaveAsync(randomAccessStream, CanvasBitmapFileFormat.Jpeg, 1f);
var fileBytes = new byte[randomAccessStream.Size];
using (var reader = new DataReader(randomAccessStream))
{
await reader.LoadAsync((uint)randomAccessStream.Size);
reader.ReadBytes(fileBytes);
}
return fileBytes;
}

Related

Correct way to passed the stream image to tensorflow-lite model using Xamarin

im currently developing Image Classification using xamarin form, the model works well and the prediction result is good, but my problem is, it is only works in PickPhotoAsync passing to MediaFile then the MediaFile > get stream > get the byte > feed to model. i just used this for testing only
What i really need to do is to get the image from SignaturePad > get stream > get the Byte > feed to model
i got wrong prediction when i change it to signaturePad. i hope someone will help me, thank you in advance
here's my code.
protected async void PickPhotoAsync(object sender, EventArgs e)
{
var file = await CrossMedia.Current.PickPhotoAsync();
HandlePhoto(file);
}
private void HandlePhoto(MediaFile file)
{
var stream = file.GetStreamWithImageRotatedForExternalStorage();
var memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
var bytes = memoryStream.ToArray();
var len = stream.Length;
var pos = stream.Position;
var read = stream.CanRead;
var lens = bytes.Length;
List<ImageClassificationModel> classifyImage = DependencyService.Get<IClassify>().Classify(bytes);
var sortedList = classifyImage.OrderByDescending(x => x.Probability);
var top = sortedList.First();//Highest Prediction Result
var max = sortedList.Max(x => x.Probability);
}
Here is my code when i change it to SignaturePad (Source of image) which i got bad result
private async void SaveImagePad(object sender, EventArgs e)
{
Stream image = await PadView.GetImageStreamAsync(SignatureImageFormat.Png);
//get the stream from SignaturePad
if (image == null)
return;
BinaryReader br = new BinaryReader(image);
Byte[] All = br.ReadBytes((int)image.Length);
byte[] acImage = (byte[])All;
//Convert To Byte before passing to classifier
List<ImageClassificationModel> classifyImage = DependencyService.Get<IClassify>().Classify(acImage);
//File.Delete(path);
var sortedList = classifyImage.OrderByDescending(x => x.Probability);
var top = sortedList.First();
var max = sortedList.Max(x => x.Probability); // i got bad and random result
}

Flutter - How to convert an NetworkImage into an ui.Image?

I need to convert a NetworkImage to an ui.Image.
I tried to use the given solution from this question with some adjusts but it isn't working.
Someone can help me?
Uint8List yourVar;
ui.Image image;
final DecoderCallback callback =
(Uint8List bytes, {int cacheWidth, int cacheHeight}) async {
yourVar = bytes.buffer.asUint8List();
var codec = await instantiateImageCodec(bytes,
targetWidth: cacheWidth, targetHeight: cacheHeight);
var frame = await codec.getNextFrame();
image = frame.image;
return image;
};
ImageProvider provider = NetworkImage(yourImageUrl);
provider.obtainKey(createLocalImageConfiguration(context)).then((key) {
provider.load(key, callback);
});
first create a field in your class:
var cache = MapCache<String, ui.Image>();
then to get ui.Image you can simply call:
var myUri = 'http:// ...';
var img = await cache.get(myUri, ifAbsent: (uri) {
print('getting not cached image from $uri');
return http.get(uri).then((resp) => decodeImageFromList(resp.bodyBytes));
});
print('image: $img');
of course you should add some http response error handling but this is the base idea...

Rgb 565 Pdf to Image

I am trying to convert a PDF page to an image, to create thumbnails. This is the code that I am using:
PdfRenderer pdfRenderer = new PdfRenderer(GetSeekableFileDescriptor(filePath));
var appDirectory = System.Environment.GetFolderPath(System.Environment.SpecialFolder.MyDocuments);
string fileName = System.IO.Path.GetFileNameWithoutExtension(filePath);
string directoryPath = System.IO.Path.Combine(appDirectory, "thumbnailsTemp", System.IO.Path.GetFileNameWithoutExtension(fileName));
if (!Directory.Exists(directoryPath))
{
Directory.CreateDirectory(directoryPath);
int pageCount = pdfRenderer.PageCount;
for (int i = 0; i < pageCount; i++)
{
Page page = pdfRenderer.OpenPage(i);
Android.Graphics.Bitmap bmp = Android.Graphics.Bitmap.CreateBitmap(page.Width, page.Height, Android.Graphics.Bitmap.Config.Rgb565 or Argb8888);
page.Render(bmp, null, null, PdfRenderMode.ForDisplay);
try
{
using (FileStream output = new FileStream(System.IO.Path.Combine(directoryPath, fileName + "Thumbnails" + i + ".png"), FileMode.Create))
{
bmp.Compress(Android.Graphics.Bitmap.CompressFormat.Png, 100, output);
}
page.Close();
}
catch (Exception ex)
{
//TODO -- GERER CETTE EXPEXPTION
throw new Exception();
}
}
return directoryPath;
}
I tried with ARGB 8888 and that was a success. But the rendering time was too slow for big PDF files. This is why I tried to improve it by changing the format to RGB 565. But my app is crashing with this Execption:
Unsuported pixel format
Any idea to fix this, or how to render a PDF to a bitmap faster? I was looking on google but didn't find a solution related to my code.
UPDATE
I did this but know, my app is crashing at this line of code :
await Task.Run(() =>
{
bytes = page.AsPNG(72);
});
My class :
public async Task<string> GetBitmaps(string filePath)
{
//TODO -- WORK ON THIS
PdfRenderer pdfRenderer = new PdfRenderer(GetSeekableFileDescriptor(filePath));
var appDirectory = System.Environment.GetFolderPath(System.Environment.SpecialFolder.MyDocuments);
string fileName = System.IO.Path.GetFileNameWithoutExtension(filePath);
string directoryPath = System.IO.Path.Combine(appDirectory, "thumbnailsTemp", System.IO.Path.GetFileNameWithoutExtension(fileName));
var stream = new MemoryStream();
using (Stream resourceStream = new FileStream(filePath, FileMode.Open))
{
resourceStream.CopyTo(stream);
}
for (int i = 0; i < pdfRenderer.PageCount; i++)
{
TallComponents.PDF.Rasterizer.Page page = new TallComponents.PDF.Rasterizer.Page(stream, i);
byte[] bytes = null;
await Task.Run(() =>
{
bytes = page.AsPNG(72);
});
using (FileStream output = new FileStream(System.IO.Path.Combine(directoryPath, fileName + "Thumbnails" + i + ".png"), FileMode.Create, FileAccess.Write))
{
output.Write(bytes, 0, bytes.Length);
}
}
return directoryPath;
}
you could draw a PDF page in app by converting a PDF page to a bitmap,here the PDF document itself is embedded as a resource.
var assembly = Assembly.GetExecutingAssembly();
var stream = new MemoryStream();
using (Stream resourceStream = assembly.GetManifestResourceStream("DrawPdf.Android.tiger.pdf"))
{
resourceStream.CopyTo(stream);
}
Page page = new Page(stream, 0);
// render PDF Page object to a Bitmap
byte[] bytes = null;
await Task.Run(() =>
{
bytes = page.AsPNG(72);
});
Bitmap bmp = global::Android.Graphics.BitmapFactory.DecodeByteArray(bytes, 0, bytes.Length);

Android post image to Facebook comment

This is a followup to my previous question: Xamarin.Forms App return data to calling App
That works perfectly and I can share images to anywhere, except to Facebook comments. When I click the camera on the content box the app can be selected, I can select the image, Set result and Finish are called, and the app closes and it sends data to Facebook, and then however I then get the error : The image could not be uploaded, try again?
I can't find any fundamental differences between posting to a status or a comment, so I'm guessing it's subtle. Any thoughts on how I can change my intent to post properly?
Adding for completeness:
Bitmap b = null;
string url;
if (!string.IsNullOrEmpty(this.saleItems[i].ImageUrl))
{
url = this.saleItems[i].ImageUrl;
}
else
{
url = await FileHelper.GetLocalFilePathAsync(this.saleItems[i].Id);
}
//download
using (var webClient = new WebClient())
{
var imageBytes = webClient.DownloadData(url);
if (imageBytes != null && imageBytes.Length > 0)
{
b = BitmapFactory.DecodeByteArray(imageBytes, 0, imageBytes.Length);
}
}
//set local path
var tempFilename = "test.png";
var sdCardPath = Android.OS.Environment.ExternalStorageDirectory.AbsolutePath;
var filePath = System.IO.Path.Combine(sdCardPath, tempFilename);
using (var os = new FileStream(filePath, FileMode.Create))
{
b.Compress(Bitmap.CompressFormat.Png, 100, os);
}
b.Dispose();
var imageUri = Android.Net.Uri.Parse($"file://{sdCardPath}/{tempFilename}");
var sharingIntent = new Intent();
sharingIntent.SetAction(Intent.ActionSend);
sharingIntent.SetType("image/*");
sharingIntent.PutExtra(Intent.ExtraText, "some txt content");
sharingIntent.PutExtra(Intent.ExtraStream, imageUri);
sharingIntent.AddFlags(ActivityFlags.GrantReadUriPermission);
//await SaleItemDataService.Instance.BuySaleItemAsync(this.saleItem);
SetResult(Result.Ok, sharingIntent);
Finish();
Use below:
Intent sharingIntent = new Intent();
string imageUri = "file://" + requestedUri;
sharingIntent.SetData(Android.Net.Uri.Parse(imageUri));

Windows 10 Store App - save URI as file

I am writing a Windows 10 Store app. In the app the User can input a Text, and then press "Read Text" and Cortana reads the text loud. That works fine.
Now I want to add the feature, to press a button called "Save" or something like that and then save Cortanas output as mp3 file. This should work via a normal save-file dialog.
This is what I got so far.
private static MediaElement mediaplayer = new MediaElement();
/// ... mediaplayer element gets content ...
Uri file = mediaplayer.Source;
Instead of an Uri element I could also get an SpeechSynthesisStream with this information.
How can I save this Uri / Stream to a file?
EDIT:
this is the final code:
var stream2 = stream.CloneStream();
//... use stream2 as mediaelement ...
if(stream != null)
{
using (var reader = new DataReader(stream))
{
FileSavePicker savePicker = new FileSavePicker();
savePicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
savePicker.FileTypeChoices.Add("WAV", new List<string>() { ".wav" });
savePicker.SuggestedFileName = "sound.wav";
StorageFile file = await savePicker.PickSaveFileAsync();
if (file != null)
{
using (var outputStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (var writer = new DataWriter(outputStream.GetOutputStreamAt(0)))
{
long writtenBytes = 0;
const int bufferSize = 8192;
uint loadedBytes = 0;
while ((loadedBytes = (await reader.LoadAsync(bufferSize))) > 0) //!!!
{
IBuffer buffer = reader.ReadBuffer(loadedBytes);
writer.WriteBuffer(buffer);
uint tmpWritten = await writer.StoreAsync(); //!!!
writtenBytes += tmpWritten;
}
}
}
}
}
}
If you're trying to write the output to a file instead (or as well as) rendering it audibly to a MediaElement, you probably want something like this in here as well.
SpeechSynthesisStream synthesisStream = await synthesizer.SynthesizeTextToStreamAsync(text);
var stream2 = synthesisStream.CloneStream();
FileSavePicker savePicker = new FileSavePicker();
savePicker.SuggestedStartLocation = PickerLocationId.MusicLibrary;
savePicker.FileTypeChoices.Add("WAV", new List<string>() { ".wav" });
savePicker.SuggestedFileName = "sound.wav";
StorageFile file = await savePicker.PickSaveFileAsync();
using (var reader = new DataReader(synthesisStream))
{
using (var outputStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (var writer = new DataWriter(outputStream.GetOutputStreamAt(0)))
{
long writtenBytes = 0;
const int bufferSize = 8192;
uint loadedBytes = 0;
while ((loadedBytes = (await reader.LoadAsync(bufferSize))) > 0) //!!!
{
IBuffer buffer = reader.ReadBuffer(loadedBytes);
writer.WriteBuffer(buffer);
uint tmpWritten = await writer.StoreAsync(); //!!!
writtenBytes += tmpWritten;
}
}
}
}
// Set the source and start playing the synthesized audio stream.
media.AutoPlay = true;
media.SetSource(stream2, synthesisStream.ContentType);
media.Play();
The one problem is that the synthesisStream isn't rewindable (so far as I can tell), so you might have to synthesize it twice, or make a second (in memory) copy of the stream if you want to make it audible at the same time.

Resources