How can I get rid of artifacts in ImageSource created with SkiaSharp - xamarin

I created an app in which I want to display text on top of google maps. I chose to use custom markers, but they can only be images, so I decided to create an image from my text utilizing SkiaSharp.
private static ImageSource CreateImageSource(string text)
{
int numberSize = 20;
int margin = 5;
SKBitmap bitmap = new SKBitmap(30, numberSize + margin * 2, SKImageInfo.PlatformColorType, SKAlphaType.Premul);
SKCanvas canvas = new SKCanvas(bitmap);
SKPaint paint = new SKPaint
{
Style = SKPaintStyle.StrokeAndFill,
TextSize = numberSize,
Color = SKColors.Red,
StrokeWidth = 1,
};
canvas.DrawText(text.ToString(), 0, numberSize, paint);
SKImage skImage = SKImage.FromBitmap(bitmap);
SKData data = skImage.Encode(SKEncodedImageFormat.Png, 100);
return ImageSource.FromStream(data.AsStream);
}
The images I create however have ugly artifacts on the top of the resulting image and my feeling is that they get worse if I create multiple images.
I built an example app, that shows the artifacts and the code I used to draw the text. It can be found here:
https://github.com/hot33331/SkiaSharpExample
How can I get rid of those artifacts. Am I using skia wrong?

I got the following answer from Matthew Leibowitz on the SkiaSharp GitHub:
The chances are you are not clearing the canvas/bitmap first.
You can either do bitmap.Erase(SKColors.Transparent) or canvas.Clear(SKColors.Transparent) (you can use any color).
The reason for this is performance. When creating a new bitmap, the computer has no way of knowing what background color you want. So, if it was to go transparent and you wanted white, then there would be two draw operations to clear the pixels (and this may be very expensive for large images).
During the allocation of the bitmap, the memory is provided, but the actual data is untouched. If there was anything there previously (which there will be), this data appears as colored pixels.

When I've seen that before, it's been because the memory passed to SkiaSharp was not zeroed. As an optimization, though, Skia assumes that the memory block passed to it is pre zeroed. Resultingly, if your first operation is a clear, it will ignore that operation, because it thinks that the state is already clean. To resolve this issue, you can manually zero the memory passed to SkiaSharp.
public static SKSurface CreateSurface(int width, int height)
{
// create a block of unmanaged native memory for use as the Skia bitmap buffer.
// unfortunately, this may not be zeroed in some circumstances.
IntPtr buff = System.Runtime.InteropServices.Marshal.AllocCoTaskMem(width * height * 4);
byte[] empty = new byte[width * height * 4];
// copy in zeroed memory.
// maybe there's a more sanctioned way to do this.
System.Runtime.InteropServices.Marshal.Copy(empty, 0, buff, width * height * 4);
// create the actual SkiaSharp surface.
var colorSpace = CGColorSpace.CreateDeviceRGB();
var bContext = new CGBitmapContext(buff, width, height, 8, width * 4, colorSpace, (CGImageAlphaInfo)bitmapInfo);
var surface = SKSurface.Create(width, height, SKColorType.Rgba8888, SKAlphaType.Premul, bitmap.Data, width * 4);
return surface;
}
Edit: btw, I assume this is a bug in SkiaSharp. The samples/apis that create the buffer for you should probably be zeroing it out. Depending on the platform it can be hard to repro as the memory alloc behaves differently. More or less likely to provide you untouched memory.

Related

Is it possible to save a generated image in Codename One?

My question is related to this previous question. What I want to achieve is to stack images (they have transparency), write a string on top, and save the photomontage / photocollage with full resolution.
#Override
protected void beforeMain(Form f) {
Image photoBase = fetchResourceFile().getImage("Voiture_4_3.jpg");
Image watermark = fetchResourceFile().getImage("Watermark.png");
f.setLayout(new LayeredLayout());
final Label drawing = new Label();
f.addComponent(drawing);
// Image mutable dans laquelle on va dessiner (fond blanc)
Image mutableImage = Image.createImage(photoBase.getWidth(), photoBase.getHeight());
drawing.getUnselectedStyle().setBgImage(mutableImage);
drawing.getUnselectedStyle().setBackgroundType(Style.BACKGROUND_IMAGE_SCALED_FIT);
// Paint all the stuff
paints(mutableImage.getGraphics(), photoBase, watermark, photoBase.getWidth(), photoBase.getHeight());
// Save the collage
Image screenshot = Image.createImage(photoBase.getWidth(), photoBase.getHeight());
f.revalidate();
f.setVisible(true);
drawing.paintComponent(screenshot.getGraphics(), true);
String imageFile = FileSystemStorage.getInstance().getAppHomePath() + "screenshot.png";
try(OutputStream os = FileSystemStorage.getInstance().openOutputStream(imageFile)) {
ImageIO.getImageIO().save(screenshot, os, ImageIO.FORMAT_PNG, 1);
} catch(IOException err) {
err.printStackTrace();
}
}
public void paints(Graphics g, Image background, Image watermark, int width, int height) {
g.drawImage(background, 0, 0);
g.drawImage(watermark, 0, 0);
g.setColor(0xFF0000);
// Upper left corner
g.fillRect(0, 0, 10, 10);
// Lower right corner
g.setColor(0x00FF00);
g.fillRect(width - 10, height - 10, 10, 10);
g.setColor(0xFF0000);
Font f = Font.createTrueTypeFont("Geometos", "Geometos.ttf").derive(220, Font.STYLE_BOLD);
g.setFont(f);
// Draw a string right below the M from Mercedes on the car windscreen (measured in Gimp)
g.drawString("HelloWorld",
(int) (848 ),
(int) (610)
);
}
This is the saved screenshot I get if I use the Iphone6 skin (the payload image is smaller than the original one and is centered). If I use the Xoom skin this is what I get (the payload image is still smaller than the original image but it has moved to the left).
So to sum it all up : why is the saved screenshot with Xoom skin different from the one I get with Iphone skin ? Is there anyway to directly save the graphics on which I paint in the paints method so that the saved image would have the original dimensions ?
Thanks a lot to anyone that could help me :-)!
Cheers,
You can save an image in Codename one using the ImageIO class. Notice that you can draw a container hierarchy into a mutable image using the paintComponent(Graphics) method.
You can do both approaches with draw image on mutable or via layouts. Personally I always prefer layouts as I like the abstraction but I wouldn't say the mutable image approach is right/wrong.
Notice that if you change/repaint a lot then mutable images are slower (this will not be noticeable for regular code or on the simulator) as they are forced to use the software renderer and can't use the GPU fully.
In the previous question it seems you placed the image with a "FIT" style which naturally drew it smaller than the containing container and then drew the image on top of it manually... This is problematic.
One solution is to draw everything manually but then you will need to do the "fit" aspect of drawing yourself. If you use layouts you should position everything based on the layouts including your drawing/text.

Finding size of images imported at runtime

I am currently loading images at runtime from directories stored in an XML file, and assigning them to RawImage components via the WWW class. While this is working fine, the image is skewed to fit into the new texture size.
I am wondering how to get an image’s original size or aspect ratio so that I can change the size of the image rect to suit. The images to be imported are at varying sizes and therefore the approach used needs to be responsive to the original size of imported images.
Any suggestions would be much appreciated. [Scripting in uJS]
Many thanks in advance, Ryan
function loadContextImage(texLocation : String)
{
if (!imageView.activeSelf)
{
imageView.SetActive(true);
}
var wwwDirectory = "file://" + texLocation; //this will probably need to change for other OS (PC = file:/ [I think?]) - **REVISE**
var newImgTex = new Texture2D(512, 512);
while(true){
var www : WWW = new WWW(wwwDirectory);
yield www;
www.LoadImageIntoTexture(newImgTex);
if (www.isDone){
break; //if done downloading image break loop
}
}
var imageRender : UI.RawImage = imageView.GetComponent.<RawImage>();
imageRender.texture = newImgTex;
}
If you cannot use an Image (for nay valid reasons), you can get the width and height of the texture:
WWW www = new WWW(url);
yield return www;
Texture2D tex = www.texture;
float aspectRatio = tex.height / tex.width;
rawImage.width = width;
rawImage.height = width * aspectRatio;
This should make the rect of the image of the appropriate ratio of the texture.
If you can use Image and preserveAspectRatio, you get it done by Unity. The result is not necessarily the same since it will keep the dimensions of the box and make the Sprite occupies as much space while keeping ratio.

Drawing on image and saving

I guess I don't really understand how Graphics objects work in Visual C++ 2010 Express.
I am grabbing a frame from a webcam, and drawing a circle on it. It works great on the screen. I simply create a Graphics object, draw the image, and draw the ellipse.
In the pictureBox_paint function, I have
Graphics^ g = e->Graphics; // from the camera
System::Drawing::Rectangle destRect = System::Drawing::Rectangle(0,0,pbCameraMonitor->Size.Width,pbCameraMonitor->Size.Height);
double slitHeightToWidth = 3;
g->DrawImage(this->currentCamImage,destRect);
int circleX, circleY;
circleX = (int) (pbCameraMonitor->Size.Width - radius/slitHeightToWidth)/2;
circleY = (int) (pbCameraMonitor->Size.Height - radius)/2;
g->DrawEllipse(Pens::Red, circleX, circleY, (int) radius/slitHeightToWidth, (int) radius);
So far so good, my ellipse gets drawn on there nicely. The destRect bit makes sure it is scaled to the pictureBox size. I simply invalidate the pictureBox every time the camera reports a new image, and I have video.
Now, on a button click I want to save this image, with the red ellipse on it. However, I don't want the rescaled version shown on the screen, I want the full res version. So, I'll grab another frame into a Bitmap^ called "grabbedFrame" and do this:
String ^photofile = "Image_" + expRecord.timestamp.ToString("s") + ".jpg"; // get a unique filename
photofile = photofile->Replace(':', '_');
Graphics^ g = Graphics::FromImage(grabbedFrame);
g->DrawEllipse(Pens::Red, 20, 20, 20, 20); // circle size fixed just for demo
grabbedFrame->Save(photofile, System::Drawing::Imaging::ImageFormat::Jpeg);
When I do that, I get a save of the image without the red circle.
Does g->DrawEllipse actually modify the Bitmap? Or just contain the Bitmap + instructions to draw? If the latter, how does the pictureBox know the Bitmap has been modified? If the former, why doesn't my save contain the modification?
How can I save the modified Bitmap?
You need to draw the loaded image into a new Bitmap, make your modifications to that bitmap, and then save it.
Something like (pseudo-code'ish):
// create bitmap and get its graphics
Bitmap^ pBmp = gcnew Bitmap(grabbedFrame->Width, grabbedFrame->Height);
Graphics^ g = Graphics::FromImage(pBmp);
// draw grabbed frame into bitmap
g->DrawImage(grabbedFrame, 0, 0, grabbedFrame->Width, grabbedFrame->Height);
// draw other stuff
g->DrawEllipse(Pens::Red, 20, 20, 20, 20);
// save the result
pBmp->Save(photofile, System::Drawing::Imaging::ImageFormat::Jpeg);

GetObject() on bitmap handle from LoadImage() sometimes returns incorrect bitmap size

We are seeing an intermittent problem in which owner drawn buttons under Windows XP that are using a bitmap as a backdrop are displaying the bitmap incorrectly. The window containing multiple buttons that are using the same bitmap file for the bitmap image used for the button backdrop will display and most of the buttons will be correct though in some cases there may be one or two buttons which are displaying the bitmap backdrop reduced to a smaller size.
If you exit the application and then restart it you may see the same behavior of the incorrect display of the icon on the buttons however it may or may not be the same buttons as previously. Nor is this behavior of incorrect display of icons on the buttons always seen. Sometimes it shows and sometimes it does not. Since once we load an icon for a button we just keep it, once the button is displayed incorrectly it will always be displayed incorrectly.
Using the debugger we have finally found that what appears to be happening is that when the GetObject() function is called, the data returned for the bitmap size is sometimes incorrect. For instance in one case the bitmap was 75x75 pixels and the size returned by GetObject() was 13x13 instead. Since this size is used as part of the drawing of the bitmap, the displayed backdrop becomes a small decoration on the button window.
The actual source area is as follows.
if (!hBitmapFocus) {
CString iconPath;
iconPath.Format(ICON_FILES_DIR_FORMAT, m_Icon);
hBitmapFocus = (HBITMAP)LoadImage(NULL, iconPath, IMAGE_BITMAP, 0, 0, LR_LOADFROMFILE);
}
if (hBitmapFocus) {
BITMAP bitmap;
int iNoBytes = GetObject(hBitmapFocus, sizeof(BITMAP), &bitmap);
if (iNoBytes < 1) {
char xBuff[128];
sprintf (xBuff, "GetObject() failed. GetLastError = %d", GetLastError ());
NHPOS_ASSERT_TEXT((iNoBytes > 0), xBuff);
}
cxSource = bitmap.bmWidth;
cySource = bitmap.bmHeight;
//Bitmaps cannot be drawn directly to the screen so a
//compatible memory DC is created to draw to, then the image is
//transfered to the screen
CDC hdcMem;
hdcMem.CreateCompatibleDC(pDC);
HGDIOBJ hpOldObject = hdcMem.SelectObject(hBitmapFocus);
int xPos;
int yPos;
//The Horizontal and Vertical Alignment
//For Images
//Are set in the Layout Manager
//the proper attribute will have to be checked against
//for now the Image is centered on the button
//Horizontal Alignment
if(btnAttributes.horIconAlignment == IconAlignmentHLeft){//Image to left
xPos = 2;
}else if(btnAttributes.horIconAlignment == IconAlignmentHRight){//Image to right
xPos = myRect.right - cxSource - 5;
}else {//Horizontal center
xPos = ((myRect.right - cxSource) / 2) - 1;
}
//Vertical Alignment
if(btnAttributes.vertIconAlignment == IconAlignmentVTop){//Image to top
yPos = 2;
}else if(btnAttributes.vertIconAlignment == IconAlignmentVBottom){//Image to bottom
yPos = myRect.bottom - cySource - 5;
}else{//Vertical Center
yPos = ((myRect.bottom - cySource) / 2) - 1;
}
pDC->BitBlt(xPos, yPos, cxSource, cySource, &hdcMem, 0, 0, SRCCOPY);
hdcMem.SelectObject(hpOldObject);
}
Using the debugger we can see that the iconPath string is correct and the bitmap is loaded as hBitmapFocus is not NULL. Next we can see that the call to GetObject() is made and the value returned for iNoBytes equals 24. For those buttons that display correctly the values in bitmap.bmWidth and bitmap.bmHeight are correct however for those that do not the values are much too small leading to an incorrect sizing when drawing the bitmap.
The variable is defined in the class header as
HBITMAP hBitmapFocus;
As part of doing the research for this I found this stack overflow question, GetObject returns strange size and I am wondering if there is some kind of an alignment issue here.
Does the bitmap variable used in the call to GetObject() need to be on some kind of an alignment boundary? While we are using packed for some of our data we are using pragma directives to only specify specific portions of code containing specific structs in include files that need to be packed on one byte boundaries.
Please read this Microsoft KB how to load a bitmap with palette information. It has a great example as well.
On the side note: I do not see anywhere in your code where you call ::DeleteObject(hBitmapFocus). It is very important to call this, as you can run out of GDI objects very quickly.
It is always a good idea to use Windows Task manager to see that your program does not exhaust the GDI resources. Just add "GDI Objects" column to the Task Manager and see that the number of objects is not constantly increasing in your app, but stays within an expected range, similar to other programs

Resizing and saving an image in WinMobile and .NET CF throws OutOfMemoryException

I have a WinMobile app which allows the user the snap a photo with the camera, and then use for for various things. The photo can be snapped at 1600x1200, 800x600 or 640x480, but it must always be resized to 400px for the longest size (the other is proportional of course). Here's the code:
private void LoadImage(string path)
{
Image tmpPhoto = new Bitmap(path);
// calculate new bitmap size...
double width = ...
double height = ...
// draw new bitmap
Image photo = new Bitmap(width, height, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
using (Graphics g = Graphics.FromImage(photo))
{
g.FillRectangle(new SolidBrush(Color.White), new Rectangle(0, 0, photo.Width, photo.Height));
int srcX = (int)((double)(tmpPhoto.Width - width) / 2d);
int srcY = (int)((double)(tmpPhoto.Height - height) / 2d);
g.DrawImage(tmpPhoto, new Rectangle(0, 0, photo.Width, photo.Height), new Rectangle(srcX, srcY, photo.Width, photo.Height), GraphicsUnit.Pixel);
}
tmpPhoto.Dispose();
// save new image and dispose
photo.Save(Path.Combine(config.TempPath, config.TempPhotoFileName), System.Drawing.Imaging.ImageFormat.Jpeg);
photo.Dispose();
}
Now the problem is that the app breaks in the photo.Save call, with an OutOfMemoryException. And I don't know why, since I dispose the tempPhoto (with the original photo from the camera) as soon as I can, and I also dispose the Graphics obj. Why does this happen? It seems impossible to me that one can't take a photo with the camera and resize/save it without making it crash :( Should I restor t C++ for such a simple thing?
Thanks.
Have you looked at memory usage with each step to see exactly where you're using the most? You omitted your calculations for width and height, but assuming they are right you would end up with photo requiring 400x300x3 (24bits) == 360k for the bitmap data itself, which is not inordinately large.
My guess is that even though you're calling Dispose, the resources aren't getting rleased, especially if you're calling this method multiple times. The CF behaves in an unexpected way with Bitmaps. I call it a bug. The CF team doesn't.

Resources