Best way to remove borders / crop image - go

I am using a Go package (Go binding to ImageMagick's MagickWand C API) to ImageMagick where I'm removing borders from images (cropping). The way I am using the trim function can be found below.
Now the problem is the fuzzy factor. For example, if I set the value to 2000, the image (here is the source) still has some white images like these:
fuzz factor value 2000 --> result
fuzz factor value 10000 --> result
I have created a small html which illustrates the problem best. It contains both images: https://dl.dropboxusercontent.com/u/15684927/image-trim-problem.html
As you can see the source has some pixels on the bottom right corner which are causing the trouble. If I set the factor to 10000, I'm afraid that I will loose pixels on other pictures. If I set it on 2000, the trimming isn't done right in pictures like these.
So my actual question is: what is the best way to "crop" / "trim" images?
package main
import "gopkg.in/gographics/imagick.v1/imagick"
func main() {
imagick.Initialize()
defer imagick.Terminate()
inputFile := "tshirt-original.jpg"
outputFile := "trimmed.jpg"
mw := imagick.NewMagickWand()
// Schedule cleanup
defer mw.Destroy()
// read image
err := mw.ReadImage(inputFile)
if err != nil {
panic(err)
}
// first trim original image
// fuzz: by default target must match a particular pixel color exactly.
// However, in many cases two colors may differ by a small amount. The fuzz
// member of image defines how much tolerance is acceptable to consider two
// colors as the same. For example, set fuzz to 10 and the color red at
// intensities of 100 and 102 respectively are now interpreted as the same
// color for the purposes of the floodfill.
mw.TrimImage(10000)
// Set the compression quality to 95 (high quality = low compression)
err = mw.SetImageCompressionQuality(95)
if err != nil {
panic(err)
}
// save
err = mw.WriteImage(outputFile)
if err != nil {
panic(err)
}
}

Basically, your problem is that you have a high-frequency, high-amplitude artifact at the edge of your image. Or, put differently, a sharp, high peak at the edge, which, if you want to use trim, forces you to use such a high a fuzz-value to overcome this, that the algorithm also considers the 'actual content' as equal to the 'background' (border).
One solution here is to use a multi-step approach, whereby you first smooth out the edge artifacts and then apply trim to the resulting image. By smoothing it out, you get rid of the high peak and smear it out into a nice rolling hill. Rolling hills, in turn, can be easily trimmed with low fuzz values. This then provides you with the desired geometry which you can use to crop the original.
Specifically, let's take the original image:
Now, let's smooth out that ridge on the edge using a blur with a radius of 10 and a sigma of 10 through convert original.jpg -blur 10x10 10x10.jpg, which yields:
Now, you might notice that the artifacts on the edge have now pretty much disappeared.
We can now do a 'virtual' trim and ask ImageMagick what the result of the trim would be through convert 10x10.jpg -fuzz 2000 -format %# info:, which, according to the documentation gives you the "trim bounding box (without actually trimming)": 1326x1560+357+578%
Taking these values (except for the percentage sign) and using them for crop geometry, gives you the convert with crop command convert original.jpg -crop 1326x1560+357+578 cropped.jpg, which gives you:
Edit:
Now, since you want this as code, using imagick, here's the solution in code. It assumes you have the file stored as './data/original.jpg' and will store it as './data/trimmed.jpg'
package main
import (
"fmt"
"gopkg.in/gographics/imagick.v2/imagick"
)
func init() {
imagick.Initialize()
}
const originalImageFilename = "data/original.jpg"
func main() {
mw := imagick.NewMagickWand()
err := mw.ReadImage(originalImageFilename)
if err != nil {
fmt.Sprint(err.Error())
return
}
// Use a clone to determine what will happen
mw2 := mw.Clone()
mw2.BlurImage(10, 10)
mw2.TrimImage(2000)
_, _, xOffset, yOffset, err := mw2.GetImagePage()
if err != nil {
fmt.Sprint(err.Error())
return
}
trimmedWidth := mw2.GetImageWidth()
trimmedHeight := mw2.GetImageHeight()
mw2.Destroy()
mw.CropImage(trimmedWidth, trimmedHeight, xOffset, yOffset)
mw.WriteImage("data/trimmed.jpg")
mw.Destroy()
}

Related

Publishing changes to window with shiny driver

I need to show a sequence of image.RGBA frames generated by a simulation I'm running, and I'm trying to use shiny to do so.
While getting familiar with the package, I tried running a simple example, very similar to one of those provided, but I do not understand the logic behind Window.Publish, and when the changes are shown to the screen.
According to the docs of Upload and Fill,
When filling a Window, there will not be any visible effect until
Publish is called.
However, when I call w.Publish after Uploading a buffer or Filling the window, I see only a black window.
Strangely, if I use the same lines to upload or fill the window inside the event loop, the window is updated: I see the background color changing when moving the mouse and the square is drawn when resizing the window, even if I do not call w.Publish inside the event loop.
// create a Window
winSize := image.Point{900, 600}
w, err := s.NewWindow(&screen.NewWindowOptions{
Width: winSize.X,
Height: winSize.Y,
Title: "Viewer",
})
if err != nil {
log.Fatalf("s.NewWindow: %+v", err)
}
defer w.Release()
// create a Buffer
size0 := image.Point{30, 30}
b, err := s.NewBuffer(size0)
if err != nil {
log.Fatalf("s.NewBuffer: %+v", err)
}
defer b.Release()
// fill the buffer with a color
white := color.RGBA{255, 255, 255, 255}
draw.Draw(b.RGBA(), b.RGBA().Bounds(), &image.Uniform{white}, image.Point{0, 0}, draw.Src)
// draw the buffer on the window
w.Upload(image.Point{40, 40}, b, b.Bounds())
// publish the changes
w.Publish()
// at this point I see a black window
var sz size.Event
for {
e := w.NextEvent()
switch e := e.(type) {
default:
case key.Event:
if e.Code == key.CodeEscape {
return
}
case mouse.Event:
// this works, the background changes and the square is drawn
v := uint8(rand.Intn(255))
w.Fill(image.Rectangle{image.Point{}, winSize}, color.RGBA{v, v, v, 255}, draw.Src)
w.Upload(image.Point{30, 30}, b, b.Bounds())
case size.Event:
// this works, the background changes and the square is drawn
sz = e
w.Fill(sz.Bounds(), color.RGBA{80, 80, 80, 255}, draw.Src)
w.Upload(image.Point{30, 30}, b, b.Bounds())
case paint.Event:
log.Printf("paint.Event: %T %+v", e, e)
}
}
Why is the window not updated when calling publish?
Why is the window updated inside the loop, even without calling publish?
Why does a size event generate a paint event? That log line I left is printed after a size event, but not a mouse event.
The same thing happens if a use a texture, uploading the buffer to the texture and then copying the texture to the window.
I only need a minimal UI with fast drawing and basic event handling and this package seemed pretty straightforward to use, but if there are other easy options I'm ok with that.
Edit:
As I am running this on Ubuntu 20.04 I believe the driver used is X11, so I looked into that implementation of
Publish and there is a comment which might explain why the call to publish is not needed to actually draw, but is useful for synchronization:
func (w *windowImpl) Publish() screen.PublishResult {
// This sync isn't needed to flush the outgoing X11 requests. Instead, it
// acts as a form of flow control. Outgoing requests can be quite small on
// the wire, e.g. draw this texture ID (an integer) to this rectangle (four
// more integers), but much more expensive on the server (blending a
// million source and destination pixels). Without this sync, the Go X11
// client could easily end up sending work at a faster rate than the X11
// server can serve.
w.s.xc.Sync()
return screen.PublishResult{}
}
But I still do not understand why drawing outside the loop does not work even with the explicit publish call.
Cheers!

What is the best way to decide if an image has alpha channel?

I need to decide if an image has alpha channel or not, so I write the code like this.
var HaveAlpha = func(Image image.Image) bool {
switch Image.ColorModel() {
case color.YCbCrModel, color.CMYKModel:
return false
case color.RGBAModel:
return !Image.(*image.RGBA).Opaque()
}
// ...
return false
}
So I need to list all the ColorModel types and use Opaque() to decide if the image has alpha channel or not (because I cannot use Opaque() method in type image.Image directly). And if an image has alpha channel but all pixels are opaque in the image (all RGBA of pixels in that image are like (*,*,*,255)), this code may return wrong answer.
Is there a right or better way to decide if an image has alpha channel or not in Golang?
You may use type assertion to check if the concrete value stored in the image.Image interface type has an Opaque() bool method, and if so, simply call that and return its result. Note that all concrete image types in the image package do have an Opaque() method, so this will cover most cases.
If the image does not have such an Opaque() method, loop over all pixels of the image and check if any of the pixel has an alpha value other than 0xff, which means it's non-opaque.
Note that Image.At() has a return type of the general color.Color interface type, which only guarantees a single method: Color.RGBA(). This RGBA() method returns the alpha-premultiplied red, green, blue and alpha components, so if a pixel has 0xff alpha value, that equals to 0xffff when "alpha-premultiplied", so that's what we need to compare to.
func Opaque(im image.Image) bool {
// Check if image has Opaque() method:
if oim, ok := im.(interface {
Opaque() bool
}); ok {
return oim.Opaque() // It does, call it and return its result!
}
// No Opaque() method, we need to loop through all pixels and check manually:
rect := im.Bounds()
for y := rect.Min.Y; y < rect.Max.Y; y++ {
for x := rect.Min.X; x < rect.Max.X; x++ {
if _, _, _, a := im.At(x, y).RGBA(); a != 0xffff {
return false // Found a non-opaque pixel: image is non-opaque
}
}
}
return true // All pixels are opaque, so is the image
}
The above Opaque() function will return true if the image does not have an alpha channel, or it has but all pixels are opaque. It returns false if and only if the image has alpha channel and there is at least 1 pixel that is not (fully) opaque.
Note: If an image does have an Opaque() method, you can be sure that it takes existing pixels and their alpha values into consideration, so for example image.RGBA.Opaque() also scans the entire image similarly to what we did above; but we had to do it in a general way, and Opaque() implementations of concrete images may be much more efficient (so it is highly recommended to use the "shipped" Opaque() method if it is available). As an example, implementation of image.YCbCr.Opaque() is a simple return true statement because YCbCr images do not have alpha channel.

PNG Encode producing corrupt image

I'm reading a framebuffer from a video game console with golang - the buffer is in the format BRGA (which I then convert to RGBA). When I pass the information into the Go PNG Encoder, the image that comes out is not valid. The code i'm using is - where:
where data is a slice of RGBA pixels - 0x398000 in length, pitch is 5120, width is 1270, and height is 720)
img := &image.RGBA{
Pix: data,
Stride: pitch,
Rect: image.Rect(0, 0, width, height),
}
os.Remove("./img.png")
file, _ := os.Create("./img.png")
defer file.Close()
filewriter := bufio.NewWriter(file)
if err := png.Encode(filewriter, img); err != nil {
panic(err)
}
The expected outcome would be:
But the actual outcome is (only renders on Windows or when view in Chrome.. weird):
I have uploaded a binary dump of the RGBA slice if anybody would like it - https://1drv.ms/u/s!Ak-aZ3z7Ea8KwvUsqdP5OgWpZqxsGA
You are not flushing the buffered writer. You should do:
filewriter := bufio.NewWriter(file)
defer filewriter.Flush()
After this fix, I get a valid image:
Not a fix, and I want to comment but can't yet due to reputation, but will add to the Mac OS discrepancy.
The MacOS part of the problem appears to be new, showing up since either the latest 10.12.3 update, or something with Safari. I haven't narrowed down the source yet. But yes, there is something new about how a Mac system will encode/decode an image, causing it to be transparent or grey as a result. A project I am on is also suffering from this problem for the past few weeks and I'm still investigating where it breaks down.

Golang Iris Web - Serve 1x1 pixel

I'm new to Golang and trying to use a framework called iris.
My problem is how to serve 1x1 gif pixel, not using c.HTML context, but the way the browser title becomes 1x1 image gif. The image will be used for tracking.
Any help will be deeply appreciated.
Attention: I'm not really familiar with iris so the solution maybe not idiomatic.
package main
import (
"github.com/kataras/iris"
"image"
"image/color"
"image/gif"
)
func main() {
iris.Get("/", func(ctx *iris.Context) {
img := image.NewRGBA(image.Rect(0, 0, 1, 1)) //We create a new image of size 1x1 pixels
img.Set(0, 0, color.RGBA{255, 0, 0, 255}) //set the first and only pixel to some color, in this case red
err := gif.Encode(ctx.Response.BodyWriter(), img, nil) //encode the rgba image to gif, using gif.encode and write it to the response
if err != nil {
panic(err) //if we encounter some problems panic
}
ctx.SetContentType("image/gif") //set the content type so the browser can identify that our response is actually an gif image
})
iris.Listen(":8080")
}
Links for better understanding:
https://golang.org/pkg/image/
https://golang.org/pkg/image/gif
https://golang.org/pkg/image/color
https://golang.org/pkg/image/
https://godoc.org/github.com/valyala/fasthttp#Response

How to convert image.RGBA (image.Image) to image.Paletted?

I'm trying to create an animated GIF from a series of arbitrary non-paletted images. In order to create a paletted image, I need to come up with a palette somehow.
// RGBA, etc. images from somewhere else
var frames []image.Image
outGif := &gif.GIF{}
for _, simage := range frames {
// TODO: Convert image to paletted image
// bounds := simage.Bounds()
// palettedImage := image.NewPaletted(bounds, ...)
// Add new frame to animated GIF
outGif.Image = append(outGif.Image, palettedImage)
outGif.Delay = append(outGif.Delay, 0)
}
gif.EncodeAll(w, outGif)
Is there an easy way in golang stdlib to accomplish this?
It seems an automatic way of intelligently generating palettes is missing from the golang stdlib (correct me if I'm wrong here). But there seems to be a stub for providing your own Quantizer, which led me to the gogif project. (Which was the apparent source of image.Gif.)
I was able to borrow the MedianCutQuantizer from that project, defined here:
https://github.com/andybons/gogif/blob/master/mediancut.go
Which results in the following:
var subimages []image.Image // RGBA, etc. images from somewhere else
outGif := &gif.GIF{}
for _, simage := range subimages {
bounds := simage.Bounds()
palettedImage := image.NewPaletted(bounds, nil)
quantizer := gogif.MedianCutQuantizer{NumColor: 64}
quantizer.Quantize(palettedImage, bounds, simage, image.ZP)
// Add new frame to animated GIF
outGif.Image = append(outGif.Image, palettedImage)
outGif.Delay = append(outGif.Delay, 0)
}
gif.EncodeAll(w, outGif)
Instead of generating your own palette, you can also use on of the predefined (https://golang.org/pkg/image/color/palette/)
...
palettedImage := image.NewPaletted(bounds, palette.Plan9)
draw.Draw(palettedImage, palettedImage.Rect, simage, bounds.Min, draw.Over)
...

Resources