Publishing changes to window with shiny driver - go

I need to show a sequence of image.RGBA frames generated by a simulation I'm running, and I'm trying to use shiny to do so.
While getting familiar with the package, I tried running a simple example, very similar to one of those provided, but I do not understand the logic behind Window.Publish, and when the changes are shown to the screen.
According to the docs of Upload and Fill,
When filling a Window, there will not be any visible effect until
Publish is called.
However, when I call w.Publish after Uploading a buffer or Filling the window, I see only a black window.
Strangely, if I use the same lines to upload or fill the window inside the event loop, the window is updated: I see the background color changing when moving the mouse and the square is drawn when resizing the window, even if I do not call w.Publish inside the event loop.
// create a Window
winSize := image.Point{900, 600}
w, err := s.NewWindow(&screen.NewWindowOptions{
Width: winSize.X,
Height: winSize.Y,
Title: "Viewer",
})
if err != nil {
log.Fatalf("s.NewWindow: %+v", err)
}
defer w.Release()
// create a Buffer
size0 := image.Point{30, 30}
b, err := s.NewBuffer(size0)
if err != nil {
log.Fatalf("s.NewBuffer: %+v", err)
}
defer b.Release()
// fill the buffer with a color
white := color.RGBA{255, 255, 255, 255}
draw.Draw(b.RGBA(), b.RGBA().Bounds(), &image.Uniform{white}, image.Point{0, 0}, draw.Src)
// draw the buffer on the window
w.Upload(image.Point{40, 40}, b, b.Bounds())
// publish the changes
w.Publish()
// at this point I see a black window
var sz size.Event
for {
e := w.NextEvent()
switch e := e.(type) {
default:
case key.Event:
if e.Code == key.CodeEscape {
return
}
case mouse.Event:
// this works, the background changes and the square is drawn
v := uint8(rand.Intn(255))
w.Fill(image.Rectangle{image.Point{}, winSize}, color.RGBA{v, v, v, 255}, draw.Src)
w.Upload(image.Point{30, 30}, b, b.Bounds())
case size.Event:
// this works, the background changes and the square is drawn
sz = e
w.Fill(sz.Bounds(), color.RGBA{80, 80, 80, 255}, draw.Src)
w.Upload(image.Point{30, 30}, b, b.Bounds())
case paint.Event:
log.Printf("paint.Event: %T %+v", e, e)
}
}
Why is the window not updated when calling publish?
Why is the window updated inside the loop, even without calling publish?
Why does a size event generate a paint event? That log line I left is printed after a size event, but not a mouse event.
The same thing happens if a use a texture, uploading the buffer to the texture and then copying the texture to the window.
I only need a minimal UI with fast drawing and basic event handling and this package seemed pretty straightforward to use, but if there are other easy options I'm ok with that.
Edit:
As I am running this on Ubuntu 20.04 I believe the driver used is X11, so I looked into that implementation of
Publish and there is a comment which might explain why the call to publish is not needed to actually draw, but is useful for synchronization:
func (w *windowImpl) Publish() screen.PublishResult {
// This sync isn't needed to flush the outgoing X11 requests. Instead, it
// acts as a form of flow control. Outgoing requests can be quite small on
// the wire, e.g. draw this texture ID (an integer) to this rectangle (four
// more integers), but much more expensive on the server (blending a
// million source and destination pixels). Without this sync, the Go X11
// client could easily end up sending work at a faster rate than the X11
// server can serve.
w.s.xc.Sync()
return screen.PublishResult{}
}
But I still do not understand why drawing outside the loop does not work even with the explicit publish call.
Cheers!

Related

How to set specific height for text in golang? (SDL2 TTF)

So, I finally understood how to set text to screen using SDL2 ttf, but I also discovered, that text covers the entire screen, while I need text to be smaller, but I haven't found anything here https://pkg.go.dev/github.com/veandco/go-sdl2/ttf
Is there a way to make text smaller?
Here's my code:
func WriteChoices() {
font, _ := ttf.OpenFont("arial.ttf", 1920)
font.SetOutline(21)
surface, _ := font.RenderUTF8Solid("Player Computer", sdl.Color{0, 0, 50, 255})
texture, _ := renderer.CreateTextureFromSurface(surface)
renderer.Copy(texture, nil, nil)
font.Close()
surface.Free()
texture.Destroy()
}
You probably can just reduce font size by loading the font like so.
func WriteChoices() {
font, _ := ttf.OpenFont("arial.ttf", 20)
font.SetOutline(21)
surface, _ := font.RenderUTF8Solid("Player Computer", sdl.Color{0, 0, 50, 255})
texture, _ := renderer.CreateTextureFromSurface(surface)
renderer.Copy(texture, nil, sdl.{W:surface.W,H:surface.H})
font.Close()
surface.Free()
texture.Destroy()
}
This may work but i cant test right now, Sorry if i just wasted your time

Golang: remove structs older than 1h from slice

So I'm building a small utility that listens on a socket and stores incoming messages as a structs in a slice:
var points []Point
type Point struct {
time time.Time
x float64
y float64
}
func main() {
received = make([]Point, 0)
l, err := net.Listen("tcp4", ":8900")
// (...)
}
func processIncomingData(data string) {
// Parse icoming data that comes as: xValue,yValue
inData = strings.Split(data, ",")
x, err := strconv.ParseFloat(inData[0], 64);
if err != nil {
fmt.Println(err)
}
y, err := strconv.ParseFloat(inData[1], 64);
if err != nil {
fmt.Println(err)
}
// Store the new Point
points = append(points, Point{
time: time.Now(),
x: x,
y: y,
})
// Remove points older than 1h ?
}
Now, as you might imagine this will quickly fill my RAM. What's the best way (faster execution) to remove points older than 1h after appening each new one? I'll be getting new points 10-15 times peer second.
Thank you.
An approach I've used several times is to start a goroutine early in the project that looks something like this:
go cleanup()
...
func cleanup() {
for {
time.Sleep(...)
// do cleanup
}
}
Then what you could do is iterate over points using time.Since(point.time) to figure out how old each piece of data is. If it's too old, there's a slice trick to remove an item from a slice given it's position:
points = append(points[:i], points[i+1:]...)
(where i is the index to remove)
Because the points are in the slice in order of the time they were added, you could speed things up by simply finding the first index that isn't an hour old and doing points = points[i:] to chop off the old points from the beginning of the slice.
You may run into problems if you get a request that accesses the array while you're cleaning it up. Adding a sync.Mutex can help with that. Just lock the mutex before the cleanup and also attempt to lock the mutex anywhere else you write to the array. This may be premature optimization though. I'd experiment without the mutex before adding it in as this would effectively make interacting with points a serial operation and slow the service down.
The time.Sleep(...) in the loop is to prevent from cleaning too often. You might be tempted to set it to an hour since you want to delete points older than that but you might end up with a situation where a point is added immediately after a cleanup. On the next cleanup it'll be 59 mins old and you don't delete it, on the NEXT cleanup it's nearly 2 hours old. My rule of thumb is that I attempt to clean up every 1/10 the amount of time I want to allow an object to stay in memory but that's rather arbitrary. This approach means an object could be at most 1h 5m 59s old when it's deleted.

PNG Encode producing corrupt image

I'm reading a framebuffer from a video game console with golang - the buffer is in the format BRGA (which I then convert to RGBA). When I pass the information into the Go PNG Encoder, the image that comes out is not valid. The code i'm using is - where:
where data is a slice of RGBA pixels - 0x398000 in length, pitch is 5120, width is 1270, and height is 720)
img := &image.RGBA{
Pix: data,
Stride: pitch,
Rect: image.Rect(0, 0, width, height),
}
os.Remove("./img.png")
file, _ := os.Create("./img.png")
defer file.Close()
filewriter := bufio.NewWriter(file)
if err := png.Encode(filewriter, img); err != nil {
panic(err)
}
The expected outcome would be:
But the actual outcome is (only renders on Windows or when view in Chrome.. weird):
I have uploaded a binary dump of the RGBA slice if anybody would like it - https://1drv.ms/u/s!Ak-aZ3z7Ea8KwvUsqdP5OgWpZqxsGA
You are not flushing the buffered writer. You should do:
filewriter := bufio.NewWriter(file)
defer filewriter.Flush()
After this fix, I get a valid image:
Not a fix, and I want to comment but can't yet due to reputation, but will add to the Mac OS discrepancy.
The MacOS part of the problem appears to be new, showing up since either the latest 10.12.3 update, or something with Safari. I haven't narrowed down the source yet. But yes, there is something new about how a Mac system will encode/decode an image, causing it to be transparent or grey as a result. A project I am on is also suffering from this problem for the past few weeks and I'm still investigating where it breaks down.

Best way to remove borders / crop image

I am using a Go package (Go binding to ImageMagick's MagickWand C API) to ImageMagick where I'm removing borders from images (cropping). The way I am using the trim function can be found below.
Now the problem is the fuzzy factor. For example, if I set the value to 2000, the image (here is the source) still has some white images like these:
fuzz factor value 2000 --> result
fuzz factor value 10000 --> result
I have created a small html which illustrates the problem best. It contains both images: https://dl.dropboxusercontent.com/u/15684927/image-trim-problem.html
As you can see the source has some pixels on the bottom right corner which are causing the trouble. If I set the factor to 10000, I'm afraid that I will loose pixels on other pictures. If I set it on 2000, the trimming isn't done right in pictures like these.
So my actual question is: what is the best way to "crop" / "trim" images?
package main
import "gopkg.in/gographics/imagick.v1/imagick"
func main() {
imagick.Initialize()
defer imagick.Terminate()
inputFile := "tshirt-original.jpg"
outputFile := "trimmed.jpg"
mw := imagick.NewMagickWand()
// Schedule cleanup
defer mw.Destroy()
// read image
err := mw.ReadImage(inputFile)
if err != nil {
panic(err)
}
// first trim original image
// fuzz: by default target must match a particular pixel color exactly.
// However, in many cases two colors may differ by a small amount. The fuzz
// member of image defines how much tolerance is acceptable to consider two
// colors as the same. For example, set fuzz to 10 and the color red at
// intensities of 100 and 102 respectively are now interpreted as the same
// color for the purposes of the floodfill.
mw.TrimImage(10000)
// Set the compression quality to 95 (high quality = low compression)
err = mw.SetImageCompressionQuality(95)
if err != nil {
panic(err)
}
// save
err = mw.WriteImage(outputFile)
if err != nil {
panic(err)
}
}
Basically, your problem is that you have a high-frequency, high-amplitude artifact at the edge of your image. Or, put differently, a sharp, high peak at the edge, which, if you want to use trim, forces you to use such a high a fuzz-value to overcome this, that the algorithm also considers the 'actual content' as equal to the 'background' (border).
One solution here is to use a multi-step approach, whereby you first smooth out the edge artifacts and then apply trim to the resulting image. By smoothing it out, you get rid of the high peak and smear it out into a nice rolling hill. Rolling hills, in turn, can be easily trimmed with low fuzz values. This then provides you with the desired geometry which you can use to crop the original.
Specifically, let's take the original image:
Now, let's smooth out that ridge on the edge using a blur with a radius of 10 and a sigma of 10 through convert original.jpg -blur 10x10 10x10.jpg, which yields:
Now, you might notice that the artifacts on the edge have now pretty much disappeared.
We can now do a 'virtual' trim and ask ImageMagick what the result of the trim would be through convert 10x10.jpg -fuzz 2000 -format %# info:, which, according to the documentation gives you the "trim bounding box (without actually trimming)": 1326x1560+357+578%
Taking these values (except for the percentage sign) and using them for crop geometry, gives you the convert with crop command convert original.jpg -crop 1326x1560+357+578 cropped.jpg, which gives you:
Edit:
Now, since you want this as code, using imagick, here's the solution in code. It assumes you have the file stored as './data/original.jpg' and will store it as './data/trimmed.jpg'
package main
import (
"fmt"
"gopkg.in/gographics/imagick.v2/imagick"
)
func init() {
imagick.Initialize()
}
const originalImageFilename = "data/original.jpg"
func main() {
mw := imagick.NewMagickWand()
err := mw.ReadImage(originalImageFilename)
if err != nil {
fmt.Sprint(err.Error())
return
}
// Use a clone to determine what will happen
mw2 := mw.Clone()
mw2.BlurImage(10, 10)
mw2.TrimImage(2000)
_, _, xOffset, yOffset, err := mw2.GetImagePage()
if err != nil {
fmt.Sprint(err.Error())
return
}
trimmedWidth := mw2.GetImageWidth()
trimmedHeight := mw2.GetImageHeight()
mw2.Destroy()
mw.CropImage(trimmedWidth, trimmedHeight, xOffset, yOffset)
mw.WriteImage("data/trimmed.jpg")
mw.Destroy()
}

How to convert image.RGBA (image.Image) to image.Paletted?

I'm trying to create an animated GIF from a series of arbitrary non-paletted images. In order to create a paletted image, I need to come up with a palette somehow.
// RGBA, etc. images from somewhere else
var frames []image.Image
outGif := &gif.GIF{}
for _, simage := range frames {
// TODO: Convert image to paletted image
// bounds := simage.Bounds()
// palettedImage := image.NewPaletted(bounds, ...)
// Add new frame to animated GIF
outGif.Image = append(outGif.Image, palettedImage)
outGif.Delay = append(outGif.Delay, 0)
}
gif.EncodeAll(w, outGif)
Is there an easy way in golang stdlib to accomplish this?
It seems an automatic way of intelligently generating palettes is missing from the golang stdlib (correct me if I'm wrong here). But there seems to be a stub for providing your own Quantizer, which led me to the gogif project. (Which was the apparent source of image.Gif.)
I was able to borrow the MedianCutQuantizer from that project, defined here:
https://github.com/andybons/gogif/blob/master/mediancut.go
Which results in the following:
var subimages []image.Image // RGBA, etc. images from somewhere else
outGif := &gif.GIF{}
for _, simage := range subimages {
bounds := simage.Bounds()
palettedImage := image.NewPaletted(bounds, nil)
quantizer := gogif.MedianCutQuantizer{NumColor: 64}
quantizer.Quantize(palettedImage, bounds, simage, image.ZP)
// Add new frame to animated GIF
outGif.Image = append(outGif.Image, palettedImage)
outGif.Delay = append(outGif.Delay, 0)
}
gif.EncodeAll(w, outGif)
Instead of generating your own palette, you can also use on of the predefined (https://golang.org/pkg/image/color/palette/)
...
palettedImage := image.NewPaletted(bounds, palette.Plan9)
draw.Draw(palettedImage, palettedImage.Rect, simage, bounds.Min, draw.Over)
...

Resources