Encoding an Image to JPEG in Go - go

I have a struct called SpriteImage which is defined like this:
type SpriteImage struct {
dimentions image.Point
lastImgPosition image.Point
sprite *image.NRGBA
}
In my flow, I first initiate a new such struct:
func NewSpriteImage(width, height int) SpriteImage {
c := color.RGBA{0xff, 0xff, 0xff, 0xff}
blankImage := imaging.New(width, height, c)
return SpriteImage{
dimentions: image.Point{X: width, Y: height},
lastImgPosition: image.Point{X: 0, Y: 0},
sprite: blankImage,
}
}
And then I add images to this SpriteImage like so:
func (s *SpriteImage) AddImage(img image.Image) error {
imgWidth := img.Bounds().Dx()
imgHeight := img.Bounds().Dy()
// Make sure new image will fit into the sprite.
if imgWidth != s.dimentions.X {
return fmt.Errorf("image width %d mismatch sprite width %d", imgWidth, s.dimentions.X)
}
spriteHeightLeft := s.dimentions.Y - s.lastImgPosition.Y
if imgHeight > spriteHeightLeft {
return fmt.Errorf("image height %d won't fit into sprite, sprite free space %d ", imgHeight, s.dimentions.Y)
}
// add image to sprite
s.sprite = imaging.Paste(s.sprite, img, s.lastImgPosition)
// update next image position within sprite
s.lastImgPosition = s.lastImgPosition.Add(image.Point{X: 0, Y: imgHeight})
return nil
}
Eventually, I want to take this SpriteImage and encode it as JPEG. But it doesn't seem to work. The native JPEG Encode function takes up an image, but I have an image.NRGBA. So I'm using github.com/disintegration/imaging lib like so:
func (s SpriteImage) GetBytes() ([]byte, error) {
var b bytes.Buffer
w := bufio.NewWriter(&b)
if s.sprite == nil {
return nil, fmt.Errorf("sprite is nil")
}
if err := imaging.Encode(w, s.sprite, imaging.JPEG); err != nil {
return nil, err
}
return b.Bytes(), nil
}
However is seems that the bytes returned are not in fact JPEG. The native Go JPEG lib will not decode those bytes to a Go image struct. If I try to decode those bytes to image like so:
m, _, err := image.Decode(reader)
if err != nil {
log.Fatal(err)
}
I'm getting err:
image: unknown format
Any ideas?

So, it looks like this may be an issue with the github.com/disintegration/imaging lib. I posted an issue about it in the repo's page.
If I change my GetBytes() function to this, I get valid JPEG:
func (s SpriteImage) GetBytes() ([]byte, error) {
var b bytes.Buffer
w := bufio.NewWriter(&b)
if s.sprite == nil {
return nil, fmt.Errorf("sprite is nil")
}
im := s.sprite
err := jpeg.Encode(w, im, &jpeg.Options{jpegCompression})
if err != nil {
fmt.Println(err)
}
return b.Bytes(), nil
}

Maybe something to check: The image: unknown format is a common error when you don't include the image/jpeg lib and specifically decode it with it:
import (
"image/jpeg"
"io"
)
...
img, err := jpeg.Decode(r)
if err != nil {
return err
}
...
Some try to use image.Decode, but you still need to include the image/jpeg lib.
I believe it works with a blind include:
import (
_ "image/jpeg"
"image"
"io"
)
...
img, err := image.Decode(r)
if err != nil {
return err
}
...

Related

How to convert VP8 interframe into image with Pion/Webrtc?

I have tried snapshot example to convert VP8 video stream to jpeg in Pion/Webrtc examples.
It works well, but interframes are not included. An error message displayed when I was fixing it.
vp8: Golden / AltRef frames are not implemented.
This error comes from golang's vp8 package.
The modified code is shown below:
if !videoKeyframe {
decoder.Init(bytes.NewReader(sample.Data), len(sample.Data))
_, err := decoder.DecodeFrameHeader()
if err != nil {
fmt.Printf("Header Error: %s\n", err)
continue
}
_, err2 := decoder.DecodeFrame()
if err2 != nil {
fmt.Printf("DecodeFrame Error: %s\n", err2)
continue
}
}
I also tried to modify gstreamer-receive example. But it's hard for me to do in C.
Is there anyway to convert interframe to jpeg easily ?
Finally, I solved with libvpx-go
I modified code as shown below.
package main
import (
"bytes"
"fmt"
"image/jpeg"
// "log"
"os"
"github.com/pion/rtp"
"github.com/pion/rtp/codecs"
"github.com/pion/webrtc/v3/pkg/media/samplebuilder"
"github.com/xlab/libvpx-go/vpx"
)
// type Frame struct {
// *image.RGBA
// Timecode time.Duration
// IsKeyframe bool
// }
type VDecoder struct {
enabled bool
src <-chan *rtp.Packet
ctx *vpx.CodecCtx
iface *vpx.CodecIface
}
type VCodec string
const (
CodecVP8 VCodec = "V_VP8"
CodecVP9 VCodec = "V_VP9"
CodecVP10 VCodec = "V_VP10"
)
func NewVDecoder(codec VCodec, src <-chan *rtp.Packet) *VDecoder {
dec := &VDecoder{
src: src,
ctx: vpx.NewCodecCtx(),
}
switch codec {
case CodecVP8:
dec.iface = vpx.DecoderIfaceVP8()
case CodecVP9:
dec.iface = vpx.DecoderIfaceVP9()
default: // others are currently disabled
log.Println("[WARN] unsupported VPX codec:", codec)
return dec
}
err := vpx.Error(vpx.CodecDecInitVer(dec.ctx, dec.iface, nil, 0, vpx.DecoderABIVersion))
if err != nil {
log.Println("[WARN]", err)
return dec
}
dec.enabled = true
return dec
}
func (v *VDecoder) Save(savePath string) { //, out chan<- Frame
// defer close(out)
i := 0
sampleBuilder := samplebuilder.New(20000, &codecs.VP8Packet{}, 90000)
for pkt := range v.src {
sampleBuilder.Push(pkt)
// Use SampleBuilder to generate full picture from many RTP Packets
sample := sampleBuilder.Pop()
if sample == nil {
continue
}
if !v.enabled {
continue
}
dataSize := uint32(len(sample.Data))
err := vpx.Error(vpx.CodecDecode(v.ctx, string(sample.Data), dataSize, nil, 0))
if err != nil {
log.Println("[WARN]", err)
continue
}
var iter vpx.CodecIter
img := vpx.CodecGetFrame(v.ctx, &iter)
if img != nil {
img.Deref()
// out <- Frame{
// RGBA: img.ImageRGBA(),
// Timecode: time.Duration(pkt.Timestamp),
// }
i++
buffer := new(bytes.Buffer)
if err = jpeg.Encode(buffer, img.ImageYCbCr(), nil); err != nil {
// panic(err)
fmt.Printf("jpeg Encode Error: %s\r\n", err)
}
fo, err := os.Create(fmt.Sprintf("%s%d%s", savePath, i, ".jpg"))
if err != nil {
fmt.Printf("image create Error: %s\r\n", err)
//panic(err)
}
// close fo on exit and check for its returned error
defer func() {
if err := fo.Close(); err != nil {
panic(err)
}
}()
if _, err := fo.Write(buffer.Bytes()); err != nil {
fmt.Printf("image write Error: %s\r\n", err)
//panic(err)
}
fo.Close()
}
}
}
Just add 2 lines of code to save sequential images
vd := NewVDecoder(CodecVP8, rtpChan)
go vd.Save("/ramdisk/image_seq/xxx")
Enjoy!
P.S. don't forget to install libvpx-go 's dependencies before running go get and make sure libvpx 's version is 1.8
If you need to convert interframe to jpeg the only way would be to decode every frame from your stream. I don't think there is any other way as interframes depend on previous frames.
Sync Solution
This is an alternative version of the same solution of CK So
It is based on libvpx-go but dose not use channels.
package main
import (
"bytes"
"fmt"
"image/jpeg"
"log"
// "log"
"os"
"github.com/xlab/libvpx-go/vpx"
)
// type Frame struct {
// *image.RGBA
// Timecode time.Duration
// IsKeyframe bool
// }
type VDecoderSync struct {
enabled bool
//src <-chan []byte this blocked the thread even if fed.
ctx *vpx.CodecCtx
iface *vpx.CodecIface
i int
}
type VCodecSync string
const (
CodecVP8Sync VCodecSync = "V_VP8"
CodecVP9Sync VCodecSync = "V_VP9"
CodecVP10Sync VCodecSync = "V_VP10"
)
func NewVDecoderSync(codec VCodecSync) *VDecoderSync {
dec := &VDecoderSync{
ctx: vpx.NewCodecCtx(),
}
switch codec {
case CodecVP8Sync:
dec.iface = vpx.DecoderIfaceVP8()
case CodecVP9Sync:
dec.iface = vpx.DecoderIfaceVP9()
default: // others are currently disabled
log.Println("[WARN] unsupported VPX codec:", codec)
return dec
}
err := vpx.Error(vpx.CodecDecInitVer(dec.ctx, dec.iface, nil, 0, vpx.DecoderABIVersion))
if err != nil {
log.Println("[WARN]", err)
return dec
}
dec.enabled = true
return dec
}
func (v *VDecoderSync) EncodeToJson(savePath string, pkt []byte, print_to_file bool) []byte {
sample := pkt
dataSize := uint32(len(sample))
err := vpx.Error(vpx.CodecDecode(v.ctx, string(sample), dataSize, nil, 0))
if err != nil {
log.Println("[WARN]", err)
return nil
}
var iter vpx.CodecIter
img := vpx.CodecGetFrame(v.ctx, &iter)
if img != nil {
img.Deref()
// out <- Frame{
// RGBA: img.ImageRGBA(),
// Timecode: time.Duration(pkt.Timestamp),
// }
buffer := new(bytes.Buffer)
if err = jpeg.Encode(buffer, img.ImageYCbCr(), nil); err != nil {
// panic(err)
fmt.Printf("jpeg Encode Error: %s\r\n", err)
}
bytes_to_send := buffer.Bytes()
if print_to_file {
v.i++
v.i = v.i % 30
fo, err := os.Create(fmt.Sprintf("%s%d%s", savePath, v.i, ".jpg"))
if err != nil {
fmt.Printf("image create Error: %s\r\n", err)
//panic(err)
}
// close fo on exit and check for its returned error
defer func() {
if err := fo.Close(); err != nil {
panic(err)
}
}()
if _, err := fo.Write(bytes_to_send); err != nil {
fmt.Printf("image write Error: %s\r\n", err)
//panic(err)
}
}
return bytes_to_send
}
return nil
}

How to load local html to headless chrome

I have local html files which I need to render them and get their screenshot.
I could not find any solution to load html codes in chromedp
Is that possible?
Yes, it is.
In chromedp documentation there is a nice example https://github.com/chromedp/examples/blob/master/screenshot/main.go. Only difference is that instead of using "https://..." in urlstring you will use "file:///<absolute_path_to_your_file>" .
Example of code, which I mostly took from the example link and used to make screenshot of html stored on my local system:
package main
import (
"context"
"io/ioutil"
"log"
"math"
"github.com/chromedp/cdproto/emulation"
"github.com/chromedp/cdproto/page"
"github.com/chromedp/chromedp"
)
func main() {
// create context
ctx, cancel := chromedp.NewContext(context.Background())
defer cancel()
//if you want to use html from your local filesystem use file:/// + absolute path to your html file
url := "file:///home/oktogen/Documents/Notebooks/2020/May/AnalysisJobs/FlaskApp/templates/index.html"
// capture screenshot of an element
var buf []byte
// capture entire browser viewport, returning png with quality=90
if err := chromedp.Run(ctx, fullScreenshot(url, 90, &buf)); err != nil {
log.Fatal(err)
}
if err := ioutil.WriteFile("fullScreenshot.png", buf, 0644); err != nil {
log.Fatal(err)
}
}
// fullScreenshot takes a screenshot of the entire browser viewport.
//
// Liberally copied from puppeteer's source.
//
// Note: this will override the viewport emulation settings.
func fullScreenshot(urlstr string, quality int64, res *[]byte) chromedp.Tasks {
return chromedp.Tasks{
chromedp.Navigate(urlstr),
chromedp.ActionFunc(func(ctx context.Context) error {
// get layout metrics
_, _, contentSize, err := page.GetLayoutMetrics().Do(ctx)
if err != nil {
return err
}
width, height := int64(math.Ceil(contentSize.Width)), int64(math.Ceil(contentSize.Height))
// force viewport emulation
err = emulation.SetDeviceMetricsOverride(width, height, 1, false).
WithScreenOrientation(&emulation.ScreenOrientation{
Type: emulation.OrientationTypePortraitPrimary,
Angle: 0,
}).
Do(ctx)
if err != nil {
return err
}
// capture screenshot
*res, err = page.CaptureScreenshot().
WithQuality(quality).
WithClip(&page.Viewport{
X: contentSize.X,
Y: contentSize.Y,
Width: contentSize.Width,
Height: contentSize.Height,
Scale: 1,
}).Do(ctx)
if err != nil {
return err
}
return nil
}),
}
}

Reading Image in Go

func (sticky *Sticky) DrawImage(W, H int) (img *image, err error) {
myImage := image.NewRGBA(image.Rect(0, 0, 10, 25))
myImage.Pix[0] = 55 // 1st pixel red
myImage.Pix[1] = 155 // 1st pixel green
return myImage ,nil
}
I am creating an image. I want to read the existing Image and return in this Function. How I can I do that?
Something like this:
func getImageFromFilePath(filePath string) (image.Image, error) {
f, err := os.Open(filePath)
if err != nil {
return nil, err
}
defer f.Close()
image, _, err := image.Decode(f)
return image, err
}
references
https://golang.org/pkg/image/#Decode
https://golang.org/pkg/os/#Open
try this:
package main
import (
"fmt"
"image"
"image/png"
"os"
)
func main() {
// Read image from file that already exists
existingImageFile, err := os.Open("test.png")
if err != nil {
// Handle error
}
defer existingImageFile.Close()
// Calling the generic image.Decode() will tell give us the data
// and type of image it is as a string. We expect "png"
imageData, imageType, err := image.Decode(existingImageFile)
if err != nil {
// Handle error
}
fmt.Println(imageData)
fmt.Println(imageType)
// We only need this because we already read from the file
// We have to reset the file pointer back to beginning
existingImageFile.Seek(0, 0)
// Alternatively, since we know it is a png already
// we can call png.Decode() directly
loadedImage, err := png.Decode(existingImageFile)
if err != nil {
// Handle error
}
fmt.Println(loadedImage)
}
references
https://www.devdungeon.com/content/working-images-go

Golang how can I upload external images without rotating them

I have been working on an image upload functionality for a few weeks and I just about have it done. I am using Golang as a backend language and it's purpose is to upload images sent from IOS devices to amazon s3 . In the process of uploading the image I also resize them and this has caused problems primarily the decode method sometimes rotates images which I do not want
file, handler, err := r.FormFile("file")
if err != nil {
fmt.Println("Error Uploading Image")
return
}
defer file.Close()
// the code below sometimes rotates an image
img,err := imaging.Decode(file)
if err != nil {
print("Imaging Open error")
}
new_image := imaging.Resize(img,400,400, imaging.Lanczos)
The library I am using is this one https://github.com/disintegration/imaging which is fantastic and the example they showed was this
src, err := imaging.Open("testdata/lena_512.png")
if err != nil {
log.Fatalf("Open failed: %v", err)
}
src = imaging.Resize(src, 256, 0, imaging.Lanczos)
That example is fine however my images are not stored locally they are coming from IOS devices is there something that I can do to fix this problem ? Some of my images are being saved like this
Some images are rotated like this and it is the Decode method doing it
I can correct it by rotating the image however some other images that do not get rotated by decode end up being rotated by the Rotate270 method .
img,err := imaging.Decode(file)
if err != nil {
print("Imaging Open error")
}
// rotate the image
img = imaging.Rotate270(img)
new_image := imaging.Resize(img,400,400, imaging.Lanczos)
This is how the image is saved and looks after I rotate it . Is there someway I can upload external images without having to use decode or just fixing the decode issue ? Imaging.Resize first parameter takes in type image.Image and here is my full code
func myImages(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
var buff bytes.Buffer
file, handler, err := r.FormFile("file")
if err != nil {
fmt.Println("Error Uploading Image")
return
}
defer file.Close()
img,err := imaging.Decode(file)
if err != nil {
print("Imaging Open error")
}
new_image := imaging.Resize(img,400,400, imaging.Lanczos)
var buf bytes.Buffer
err = imaging.Encode(&buf,new_image, imaging.JPEG)
if err != nil {
log.Println(err)
return
}
}
Unfortunately the go stdlib image package does not handle images which are marked as rotated with exif tags (like those here taken on a device upside down). The bug about this is here:
https://github.com/golang/go/issues/4341
You can see an example of handling this in camlistore, but it is quite involved:
https://camlistore.googlesource.com/camlistore/+/master/pkg/images/images.go
First you have to decode exif options (DecodeOpts), then you have to check which orientation it has (out of 8 possible), then rotate as necessary. It's painful and there is no easy solution as yet. You can use this package to read exif data though:
https://github.com/rwcarlsen/goexif
I had the same problem and solved it with a help of two libraries:
go get -u github.com/disintegration/imaging
go get -u github.com/rwcarlsen/goexif/exif
with exif you can obtain the actual orientation of the image and with imaging you can perform the operations which are needed to revert the image to orientation 1.
Since a replaced image has no exif information left, the result is then shown correctly.
// ReadImage makes a copy of image (jpg,png or gif) and applies
// all necessary operation to reverse its orientation to 1
// The result is a image with corrected orientation and without
// exif data.
func ReadImage(fpath string) *image.Image {
var img image.Image
var err error
// deal with image
ifile, err := os.Open(fpath)
if err != nil {
logrus.Warnf("could not open file for image transformation: %s", fpath)
return nil
}
defer ifile.Close()
filetype, err := GetSuffix(fpath)
if err != nil {
return nil
}
if filetype == "jpg" {
img, err = jpeg.Decode(ifile)
if err != nil {
return nil
}
} else if filetype == "png" {
img, err = png.Decode(ifile)
if err != nil {
return nil
}
} else if filetype == "gif" {
img, err = gif.Decode(ifile)
if err != nil {
return nil
}
}
// deal with exif
efile, err := os.Open(fpath)
if err != nil {
logrus.Warnf("could not open file for exif decoder: %s", fpath)
}
defer efile.Close()
x, err := exif.Decode(efile)
if err != nil {
if x == nil {
// ignore - image exif data has been already stripped
}
logrus.Errorf("failed reading exif data in [%s]: %s", fpath, err.Error())
}
if x != nil {
orient, _ := x.Get(exif.Orientation)
if orient != nil {
logrus.Infof("%s had orientation %s", fpath, orient.String())
img = reverseOrientation(img, orient.String())
} else {
logrus.Warnf("%s had no orientation - implying 1", fpath)
img = reverseOrientation(img, "1")
}
imaging.Save(img, fpath)
}
return &img
}
// reverseOrientation amply`s what ever operation is necessary to transform given orientation
// to the orientation 1
func reverseOrientation(img image.Image, o string) *image.NRGBA {
switch o {
case "1":
return imaging.Clone(img)
case "2":
return imaging.FlipV(img)
case "3":
return imaging.Rotate180(img)
case "4":
return imaging.Rotate180(imaging.FlipV(img))
case "5":
return imaging.Rotate270(imaging.FlipV(img))
case "6":
return imaging.Rotate270(img)
case "7":
return imaging.Rotate90(imaging.FlipV(img))
case "8":
return imaging.Rotate90(img)
}
logrus.Errorf("unknown orientation %s, expect 1-8", o)
return imaging.Clone(img)
}
You can find this implementation also here:
https://github.com/Macilias/go-images-orientation

Looking for better way to save an in memory image to file

The goal of the code is to download an image, stick it to a larger parent image and save the result.
After quite a few failures I ended up with the following code that does work.
However, is there a better way than using bytes.Buffer and a writer to save the target image to a file / pass it to an httpResponse?
package main
import (
"image"
"image/draw"
"image/jpeg"
"os"
// "image/color"
// "io/ioutil"
// "fmt"
"bufio"
"bytes"
"log"
"net/http"
)
func main() {
// Fetch an image.
resp, err := http.Get("http://katiebrookekennels.com/wp-content/uploads/2014/10/dog-bone4.jpg")
if err != nil {
log.Panic(err)
}
defer resp.Body.Close()
// Keep an in memory copy.
myImage, err := jpeg.Decode(resp.Body)
if err != nil {
log.Panic(err)
}
// Prepare parent image where we want to position child image.
target := image.NewRGBA(image.Rect(0, 0, 800, 800))
// Draw white layer.
draw.Draw(target, target.Bounds(), image.White, image.ZP, draw.Src)
// Draw child image.
draw.Draw(target, myImage.Bounds(), myImage, image.Point{0, 0}, draw.Src)
// Encode to jpeg.
var imageBuf bytes.Buffer
err = jpeg.Encode(&imageBuf, target, nil)
if err != nil {
log.Panic(err)
}
// Write to file.
fo, err := os.Create("img.jpg")
if err != nil {
panic(err)
}
fw := bufio.NewWriter(fo)
fw.Write(imageBuf.Bytes())
}
jpeg.Encode() expects an io.Writer to write the encoded image to (in JPEG format). Both *os.File and http.ResponseWriter implement io.Writer too, so instead of a bytes.Buffer, you can directly pass a file or HTTP response writer too.
To save an image directly to a file:
f, err := os.Create("img.jpg")
if err != nil {
panic(err)
}
defer f.Close()
if err = jpeg.Encode(f, target, nil); err != nil {
log.Printf("failed to encode: %v", err)
}
To serve it as an image:
// w is of type http.ResponseWriter:
if err := jpeg.Encode(w, target, nil); err != nil {
log.Printf("failed to encode: %v", err)
}
You can see some examples in these questions (saving/loading JPEG and PNG images):
Draw a rectangle in Golang?
How to add a simple text label to an image in Go?
Change color of a single pixel - Golang image

Resources