Used a decimal point of 200 as the precision, I need to calculate a number from atto to decimal number similar screenshot.
To get the values at precision of nano and atto you can use %.9f and %.18f in fmt.Printf() respectively,I created a small program to get your value of 0.000000000000099707 as follows:
package main
import (
"fmt"
"math"
)
func main() {
powr := math.Pow(10, -18)
numb := 99707 * powr
fmt.Println("number", numb)
fmt.Printf("\nthe value in atto %.18f\n", numb)
}
Output:
number 9.970700000000001e-14
the value in atto 0.000000000000099707
You can use the github.com/shopspring/decimal package for this as well. This library can represents numbers up to 2^31 (2147483648) digits. Here is a simple code to do the calculation:
d := decimal.NewFromInt(99707)
d10 := decimal.NewFromInt(10)
dpow := decimal.NewFromInt(-18)
d10pow := d10.Pow(dpow)
dmul := d.Mul(d10pow)
fmt.Println(dmul)
This can simplified to:
d := decimal.NewFromInt(99707).Mul(decimal.NewFromInt(10).Pow(decimal.NewFromInt(-18)))
fmt.Println(d)
Output: 0.000000000000099707
See playground
I was interested in how to do this so I found the apd package from cockroach that handles arbitrary precision calculations. You can use it like this:
import (
"fmt"
"github.com/cockroachdb/apd"
)
func main() {
// 99707 * 10^(-18)
n1 := apd.New(99707, 0)
n2 := apd.New(10, 0)
n3 := apd.New(-18, 0)
c := apd.BaseContext.WithPrecision(200)
res := apd.New(0,0)
ctx, err := c.Pow(res, n2, n3)
if err != nil {
panic(err)
}
ctx, err = c.Mul(res, res, n1)
if err != nil {
panic(err)
}
fmt.Println(ctx.Inexact(), res.Text('f'))
}
And it will output:
false 0.000000000000099707
You will have to be careful with the loss of precision that may happen and look at the inexact field.
Related
With following IDL files my intention is to measure the serialization speed of Flatbuffer . I am using golang for my analysis
namespace MyFlat;
struct Vertices {
x : double;
y :double;
}
table Polygon {
polygons : [Vertices];
}
table Layer {
polygons : [Polygon];
}
root_type Layer;
Here is the code I have written for calculation
package main
import (
"MyFlat"
"fmt"
"io/ioutil"
"log"
"strconv"
"time"
flatbuffers "github.com/google/flatbuffers/go"
)
func calculation(size int, vertices int) {
b := flatbuffers.NewBuilder(0)
var polyoffset []flatbuffers.UOffsetT
rawSize := ((16 * vertices) * size) / 1024
var vec1 flatbuffers.UOffsetT
var StartedAtMarshal time.Time
var EndedAtMarshal time.Time
StartedAtMarshal = time.Now()
for k := 0; k < size; k++ {
MyFlat.PolygonStartPolygonsVector(b, vertices)
for i := 0; i < vertices; i++ {
MyFlat.CreateVertices(b, 2.0, 2.4)
}
vec1 = b.EndVector(vertices)
MyFlat.PolygonStart(b)
MyFlat.PolygonAddPolygons(b, vec1)
polyoffset = append(polyoffset, MyFlat.PolygonEnd(b))
}
MyFlat.LayerStartPolygonsVector(b, size)
for _, offset := range polyoffset {
b.PrependUOffsetT(offset)
}
vec := b.EndVector(size)
MyFlat.LayerStart(b)
MyFlat.LayerAddPolygons(b, vec)
finalOffset := MyFlat.LayerEnd(b)
b.Finish(finalOffset)
EndedAtMarshal = time.Now()
SeElaprseTime := EndedAtMarshal.Sub(StartedAtMarshal).String()
mybyte := b.FinishedBytes()
file := "/tmp/myflat_" + strconv.Itoa(size) + ".txt"
if err := ioutil.WriteFile(file, mybyte, 0644); err != nil {
log.Fatalln("Failed to write address book:", err)
}
StartedAt := time.Now()
layer := MyFlat.GetRootAsLayer(mybyte, 0)
size = layer.PolygonsLength()
obj := &MyFlat.Polygon{}
layer.Polygons(obj, 1)
for i := 0; i < obj.PolygonsLength(); i++ {
objVertices := &MyFlat.Vertices{}
obj.Polygons(objVertices, i)
fmt.Println(objVertices.X(), objVertices.Y())
}
EndedAt := time.Now()
DeElapseTime := EndedAt.Sub(StartedAt).String()
fmt.Println(size, ",", vertices, ", ", SeElaprseTime, ",", DeElapseTime, ",", (len(mybyte) / 1024), ",", rawSize)
}
func main() {
data := []int{500000, 1000000, 1500000, 3000000, 8000000}
for _, size := range data {
//calculation(size, 5)
//calculation(size, 10)
calculation(size, 20)
}
}
Problem is I find it serialization is quite slow compared to protobuff with similar idl.
For 3M polygons serialization its taking almost 4.1167037s. Where in protobuf its taking half. Deserilization time for flatbuf is very less (in micro sec). In protobuf its quite high. But still if I add both flatbuf performance is lower.
Do you see any optimized way to serialize it. Flatbuffer is having a method createBinaryVector for byte vector but there is no direct way to serialize vector of polygon from a existing a user defined type vector.
I am adding protobuf code also
syntax = 'proto3';
package myproto;
message Polygon {
repeated double v_x = 1 ;
repeated double v_y = 2 ;
}
message CADData {
repeated Polygon polygon = 1;
string layer_name = 2;
}
Go Code with protobuf
package main
import (
"fmt"
"io/ioutil"
"log"
"math/rand"
"myproto"
"strconv"
"time"
"github.com/golang/protobuf/proto"
)
func calculation(size int, vertices int) {
var comp []*myproto.Polygon
var vx []float64
var vy []float64
for i := 0; i < vertices; i++ {
r := 0 + rand.Float64()*(10-0)
vx = append(vx, r)
vy = append(vy, r/2)
}
rawSize := ((16 * vertices) * size) / 1024
StartedAtMarshal := time.Now()
for i := 0; i < size; i++ {
comp = append(comp, &myproto.Polygon{
VX: vx,
VY: vy,
})
}
pfs := &myproto.CADData{
LayerName: "Layer",
Polygon: comp,
}
data, err := proto.Marshal(pfs)
if err != nil {
log.Fatal("marshaling error: ", err)
}
EndedAtMarshal := time.Now()
SeElaprseTime := EndedAtMarshal.Sub(StartedAtMarshal).String()
file := "/tmp/myproto_" + strconv.Itoa(size) + ".txt"
if err := ioutil.WriteFile(file, data, 0644); err != nil {
log.Fatalln("Failed to write address book:", err)
}
StartedAt := time.Now()
serialized := &myproto.CADData{}
proto.Unmarshal(data, serialized)
EndedAt := time.Now()
DeElapseTime := EndedAt.Sub(StartedAt).String()
fmt.Println(size, ",", vertices, ", ", SeElaprseTime, ",", DeElapseTime, ",", (len(data) / 1024), ",", rawSize)
}
func main() {
data := []int{500000, 1000000, 1500000, 3000000, 8000000}
for _, size := range data {
// calculation(size, 5)
//calculation(size, 10)
calculation(size, 20)
}
}
The time you give, is that for serialization, de-serialization, or both?
Your de-serialization code is likely entirely dominated by fmt.Println. Why don't you instead do sum += objVertices.X() + objVertices.Y() and print sum after timing is done? Can you pull objVertices := &MyFlat.Vertices{} outside of the loop?
You didn't post your protobuf code. Are you including in the timing the time to create the tree of objects which is being serialized (which is required for use in Protobuf but not in FlatBuffers)? Similarly, are you doing the timed (de-)serialization at least a 1000x or so, so you can include the cost of GC (Protobuf allocates a LOT of objects, FlatBuffers allocates few/none) in your comparison?
If after you do the above, it is still slower, post on the FlatBuffers github issues, the authors of the Go port may be able to help further. Make sure you post full code for both systems, and full timings.
Note generally: the design of FlatBuffers is such that it will create the biggest performance gap with Protobuf in C/C++. That said, it should still be a lot faster in Go also. There are unfortunate things about Go however that prevent it from maximizing the performance potential.
b := flatbuffers.NewBuilder(0)
I'm not sure what the "grows automatically" behavior is in Go for flatbuffers, but I'm pretty sure requiring the buffer to grow automatically is not the preferred pattern. Could you try doing your same timing comparison after initializing the buffer with flatbuffers.NewBuilder(moreBytesThanTheMessageNeeds)?
In the following code after one recursion the inputs are not read(from stdin). Output is incorrect if N is greater than 1.
X is read as 0 after one recursive call and hence the array is not read after that.
Program is supposed to print sum of squares of positive numbers in the array. P.S has to done only using recursion
package main
// Imports
import (
"fmt"
"bufio"
"os"
"strings"
"strconv"
)
// Global Variables
var N int = 0;
var X int = 0;
var err error;
var out int = 0;
var T string = "0"; // All set to 0 just in case there is no input, so we don't crash with nil values.
func main() {
// Let's grab our input.
fmt.Print("Enter N: ")
fmt.Scanln(&N)
// Make our own recursion.
loop()
}
func loop() {
if N == 0 {return}
// Grab our array length.
fmt.Scanln(&X)
tNum := make([]string, X)
// Grab our values and put them into an array.
in := bufio.NewReader(os.Stdin)
T, err = in.ReadString('\n')
tNum = strings.Fields(T)
// Parse the numbers, square, and add.
add(tNum)
// Output and reset.
fmt.Print(out)
out = 0;
N--
loop()
}
// Another loop, until X is 0.
func add(tNum []string) {
if X == 0 {return}
// Parse a string to an integer.
i, err := strconv.Atoi(tNum[X-1])
if err != nil {}
// If a number is negative, make it 0, so when we add its' square, it does nothing.
if (i < 0) {
i = 0;
}
// Add to our total!
out = out + i*i
X--
add(tNum)
}
Input:
2
4
2 4 6 8
3
1 3 9
Output:
1200
Expected output:
120
91
bufio.Reader, like the name suggests, use a buffer to store what is in the reader (os.Stdin here), which means, each time you create a bufio.Reader and read it once, there are more than what is read stored into the buffer, and thus the next time you read from the reader (os.Stdin), you do not read from where you left.
You should only have one bufio.Reader for os.Stdin. Make it global (if that is a requirement) or make it an argument. In fact, bufio package has a Scanner type that can splits spaces and new lines so you don't need to call strings.Fields.
I think you should practise doing this yourself, but here is a playground link: https://play.golang.org/p/7zBDYwqWEZ0
Here is an example that illustrates the general principles.
// Print the sum of the squares of positive numbers in the input.
package main
import (
"bufio"
"fmt"
"io"
"os"
"strconv"
"strings"
)
func sumOfSquares(sum int, s *bufio.Scanner, err error) (int, *bufio.Scanner, error) {
if err != nil {
return sum, s, err
}
if !s.Scan() {
err = s.Err()
if err == nil {
err = io.EOF
}
return sum, s, err
}
for _, f := range strings.Fields(s.Text()) {
i, err := strconv.Atoi(f)
if err != nil || i <= 0 {
continue
}
sum += i * i
}
return sumOfSquares(sum, s, nil)
}
func main() {
sum := 0
s := bufio.NewScanner(os.Stdin)
sum, s, err := sumOfSquares(sum, s, nil)
if err != nil && err != io.EOF {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println(sum)
}
Input:
2
4
2 4 6 8
3
1 3 9
Output:
240
While running:
package main
import (
"fmt"
"math/big"
)
func main() {
a := big.NewFloat(float64(2.1234))
fmt.Println(a.Text(102,18))
}
I expected 2.123400000000000000 as output, but instead got 2.123400000000000176.
Can someone explain me why I do not have the expected number?
big.NewFloat(float64(2.1234))
float64(2.1234) converts to Go float64 (IEEE-754 64-bit floating-point), which has 53 bits of precision.
For example,
package main
import (
"fmt"
"math/big"
)
func main() {
// 53 bits of precision (float64)
a := big.NewFloat(float64(2.1234))
fmt.Println(a.Text(102, 18))
x := "2.1234"
// 53 bits of precision
f, _, err := big.ParseFloat(x, 10, 53, big.ToNearestEven)
if err != nil {
panic(err)
}
fmt.Println(f.Text(102, 18))
// 256 bits of precision
f, _, err = big.ParseFloat(x, 10, 256, big.ToNearestEven)
if err != nil {
panic(err)
}
fmt.Println(f.Text(102, 18))
}
Playground: https://play.golang.org/p/z5iK90lQcD9
Output:
2.123400000000000176
2.123400000000000176
2.123400000000000000
In go image package, i don't see any methods which can be used to compare two images? Is it possible to do image comparison in go similar to ImageMagick?
If you are trying to compare two images and just need to boil it down to a single number, the following will work. This is useful in (for example) genetic algorithms, where you want to compare a set of candidates and choose the one that differs the least from a reference image:
Visit every pixel, break it down into its parts: R, G, B, & A (in go: image.At(x,y).RGBA())
Subtract the RGBA vals from their corresponding pixel vals in the reference image.
Square the differences, add them up.
Take the square root of the total sum.
This number will give you a rough idea of how much the images differ.
If you know that the two images are both instances of image.RGBA (or you can convert them), then you can do something even faster: just grab the bytes directly from RGBA.Pix. That's what I do here, and it's roughly 10x faster than doing img.At(x,y).RGBA() for every pixel pair:
func FastCompare(img1, img2 *image.RGBA) (int64, error) {
if img1.Bounds() != img2.Bounds() {
return 0, fmt.Errorf("image bounds not equal: %+v, %+v", img1.Bounds(), img2.Bounds())
}
accumError := int64(0)
for i := 0; i < len(img1.Pix); i++ {
accumError += int64(sqDiffUInt8(img1.Pix[i], img2.Pix[i]))
}
return int64(math.Sqrt(float64(accumError))), nil
}
func sqDiffUInt8(x, y uint8) uint64 {
d := uint64(x) - uint64(y)
return d * d
}
Try https://github.com/vitali-fedulov/images3. I wrote
this package to be able to find near duplicates. There is a live web-demo with the same algorithm, so you can get an idea how well the package suites your needs.
Inspired by George's answer.
The function below is not so fast, but it allows you to visually assess the difference in images.
func ImgCompare(img1, img2 image.Image) (int64, image.Image, error) {
bounds1 := img1.Bounds()
bounds2 := img2.Bounds()
if bounds1 != bounds2 {
return math.MaxInt64, nil, fmt.Errorf("image bounds not equal: %+v, %+v", img1.Bounds(), img2.Bounds())
}
accumError := int64(0)
resultImg := image.NewRGBA(image.Rect(
bounds1.Min.X,
bounds1.Min.Y,
bounds1.Max.X,
bounds1.Max.Y,
))
draw.Draw(resultImg, resultImg.Bounds(), img1, image.Point{0, 0}, draw.Src)
for x := bounds1.Min.X; x < bounds1.Max.X; x++ {
for y := bounds1.Min.Y; y < bounds1.Max.Y; y++ {
r1, g1, b1, a1 := img1.At(x, y).RGBA()
r2, g2, b2, a2 := img2.At(x, y).RGBA()
diff := int64(sqDiffUInt32(r1, r2))
diff += int64(sqDiffUInt32(g1, g2))
diff += int64(sqDiffUInt32(b1, b2))
diff += int64(sqDiffUInt32(a1, a2))
if diff > 0 {
accumError += diff
resultImg.Set(
bounds1.Min.X+x,
bounds1.Min.Y+y,
color.RGBA{R: 255, A: 255})
}
}
}
return int64(math.Sqrt(float64(accumError))), resultImg, nil
}
func sqDiffUInt32(x, y uint32) uint64 {
d := uint64(x) - uint64(y)
return d * d
}
With two of the current answers here, the images need to be the same size, or the comparison fails. A third answer here uses vitali-fedulov/images, which doesn't have any method to get the
difference between two images, only a Similar function that returns a bool determining if two images are similar. Further, the answer at Rosetta Code also fails if the images are different sizes.
So if I was to implement my own solution, first I would need to scale down the larger image. I found x/image/draw and nfnt/resize for that purpose, but I thought maybe I could find something, to kill two birds with one stone. To that end, I did find some packages that scale the images as needed, take a hash of each, and get the difference of the hashes. Here is corona10/goimagehash:
package main
import (
"github.com/corona10/goimagehash"
"image/jpeg"
"os"
)
func hash(name string) (*goimagehash.ImageHash, error) {
f, err := os.Open(name)
if err != nil {
return nil, err
}
defer f.Close()
i, err := jpeg.Decode(f)
if err != nil {
return nil, err
}
return goimagehash.AverageHash(i)
}
Example:
package main
func main() {
a, err := hash("mb.jpg")
if err != nil {
panic(err)
}
b, err := hash("hqdefault.jpg")
if err != nil {
panic(err)
}
d, err := a.Distance(b)
if err != nil {
panic(err)
}
println(d)
}
How do you convert a long string of digits (50 digits) into an integer in Go?
I am getting the output for the code below:
number = 2147483647
err = strconv.ParseInt: parsing "37107287533902102798797998220837590246510135740250
": value out of range
It seems to be able to convert numbers only up to 2147483647.
package main
import "fmt"
import "io/ioutil"
import "strings"
import "strconv"
var (
number int64
)
func main() {
fData,err := ioutil.ReadFile("one-hundred_50.txt")
if err != nil {
fmt.Println("Err is ",err)
}
strbuffer := string(fData)
lines := strings.Split(strbuffer, "\n")
for i, line := range lines {
fmt.Printf("%d: %s\n", i, line)
number, err := strconv.Atoi(line)
fmt.Println("number = ", number)
fmt.Println("err = ", err)
}
}
You want the math/big package, which provides arbitrary-precision integer support.
import "math/big"
func main() {
// ...
for i, line := range lines {
bi := big.NewInt(0)
if _, ok := bi.SetString(line, 10); ok {
fmt.Printf("number = %v\n", bi)
} else {
fmt.Printf("couldn't interpret line %#v\n", line)
}
}
}
Here's a quick example of it working.