In Python, using the struct module, I can do something like this to get a packed representation of a value as a string:
import struct
print struct.pack('L', 64)
"#\x00\x00\x00\x00\x00\x00\x00"
struct.unpack('L', '#\x00\x00\x00\x00\x00\x00\x00')
(64,)
I'd like to do something similar in Go, except I'm a little unclear on how to do so. I know I can do something like this:
import (
"encoding/binary"
"fmt"
)
bs := make([]byte, 8)
binary.PutUvarint(bs, uint64(64))
fmt.Printf("%s", bs)
"#"
But that's very different and probably not what I want.
Yes, "encoding/binary" is what you want, you just don't want the variable length format.
https://play.golang.org/p/e81LuPO_JR
bs := make([]byte, 8)
binary.LittleEndian.PutUint64(bs, uint64(64))
fmt.Printf("%#v\n", bs)
i := binary.LittleEndian.Uint64(bs)
fmt.Println(i)
There are a couple interesting solutions.
1st is gist that's simple to follow and its limitations are clear:
https://gist.github.com/cloveryume/9a59e8d77f5836f11720#file-golang_struct_packed-go
2nd is a package that tries to the job of packing:
https://github.com/lunixbochs/struc
Related
As suggested here names of people should be capitalized like John William Smith.
I'm writing a small software in Golang which gets last and first name from user's form inputs.
Until Go 1.18 I was using:
lastname = strings.Title(strings.ToLower(strings.TrimSpace(lastname)))
firstname = strings.Title(strings.ToLower(strings.TrimSpace(firstname)))
It works but now Go 1.18 has deprecated strings.Title().
They suggest to use golang.org/x/text/cases instead.
So I think I should change my code in something like this:
caser := cases.Title(language.Und)
lastname = caser.Title(strings.ToLower(strings.TrimSpace(lastname)))
firstname = caser.Title(strings.ToLower(strings.TrimSpace(firstname)))
It works the same as before.
The difference is for Dutch word like ijsland that should be titled as IJsland and not Ijsland.
The question
In the line caser := cases.Title(language.Und) I'm using Und because I don't know what language Tag to use.
Should I use language.English or language.AmericanEnglish or other?
So far it was like strings.Title() was using Und or English?
As mentioned in documentation strings.Title is deprecated and you should use cases.Title instead.
Deprecated: The rule Title uses for word boundaries does not handle
Unicode punctuation properly. Use golang.org/x/text/cases instead.
Here is an example code of how to use it as from two perspectives:
// Straightforward approach
caser := cases.Title(language.BrazilianPortuguese)
titleStr := caser.String(str)
// Transformer interface aware approach
src := []byte(s)
dest := []byte(s) // dest can also be `dest := src`
caser := cases.Title(language.BrazilianPortuguese)
_, _, err := caser.Transform(dest, src, true)
Make sure to take a look on the transform.Transformer.Transform and cases.Caser in order to understand what each parameter and return values mean, as well as the tool's limitations. For example:
A Caser may be stateful and should therefore not be shared between
goroutines.
Regarding what language to use, you should be aware of their difference in the results, besides that, you should be fine with any choice. Here is a copy from 煎鱼's summary on the differences that cleared it for me:
Go Playground: https://go.dev/play/p/xp59r1BkC9L
func main() {
src := []string{
"hello world!",
"i with dot",
"'n ijsberg",
"here comes O'Brian",
}
for _, c := range []cases.Caser{
cases.Lower(language.Und),
cases.Upper(language.Turkish),
cases.Title(language.Dutch),
cases.Title(language.Und, cases.NoLower),
} {
fmt.Println()
for _, s := range src {
fmt.Println(c.String(s))
}
}
}
With the following output
hello world!
i with dot
'n ijsberg
here comes o'brian
HELLO WORLD!
İ WİTH DOT
'N İJSBERG
HERE COMES O'BRİAN
Hello World!
I With Dot
'n IJsberg
Here Comes O'brian
Hello World!
I With Dot
'N Ijsberg
Here Comes O'Brian
So far it was like strings.Title() was using Und or English?
strings.Title() works based on ASCII, where cases.Title() works based on Unicode, there is no way to get the exact same behavior.
Should I use language.English or language.AmericanEnglish or other?
language.English, language.AmericanEnglish and language.Und all seem to have the same Title rules. Using any of them should get you the closest to the original strings.Title() behavior as you are going to get.
The whole point of using this package with Unicode support is that it is objectively more correct. So pick a tag appropriate for your users.
strings.Title(str) was deprecated, should change to cases.Title(language.Und, cases.NoLower).String(str)
package main
import (
"fmt"
"strings"
"golang.org/x/text/cases"
"golang.org/x/text/language"
)
func main() {
fmt.Println(strings.Title("abcABC")) // AbcABC
fmt.Println(cases.Title(language.Und, cases.NoLower).String("abcABC")) // AbcABC
}
Playground : https://go.dev/play/p/i0Eqh3QfxTx
Here is a straightforward example of how to capitalize the initial letter of each string value in the variable using the golang.org/x/text package.
package main
import (
"fmt"
"golang.org/x/text/cases"
"golang.org/x/text/language"
)
func main() {
sampleStr := "with value lower, all the letters are lowercase. this is good for poetry perhaps"
caser := cases.Title(language.English)
fmt.Println(caser.String(sampleStr))
}
Output : With Value Lower, All The Letters Are Lowercase. This Is Good For Poetry Perhaps
Playground Example: https://go.dev/play/p/_J8nGVuhYC9
I am trying to convert a string to integer and then to calculate its log.
My first approach was to convert the string using strconv library, but I got an error about the length of the string to be converted.
After that, I used math/big library which worked fine. Now I am not able to apply math.Log()on the resulted big integer.
Code:
package main
import (
"fmt"
"math"
"math/big"
)
func main() {
bb := "11948904162160164791281681976941230184120142151411311314211115130161285142991119211447"
bi := big.NewInt(0)
if _, ok := bi.SetString(bb, 10); ok {
fmt.Println(math.Log(bi))
} else {
fmt.Printf("error parsing line %#v\n", bb)
}
}
Error:
cannot use bi (type *big.Int) as type float64 in argument to math.Log
There are very few situations in which you'd need a precision greater than the one provided by the standard float64 type.
But just to satisfy any "midnight crazy ideas" (or even some very in-depth scientific research!) anyone might run into, Rob Pike's implementations of some operations with big floats are probably the best you can get right now with Go. The log function can be found here.
Is there an idiomatic way to read a file from the system starting from a (file scheme) url and not a path?
I tried this first:
fileUrlStr := "file:///path/to/file.json"
jsonBuffer, _ := ioutil.ReadFile(fileUrlStr)
This is my current (mostly working version) but I'm concerned there are some gotchas that I'm missing, so I'm hoping there's a more tried and true way to do it:
fileUrlStr := "file:///path/to/file.json"
fileUrl, _ := url.Parse(fileUrlStr)
jsonBuffer, _ := ioutil.ReadFile(fileUrl.Path)
(Bonus if I can support both file:///Users/jdoe/temp.json and file:///c:/WINDOWS/clock.json without having to add code-paths accounting for them)
Using net/url, the solution that you were using, is the correct one.
It's properly deals with hostnames and paths across platforms and also gives you a chance to check the url scheme is the file scheme.
package main
import (
"fmt"
"net/url"
)
func main() {
for _, path := range []string{
"file:///path/to/file.json",
"file:///c:/WINDOWS/clock.json",
"file://localhost/path/to/file.json",
"file://localhost/c:/WINDOWS/clock.avi",
// A case that you probably don't need to handle given the rarity,
// but is a known legacy win32 issue when translating \\remotehost\share\dir\file.txt
"file:////remotehost/share/dir/file.txt",
} {
u, _ := url.ParseRequestURI(path)
fmt.Printf("url:%v\nscheme:%v host:%v Path:%v\n\n", u, u.Scheme, u.Host, u.Path)
}
}
Go has two packages for random numbers:
crypto/rand, which provides a way to get random bytes
math/rand, which has a nice algorithm for shuffling ints
I want to use the Perm algorithm from math/rand, but provide it with high-quality random numbers.
Since the two rand packages are part of the same standard library there should be a way to combine them in a way so that crypto/rand provides a good source of random numbers that is used by math/rand.Perm to generate a permutation.
Here (and on the Playground) is the code I wrote to connect these two packages:
package main
import (
cryptoRand "crypto/rand"
"encoding/binary"
"fmt"
mathRand "math/rand"
)
type cryptoSource struct{}
func (s cryptoSource) Int63() int64 {
bytes := make([]byte, 8, 8)
cryptoRand.Read(bytes)
return int64(binary.BigEndian.Uint64(bytes) >> 1)
}
func (s cryptoSource) Seed(seed int64) {
panic("seed")
}
func main() {
rnd := mathRand.New(&cryptoSource{})
perm := rnd.Perm(52)
fmt.Println(perm)
}
This code works. Ideally I don't want to define the cryptoSource type myself but just stick together the two rand packages so that they work together. So is there a predefined version of this cryptoSource type somewhere?
That's basically what you need to do. It's not often that you need a cryptographically secure source of randomness for the common usage of math/rand, so there's no adaptor provided. You can make the implementation slightly more efficient by allocating the buffer space directly in the value, rather than allocating a new slice on every call. However in the unlikely event that reading the OS random source fails, this will need to panic to prevent returning invalid results.
type cryptoSource [8]byte
func (s *cryptoSource) Int63() int64 {
_, err := cryptoRand.Read(s[:])
if err != nil {
panic(err)
}
return int64(binary.BigEndian.Uint64(s[:]) & (1<<63 - 1))
}
Using Go (golang) I'd like to take a string with a formula and evaluate it with pre-defined values. Here's a way to do it with python's parser module:
x = 8
code = parser.expr("(x + 2) / 10").compile()
print eval(code)
# prints 1
Any idea how to do it with Go?
This package will probably work for your needs: https://github.com/Knetic/govaluate
expression, err := govaluate.NewEvaluableExpression("(x + 2) / 10");
parameters := make(map[string]interface{}, 8)
parameters["x"] = 8;
result, err := expression.Evaluate(parameters);
You will probably need to resort to a library that interprets math statements or have to write your own parser. Python being a dynamic language can parse and execute python code at runtime. Standard Go cannot do that.
If you want to write a parser on your own, the go package will be of help. Example (On play):
import (
"go/ast"
"go/parser"
"go/token"
)
func main() {
fs := token.NewFileSet()
tr, _ := parser.ParseExpr("(3-1) * 5")
ast.Print(fs, tr)
}
The resulting AST (Abstract Syntax Tree) can then be traversed and interpreted as you choose (handling '+' tokens as addition for the now stored values, for example).
I have made my own equation evaluator, using Djikstra's Shunting Yard Algorithm.
It supports all operators, nested parenthesis, functions and even user defined variables.
It is written in pure go
https://github.com/marcmak/calc
go-exprtk package will probably meet all kinds of your needs to evaluate any kind of mathematical expression dynamically.
package main
import (
"fmt"
"github.com/Pramod-Devireddy/go-exprtk"
)
func main() {
exprtkObj := exprtk.NewExprtk()
exprtkObj.SetExpression("(x + 2) / 10")
exprtkObj.AddDoubleVariable("x")
exprtkObj.CompileExpression()
exprtkObj.SetDoubleVariableValue("x", 8)
fmt.Println(exprtkObj.GetEvaluatedValue())
}
This package has many capabilities
There is no such module in Go. You have to build your own. You could use subpackages of the go package, but they might be overkill for your application.
For expression or program evaluation, you can build a lexer and parser using lex and yacc, and specify exactly the syntax and semantics of your mini-language. A calculator has always been a standard yacc example, and the go versions of lex and yacc are no different.
Here's a pointer to the calc example: https://github.com/golang-samples/yacc/tree/master/simple
With this code you can evaluate dynamically any formula and return true or false:
package main
import (
"go/token"
"go/types"
)
func main() {
fs := token.NewFileSet()
tv, err := types.Eval(fs, nil, token.NoPos, "(1 + 4) >= 5")
if err != nil {
panic(err)
}
println(tv.Value.String())
}
There's nothing built in that could do that (remember, Go is not a dynamic language).
However, you can always use bufio.Scanner and build your own parser.
Googling around I found this: https://github.com/sbinet/go-eval
It appears to be an eval loop for Go.
go get github.com/sbinet/go-eval/cmd/go-eval
go install github.com/sbinet/go-eval/cmd/go-eval
go-eval