Is it good practice (at least general practice) to have log.SetFlags(log.LstdFlags | log.Lshortfile) in production in Go? I wonder if there is whether performance or security issue by doing it in production. Since it is not default setting of log package in Go. Still can't find any official reference or even opinion article regarding that matter.
As for the performance. Yes, it has an impact, however, it is imho negligible for various reasons.
Testing
Code
package main
import (
"io/ioutil"
"log"
"testing"
)
func BenchmarkStdLog(b *testing.B) {
// We do not want to benchmark the shell
stdlog := log.New(ioutil.Discard, "", log.LstdFlags)
for i := 0; i < b.N; i++ {
stdlog.Println("foo")
}
}
func BenchmarkShortfile(b *testing.B) {
slog := log.New(ioutil.Discard, "", log.LstdFlags|log.Lshortfile)
for i := 0; i < b.N; i++ {
slog.Println("foo")
}
}
Result
goos: darwin
goarch: amd64
pkg: stackoverflow.com/go/logbench
BenchmarkStdLog-4 3803840 277 ns/op 4 B/op 1 allocs/op
BenchmarkShortfile-4 1000000 1008 ns/op 224 B/op 3 allocs/op
Your mileage may vary, but the order of magnitude should be roughly equal.
Why I think the impact is negligible
It is unlikely that your logging will be the bottleneck of your application, unless you write a shitload of logs. In 99 times out of 100, it is not the logging which is the bottleneck.
Get your application up and running, load test and profile it. You can still optimize then.
Hint: make sure you can scale out.
Related
EDIT: Turns out this is not easy to reproduce. I think it may be a Host OS or Linux Distro issue. Trying it with different distros in Docker, but running on the same host, produces the same results.
EDIT 2: I altered the code in main.go. Based on the comments saying that it was the GC causing this. Now it should definitely be rewriting the value over the previous one. So there shouldn't be any GC.
Basically there's 2 things I am trying to understand.
The main thing I am trying to understand why these two functions, which do the same thing in the end, are so different in speed. One of them I make the array the size of the number of hashes I want (which uses more memory), and the other one is just a for loop.
How could I speed up the regular for loop function to gain the speed benefit without using the extra memory? (if possible)
Results of Benchmark on my system:
The Array function completes in 3.6 seconds and the loop function takes 6.4 seconds.
go version go1.19.2 linux/amd64
goos: linux
goarch: amd64
pkg: github.com/gngenius02/shardedmapdb
cpu: AMD Ryzen 9 5900X 12-Core Processor
BenchmarkGetHashesArray10Million-24 1 3662003126 ns/op 2080037632 B/op 30000051 allocs/op
BenchmarkGetHashesLoop10Million-24 1 6462627155 ns/op 1920001352 B/op 30000022 allocs/op
PASS
The two functions in question are GetHashUsingArray and GetHashUsingLoop.
main.go:
package main
import (
"crypto/sha256"
"encoding/hex"
)
type HashArray []string
type HS struct {
LastHash string
HashList HashArray
}
func (h *HS) GetHashUsingArray() {
hashit := func(s string) string {
digest := sha256.Sum256([]byte(s))
return hex.EncodeToString(digest[:])
}
hl := h.HashList
for i := 1; i < len(hl); i++ {
(hl)[i] = hashit((hl)[i-1])
}
h.LastHash = hl[len(hl)-1]
}
func GetHashUsingLoop(s string, loops int) string {
hashit := func(s *string) {
digest := sha256.Sum256([]byte(*s))
*s = hex.EncodeToString(digest[:])
}
hash := s
for i := 0; i < loops; i++ {
hashit(&hash)
}
return hash
}
func main() {}
main_test.go:
package main
import (
"testing"
)
func BenchmarkGetHashUsingArray10Million(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
firstValue := "abc"
hs := HS{"", make(HashArray, 10_000_001)}
hs.HashList[0] = firstValue
hs.GetHashUsingArray()
if hs.LastHash != "bf34d93b4be2a313b06cdf9d805c5f3d140abd872c37199701fb1e43fe479923" {
b.Error("Unexpected Result: " + hs.LastHash)
}
}
}
func BenchmarkGetHashUsingLoop10Million(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
firstValue := "abc"
result := GetHashUsingLoop(firstValue, 10_000_000)
if result != "bf34d93b4be2a313b06cdf9d805c5f3d140abd872c37199701fb1e43fe479923" {
b.Error("Unexpected result: " + result)
}
}
}
I think it's somehow relates to either to your own hardware or go version. On my machine results are completely different:
go version go1.18.1 darwin/arm64
% go test -bench=. -v
goos: darwin
goarch: arm64
pkg: github.com/foo/bar
BenchmarkGetHashUsingArray10Million
BenchmarkGetHashUsingArray10Million-8 1 1658941791 ns/op 2080012144 B/op 30000012 allocs/op
BenchmarkGetHashUsingLoop10Million
BenchmarkGetHashUsingLoop10Million-8 1 1391175042 ns/op 1920005816 B/op 30000063 allocs/op
PASS
maybe worth checking with go 1.18 to narrow the scope. I.e. if it will be the same difference with go 1.19 then it's hardware. If difference is not that huge then it's something that was introduced in go 1.19
Update
Actually it seems the benchmark was incorrectly setup I have followed the resource shared by user #Luke Joshua Park and now it works.
package main
import "testing"
func benchmarkBcrypt(i int, b *testing.B){
for n:= 0; n < b.N; n++ {
HashPassword("my pass", i)
}
}
func BenchmarkBcrypt9(b *testing.B){
benchmarkBcrypt(9, b)
}
func BenchmarkBcrypt10(b *testing.B){
benchmarkBcrypt(10, b)
}
func BenchmarkBcrypt11(b *testing.B){
benchmarkBcrypt(11, b)
}
func BenchmarkBcrypt12(b *testing.B){
benchmarkBcrypt(12, b)
}
func BenchmarkBcrypt13(b *testing.B){
benchmarkBcrypt(13, b)
}
func BenchmarkBcrypt14(b *testing.B){
benchmarkBcrypt(14, b)
}
Output:
BenchmarkBcrypt9-4 30 39543095 ns/op
BenchmarkBcrypt10-4 20 79184657 ns/op
BenchmarkBcrypt11-4 10 158688315 ns/op
BenchmarkBcrypt12-4 5 316070133 ns/op
BenchmarkBcrypt13-4 2 631838101 ns/op
BenchmarkBcrypt14-4 1 1275047344 ns/op
PASS
ok go-playground 10.670s
Old incorrect benchmark
I have a small set on benchmark test in golang and am curios of to what is a recommended bcrypt cost to use as of May 2018.
This is my benchrmark file:
package main
import "testing"
func BenchmarkBcrypt10(b *testing.B){
HashPassword("my pass", 10)
}
func BenchmarkBcrypt12(b *testing.B){
HashPassword("my pass", 12)
}
func BenchmarkBcrypt13(b *testing.B){
HashPassword("my pass", 13)
}
func BenchmarkBcrypt14(b *testing.B){
HashPassword("my pass", 14)
}
func BenchmarkBcrypt15(b *testing.B){
HashPassword("my pass", 15)
}
and this is HashPassword() func inside main.go:
import (
"golang.org/x/crypto/bcrypt"
)
func HashPassword(password string, cost int) (string, error) {
bytes, err := bcrypt.GenerateFromPassword([]byte(password), cost)
return string(bytes), err
}
The current output is:
go test -bench=.
BenchmarkBcrypt10-4 2000000000 0.04 ns/op
BenchmarkBcrypt12-4 2000000000 0.16 ns/op
BenchmarkBcrypt13-4 2000000000 0.32 ns/op
BenchmarkBcrypt14-4 1 1281338532 ns/op
BenchmarkBcrypt15-4 1 2558998327 ns/op
PASS
It seems that for a bcrypt with cost of 13 the time it takes is 0.32 nanoseconds, and for cost 14 the time is 1281338532ns or ~1.2 seconds
Which I believe is too much. What do is the best bcrypt cost to use for the current year 2018.
I'm not certain what's going on with Benchmark here. If you just time these, it works fine, and you can work out the right answer for you.
package main
import (
"golang.org/x/crypto/bcrypt"
"time"
)
func main() {
cost := 10
start := time.Now()
bcrypt.GenerateFromPassword([]byte("password"), cost)
end := time.Now()
print(end.Sub(start) / time.Millisecond)
}
For a work factor of 10, on my MacBook Pro I get 78ms. A work factor of 11 is 154ms, and 12 is 334ms. So we're seeing roughly doubling, as expected.
The goal is not a work factor; it's a time. You want as long as you can live with. In my experience (mostly working on client apps), 80-100ms is a nice target because compared to a network request it's undetectable to the user, while being massive in terms of brute-force attacks (so the default of 10 is ideal for my common use).
I generally avoid running password stretching on servers if I can help it, but this scale can be a reasonable trade-off between server impact and security. Remember that attackers may use something dramatically faster than a MacBook Pro, and may use multiple machines in parallel; I pick 80-100ms because of user experience trade-offs. (I perform password stretching on the client when I can get away with it, and then apply a cheap hash like SHA-256 on the server.)
But if you don't do this very often, or can spend more time on it, then longer is of course better, and on my MacBook Pro a work factor of 14 is about 1.2s, which I would certainly accept for some purposes.
But there's a reason that 10 is still the default. It's not an unreasonable value.
Measuring time around a function is easy in Go.
But what if you need to measure it 5000 times per second in parallel?
I'm referring to Correctly measure time duration in Go which contains great answers about how to measure time in Go.
What is the cost of using time.Now() 5000 times per second or more?
While it may depend on the underlying OS, let's consider on linux.
Time measurement depends on the programming language and its implementation, the operating system and its implementation, the hardware architecture, implementation, and speed, and so on.
You need to focus on facts, not speculation. In Go, start with some benchmarks. For example,
since_test.go:
package main
import (
"testing"
"time"
)
var now time.Time
func BenchmarkNow(b *testing.B) {
for N := 0; N < b.N; N++ {
now = time.Now()
}
}
var since time.Duration
var start time.Time
func BenchmarkSince(b *testing.B) {
for N := 0; N < b.N; N++ {
start = time.Now()
since = time.Since(start)
}
}
Output:
$ go test since_test.go -bench=. -benchtime=1s
goos: linux
goarch: amd64
BenchmarkNow-4 30000000 47.5 ns/op
BenchmarkSince-4 20000000 98.1 ns/op
PASS
ok command-line-arguments 3.536s
$ go version
go version devel +48c4eeeed7 Sun Mar 25 08:33:21 2018 +0000 linux/amd64
$ uname -srvio
Linux 4.13.0-37-generic #42-Ubuntu SMP Wed Mar 7 14:13:23 UTC 2018 x86_64 GNU/Linux
$ cat /proc/cpuinfo | grep 'model name' | uniq
model name : Intel(R) Core(TM) i7-7500U CPU # 2.70GHz
$
Now, ask yourself if 5,000 times per second is necessary, practical, and reasonable.
What are your benchmark results?
I've been trying to benchmark a Radix Tree implementation I wrote for sake of practice with Golang.
But I encountered a problem on "How should I benchmark it?". In the code below shows two cases or lets say different ways I would like to benchmark the LookUp func.
Case 1: Use one single slice of bytes which exist on the tree meaning it will be successful LookUp through all children nodes etc...
Case 2: Use a func to generate that random slice from the existing data in the tree meaning it will be successful LookUp as well...
I know the time expend will depend on the tree depth... I think Case 2 is close to a real world implementation or not?
QUESTION: Which case is more efficient or useful to benchmark?
Benchmark:
func BenchmarkLookUp(b *testing.B) {
radix := New()
insertData(radix, sampleData2)
textToLookUp := randomBytes()
for i := 0; i < b.N; i++ {
radix.LookUp(textToLookUp) // Case 1
//radix.LookUp(randomBytes()) // Case 2
}
}
func randomBytes() []byte {
strings := sampleData2()
return []byte(strings[random(0, len(strings))])
}
func sampleData2() []string {
return []string{
"romane",
"romanus",
"romulus",
...
}
}
Result Case 1:
PASS
BenchmarkLookUp-4 10000000 146 ns/op
ok github.com/falmar/goradix 2.068s
PASS
BenchmarkLookUp-4 10000000 149 ns/op
ok github.com/falmar/goradix 2.244s
Result Case 2:
PASS
BenchmarkLookUp-4 3000000 546 ns/op
ok github.com/falmar/goradix 3.094s
PASS
BenchmarkLookUp-4 3000000 538 ns/op
ok github.com/falmar/goradix 4.481s
Results when there is no match:
PASS
BenchmarkLookUp-4 10000000 194 ns/op
ok github.com/falmar/goradix 3.189s
PASS
BenchmarkLookUp-4 10000000 191 ns/op
ok github.com/falmar/goradix 3.243s
If your benchmark is random, that would make it very difficult to compare the performance between different implementations from one run to the next.
Instead, statically implement a few different benchmark cases that stress different areas of your algorithm. The cases should represent different scenarios, such as the case when there are no matches (as you already have), the case where there are many items in the source data that will be returned in a lookup, the case where there are many items and only 1 item will be returned, etc etc.
I have 2 methods to trim the domain suffix from a subdomain and I'd like to find out which one is faster. How do I do that?
2 string trimming methods
You can use the builtin benchmark capabilities of go test.
For example (on play):
import (
"strings"
"testing"
)
func BenchmarkStrip1(b *testing.B) {
for br := 0; br < b.N; br++ {
host := "subdomain.domain.tld"
s := strings.Index(host, ".")
_ = host[:s]
}
}
func BenchmarkStrip2(b *testing.B) {
for br := 0; br < b.N; br++ {
host := "subdomain.domain.tld"
strings.TrimSuffix(host, ".domain.tld")
}
}
Store this code in somename_test.go and run go test -test.bench='.*'. For me this gives
the following output:
% go test -test.bench='.*'
testing: warning: no tests to run
PASS
BenchmarkStrip1 100000000 12.9 ns/op
BenchmarkStrip2 100000000 16.1 ns/op
ok 21614966 2.935s
The benchmark utility will attempt to do a certain number of runs until a meaningful time is
measured which is reflected in the output by the number 100000000. The code was run
100000000 times and each operation in the loop took 12.9 ns and 16.1 ns respectively.
So you can conclude that the code in BenchmarkStrip1 performed better.
Regardless of the outcome, it is often better to profile your program to see where the
real bottleneck is instead of wasting your time with micro benchmarks like these.
I would also not recommend writing your own benchmarking as there are some factors you might
not consider such as the garbage collector and running your samples long enough.