Relatively new to Go. Seems trivial, but I can't figure out how to detect the OS version. I know I can use runtime.GOOS and runtime.GOARCH to get the platform and architecture, but say I know I'm on linux but I want to find if I'm on RH6 vice RH7, etc. is what I'm trying to figure out.
So there's this obscure Uname method in the syscall package that basically does it, at least on Linux. The struct it fills is a bit clunky and undocumented, but you can get the gist of it:
import (
"fmt"
"syscall"
)
// A utility to convert the values to proper strings.
func int8ToStr(arr []int8) string {
b := make([]byte, 0, len(arr))
for _, v := range arr {
if v == 0x00 {
break
}
b = append(b, byte(v))
}
return string(b)
}
func main() {
var uname syscall.Utsname
if err := syscall.Uname(&uname); err == nil {
// extract members:
// type Utsname struct {
// Sysname [65]int8
// Nodename [65]int8
// Release [65]int8
// Version [65]int8
// Machine [65]int8
// Domainname [65]int8
// }
fmt.Println(int8ToStr(uname.Sysname[:]),
int8ToStr(uname.Release[:]),
int8ToStr(uname.Version[:]))
}
}
BTW This doesn't work on the playground, probably because of the sandbox limitations, but works on Linux. Haven't tested other systems.
Related
I have this function to convert string to slice of bytes without copying
func StringToByteUnsafe(s string) []byte {
strh := (*reflect.StringHeader)(unsafe.Pointer(&s))
var sh reflect.SliceHeader
sh.Data = strh.Data
sh.Len = strh.Len
sh.Cap = strh.Len
return *(*[]byte)(unsafe.Pointer(&sh))
}
That works fine, but with very specific setup gives very strange behavior:
The setup is here: https://github.com/leviska/go-unsafe-gc/blob/main/pkg/pkg_test.go
What happens:
Create a byte slice
Convert it into temporary (rvalue) string and with unsafe convert it into byte slice again
Then, copy this slice (by reference)
Then, do something with the second slice inside goroutine
Print the pointers before and after
And I have this output on my linux mint laptop with go 1.16:
go test ./pkg -v -count=1
=== RUN TestSomething
0xc000046720 123 0xc000046720 123
0xc000076f20 123 0xc000046721 z
--- PASS: TestSomething (0.84s)
PASS
ok github.com/leviska/go-unsafe-gc/pkg 0.847s
So, the first slice magically changes its address, while the second isn't
If we remove the goroutine with runtime.GC() (and may be play with the code a little bit), we can get the both pointers to change the value (to the same one).
If we change the unsafe cast to just []byte() everything works without changing the addresses. Also, if we change it to the unsafe cast from here https://stackoverflow.com/a/66218124/5516391 everything works the same.
func StringToByteUnsafe(str string) []byte { // this works fine
var buf = *(*[]byte)(unsafe.Pointer(&str))
(*reflect.SliceHeader)(unsafe.Pointer(&buf)).Cap = len(str)
return buf
}
I run it with GOGC=off and got the same result. I run it with -race and got no errors.
If you run this as main package with main function, it seems to work correctly. Also if you remove the Convert function. My guess is that compiler optimizes stuff in this cases.
So, I have several questions about this:
What the hell is happening? Looks like a weird UB
Why and how go runtime magically changes the address of the variable?
Why in concurentless case it can change both addresses, while in concurrent can't?
What's the difference between this unsafe cast and the cast from stackoverflow answer? Why it does work?
Or is this just a compiler bug?
A copy of the full code from github, you need to put it in some package and run as test:
import (
"fmt"
"reflect"
"sync"
"testing"
"unsafe"
)
func StringToByteUnsafe(s string) []byte {
strh := (*reflect.StringHeader)(unsafe.Pointer(&s))
var sh reflect.SliceHeader
sh.Data = strh.Data
sh.Len = strh.Len
sh.Cap = strh.Len
return *(*[]byte)(unsafe.Pointer(&sh))
}
func Convert(s []byte) []byte {
return StringToByteUnsafe(string(s))
}
type T struct {
S []byte
}
func Copy(s []byte) T {
return T{S: s}
}
func Mid(a []byte, b []byte) []byte {
fmt.Printf("%p %s %p %s\n", a, a, b, b)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
b = b[1:2]
wg.Done()
}()
wg.Wait()
fmt.Printf("%p %s %p %s\n", a, a, b, b)
return b
}
func TestSomething(t *testing.T) {
str := "123"
a := Convert([]byte(str))
b := Copy(a)
Mid(a, b.S)
}
Answer from the github issue https://github.com/golang/go/issues/47247
The backing store of a is allocated on stack, because it does not
escape. And goroutine stacks can move dynamically. b, on the other
hand, escapes to heap, because it is passed to another goroutine. In
general, we don't assume the address of an object don't change.
This works as intended.
And my version is incorrect because
it uses reflect.SliceHeader as plain struct. You can run go vet on it,
and go vet will warn you.`
I'm trying to make a general purpose debug printer for complex data types because %v has a tendency to just print pointer values rather than what they point at. I've got it working with everything up until I have to deal with structs containing reflect.Value fields.
The following demo code runs without error: (https://play.golang.org/p/qvdRKc40R8k)
package main
import (
"fmt"
"reflect"
)
type MyStruct struct {
i int
R reflect.Value
}
func printContents(value interface{}) {
// Omitted: check if value is actually a struct
rv := reflect.ValueOf(value)
for i := 0; i < rv.NumField(); i++ {
fmt.Printf("%v: ", rv.Type().Field(i).Name)
field := rv.Field(i)
switch field.Kind() {
case reflect.Int:
fmt.Printf("%v", field.Int())
case reflect.Struct:
// Omitted: check if field is actually a reflect.Value to an int
fmt.Printf("reflect.Value(%v)", field.Interface().(reflect.Value).Int())
}
fmt.Printf("\n")
}
}
func main() {
printContents(MyStruct{123, reflect.ValueOf(456)})
}
This prints:
i: 123
R: reflect.Value(456)
However, if I change MyStruct's R field name to r, it fails:
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
Of course, it's rightly failing because this would otherwise be a way to get an unexported field into proper goland, which is a no-no.
But this leaves me in a quandry: How can I gain access to whatever the unexported reflect.Value refers to without using Interface() so that I can walk its contents and print? I've looked through the reflect documentation and haven't found anything that looks helpful...
After some digging, I've found a solution:
The only way to get at the inner reflect.Value is to call Interface() and type assert it, but this will panic if called on an unexported field. The only way around this is to use the unsafe package to clear the read-only flag so that the Interface() method will think it's exported when it's not (basically, we subvert the type system):
type flag uintptr // reflect/value.go:flag
type flagROTester struct {
A int
a int // reflect/value.go:flagStickyRO
int // reflect/value.go:flagEmbedRO
// Note: flagRO = flagStickyRO | flagEmbedRO
}
var flagOffset uintptr
var maskFlagRO flag
var hasExpectedReflectStruct bool
func initUnsafe() {
if field, ok := reflect.TypeOf(reflect.Value{}).FieldByName("flag"); ok {
flagOffset = field.Offset
} else {
log.Println("go-describe: exposeInterface() is disabled because the " +
"reflect.Value struct no longer has a flag field. Please open an " +
"issue at https://github.com/kstenerud/go-describe/issues")
hasExpectedReflectStruct = false
return
}
rv := reflect.ValueOf(flagROTester{})
getFlag := func(v reflect.Value, name string) flag {
return flag(reflect.ValueOf(v.FieldByName(name)).FieldByName("flag").Uint())
}
flagRO := (getFlag(rv, "a") | getFlag(rv, "int")) ^ getFlag(rv, "A")
maskFlagRO = ^flagRO
if flagRO == 0 {
log.Println("go-describe: exposeInterface() is disabled because the " +
"reflect flag type no longer has a flagEmbedRO or flagStickyRO bit. " +
"Please open an issue at https://github.com/kstenerud/go-describe/issues")
hasExpectedReflectStruct = false
return
}
hasExpectedReflectStruct = true
}
func canExposeInterface() bool {
return hasExpectedReflectStruct
}
func exposeInterface(v reflect.Value) interface{} {
pFlag := (*flag)(unsafe.Pointer(uintptr(unsafe.Pointer(&v)) + flagOffset))
*pFlag &= maskFlagRO
return v.Interface()
}
There are caveats, in that unsafe isn't allowed or desirable in all environments, not to mention that subverting the type system is rarely the right thing to do. It's recommended that you make such code conditional on build tags, and include a safe alternative.
I'm using golang to call a Dll function like char* fn(), the dll is not written by myself and I cannot change it. Here's my code:
package main
import (
"fmt"
"syscall"
"unsafe"
)
func main() {
dll := syscall.MustLoadDLL("my.dll")
fn := dll.MustFindProc("fn")
r, _, _ := fn.Call()
p := (*byte)(unsafe.Pointer(r))
// define a slice to fill with the p string
data := make([]byte, 0)
// loop until find '\0'
for *p != 0 {
data = append(data, *p) // append 1 byte
r += unsafe.Sizeof(byte(0)) // move r to next byte
p = (*byte)(unsafe.Pointer(r)) // get the byte value
}
name := string(data) // convert to Golang string
fmt.Println(name)
}
I have some questions:
Is there any better way of doing this? There're hundred of dll functions like this, I'll have to write the loop for all functions.
For very-long-string like 100k+ bytes, will append() cause performance issue?
Solved. the unsafe.Pointer(r) causes linter govet shows warning possible misuse of unsafe.Pointer, but the code runs fine, how to avoid this warning? Solution: This can be solved by adding -unsafeptr=false to govet command line, for vim-ale, add let g:ale_go_govet_options = '-unsafeptr=false'.
Casting uintptr as upointer is haram.
You must read the rules:
https://golang.org/pkg/unsafe/#Pointer
But there's hacky way, that shouldn't produce warning:
//go:linkname gostringn runtime.gostringn
func gostringn(p uintptr, l int) string
//go:linkname findnull runtime.findnull
//go:nosplit
func findnull(s uintptr) int
// ....
name := gostringn(r, findnull(r))
Functions takes pointer, but we link them from runtime as uintptr because they have same sizeof.
Might work in theory. But is also frowned upon.
Getting back to your code, as JimB said, you could do it one line with:
name := C.GoString((*C.char)(unsafe.Pointer(r)))
I got the following solution by tracking the os.Args of the go source code, But I am based on go1.17. If you are in another version, you can read the source code to solve it.
func UintPtrToString(r uintptr) string {
p := (*uint16)(unsafe.Pointer(r))
if p == nil {
return ""
}
n, end, add := 0, unsafe.Pointer(p), unsafe.Sizeof(*p)
for *(*uint16)(end) != 0 {
end = unsafe.Add(end, add)
n++
}
return string(utf16.Decode(unsafe.Slice(p, n)))
}
I want to write some code like this:
var myValue interface{}
func GetMyValue() interface{} {
return atomic.Load(myValue)
}
func StoreMyValue(newValue interface{}) {
atomic.Store(myValue, newValue)
}
It seems like that i can use LoadUintptr(addr *uintptr) (val uintptr) and StoreUintptr(addr *uintptr, val uintptr) in atomic package to achive this,but i do not know how to convert between uintptr,unsafe.Pointer and interface{}.
If i do it like this:
var V interface{}
func F(v interface{}) {
p := unsafe.Pointer(&V)
atomic.StorePointer(&p, unsafe.Pointer(&v))
}
func main() {
V = 1
F(2)
fmt.Println(V)
}
the V will always be 1
If I'm not mistaken you want atomic Value. You can store and fetch values atomically with it (signatures are interface{} but you should put same type into it). It does some unsafe pointer stuff under the hood like what you wanted to do.
Sample from docs:
var config Value // holds current server configuration
// Create initial config value and store into config.
config.Store(loadConfig())
go func() {
// Reload config every 10 seconds
// and update config value with the new version.
for {
time.Sleep(10 * time.Second)
config.Store(loadConfig())
}
}()
// Create worker goroutines that handle incoming requests
// using the latest config value.
for i := 0; i < 10; i++ {
go func() {
for r := range requests() {
c := config.Load()
// Handle request r using config c.
_, _ = r, c
}
}()
}
Here's a way to use atomic.StorePointer and atomic.LoadPointer (based on your example):
package main
import (
"fmt"
"sync/atomic"
"unsafe"
)
var addr unsafe.Pointer
func GetMyValue() *interface{} {
return (*interface{})(atomic.LoadPointer(&addr))
}
func StoreMyValue(newValue *interface{}) {
atomic.StorePointer(&addr, unsafe.Pointer(newValue))
}
func main() {
var i interface{}
i = 1
StoreMyValue(&i)
fmt.Println("before:", *GetMyValue())
i = 2
StoreMyValue(&i)
fmt.Println("after", *GetMyValue())
}
Playground link
Note that this will not make your object thread-safe. Only the pointer is stored/loaded atomically. Also, I would avoid using interface{} and prefer concrete types whenever possible.
As an alternative to using 'any' (interface{}), Go 1.19 (Q3 2022) comes with new types in the sync/atomic package that make it easier to use atomic values, such as atomic.Int64 and atomic.Pointer[T].
That would be easier than using atomic.StorePointer.
This comes from issue 50860 "sync/atomic: add typed atomic values".
And CL 381317
Pointer[T] also avoids conversions using unsafe.Pointer at call sites.
You cannot do this.
You will have to protect the store/load with a mutex.
The internal representation of an interface is not specified by the language and might (is) to large to be handled by package atomic.
How do I obtain the underlying syscall.Handle for a *net.UDPConn on Windows? I want this handle to set the IP_MULTICAST_TTL via syscall.SetsockoptInt. On Linux I do the following:
func setTTL(conn *net.UDPConn, ttl int) error {
f, err := conn.File()
if err != nil {
return err
}
defer f.Close()
fd := int(f.Fd())
return syscall.SetsockoptInt(fd, syscall.SOL_IP, syscall.IP_MULTICAST_TTL, ttl)
}
But on Windows, the implicit dup inside *net.UDPConn's File() fails with:
04:24:49 main.go:150: dup: not supported by windows
And in the source code is marked as a to-do. How can I get this handle? Is there some other way to set the TTL if not?
Update0
I've submitted the shortcomings to the Go issue tracker:
SetTTL for *net.UDPConn
Implement dup for netFD on Windows
The short answer is impossible. But since that isn't an answer you want to hear, I will give you the right way and wrong way to solve the problem.
The right way:
implement dup() for Windows.
submit to Go as a changeset
wait for it to be released to use it
Obviously the right way has some issues... but I highly recommend doing it. Go needs windows developers to fix up these types of serious problems. The only reason this can't be done in Windows is no one implemented the function
The wrong way:
Until the patch you write gets accepted and released, you can fake it through unsafe. The way the following code works by mirroring the exact structure of a net.UDPConn. This included copying over all structs from net that make up a UDPConn. Then unsafe is used to assert that the local UDPConn is the same as net's UDPConn. The compiler can not check this and takes your word for it. Were the internals of net to ever change, it would compile but god knows what it would do.
All code is untested.
package reallyunsafenet
import (
"net"
"sync"
"syscall"
"unsafe"
)
// copied from go/src/pkg/net/fd_windows.go
type ioResult struct {
qty uint32
err error
}
// copied from go/src/pkg/net/fd_windows.go
type netFD struct {
// locking/lifetime of sysfd
sysmu sync.Mutex
sysref int
closing bool
// immutable until Close
sysfd syscall.Handle
family int
sotype int
isConnected bool
net string
laddr net.Addr
raddr net.Addr
resultc [2]chan ioResult
errnoc [2]chan error
// owned by client
rdeadline int64
rio sync.Mutex
wdeadline int64
wio sync.Mutex
}
// copied from go/src/pkg/net/udpsock_posix.go
type UDPConn struct {
fd *netFD
}
// function to get fd
func GetFD(conn *net.UDPConn) syscall.Handle {
c := (*UDPConn)(unsafe.Pointer(conn))
return c.fd.sysfd
}