Good evening,
I've been trying to build a golang application which scans values in memory but am struggling trying to understand how to address specific memory locations. I know that when accessing memory within the application you can use *variablename to deference and get the address location, but how would I provide an address location and print the value to the screen or grab the next allocated object of any size from RAM and print it's value?
Thanks in advance for any help you may be willing to share
I don't know how much useful this will be, but here is a sample code.
package main
import (
"fmt"
"unsafe"
)
func main() {
var i int = 1
fmt.Println("Address : ", &i, " Value : ", i)
var address *int
address = &i // getting the starting address
loc := (uintptr)(unsafe.Pointer(address))
p := unsafe.Pointer(loc)
// verification - it should print 1
var val int = *((* int)(p))
fmt.Println("Location : ", loc, " Val :",val) // it does print !!
// lets print 1000 bytes starting from address of variable i
// first memory location contains 1 as expected
printValueAtMemoryLocation(loc, 1000)
// now lets test for some arbitrary memory location
// not so random ! wanted to reduce the diff value also any arbitrary memory location you can't read !!
memoryToReach := 842350500000
loc = changeToInputLocation(loc, memoryToReach)
fmt.Println("Loc is now at : ", loc)
// lets print 1000 bytes starting from the memory location "memoryToReach"
printValueAtMemoryLocation(loc, 1000)
}
func changeToInputLocation(location uintptr, locationToreach int) uintptr {
var diff,i int
diff = locationToreach - int(location)
fmt.Println("We need to travel ", diff, " memory locations !")
if diff < 0 {
i= diff * -1
for i > 0 {
location--
i--
}
} else {
i= diff
for i > 0 {
location++
i--
}
}
return location
}
func printValueAtMemoryLocation(location uintptr, next int) {
var v byte
p := unsafe.Pointer(location)
fmt.Println("\n")
for i:=1; i<next; i++ {
p = unsafe.Pointer(location)
v = *((*byte)(p))
fmt.Print(v," ")
//fmt.Println("Loc : ", loc, " --- Val : ", v)
location++
}
fmt.Println("\n")
}
Using "unsafe" package is not a good idea, also you can not read any arbitrary location I believe.
For me when I tried some other random locations where, most probably, I didn't have read access, it threw me error like this:
unexpected fault address 0xc41ff8f780
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0xc41ff8f780 pc=0x1093ec0]
But hopefully, it can be of some value to you.
Related
Pointer assignment of index is being consistently inconsistent within addData(..). I expect memory address is moving around as underlying array increases in size.
Behavior: I assign to variable A, then assign B = A*0.2, then assign y = sig(B), finally B = y. Sometimes on the next loop B == y || B == A*0.2. It is perfectly consistent across multiple executions.
I made a simpler and more full version of the code.
package main
import(
"fmt"
"math"
)
func main(){
//Structure setup
l := lots{}; l.addData(2); l.addData(2); l.addData(2); l.addData(2)
l.val[0].y[0] = 0.20700021
l.val[0].y[1] = 0.30003001
l.propagate()
}
type data struct{
y []float64
}
type pair struct {
one *data
two *data
}
// lots is the biggest part of the structure
// the problem seems to occure when this is introduced
type lots struct{
val []data
join []pair
}
// addData appends a data struct and a pair struct to
// the corresponding parts of lots struct
func (l *lots)addData(size int){
l.val = append(l.val, data{make([]float64, size)})
// should be skipped first call only
if(len(l.join) < len(l.val)-1){
fmt.Println("len of l.val: ", len(l.val))
l.join = append(l.join, pair{})
l.join[len(l.join)-1].one = &l.val[len(l.val)-2]
l.join[len(l.join)-1].two = &l.val[len(l.val)-1]
}
}
// propagate
func (l *lots)propagate(){
for _, v := range l.join{
v.travel()
}
}
// travel modifies values going from p.one -> p.two
func (p *pair) travel(){
fmt.Println("p.one.y: ", p.one.y)
p.mathy()
fmt.Println("p.two.y: ", p.two.y)
p.two.y = sigmoid(p.two.y)
fmt.Println("p.two.y: ", p.two.y)
}
func (p *pair) mathy(){
for i := range p.one.y {
p.two.y[i] = p.one.y[i] * p.one.y[i]
}
}
// sigmoid seems to be causing some problems.
// Works fine on it's own though
func sigmoid(x []float64)(y []float64){
y = make([]float64, len(x))
for i := range x{
y[i] = 1./(1.+math.Exp(-x[i]))
}
return
}
I expect the #'s of p.two.y: [#'s] to equal the following lines #'s of p.one.y: [#'s] all the time. The output I am getting is not consistently equal. Sometimes the p.one.y: [#'s] #'s are equal to the p.two.y: [#'s] from the line preceding line; that value was over written and then that value came back.
p.one.y: [0.20700021 0.30003001]
p.two.y: [0.04284908694004409 0.0900180069006001]/////// bad
p.two.y: [0.5107106330188076 0.5224893174114301] // overwritten
p.one.y: [0.04284908694004409 0.0900180069006001]/////// reappeared
p.two.y: [0.0018360442515954571 0.008103241566356488]
p.two.y: [0.5004590109339528 0.5020257993066767]//// overwritten
p.one.y: [0.5004590109339528 0.5020257993066767]//// good
p.two.y: [0.25045922162499035 0.25202990316950763]
p.two.y: [0.5622895277500193 0.5626760660176802]
I have tried to reduce function nesting, and it worked when I put everything into the Propagate() function and assigned directly to p.two.y[i] with the sigmoid function. (below)
// propagate
func (l *lots)propagate(){
for _, p := range l.join{
fmt.Println("p.one.y: ", p.one.y)
for i := range p.one.y {
p.two.y[i] = p.one.y[i] * p.one.y[i]
}
fmt.Println("p.two.y: ", p.two.y)
// using this extra variable causes the problem of inconsistent assignment
//y := make([]float64, len(p.two.y))
for i := range p.two.y{
//y[i] = 1./(1.+math.Exp(-p.two.y[i]))
p.two.y[i] = 1./(1.+math.Exp(-p.two.y[i]))
}
//p.two.y = y
fmt.Println("p.two.y: ", p.two.y)
}
}
This version provides good data, but takes away so much of the specialization I like.
p.one.y: [0.20700021 0.30003001]
p.two.y: [0.04284908694004409 0.0900180069006001]
p.two.y: [0.5107106330188076 0.5224893174114301]////
p.one.y: [0.5107106330188076 0.5224893174114301]//// Good
p.two.y: [0.2608253506784712 0.27299508680906215]
p.two.y: [0.564839170446528 0.5678280461350629]////
p.one.y: [0.564839170446528 0.5678280461350629]//// Good
p.two.y: [0.3190432884707219 0.3224286899775631]
p.two.y: [0.5790910765397528 0.5799160282084651]
The problem is with the pointer assignments as you build the slice you are trying to reference. The addresses keep changing.
func main(){
var lump []int
// A loop to build a slice of `int`'s from 0 size to 8 size
// and print each index address
for i:= 0; i < 8; i++{
lump = append(lump, int(i))
fmt.Printf("addr of lump[%v]: %p\n",i, &lump[i])
}
fmt.Println()
// A loop to look at the addresses of each index
for i := range lump{
fmt.Printf("addr of lump[%v]: %p\n",i, &lump[i])
}
}
Check out the addresses not being created in sequential memory locations.
//while building the slice
// notice the addresses making big jumps
addr of lump[0]: 0xc00000a0c8
addr of lump[1]: 0xc00000a0f8
addr of lump[2]: 0xc00000e3b0
addr of lump[3]: 0xc00000e3b8
addr of lump[4]: 0xc00000c2e0
addr of lump[5]: 0xc00000c2e8
addr of lump[6]: 0xc00000c2f0
addr of lump[7]: 0xc00000c2f8
//after building the slice
// notice all address being sequential
addr of lump[0]: 0xc00000c2c0
addr of lump[1]: 0xc00000c2c8
addr of lump[2]: 0xc00000c2d0
addr of lump[3]: 0xc00000c2d8
addr of lump[4]: 0xc00000c2e0
addr of lump[5]: 0xc00000c2e8
addr of lump[6]: 0xc00000c2f0
addr of lump[7]: 0xc00000c2f8
You could move to C/C++ where you could handle all the memory adjustments as the array increases in size. Or build one slice then the other.
I'm using golang to call a Dll function like char* fn(), the dll is not written by myself and I cannot change it. Here's my code:
package main
import (
"fmt"
"syscall"
"unsafe"
)
func main() {
dll := syscall.MustLoadDLL("my.dll")
fn := dll.MustFindProc("fn")
r, _, _ := fn.Call()
p := (*byte)(unsafe.Pointer(r))
// define a slice to fill with the p string
data := make([]byte, 0)
// loop until find '\0'
for *p != 0 {
data = append(data, *p) // append 1 byte
r += unsafe.Sizeof(byte(0)) // move r to next byte
p = (*byte)(unsafe.Pointer(r)) // get the byte value
}
name := string(data) // convert to Golang string
fmt.Println(name)
}
I have some questions:
Is there any better way of doing this? There're hundred of dll functions like this, I'll have to write the loop for all functions.
For very-long-string like 100k+ bytes, will append() cause performance issue?
Solved. the unsafe.Pointer(r) causes linter govet shows warning possible misuse of unsafe.Pointer, but the code runs fine, how to avoid this warning? Solution: This can be solved by adding -unsafeptr=false to govet command line, for vim-ale, add let g:ale_go_govet_options = '-unsafeptr=false'.
Casting uintptr as upointer is haram.
You must read the rules:
https://golang.org/pkg/unsafe/#Pointer
But there's hacky way, that shouldn't produce warning:
//go:linkname gostringn runtime.gostringn
func gostringn(p uintptr, l int) string
//go:linkname findnull runtime.findnull
//go:nosplit
func findnull(s uintptr) int
// ....
name := gostringn(r, findnull(r))
Functions takes pointer, but we link them from runtime as uintptr because they have same sizeof.
Might work in theory. But is also frowned upon.
Getting back to your code, as JimB said, you could do it one line with:
name := C.GoString((*C.char)(unsafe.Pointer(r)))
I got the following solution by tracking the os.Args of the go source code, But I am based on go1.17. If you are in another version, you can read the source code to solve it.
func UintPtrToString(r uintptr) string {
p := (*uint16)(unsafe.Pointer(r))
if p == nil {
return ""
}
n, end, add := 0, unsafe.Pointer(p), unsafe.Sizeof(*p)
for *(*uint16)(end) != 0 {
end = unsafe.Add(end, add)
n++
}
return string(utf16.Decode(unsafe.Slice(p, n)))
}
I would like to find the remaining IPs in a range (CIDR).
For example:
prefixLen, bits := n.Mask.Size()
return 1 << (uint64(bits) - uint64(prefixLen)), nil
This return the # of IPs in the range where n is *net.IPNet.
But say if I have few addresses that are already given out and want to know how many are left 'starting' from some IP addresses, how can I do it?
Here's something you could do
package main
import (
"fmt"
"net"
)
func inet_aton(ip string) uint32 {
var rv uint32
b := net.ParseIP(ip).To4()
rv=uint32(b[0]) << 24
rv=rv+uint32(b[1]) << 16
rv=rv+uint32(b[2]) << 8
rv=rv+uint32(b[3])
return rv
}
func main() {
start := inet_aton("192.168.0.1") //192.168.2.0/20 start
fin := inet_aton("192.168.15.255") //ditto end
here := inet_aton("192.168.15.248") //current allocation
fmt.Printf("you have %d remaining\n",(fin-start) - (here-start))
}
Here is a way to do it that works for both IPv4 and IPv6 using the IPAddress Go library. Disclaimer: I am the project manager.
func remaining(cidr string) (used, remaining, total *big.Int) {
cidrStr := ipaddr.NewIPAddressString(cidr)
cidrBlock := cidrStr.GetAddress().ToPrefixBlock()
host := cidrStr.GetHostAddress()
used = new(big.Int).Sub(host.GetValue(), cidrBlock.GetValue())
remaining = new(big.Int).Sub(cidrBlock.GetUpperValue(), host.GetValue())
total = cidrBlock.GetCount()
fmt.Printf("\nHost %s is address number %d of %d addresses in block %s.\n"+
"There remains %d addresses.\n", host, used, total, cidrBlock,
remaining)
return
}
If you want to stick with IPv4, then you can replace big.Int with unsigned int types.
func remainingIPv4(cidr string) (used, remaining uint32, total uint64) {
cidrStr := ipaddr.NewIPAddressString(cidr)
cidrBlock := cidrStr.GetAddress().ToPrefixBlock().ToIPv4()
host := cidrStr.GetHostAddress().ToIPv4()
used = host.Uint32Value() - cidrBlock.Uint32Value()
remaining = cidrBlock.UpperUint32Value() - host.Uint32Value()
total = cidrBlock.GetIPv4Count()
fmt.Printf("\nHost %s is address number %d of %d addresses in block %s.\n"+
"There remains %d addresses.\n", host, used, total, cidrBlock,
remaining)
return
}
Example:
package main
import (
"fmt"
"github.com/seancfoley/ipaddress-go/ipaddr"
"math/big"
)
func main() {
remaining("192.168.2.0/20")
remaining("2001:0db8:85a3::8a2e:0370:7334/64")
remainingIPv4("192.168.2.0/20")
}
Example output:
Host 192.168.2.0 is address number 512 of 4096 addresses in block 192.168.0.0/20.
There remains 3583 addresses.
Host 2001:db8:85a3::8a2e:370:7334 is address number 151930230829876 of 18446744073709551616 addresses in block 2001:db8:85a3::/64.
There remains 18446592143478721739 addresses.
Host 192.168.2.0 is address number 512 of 4096 addresses in block 192.168.0.0/20.
There remains 3583 addresses.
I am new to Golang and would like to model physical devices for measuring quantities such as light intensity, mass, electric current and so forth. So as a starting point I will define a device struct as follows:
const (
// Light can be measured in the form of luminosity
Light = 1<< iota
Mass
ElectricalCurrent
)
type PhysicalDevice struct{
Owner string
ID string
Description string
}
I am a confused now on how to express the device's capabilities (what it can measure) and the units of measurement. For example I would like to express that a physical device can measure electrical currents in amperes. However, I also want to express that a PhysicalDevice can measure more than one quantity. For example it could measure electrical current and temperature.
The PhysicalDevice's capabilities are not known in advance and can contain an arbitrary combination of capabilities.
I was thinking of using something equivalent to a C++ bitset for expressing the physical quantities a device can measure (would this be the right approach in the first place?).
I did not find the Go bitset type and not sure how to express that. I also need to map the measured physical quantity to a corresponding unit.
You should understand that trying to replicate another language's features in Go is generally regarded as a bad idea. There is a 'go' way of doing things.
You might want to consider iota and bitmask operations like this example on the Go playground. I have included the code here as well (in all its plagiarized glory):
package main
import "fmt"
func main() {
TestAddFlag()
TestHasFlag()
TestClearFlag()
TestToggleFlag()
fmt.Println("all tests passed")
}
type Bitmask uint32
func (f Bitmask) HasFlag(flag Bitmask) bool { return f&flag != 0 }
func (f *Bitmask) AddFlag(flag Bitmask) { *f |= flag }
func (f *Bitmask) ClearFlag(flag Bitmask) { *f &= ^flag }
func (f *Bitmask) ToggleFlag(flag Bitmask) { *f ^= flag }
const (
TESTFLAG_ONE Bitmask = 1 << iota
TESTFLAG_TWO
TESTFLAG_THREE
)
func TestAddFlag() {
var mainFlag Bitmask = TESTFLAG_TWO
mainFlag.AddFlag(TESTFLAG_THREE)
if mainFlag&(1<<TESTFLAG_THREE) != 0 {
panic("failed")
}
}
func TestClearFlag() {
var mainFlag Bitmask = TESTFLAG_ONE | TESTFLAG_THREE
mainFlag.ClearFlag(TESTFLAG_THREE)
if mainFlag&(1<<TESTFLAG_ONE) != 0 {
panic("failed")
}
}
func TestHasFlag() {
var mainFlag Bitmask = TESTFLAG_ONE | TESTFLAG_THREE
if !mainFlag.HasFlag(TESTFLAG_THREE) {
panic("failed")
}
}
func TestToggleFlag() {
flag := TESTFLAG_ONE | TESTFLAG_THREE
flag.ToggleFlag(TESTFLAG_ONE)
if flag.HasFlag(TESTFLAG_ONE) {
panic("failed")
}
flag.ToggleFlag(TESTFLAG_ONE)
if !flag.HasFlag(TESTFLAG_ONE) {
panic("failed")
}
}
This approach is commonly used in the standard library.
Define the capabilities using = 1 << iota:
const (
Light Capability = 1 << iota
Mass
ElectricalCurrent
Energy
)
Notice that the expression is only needed on the first constant. The same expression (but with an updated value of iota) will be used on the consecutive lines in the same group.
Here's a complete working example:
package main
import (
"fmt"
"strings"
)
type Capability int
const (
// Light can be measured in the form of luminosity
Light Capability = 1 << iota
Mass
ElectricalCurrent
Energy
)
func (c Capability) String() string {
var caps []string
if c&Light > 0 {
caps = append(caps, "Light")
}
if c&Mass > 0 {
caps = append(caps, "Mass")
}
if c&ElectricalCurrent > 0 {
caps = append(caps, "ElectricalCurrent")
}
if c&Energy > 0 {
caps = append(caps, "Energy")
}
return strings.Join(caps, "|")
}
type PhysicalDevice struct {
Owner string
ID string
Description string
Capability Capability
}
func (pd PhysicalDevice) String() string {
return "Owner: " + pd.Owner + "\n" +
"ID: " + pd.ID + "\n" +
"Description: " + pd.Description + "\n" +
"Capability: " + pd.Capability.String() + "\n"
}
func main() {
dev := PhysicalDevice{
Owner: "Albert Einstein",
ID: "E=mc^2",
Description: "My well-known formula as a device",
Capability: Energy | Mass | Light,
}
fmt.Println(dev)
}
The code can be found on The Go Playground.
I'm working on a small Go program that receives an ascii message over UDP. I want to look up the first field in the message and see if it exist in a map. Go thinks the key does not exist in the map but it does. I can add the key to the map and it creates a new entry, so I have two entries with the same key. I'm I doing something wrong or is this a bug?
EDIT:
I've simplified the test down to remove the UDP and YAML.
https://play.golang.org/p/2Bg8UjhfWC
package main
import (
"fmt"
"strings"
)
type TestCase struct {
Test string
Result string
}
func main() {
tcmap := make(map[string]TestCase)
tcmap["adc"] = TestCase{Test: "/bar"}
fmt.Printf("TestMap: ----------\n%v\n\n", tcmap)
buf := make([]byte, 1024)
buf[0] = 'a'//0x61
buf[1] = 'd'//0x64
buf[2] = 'c'//0x63
fmt.Printf("Received: ---------\n%v\n\n", string(buf[0:3]))
fmt.Printf("Compare hex:-------\n|%x| |%x|\n\n", buf[0:3], "adc")
// Get the first field from the message
testname := strings.Split(strings.Trim(string(buf), " "), " ")[0]
fmt.Printf("Test Name: |%v|\n", testname)
// Does the key exist in the map?
if t, ok := tcmap[testname]; ok {
fmt.Printf("Test found: %v\n", t)
} else {
fmt.Printf("Test NOT found\n")
}
// Add testname to map, does it replace existing?
tcmap[testname] = TestCase{Test: "/foo"}
fmt.Printf("\nMAP: ---------\n%v\n\n", tcmap)
fmt.Printf("KEY adc:---------\n%v\n\n", tcmap["adc"])
for k,v := range tcmap {
fmt.Printf("%v: %v\n", k, v)
}
}
Output:
TestMap: ----------
map[adc:{/bar }]
Received: ---------
adc
Compare hex:-------
|616463| |616463|
Test Name: |adc|
Test NOT found
MAP: ---------
map[adc:{/bar } adc:{/foo }]
KEY adc:---------
{/bar }
adc: {/bar }
adc: {/foo }
As pointed out by Alexander, the issue is the length between the two keys are different. One key has the length of 3 and the other has the length of 1024. The first three bytes were the same and on the longer key the remaining bytes were 0x00.
So the string output of the two keys make it appear the two are identical, but this was fooling me. The length of the keys was different.
One of the keys has a trailing newline. If you use strings.TrimSpace instead of strings.Trim you'll see that the trailing newline is trimmed and there is no duplicate.