Calling kernel32's ReadProcessMemory in Go - go

I'm trying to manipulate processes on Windows using Go language,
and I'm starting off by reading other process' memory by using ReadProcessMemory.
However, for most of the addresses I get Error: Only part of a ReadProcessMemory or WriteProcessMemory request was completed. error. Maybe my list of arguments is wrong, but I can't find out why.
Can anyone point out what I am doing wrong here?
package main
import (
"fmt"
)
import (
windows "golang.org/x/sys/windows"
)
func main() {
handle, _ := windows.OpenProcess(0x0010, false, 6100) // 0x0010 PROCESS_VM_READ, PID 6100
procReadProcessMemory := windows.MustLoadDLL("kernel32.dll").MustFindProc("ReadProcessMemory")
var data uint = 0
var length uint = 0
for i := 0; i < 0xffffffff; i += 2 {
fmt.Printf("0x%x\n", i)
// BOOL ReadProcessMemory(HANDLE hProcess, LPCVOID lpBaseAddress, LPVOID lpBuffer, DWORD nSize, LPDWORD lpNumberOfBytesRead)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i), uintptr(data), 2, uintptr(length)) // read 2 bytes
if (ret == 0) {
fmt.Println(" Error:", e)
} else {
fmt.Println(" Length:", length)
fmt.Println(" Data:", data)
}
}
windows.CloseHandle(handle)
}

uintptr(data) is incorrect: it takes the value from data (0 of type uint) and converts that to unitptr type — yielding the same value converted to another type — producing, on x86, a null pointer.
Note that Go is not C, and you can't really play dirty games with pointers in it, or, rather, you can, but only through using the unsafe built-in package and its Pointer type which is like void* (pointing somewhere in a data memory block) in C.
What you need is something like
import "unsafe"
var (
data [2]byte
length uint32
)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i),
uintptr(unsafe.Pointer(&data[0])),
2, uintptr(unsafe.Pointer(&length))) // read 2 bytes
Observe what was done here:
A variable of type "array of two bytes" is declared;
The address of the first element of this array is taken;
That address is type-converted to the type unsafe.Pointer;
The obtained value is then type-converted to uintptr.
The last two steps are needed because Go features garbage collection:
In Go, when you take an address of a value in memory and store it in a variable, the GC knows about this "implicit" pointer and the value which address was taken won't be garbage-collected even if it becomes unreachable with that value holding its address being the only reference left.
Even if you make that address value lose the type information it maintains — through type-converting it to unsafe.Pointer, the new value is still considered by GC and behaves like "normal" values containing addresses — as explained above.
By type-converting such value to uintptr you make GC stop considering it as a pointer. Hence this type is there only for FFI/interop.
In other words, in
var data [2]byte
a := &data[0]
p := unsafe.Pointer(a)
i := uintptr(p)
there are only three references to the value in data: that variable itself, a and p, but not i.
You should consider these rules when dealing with calling outside code because you should never ever pass around unitptr-typed values: they're only for marshaling data to the called functions and unmarshaling it back, and have to be used "on the spot" — in the same scope as the values they are type-converted from/to.
Also observe that in Go, you can't just take the address of a variable of an integer type and supply that address to a function which expects a pointer to a memory block of an appropriate size. You have to deal with byte arrays and after the data has been written by the called function, you need to explicitly convert it to a value of the type you need. That's why there's no "type casts" in Go but only "type conversions": you can't reinterpret the data type of a value through type-conversion, with the uintptr(unsafe.Pointer) (and back) being a notable exception for the purpose of FFI/interop, and even in this case you basically convert a pointer to a pointer, just transfer it through the GC boundary.
To "serialize" and "deserialize" a value of an integer type you might use the encoding/binary standard package or hand-roll no-brainer simple functions which do bitwise shifts and or-s and so on ;-)
2015-10-05, updated as per the suggestion of James Henstridge.
Note that after the function returns, and ret signalizes there's no error
you have to check the value of the length variable.

Related

Go vet reports "possible misuse of reflect.SliceHeader"

I have the following code snippet which "go vet" complains about with the warning "possible misuse of reflect.SliceHeader". I can not find very much information about this warning other then this. After reading that it is not very clear to me what is needed to do this in a way that makes go vet happy - and without possible gc issues.
The goal of the snippet is to have a go function copy data to memory which is managed by an opaque C library. The Go function expects a []byte as a parameter.
func Callback(ptr unsafe.Pointer, buffer unsafe.Pointer, size C.longlong) C.longlong {
...
sh := &reflect.SliceHeader{
Data: uintptr(buffer),
Len: int(size),
Cap: int(size),
}
buf := *(*[]byte)(unsafe.Pointer(sh))
err := CopyToSlice(buf)
if err != nil {
log.Fatal("failed to copy to slice")
}
...
}
https://pkg.go.dev/unsafe#go1.19.4#Pointer
Pointer represents a pointer to an arbitrary type. There are four
special operations available for type Pointer that are not available
for other types:
A pointer value of any type can be converted to a Pointer.
A Pointer can be converted to a pointer value of any type.
A uintptr can be converted to a Pointer.
A Pointer can be converted to a uintptr.
Pointer therefore allows a program to defeat the type system and read
and write arbitrary memory. It should be used with extreme care.
The following patterns involving Pointer are valid. Code not using
these patterns is likely to be invalid today or to become invalid in
the future. Even the valid patterns below come with important caveats.
Running "go vet" can help find uses of Pointer that do not conform to
these patterns, but silence from "go vet" is not a guarantee that the
code is valid.
(6) Conversion of a reflect.SliceHeader or reflect.StringHeader Data
field to or from Pointer.
As in the previous case, the reflect data structures SliceHeader and
StringHeader declare the field Data as a uintptr to keep callers from
changing the result to an arbitrary type without first importing
"unsafe". However, this means that SliceHeader and StringHeader are
only valid when interpreting the content of an actual slice or string
value.
var s string
hdr := (*reflect.StringHeader)(unsafe.Pointer(&s)) // case 1
hdr.Data = uintptr(unsafe.Pointer(p)) // case 6 (this case)
hdr.Len = n
In this usage hdr.Data is really an alternate way to refer to the
underlying pointer in the string header, not a uintptr variable
itself.
In general, reflect.SliceHeader and reflect.StringHeader should be used only as *reflect.SliceHeader and *reflect.StringHeader pointing at actual slices or strings, never as plain structs. A program should not declare or allocate variables of these struct types.
// INVALID: a directly-declared header will not hold Data as a reference.
var hdr reflect.StringHeader
hdr.Data = uintptr(unsafe.Pointer(p))
hdr.Len = n
s := *(*string)(unsafe.Pointer(&hdr)) // p possibly already lost
It looks like JimB (from the comments) hinted upon the most correct answer, though he didn't post it as an answer and he didn't include an example. The following passes go vet, staticcheck, and golangci-lint - and doesn't segfault so I think it is the correct answer.
func Callback(ptr unsafe.Pointer, buffer unsafe.Pointer, size C.longlong) C.longlong {
...
buf := unsafe.Slice((*byte)(buffer), size)
err := CopyToSlice(buf)
if err != nil {
log.Fatal("failed to copy to slice")
}
...
}

Reading ETW providers using go

I'm trying to access the EnumerateTraceGuids function from Advapi32.dll in go.
I'm at the very early stage and still trying to decipher what is it that I must do. I have the following code that keeps giving me Error: 87, meaning ERROR_INVALID_PARAMETER.
I've used this file as a starting point though it's only writing and not reading :
https://github.com/moby/moby/blob/master/daemon/logger/etwlogs/etwlogs_windows.go
Official documentation for the function I'm trying to call is here :
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363713(v=vs.85).aspx
It requires GuidPropertiesArray [in, out] An array of pointers to TRACE_GUID_PROPERTIES structures. This structure is the following (https://msdn.microsoft.com/en-us/library/windows/desktop/aa364143(v=vs.85).aspx)
typedef struct _TRACE_GUID_PROPERTIES {
GUID Guid;
ULONG GuidType;
ULONG LoggerId;
ULONG EnableLevel;
ULONG EnableFlags;
BOOLEAN IsEnable;
} TRACE_GUID_PROPERTIES, *PTRACE_GUID_PROPERTIES;
I have the following code to try and do this :
package main
import (
"errors"
"fmt"
"syscall"
"unsafe"
"github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
)
const (
win32CallSuccess = 0
MaxProv = 50
nbProviders = 50
)
var (
modAdvapi32 = windows.NewLazySystemDLL("Advapi32.dll")
procEnumerateTraceGuids = modAdvapi32.NewProc("EnumerateTraceGuids")
)
type ulong int32
type TRACE_GUID_PROPERTIES struct {
Guid syscall.GUID
GuidType ulong
LoggerId ulong
EnableLevel ulong
EnableFlags ulong
IsEnable bool
}
func callEnumerateTraceGuids() error {
GuidPropertiesArray:= make([]TRACE_GUID_PROPERTIES, 1)
ptr := &GuidPropertiesArray[0]
ret, _, _ := procEnumerateTraceGuids.Call(uintptr(unsafe.Pointer(&ptr)), MaxProv, nbProviders)
if ret != win32CallSuccess {
errorMessage := fmt.Sprintf("Failed to register ETW provider. Error: %d", ret)
logrus.Error(errorMessage)
return errors.New(errorMessage)
}
return nil
}
func main() {
callEnumerateTraceGuids()
}
At this point I'm not sure what is it that I must do. I've tried a lot of variation of initializing the array without success.
Hoping someone can point me in the right direction.
Thanks !
Edit : Changed code based on comments but still getting the same error.
PS : This is my first time posting to stackoverflow and I've already been told that I'm lazy less than 12 hours after posting my question (yay!) so not sure I'm asking this right...I am not too familiar with go and never called windows DLL from go before and since I keep hitting that ERROR_INVALID_PARAMETER I thought of reaching out to try and pass this first wall to be able to grasp some concepts at the same time. Hope this helps understands my request (ie. I come in peace).
OK, I had a bit of free time and an access to a Windows XP box,
so I've decided to dust off my Windows programming skills
and wrote a working solution:
package main
import (
"golang.org/x/sys/windows"
"log"
"syscall"
"unsafe"
)
var (
modAdvapi32 = windows.NewLazySystemDLL("advapi32")
procEnumerateTraceGuids = modAdvapi32.NewProc("EnumerateTraceGuids")
)
type traceGuidProperties struct {
guid syscall.GUID
guidType uint32
loggerId uint32
enableLevel uint32
enableFlags uint32
isEnable uint32
}
func enumerateTraceGuids(ptr **traceGuidProperties, count uint32, out *uint32) error {
rc, _, _ := procEnumerateTraceGuids.Call(uintptr(unsafe.Pointer(ptr)),
uintptr(count), uintptr(unsafe.Pointer(out)))
if rc != 0 {
return syscall.Errno(rc)
}
return nil
}
func enumTraceGuids() ([]*traceGuidProperties, error) {
var errMoreData = syscall.Errno(234)
var (
dummyProps traceGuidProperties
dummyPtr = &dummyProps
count uint32
)
err := enumerateTraceGuids(&dummyPtr, 0, &count)
if err != errMoreData {
return nil, err
}
items := make([]*traceGuidProperties, count)
for i := range items {
items[i] = new(traceGuidProperties)
}
for {
err = enumerateTraceGuids(&items[0], count, &count)
if err == nil {
break
}
if err != errMoreData {
return nil, err
}
for i := 0; i < int(count)-len(items); i++ {
items = append(items, new(traceGuidProperties))
}
}
return items[:count], nil
}
func main() {
log.SetFlags(0)
data, err := enumTraceGuids()
if err != nil {
log.Fatal(err)
}
log.Printf("len(data)=%d\n", len(data))
for i := range data {
log.Println(*(data[i]))
}
}
The key points:
I was wrong when I told you that
«you … should allocate an array of structs (not pointers)»—in fact
EnumerateTraceGuids indeed expects an array of pointers.
As hinted here,
there are two subtleties with how EnumerateTraceGuids works:
Contrary to what its documentation states,
it actually supports being called with its PropertyArrayCount
parameter set to 0, in which case it's expected to return ERROR_MORE_DATA
while having set GuidCount to the number of elements of the input
array required for the (next) call to complete successfully.
IOW, that way we know how many trace GUIDs the system currently
"knows about".
Still, even in this case, the function performs validity check
on the input array (see below).
As it turns out, the function expects an array of pointers to
TRACE_GUID_PROPERTIES blocks allocated by you.
In other words, if it says you it knows about 10 trace GUIDs,
you have to allocate 10 values of type TRACE_GUID_PROPERTIES,
then make an array of 10 pointers to those values and pass a pointer
to the 1st element of that array to the function.
Notice that there's an inherent race between changes occuring
in the system (those traces added or removed for any number of reasons)
and the calls to EnumerateTraceGuids.
This means if the first call to this function told you it "knows"
about 10 trace GUIDs, on the next call it may turn out
there's already 20 trace GUIDs, or 5 GUIDs
(or any other number of them FWIW).
So we account for both of these possibilities in the following way:
First we do a call with a pointer to a single (but valid)
TRACE_GUID_PROPERTIES value, allocated statically
(hence the function "sees" what looks like an array of a single element),
while telling the function the input "array" has zero elements.
We expect the function to fail with ERROR_MORE_DATA
and put the actual number of trace GUIDs it "knows" about into the variable
we've supplied it a pointer to.
We allocate that much TRACE_GUID_PROPERTIES memory blocks
the function indicated on the first call.
For this, we use the new() built-in function which behaves somewhat
like malloc() in the standard C library—it allocates the memory for
a value of the specified type and returns a pointer to the allocated
memory block.
We create an array of pointers to these allocated memory blocks
and call EnumerateTraceGuids again.
If it succeeds, we handle the possibility it returned less
elements than we've allocated, and reslice our slice.
If it fails with ERROR_MORE_DATA, we extend our slice with
whatever the number of elements is needed (allocating memory for their
TRACE_GUID_PROPERTIES blocks first), and try calling the function again.
The "magic number" 234 is the actual code for the ERROR_MORE_DATA value.
Sorry for the initial confusion.

encode object to bytes by golang unsafe?

func Encode(i interface{}) ([]byte, error) {
buffer := bytes.NewBuffer(make([]byte, 0, 1024))
// size := unsafe.Sizeof(i)
size := reflect.TypeOf(i).Size()
fmt.Println(size)
ptr := unsafe.Pointer(&i)
startAddr := uintptr(ptr)
endAddr := startAddr + size
for i := startAddr; i < endAddr; i++ {
bytePtr := unsafe.Pointer(i)
b := *(*byte)(bytePtr)
buffer.WriteByte(b)
}
return buffer.Bytes(), nil
}
func TestEncode(t *testing.T) {
test := Test{10, "hello world"}
b, _ := Encode(test)
ptr := unsafe.Pointer(&b)
newTest := *(*Test)(ptr)
fmt.Println(newTest.X)
}
I am learning how to use golang unsafe and wrote this function for encoding any object. I meet with two problems, first, dose unsafe.Sizeof(obj) always return obj's pointer size? Why it different from reflect.TypeOf(obj).Size()? Second, I want to iterate the underlying bytes of obj and convert it back to obj in TestEncode function by unsafe.Pointer(), but the object's values all corrupt, why?
First, unsafe.Sizeof returns the bytes that needs to store the type. It is a little bit tricky, but it does not mean bytes that needs to store the data.
For example, a slice, as it is well known, stores 3 4-byte ints on a 32bit machine. One uintptr for memory address of the underlying array, and two int32 for len and cap. So no matter how long a slice is or what type it is of, a slice takes always 12 bytes on a 32 bit machine. Likely, a string uses 8 bytes: 1 uintptr for address and 1 int32 for len.
As for difference between reflect.TypeOf().Size, it is about interface. reflect.TypeOf looks into the interface and gets an concrete type, and reports bytes needed about the concrete type, while unsafe.Sizeof just returns 8 for an interface type: 2 uintptr for a pointer to the data and a pointer to the method lists.
Second part is quite clear now. For one, unsafe.Pointer is taking the address of the interface, instead of the concrete type. Two, in TestEncode, unsafe.Pointer is taking address to the 12-byte slice "header". There might be other errors, but with the two mentioned, they are meaningless to spot.
Note: I avoid talking about orders of the uintptr and int32 not only because I don't know, but also becuase they are not documented, unsafe, and implentation depended.
Note 2: Conclusion: Don't try to dump memory of a Go data.
Note 3: I change everything to 32 bit becuase playground is using it, so it is easier to check.

When is it safe in Go to reference an object only through a `uintptr`?

The Go Programming Language says in Section 13.2 that this is code is safe and x will always
be visible to the garbage collector:
pb := (*int16)(unsafe.Pointer(
uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))
*pb = 42
And that this code is unsafe, because x is temporarily not visible to the
garbage collector, which could move it, making pb a dangling pointer:
tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)
pb := (*int16)(unsafe.Pointer(tmp))
*pb = 42
But I can't see the difference between these two examples.
In the case described as safe, after uintptr has been called, the only
reference to the x is the uintptr value, isn't it? There's a Pointer
to it on the same line, but it was an argument to uintptr, which has run,
so nothing is referencing the arguments, and so the Pointer is not live and the uintptr is the only reference to the object.
I can't see how storing the uintptr in a local variable instead of as an
expression intermediate value makes it any more safe. Aren't local variables
like tmp removed in compiler phases anyway, becoming anonymous dataflow edges,
so that the generated code should be semantically equivalent? Or does Go have
some rules for when garbage collection can run? Such as having safepoints only
between statements? But the code in the first example has method calls so I
would presume they would always be safepoints?
Found the reference I hinted at in my comments here
A uintptr is an integer, not a reference. Converting a Pointer to a uintptr creates an integer value with no pointer semantics. Even if a uintptr holds the address of some object, the garbage collector will not update that uintptr's value if the object moves, nor will that uintptr keep the object from being reclaimed.
What this means is that this expression:
pb := (*int16)(unsafe.Pointer(
uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))
*pb = 42
Is safe because you're creating a uintptr, which is seen as an integer, not a reference, but it's immediately assigned (unless there's a race condition somewhere else, the object that x references cannot be GC'ed) until after the assignment). The uintptr (again: integer type) is also immediately cast to a pointer, turning it into a reference so the GC will manage pb. This means that:
uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)): all safe, because x clearly is an existing reference to an object
pb is assigned an integer that is (through the cast) marked as a reference to an int16 object
However, when you write this:
tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)
pb := (*int16)(unsafe.Pointer(tmp))
There is a chance that, between assigning tmp (remember integer, not reference), the actual object in memory is moved. As it says in the docs: tmp will not be updated. Thus, when you assign pb, you could end up with an invalid pointer.
think of tmp in this case as x in the first case. Rather than being a reference to an object, it's as if you wrote
tmp := 123456 // a random integer
pb := (*int16) (unsafe.Pointer(tmp)) // not safe, obviously
For example:
var pb *int16
tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)
go func() {
time.Sleep(1 * time.Second)
pb = (*int16)(unsafe.Pointer(tmp))
}()
// original value of x could be GC'ed here, before the goroutine starts, or the time.Sleep call returns
x = TypeOfX{
b: 123,
}

Why the pointer is losing its value in this Go program

I don't understand why the pointer s is nil even after the input() method initialised it. Any idea?
package main
import "fmt"
type ps string
func(s *ps)input(){
x := ps("a")
s = &x
}
func(s *ps)output(){
}
func main() {
var v *ps
v.input()
if v == nil{
fmt.Println("v shouldn't be nil")
}
}
Playground http://play.golang.org/p/jU2hoMP7TS
You need two things--main needs to allocate space for a ps that input can write into, which you can do by replacing var v *ps with v := new(ps). The string will be "", but it doesn't matter what it is, just that there's space set aside in memory for a string header that input can write to. As Momer said, otherwise the pointer's nil and your program panics trying to dereference it.
And in order to assign through a pointer, input needs to use *s = x. Since *s is, informally, "get what s points to", you can read that as "change what s points to to x". Usually the automatic ref/deref behavior around the dot operator and method calls saves you from that, but when you assign through a pointer type or do other operations (arithmetic, indexing, etc.) the dereference needs to be there in the code.
v value (0) is passed into v.input. Passed value is stored in a local variable s. s value is modified. No one is saving new s value back into v.
If you want something modified in your function, you must pass pointer to the value. (or reference for slices, maps and so on).
If you want to change pointer value, you should pass pointer to your pointer.
Alex

Resources