Reading ETW providers using go - winapi

I'm trying to access the EnumerateTraceGuids function from Advapi32.dll in go.
I'm at the very early stage and still trying to decipher what is it that I must do. I have the following code that keeps giving me Error: 87, meaning ERROR_INVALID_PARAMETER.
I've used this file as a starting point though it's only writing and not reading :
https://github.com/moby/moby/blob/master/daemon/logger/etwlogs/etwlogs_windows.go
Official documentation for the function I'm trying to call is here :
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363713(v=vs.85).aspx
It requires GuidPropertiesArray [in, out] An array of pointers to TRACE_GUID_PROPERTIES structures. This structure is the following (https://msdn.microsoft.com/en-us/library/windows/desktop/aa364143(v=vs.85).aspx)
typedef struct _TRACE_GUID_PROPERTIES {
GUID Guid;
ULONG GuidType;
ULONG LoggerId;
ULONG EnableLevel;
ULONG EnableFlags;
BOOLEAN IsEnable;
} TRACE_GUID_PROPERTIES, *PTRACE_GUID_PROPERTIES;
I have the following code to try and do this :
package main
import (
"errors"
"fmt"
"syscall"
"unsafe"
"github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
)
const (
win32CallSuccess = 0
MaxProv = 50
nbProviders = 50
)
var (
modAdvapi32 = windows.NewLazySystemDLL("Advapi32.dll")
procEnumerateTraceGuids = modAdvapi32.NewProc("EnumerateTraceGuids")
)
type ulong int32
type TRACE_GUID_PROPERTIES struct {
Guid syscall.GUID
GuidType ulong
LoggerId ulong
EnableLevel ulong
EnableFlags ulong
IsEnable bool
}
func callEnumerateTraceGuids() error {
GuidPropertiesArray:= make([]TRACE_GUID_PROPERTIES, 1)
ptr := &GuidPropertiesArray[0]
ret, _, _ := procEnumerateTraceGuids.Call(uintptr(unsafe.Pointer(&ptr)), MaxProv, nbProviders)
if ret != win32CallSuccess {
errorMessage := fmt.Sprintf("Failed to register ETW provider. Error: %d", ret)
logrus.Error(errorMessage)
return errors.New(errorMessage)
}
return nil
}
func main() {
callEnumerateTraceGuids()
}
At this point I'm not sure what is it that I must do. I've tried a lot of variation of initializing the array without success.
Hoping someone can point me in the right direction.
Thanks !
Edit : Changed code based on comments but still getting the same error.
PS : This is my first time posting to stackoverflow and I've already been told that I'm lazy less than 12 hours after posting my question (yay!) so not sure I'm asking this right...I am not too familiar with go and never called windows DLL from go before and since I keep hitting that ERROR_INVALID_PARAMETER I thought of reaching out to try and pass this first wall to be able to grasp some concepts at the same time. Hope this helps understands my request (ie. I come in peace).

OK, I had a bit of free time and an access to a Windows XP box,
so I've decided to dust off my Windows programming skills
and wrote a working solution:
package main
import (
"golang.org/x/sys/windows"
"log"
"syscall"
"unsafe"
)
var (
modAdvapi32 = windows.NewLazySystemDLL("advapi32")
procEnumerateTraceGuids = modAdvapi32.NewProc("EnumerateTraceGuids")
)
type traceGuidProperties struct {
guid syscall.GUID
guidType uint32
loggerId uint32
enableLevel uint32
enableFlags uint32
isEnable uint32
}
func enumerateTraceGuids(ptr **traceGuidProperties, count uint32, out *uint32) error {
rc, _, _ := procEnumerateTraceGuids.Call(uintptr(unsafe.Pointer(ptr)),
uintptr(count), uintptr(unsafe.Pointer(out)))
if rc != 0 {
return syscall.Errno(rc)
}
return nil
}
func enumTraceGuids() ([]*traceGuidProperties, error) {
var errMoreData = syscall.Errno(234)
var (
dummyProps traceGuidProperties
dummyPtr = &dummyProps
count uint32
)
err := enumerateTraceGuids(&dummyPtr, 0, &count)
if err != errMoreData {
return nil, err
}
items := make([]*traceGuidProperties, count)
for i := range items {
items[i] = new(traceGuidProperties)
}
for {
err = enumerateTraceGuids(&items[0], count, &count)
if err == nil {
break
}
if err != errMoreData {
return nil, err
}
for i := 0; i < int(count)-len(items); i++ {
items = append(items, new(traceGuidProperties))
}
}
return items[:count], nil
}
func main() {
log.SetFlags(0)
data, err := enumTraceGuids()
if err != nil {
log.Fatal(err)
}
log.Printf("len(data)=%d\n", len(data))
for i := range data {
log.Println(*(data[i]))
}
}
The key points:
I was wrong when I told you that
«you … should allocate an array of structs (not pointers)»—in fact
EnumerateTraceGuids indeed expects an array of pointers.
As hinted here,
there are two subtleties with how EnumerateTraceGuids works:
Contrary to what its documentation states,
it actually supports being called with its PropertyArrayCount
parameter set to 0, in which case it's expected to return ERROR_MORE_DATA
while having set GuidCount to the number of elements of the input
array required for the (next) call to complete successfully.
IOW, that way we know how many trace GUIDs the system currently
"knows about".
Still, even in this case, the function performs validity check
on the input array (see below).
As it turns out, the function expects an array of pointers to
TRACE_GUID_PROPERTIES blocks allocated by you.
In other words, if it says you it knows about 10 trace GUIDs,
you have to allocate 10 values of type TRACE_GUID_PROPERTIES,
then make an array of 10 pointers to those values and pass a pointer
to the 1st element of that array to the function.
Notice that there's an inherent race between changes occuring
in the system (those traces added or removed for any number of reasons)
and the calls to EnumerateTraceGuids.
This means if the first call to this function told you it "knows"
about 10 trace GUIDs, on the next call it may turn out
there's already 20 trace GUIDs, or 5 GUIDs
(or any other number of them FWIW).
So we account for both of these possibilities in the following way:
First we do a call with a pointer to a single (but valid)
TRACE_GUID_PROPERTIES value, allocated statically
(hence the function "sees" what looks like an array of a single element),
while telling the function the input "array" has zero elements.
We expect the function to fail with ERROR_MORE_DATA
and put the actual number of trace GUIDs it "knows" about into the variable
we've supplied it a pointer to.
We allocate that much TRACE_GUID_PROPERTIES memory blocks
the function indicated on the first call.
For this, we use the new() built-in function which behaves somewhat
like malloc() in the standard C library—it allocates the memory for
a value of the specified type and returns a pointer to the allocated
memory block.
We create an array of pointers to these allocated memory blocks
and call EnumerateTraceGuids again.
If it succeeds, we handle the possibility it returned less
elements than we've allocated, and reslice our slice.
If it fails with ERROR_MORE_DATA, we extend our slice with
whatever the number of elements is needed (allocating memory for their
TRACE_GUID_PROPERTIES blocks first), and try calling the function again.
The "magic number" 234 is the actual code for the ERROR_MORE_DATA value.
Sorry for the initial confusion.

Related

golang get char* as return value from dll

I'm using golang to call a Dll function like char* fn(), the dll is not written by myself and I cannot change it. Here's my code:
package main
import (
"fmt"
"syscall"
"unsafe"
)
func main() {
dll := syscall.MustLoadDLL("my.dll")
fn := dll.MustFindProc("fn")
r, _, _ := fn.Call()
p := (*byte)(unsafe.Pointer(r))
// define a slice to fill with the p string
data := make([]byte, 0)
// loop until find '\0'
for *p != 0 {
data = append(data, *p) // append 1 byte
r += unsafe.Sizeof(byte(0)) // move r to next byte
p = (*byte)(unsafe.Pointer(r)) // get the byte value
}
name := string(data) // convert to Golang string
fmt.Println(name)
}
I have some questions:
Is there any better way of doing this? There're hundred of dll functions like this, I'll have to write the loop for all functions.
For very-long-string like 100k+ bytes, will append() cause performance issue?
Solved. the unsafe.Pointer(r) causes linter govet shows warning possible misuse of unsafe.Pointer, but the code runs fine, how to avoid this warning? Solution: This can be solved by adding -unsafeptr=false to govet command line, for vim-ale, add let g:ale_go_govet_options = '-unsafeptr=false'.
Casting uintptr as upointer is haram.
You must read the rules:
https://golang.org/pkg/unsafe/#Pointer
But there's hacky way, that shouldn't produce warning:
//go:linkname gostringn runtime.gostringn
func gostringn(p uintptr, l int) string
//go:linkname findnull runtime.findnull
//go:nosplit
func findnull(s uintptr) int
// ....
name := gostringn(r, findnull(r))
Functions takes pointer, but we link them from runtime as uintptr because they have same sizeof.
Might work in theory. But is also frowned upon.
Getting back to your code, as JimB said, you could do it one line with:
name := C.GoString((*C.char)(unsafe.Pointer(r)))
I got the following solution by tracking the os.Args of the go source code, But I am based on go1.17. If you are in another version, you can read the source code to solve it.
func UintPtrToString(r uintptr) string {
p := (*uint16)(unsafe.Pointer(r))
if p == nil {
return ""
}
n, end, add := 0, unsafe.Pointer(p), unsafe.Sizeof(*p)
for *(*uint16)(end) != 0 {
end = unsafe.Add(end, add)
n++
}
return string(utf16.Decode(unsafe.Slice(p, n)))
}

GoRoutines and passing struct to original context

I have a configuration that defines a number of instances (SomeConfigItems) which have a thing() created for each of them.
That thing is a struct returned by an included package, which contains, among other things, a Price (float64) and a nested struct. The nested struct maintains a map of trades.
The problem is that I am able to loop through the thing.Streams.Trades and see all trades happening in real time from my main()'s for{} loop. I am not able to see an updated thing.Price even though it is set in the Handler on occasion.
I am having a hard time understanding how the nested structs can contain data but not Price. I feel as though I am missing something with scoping, goroutines, or possibly pointers for instantiation of new objects.
Any help would be appreciated, I will continue reading in the meantime. I've reduced the code to what seems relevant.
main.go:
package main
import thing
var Things []thing.Handler
for _, name := range SomeConfigItems {
handler := thing.New(name)
Things = append(Things, handler)
}
for {
for _, t := range Things {
log.Info("Price: ", t.Price) // This is set to 0 every iteration, but I can actively data in thing.Streams.Trades
}
}
thing.go:
package thing
import streams
type Handler struct {
Name string
Price float64
Streams streams.Streams
}
func New(name string) (h Handler, err error) {
stream, err := streams.New(strings.ToLower(name))
h = Handler{
Name: name,
Price: "0.0"
Streams: stream,
}
go h.handler()
return h, err
}
func (bot *Handler) handler() {
var currentPrice float64
for {
currentPrice = external.GetPrice(bot.Name).Price // Validated that this returns a float64
bot.Price = currentPrice // Verified that this is updated immediately after in this context.
// Unable to see Price updated from outer context.
}
}
streams.go:
package streams
type Streams struct {
Trades
}
type State struct {
Price string `json:"p"`
Quantity string `json:"q"`
}
type Trades struct {
Trades map[float64]float64
TradeMutex sync.Mutex
Updates chan State
}
func New(name string) (s Streams, err error) {
p := newTradeStream(name)
s = Streams{
Trades: p,
}
return s, err
}
func newTradeStream(name string) (ts Trades) {
ts = Trades{}
ts.Trades = make(map[float64]float64, MaxDepth)
ts.Updates = make(chan State, 500)
// ... Other watchdog code
return ts
}
Note:
I am added some debug logging in multiple locations. From within the Bot Handler, the price was printed (successfully), then updated, and then printed (successfully) again -- Showing no gap in the setting of Price from within the handler() function.
When adding the same type of debugging to the main() for{} loop, I tried setting an incrementing counter and assigning the value of thing.Price -- Printing thing.Price on each loop results in 0, even if I set the price (and validate it gets set) in the same loop, it is back to 0 on the next iteration.
This behavior is why I think that I am missing something very fundamental.
In Go, arguments are passed to functions by value -- meaning what the function gets is a copy of the value, not a reference to the variable. The same is true of the function receiver, and also the return list.
It's not the most elegant description, but for the sake of explanation, let's call this the "function wall." If the value being passed one way or the other is a pointer, the function still gets a copy, but it's a copy of a memory address, and so the pointer can be used to change the value of the variable on the other side of the wall. If it is a reference type, which uses a pointer in the implementation of the type, then again a change to the thing being pointed to can cross that wall. But otherwise the change does not cross the wall, which is one reason so many Go functions are written to return values instead of just modifying values.
Here's a runnable example:
package main
import (
"fmt"
)
type Car struct {
Color string
}
func (c Car) Change() { // c was passed by value, it's a copy
c.Color = "Red"
}
func main() {
ride := Car{"Blue"}
ride.Change()
fmt.Println(ride.Color)
}
Prints "Blue"
But two small changes:
func (c *Car) Change() { // here
c.Color = "Red"
}
func main() {
ride := &Car{"Blue"} // and here
ride.Change()
fmt.Println(ride.Color)
}
And now it prints "Red". Struct is not a reference type. So if you want modifications to a struct to cross the wall without using the return list to do it, use a pointer. Of course this only applies to values being passed via argument, return list, or receiver; and not to variables that are in scope on both sides of the wall; or to modifying the underlying value behind a reference type.
See also "Pointers Versus Values" in Effective Go, and "Go Data Structures" by Russ Cox.

Calling EnumServicesStatusEx in Go, memory allocation?

I'm writing an app that interacts with the Windows API from a Windows Service.
After loads of help from #chowey here, I sort of got the hang of things and started a basic library which I've put on GitHub here.
I've now moved on to "Services", with the requirement to list all Windows Services on a machine, start, stop, restart them. The start/stop/restart look pretty straight forward once you've got a service handle to work with, but I'm struggling with getting a list of installed services.
EnumServicesStatusEx in Advapi32.dll is the function I need to call, but it requires a pointer to pre-allocated memory for an array of ENUM_SERVICE_STATUS_PROCESS structs.
You can call the function with a null pointer and it will return the memory allocation size required, but I don't believe there is a way to directly allocate memory in Go.
At first I thought I could get the memory allocation requirement, divide it by the SizeOf the struct using the unsafe package, create a slice containing that number of elements, then pass a pointer to the first element to the function, but it says the memory needs to include space for the string data, which this wouldn't.
Does anyone know how this could be accomplished, pretty please? :).
After the suggestions from #alex, I've got the following example code working.
Looks like we create a byte slice of the right size then use the unsafe class to cast to our struct type.
_, _, _ = svcEnumServicesStatusEx.Call(
uintptr(handle),
uintptr(uint32(SVC_SC_ENUM_PROCESS_INFO)),
uintptr(uint32(SVC_SERVICE_WIN32)),
uintptr(uint32(SVC_SERVICE_STATE_ALL)),
uintptr(0),
0,
uintptr(unsafe.Pointer(&bytesReq)),
uintptr(unsafe.Pointer(&numReturned)),
uintptr(unsafe.Pointer(&resumeHandle)),
uintptr(0),
)
if bytesReq > 0 {
var buf []byte = make([]byte, bytesReq)
ret, _, _ := svcEnumServicesStatusEx.Call(
uintptr(handle),
uintptr(uint32(SVC_SC_ENUM_PROCESS_INFO)),
uintptr(uint32(SVC_SERVICE_WIN32)),
uintptr(uint32(SVC_SERVICE_STATE_ALL)),
uintptr(unsafe.Pointer(&buf[0])),
uintptr(bytesReq),
uintptr(unsafe.Pointer(&bytesReq)),
uintptr(unsafe.Pointer(&numReturned)),
uintptr(unsafe.Pointer(&resumeHandle)),
uintptr(0),
)
if ret > 0 {
var sizeTest ENUM_SERVICE_STATUS_PROCESS
iter := uintptr(unsafe.Pointer(&buf[0]))
for i := uint32(0); i < numReturned; i++ {
var data *ENUM_SERVICE_STATUS_PROCESS = (*ENUM_SERVICE_STATUS_PROCESS)(unsafe.Pointer(iter))
fmt.Printf("Service Name: %s - Display Name: %s - %#v\r\n", syscall.UTF16ToString((*[4096]uint16)(unsafe.Pointer(data.lpServiceName))[:]), syscall.UTF16ToString((*[4096]uint16)(unsafe.Pointer(data.lpDisplayName))[:]), data.ServiceStatusProcess)
iter = uintptr(unsafe.Pointer(iter + unsafe.Sizeof(sizeTest)))
}
} else {
return nil, fmt.Errorf("Failed to get Service List even with allocated memory.")
}
} else {
return nil, fmt.Errorf("Unable to get size of required memory allocation.")
}

Calling kernel32's ReadProcessMemory in Go

I'm trying to manipulate processes on Windows using Go language,
and I'm starting off by reading other process' memory by using ReadProcessMemory.
However, for most of the addresses I get Error: Only part of a ReadProcessMemory or WriteProcessMemory request was completed. error. Maybe my list of arguments is wrong, but I can't find out why.
Can anyone point out what I am doing wrong here?
package main
import (
"fmt"
)
import (
windows "golang.org/x/sys/windows"
)
func main() {
handle, _ := windows.OpenProcess(0x0010, false, 6100) // 0x0010 PROCESS_VM_READ, PID 6100
procReadProcessMemory := windows.MustLoadDLL("kernel32.dll").MustFindProc("ReadProcessMemory")
var data uint = 0
var length uint = 0
for i := 0; i < 0xffffffff; i += 2 {
fmt.Printf("0x%x\n", i)
// BOOL ReadProcessMemory(HANDLE hProcess, LPCVOID lpBaseAddress, LPVOID lpBuffer, DWORD nSize, LPDWORD lpNumberOfBytesRead)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i), uintptr(data), 2, uintptr(length)) // read 2 bytes
if (ret == 0) {
fmt.Println(" Error:", e)
} else {
fmt.Println(" Length:", length)
fmt.Println(" Data:", data)
}
}
windows.CloseHandle(handle)
}
uintptr(data) is incorrect: it takes the value from data (0 of type uint) and converts that to unitptr type — yielding the same value converted to another type — producing, on x86, a null pointer.
Note that Go is not C, and you can't really play dirty games with pointers in it, or, rather, you can, but only through using the unsafe built-in package and its Pointer type which is like void* (pointing somewhere in a data memory block) in C.
What you need is something like
import "unsafe"
var (
data [2]byte
length uint32
)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i),
uintptr(unsafe.Pointer(&data[0])),
2, uintptr(unsafe.Pointer(&length))) // read 2 bytes
Observe what was done here:
A variable of type "array of two bytes" is declared;
The address of the first element of this array is taken;
That address is type-converted to the type unsafe.Pointer;
The obtained value is then type-converted to uintptr.
The last two steps are needed because Go features garbage collection:
In Go, when you take an address of a value in memory and store it in a variable, the GC knows about this "implicit" pointer and the value which address was taken won't be garbage-collected even if it becomes unreachable with that value holding its address being the only reference left.
Even if you make that address value lose the type information it maintains — through type-converting it to unsafe.Pointer, the new value is still considered by GC and behaves like "normal" values containing addresses — as explained above.
By type-converting such value to uintptr you make GC stop considering it as a pointer. Hence this type is there only for FFI/interop.
In other words, in
var data [2]byte
a := &data[0]
p := unsafe.Pointer(a)
i := uintptr(p)
there are only three references to the value in data: that variable itself, a and p, but not i.
You should consider these rules when dealing with calling outside code because you should never ever pass around unitptr-typed values: they're only for marshaling data to the called functions and unmarshaling it back, and have to be used "on the spot" — in the same scope as the values they are type-converted from/to.
Also observe that in Go, you can't just take the address of a variable of an integer type and supply that address to a function which expects a pointer to a memory block of an appropriate size. You have to deal with byte arrays and after the data has been written by the called function, you need to explicitly convert it to a value of the type you need. That's why there's no "type casts" in Go but only "type conversions": you can't reinterpret the data type of a value through type-conversion, with the uintptr(unsafe.Pointer) (and back) being a notable exception for the purpose of FFI/interop, and even in this case you basically convert a pointer to a pointer, just transfer it through the GC boundary.
To "serialize" and "deserialize" a value of an integer type you might use the encoding/binary standard package or hand-roll no-brainer simple functions which do bitwise shifts and or-s and so on ;-)
2015-10-05, updated as per the suggestion of James Henstridge.
Note that after the function returns, and ret signalizes there's no error
you have to check the value of the length variable.

Go: Call a function passing pointer to array

I have spent a bit of time attempting to do this, and I think I need a global array (not slice), and I want to pass it as a pointer, not by value. The function receiving the pointer needs to test for nil, and if nil, read the array from disk, using eg: "baForm, oOserr = ioutil.ReadFile(sFormName)". The calling function may pass either a global array or an array local to the calling function which I presume will be garbage collected.
The reason for doing this is that I want a standard function to read the forms from disk, and often-used forms are stored globally. Despite whether some may think there is a better way, I still want to know how to achieve this ie: a) have global or local array, b) not pass by value, c) The global arrays will be read only once from disk and the local arrays will be read each time the function is called. TIA.
From reading your description, I can't see why passing a pointer to an array is any way better than passing a slice -- but it's up to you.
You can pass a pointer just like you do in C -- attach an asterisk (*) to the declaration, and perpend an ampersand (&) to the value when you call the function.
Just remember that in Go, the array size is part of its type. This means that your function declaration will have the array size embedded into it, so you can't call the function using an array of any different size. This reason alone is generally enough to warrant using a slice instead of an array.
Here's a sample program that maintains a dynamic forms buffer based on use count. If the ReadForm function finds a form, it returns the address of the form and a nil error.
package main
import (
"fmt"
"io/ioutil"
"math"
"os"
"sync"
)
type Form struct {
Name string
useCount int64
Data []byte
}
// The capacity of the forms buffer.
const formsCap = 2
// The forms buffer.
var (
forms = make(map[string]*Form, formsCap)
formsLock sync.RWMutex
)
func ReadForm(name string) (form *Form, err os.Error) {
formsLock.RLock()
form, ok := forms[name]
formsLock.RUnlock()
if !ok {
form = &Form{name, 0, nil}
}
if form.Data == nil {
data, err := ioutil.ReadFile(name + ".form")
if err != nil {
return nil, err
}
form = &Form{name, 0, data}
formsLock.Lock()
if len(forms) >= formsCap {
minForm := &Form{useCount: math.MaxInt64}
for i, f := range forms {
if f.useCount < minForm.useCount {
minForm = f
}
}
minform.Data = nil
}
forms[name] = form
formsLock.Unlock()
}
form.useCount++
return form, nil
}
func main() {
// form files are named name.form e.g. form1.form
for _, name := range []string{"form1", "form2", "form3"} {
f, err := ReadForm(name)
if err != nil {
fmt.Println(err)
} else {
fmt.Println(string(f.Data))
}
}
fmt.Println(len(forms), forms)
}
EDIT: Map operations are not atomic. Revise the sample program to use a mutex for concurrent access to the forms map.

Resources