How can I retrieve an image data buffer from clipboard memory (uintptr)? - windows

I'm trying to use syscall with user32.dll to get the contents of the clipboard. I expect it to be image data from a Print Screen.
Right now I've got this:
if opened := openClipboard(0); !opened {
fmt.Println("Failed to open Clipboard")
}
handle := getClipboardData(CF_BITMAP)
// get buffer
img, _, err := Decode(buffer)
I need to get the data into a readable buffer using the handle.
I've had some inspiration from AllenDang/w32 and atotto/clipboard on github. The following would work for text, based on atotto's implementation:
text := syscall.UTF16ToString((*[1 << 20]uint16)(unsafe.Pointer(handle))[:])
But how can I get a buffer containing image data I can decode?
[Update]
Going by the solution #kostix provided, I hacked together a half working example:
image.RegisterFormat("bmp", "bmp", bmp.Decode, bmp.DecodeConfig)
if opened := w32.OpenClipboard(0); opened == false {
fmt.Println("Error: Failed to open Clipboard")
}
//fmt.Printf("Format: %d\n", w32.EnumClipboardFormats(w32.CF_BITMAP))
handle := w32.GetClipboardData(w32.CF_DIB)
size := globalSize(w32.HGLOBAL(handle))
if handle != 0 {
pData := w32.GlobalLock(w32.HGLOBAL(handle))
if pData != nil {
data := (*[1 << 25]byte)(pData)[:size]
// The data is either in DIB format and missing the BITMAPFILEHEADER
// or there are other issues since it can't be decoded at this point
buffer := bytes.NewBuffer(data)
img, _, err := image.Decode(buffer)
if err != nil {
fmt.Printf("Failed decoding: %s", err)
os.Exit(1)
}
fmt.Println(img.At(0, 0).RGBA())
}
w32.GlobalUnlock(w32.HGLOBAL(pData))
}
w32.CloseClipboard()
AllenDang/w32 contains most of what you'd need, but sometimes you need to implement something yourself, like globalSize():
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procGlobalSize = modkernel32.NewProc("GlobalSize")
)
func globalSize(hMem w32.HGLOBAL) uint {
ret, _, _ := procGlobalSize.Call(uintptr(hMem))
if ret == 0 {
panic("GlobalSize failed")
}
return uint(ret)
}
Maybe someone will come up with a solution to get the BMP data. In the meantime I'll be taking a different route.

#JimB is correct: user32!GetClipboardData() returns a HGLOBAL, and a comment example over there suggests using kernel32!GlobalLock() to a) globally lock that handle, and b) yield a proper pointer to the memory referred to by it.
You will need to kernel32!GlobalUnlock() the handle after you're done with it.
As to converting pointers obtained from Win32 API functions to something readable by Go, the usual trick is casting the pointer to an insanely large slice. To cite the "Turning C arrays into Go slices" of "the Go wiki article on cgo":
To create a Go slice backed by a C array (without copying the original
data), one needs to acquire this length at runtime and use a type
conversion to a pointer to a very big array and then slice it to the
length that you want (also remember to set the cap if you're using Go 1.2 > or later), for example (see http://play.golang.org/p/XuC0xqtAIC for a
runnable example):
import "C"
import "unsafe"
...
var theCArray *C.YourType = C.getTheArray()
length := C.getTheArrayLength()
slice := (*[1 << 30]C.YourType)(unsafe.Pointer(theCArray))[:length:length]
It is important to keep in mind that the Go garbage collector will not
interact with this data, and that if it is freed from the C side of
things, the behavior of any Go code using the slice is nondeterministic.
In your case it will be simpler:
h := GlobalLock()
defer GlobalUnlock(h)
length := somehowGetLengthOfImageInTheClipboard()
slice := (*[1 << 30]byte)(unsafe.Pointer((uintptr(h)))[:length:length]
Then you need to actually read the bitmap.
This depends on the format of the Device-Independent Bitmap (DIB) available for export from the clipboard.
See this and this for a start.
As usually, definitions of BITMAPINFOHEADER etc are easily available online in the MSDN site.

Related

Why copyBuffer implements while loop

I am trying to understand how copyBuffer works under the hood, but what is not clear to me is the use of while loop
for {
nr, er := src.Read(buf)
//...
}
Full code below:
// copyBuffer is the actual implementation of Copy and CopyBuffer.
// if buf is nil, one is allocated.
func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
// If the reader has a WriteTo method, use it to do the copy.
// Avoids an allocation and a copy.
if wt, ok := src.(WriterTo); ok {
return wt.WriteTo(dst)
}
// Similarly, if the writer has a ReadFrom method, use it to do the copy.
if rt, ok := dst.(ReaderFrom); ok {
return rt.ReadFrom(src)
}
size := 32 * 1024
if l, ok := src.(*LimitedReader); ok && int64(size) > l.N {
if l.N < 1 {
size = 1
} else {
size = int(l.N)
}
}
if buf == nil {
buf = make([]byte, size)
}
for {
nr, er := src.Read(buf)
if nr > 0 {
nw, ew := dst.Write(buf[0:nr])
if nw > 0 {
written += int64(nw)
}
if ew != nil {
err = ew
break
}
if nr != nw {
err = ErrShortWrite
break
}
}
if er != nil {
if er != EOF {
err = er
}
break
}
}
return written, err
}
It writes to nw, ew := dst.Write(buf[0:nr]) when nr is the number of bytes read, so why is the while loop necessary?
Let's assume that src does not implement WriterTo and dst does not implement ReaderFrom, since otherwise we would not get down to the for loop at all.
Let's further assume, for simplicity, that src does not implement LimitedReader, so that size is 32 * 1024: 32 kBytes. (There is no real loss of generality here as LimitedReader just allows the source to pick an even smaller number, at least in this case.)
Finally, let's assume buf is nil. (Or, if it's not nil, let's assume it has a capacity of 32768 bytes. If it has a large capacity, we can just change the rest of the assumptions below, so that src has more bytes than there are in the buffer.)
So: we enter the loop with size holding the size of the temporary buffer buf, which is 32k. Now suppose the source is a file that holds 64k. It will take at least two src.Read() calls to read it! Clearly we need an outer loop. That's the overall for here.
Now suppose that src.Read() really does read the full 32k, so that nr is also 32 * 1024. The code will now call dst.Write(), passing the full 32k of data. Unlike src.Read()—which is allowed to only read, say, 1k instead of the full 32k—the next chunk of code requires that dst.Write() write all 32k. If it doesn't, the loop will break with err set to ErrShortWrite.
(An alternative would have been to keep calling dst.Write() with the remaining bytes, so that dst.Write() could write only 1k of the 32k, requiring 32 calls to get it all written.)
Note that src.Read() can choose to read only, say, 1k instead of 32k. If the actual file is 64k, it will then take 64 trips, rather than 2, through the outer loop. (An alternative choice would have been to force such a reader to implement the LimitedReaderinterface. That's not as flexible, though, and is not what LimitedReader is intended for.)
func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error)
when the total data size to copy if larger than len(buf), nr, er := src.Read(buf) will try read at most len(buf) data every time.
that's how copyBuffer works:
for {
copy `len(buf)` data from `src` to `dst`;
if EOF {
//done
break;
}
if other Errors {
return Error
}
}
In the normal case, you would just call Copy rather than CopyBuffer.
func Copy(dst Writer, src Reader) (written int64, err error) {
return copyBuffer(dst, src, nil)
}
The option to have a user-supplied buffer is, I think, just for extreme optimization scenarios. The use of the word "Buffer" in the name is possibly a source of confusion since the function is not copying the buffer -- just using it internally.
There are two reasons for the looping...
The buffer might not be large enough to copy all of the data (the size of which is not necessarily known in advance) in one pass.
Reader, though not 'Writer', may return partial results when it makes sense to do so.
Regarding the second item, consider that the Reader does not necessarily represent a fixed file or data buffer. It could, instead, be a live stream from some other thread or process. As such, there are many valid scenarios for stream data to be read and processed on an as-available basis. Although CopyBuffer doesn't do this, it still has to work with such behaviors from any Reader.

Is it a good idea to use a list of *bufio.Scanner for files to be read later in golang?

I have a list of delimited files to be read after I obtained their path. Instead of saving path as a string, I'm wondering can I simply store a list of *bufio.Scanner so those will be much easier to be read later (and code will be cleaner too)? Here is a quick example:
func main(){
scannerList := read(filenameList)
dowork(scannerList)
}
func read(filenameList []string) (scannerList []*bufio.Scanner){
for _, filename := range filenameList{
op, _ := os.Open(filename)
defer op.Close()
scanner := bufio.NewScanner(op)
scannerList = append(scannerList, scanner)
}
return
}
func dowork(scannerList []*bufio.Scanner){
for _, scanner := range scannerList{
for scanner.Scan(){
//read stuff
}
//do stuff
}
}
My code similar to above example compiles, but I don't know if this is recommended (or works). Any comments? Thanks!
A Scanner is a complicated structure, and one that embeds a buffer. The buffer can grow dynamically (depending on what the scan function requests) up to 64kB (MaxScanTokenSize).
So in general it is not a good idea to keep redundant Scanners around, as the buffers cannot be released until the Scanners are discarded. But perhaps a few extra kilobytes of memory don't matter much in your case.

Flatbuffers GoLang- Unable to understand my mistake while serializing and deserializing data resulting in not able to retrieve the data

I am new to Flatbuffers and GoLang. I am trying to implement a function that take converts an object to flatbuffer and retrieves the same object. Here is my code.
Updated Code
func getannouncements(){
annList := SR.GetFromDB().GetAllAnnouncementList()
fmt.Println(annList)
builder := flatbuffers.NewBuilder(1024)
var thisobjlist [12] flatbuffers.UOffsetT
for i,j := range annList{
annTitle := builder.CreateString(j.AnnTitle)
annText := builder.CreateString(j.AnnText)
annDate := builder.CreateString(j.AnnDate)
fb.AnnouncementStart(builder)
fb.AnnouncementAddAnnId(builder,int32(j.AnnID))
fb.AnnouncementAddAnnTitle(builder,annTitle)
fb.AnnouncementAddAnnText(builder, annText)
fb.AnnouncementAddAnnActive(builder,CR.BoolToByte(j.AnnActive))
fb.AnnouncementAddAnnDate(builder,annDate)
thisobj:= fb.AnnouncementEnd(builder)
thisobjlist[i] = thisobj
}
fb.AnnouncementListStartAnnListVector(builder,len(annList))
for _,j:=range thisobjlist{
builder.PlaceUOffsetT(j)
}
finalObj := fb.AnnouncementListEnd(builder)
builder.Finish(finalObj)
buf:= builder.FinishedBytes()
fmt.Println(buf)
/*bufItem := new(bytes.Buffer)
binary.Write(bufItem, binary.LittleEndian, buf)
buf1 := bufItem.Bytes()
buffyRead := bytes.NewReader(buf1)
var buffy []byte
binary.Read(buffyRead, binary.LittleEndian, &buffy)*/
Anncmt:= fb.GetRootAsAnnouncementList(buf,0)
anns := new(fb.Announcement)
if Anncmt.AnnList(anns,1){
thisLists := anns.AnnTitle()
fmt.Println(thisLists)
}
fmt.Println(Anncmt)
}
Schema File
namespace FlatBufs;
table Announcement{
AnnId:int;
AnnTitle:string;
AnnText:string;
AnnDate:string;
AnnActive:bool= false;
}
table AnnouncementList{
AnnName:string;
AnnList:[Announcement];
}
root_type AnnouncementList
The buf object is a byte array. However when I am generating the AnnGot obj I am still getting the almost same byte array as buf. So, When I read various posts on internet on this topic, I tried to convert that buf to binary type and then retrieve buf and tried to retrieve the data (as in the commented part of code). This time the object buffy doesn't have any data in it. I am still not clear what mistake I am making in this entire code.
Please point me in the right direction. Any help on this is much appreciated.
Thanks,
Tom
Not sure why you're using two builders. Each builder is creating one FlatBuffer, so things you store in one won't be available in the other.
A smaller problem is that you are nesting things (the Go implementation should guard against that, I believe?), i.e. you should finish building all announcements before you create a vector of them.. in your case this will all be mixed up.
You also appear to make your root an AnnouncementList, but then you try to access it as an Announcement.. this won't work. It be easier to see if we could see your schema.

Calling EnumServicesStatusEx in Go, memory allocation?

I'm writing an app that interacts with the Windows API from a Windows Service.
After loads of help from #chowey here, I sort of got the hang of things and started a basic library which I've put on GitHub here.
I've now moved on to "Services", with the requirement to list all Windows Services on a machine, start, stop, restart them. The start/stop/restart look pretty straight forward once you've got a service handle to work with, but I'm struggling with getting a list of installed services.
EnumServicesStatusEx in Advapi32.dll is the function I need to call, but it requires a pointer to pre-allocated memory for an array of ENUM_SERVICE_STATUS_PROCESS structs.
You can call the function with a null pointer and it will return the memory allocation size required, but I don't believe there is a way to directly allocate memory in Go.
At first I thought I could get the memory allocation requirement, divide it by the SizeOf the struct using the unsafe package, create a slice containing that number of elements, then pass a pointer to the first element to the function, but it says the memory needs to include space for the string data, which this wouldn't.
Does anyone know how this could be accomplished, pretty please? :).
After the suggestions from #alex, I've got the following example code working.
Looks like we create a byte slice of the right size then use the unsafe class to cast to our struct type.
_, _, _ = svcEnumServicesStatusEx.Call(
uintptr(handle),
uintptr(uint32(SVC_SC_ENUM_PROCESS_INFO)),
uintptr(uint32(SVC_SERVICE_WIN32)),
uintptr(uint32(SVC_SERVICE_STATE_ALL)),
uintptr(0),
0,
uintptr(unsafe.Pointer(&bytesReq)),
uintptr(unsafe.Pointer(&numReturned)),
uintptr(unsafe.Pointer(&resumeHandle)),
uintptr(0),
)
if bytesReq > 0 {
var buf []byte = make([]byte, bytesReq)
ret, _, _ := svcEnumServicesStatusEx.Call(
uintptr(handle),
uintptr(uint32(SVC_SC_ENUM_PROCESS_INFO)),
uintptr(uint32(SVC_SERVICE_WIN32)),
uintptr(uint32(SVC_SERVICE_STATE_ALL)),
uintptr(unsafe.Pointer(&buf[0])),
uintptr(bytesReq),
uintptr(unsafe.Pointer(&bytesReq)),
uintptr(unsafe.Pointer(&numReturned)),
uintptr(unsafe.Pointer(&resumeHandle)),
uintptr(0),
)
if ret > 0 {
var sizeTest ENUM_SERVICE_STATUS_PROCESS
iter := uintptr(unsafe.Pointer(&buf[0]))
for i := uint32(0); i < numReturned; i++ {
var data *ENUM_SERVICE_STATUS_PROCESS = (*ENUM_SERVICE_STATUS_PROCESS)(unsafe.Pointer(iter))
fmt.Printf("Service Name: %s - Display Name: %s - %#v\r\n", syscall.UTF16ToString((*[4096]uint16)(unsafe.Pointer(data.lpServiceName))[:]), syscall.UTF16ToString((*[4096]uint16)(unsafe.Pointer(data.lpDisplayName))[:]), data.ServiceStatusProcess)
iter = uintptr(unsafe.Pointer(iter + unsafe.Sizeof(sizeTest)))
}
} else {
return nil, fmt.Errorf("Failed to get Service List even with allocated memory.")
}
} else {
return nil, fmt.Errorf("Unable to get size of required memory allocation.")
}

Most efficient way to read Zlib compressed file in Golang?

I'm reading in and at the same time parsing (decoding) a file in a custom format, which is compressed with zlib. My question is how can I efficiently uncompress and then parse the uncompressed content without growing the slice? I would like to parse it whilst reading it into a reusable buffer.
This is for a speed-sensitive application and so I'd like to read it in as efficiently as possible. Normally I would just ioutil.ReadAll and then loop again through the data to parse it. This time I'd like to parse it as it's read, without having to grow the buffer into which it is read, for maximum efficiency.
Basically I'm thinking that if I can find a buffer of the perfect size then I can read into this, parse it, and then write over the buffer again, then parse that, etc. The issue here is that the zlib reader appears to read an arbitrary number of bytes each time Read(b) is called; it does not fill the slice. Because of this I don't know what the perfect buffer size would be. I'm concerned that it might break up some of the data that I wrote into two chunks, making it difficult to parse because one say uint64 could be split from into two reads and therefore not occur in the same buffer read - or perhaps that can never happen and it's always read out in chunks of the same size as were originally written?
What is the optimal buffer size, or is there a way to calculate this?
If I have written data into the zlib writer with f.Write(b []byte) is it possible that this same data could be split into two reads when reading back the compressed data (meaning I will have to have a history during parsing), or will it always come back in the same read?
You can wrap your zlib reader in a bufio reader, then implement a specialized reader on top that will rebuild your chunks of data by reading from the bufio reader until a full chunk is read. Be aware that bufio.Read calls Read at most once on the underlying Reader, so you need to call ReadByte in a loop. bufio will however take care of the unpredictable size of data returned by the zlib reader for you.
If you do not want to implement a specialized reader, you can just go with a bufio reader and read as many bytes as needed with ReadByte() to fill a given data type. The optimal buffer size is at least the size of your largest data structure, up to whatever you can shove into memory.
If you read directly from the zlib reader, there is no guarantee that your data won't be split between two reads.
Another, maybe cleaner, solution is to implement a writer for your data, then use io.Copy(your_writer, zlib_reader).
OK, so I figured this out in the end using my own implementation of a reader.
Basically the struct looks like this:
type reader struct {
at int
n int
f io.ReadCloser
buf []byte
}
This can be attached to the zlib reader:
// Open file for reading
fi, err := os.Open(filename)
if err != nil {
return nil, err
}
defer fi.Close()
// Attach zlib reader
r := new(reader)
r.buf = make([]byte, 2048)
r.f, err = zlib.NewReader(fi)
if err != nil {
return nil, err
}
defer r.f.Close()
Then x number of bytes can be read straight out of the zlib reader using a function like this:
mydata := r.readx(10)
func (r *reader) readx(x int) []byte {
for r.n < x {
copy(r.buf, r.buf[r.at:r.at+r.n])
r.at = 0
m, err := r.f.Read(r.buf[r.n:])
if err != nil {
panic(err)
}
r.n += m
}
tmp := make([]byte, x)
copy(tmp, r.buf[r.at:r.at+x]) // must be copied to avoid memory leak
r.at += x
r.n -= x
return tmp
}
Note that I have no need to check for EOF because I my parser should stop itself at the right place.

Resources