I am trying to use Buffer package and copy the following code from Buffer documentation.
package main
import (
"bytes"
"fmt"
"os"
)
func main() {
var b bytes.Buffer // A Buffer needs no initialization.
b.Write([]byte("Hello "))
fmt.Fprintf(&b, "world!")
b.WriteTo(os.Stdout)
}
Why do Buffer here, not to be initialize?
As you can see here Buffer consists just of some ints, the buf slice and some arrays. All of them need no initialization, since go has zero values.
You can read more about slices and arrays and how they work here.
It is initialized. When you do not specifically initialize a variable, go will initialize it to its zero value. That means all the internal fields of a bytes.Buffer gets the value 0, or similar for the relevant types (e.g. nil for pointers).
The authors then implemented bytes.Buffer so all values being 0 is a meaningful starting point(It means an empty buffer), so programmers doesn't need to explicitly initialize it in order to start using a Buffer.
This due to the fact that when you call the Fprintf method, the bytes.Buffer.Write method is implicitely called, and as per the doc:
Write appends the contents of p to the buffer, growing the buffer as needed.
If you look at the source code, Write calls the grow function: func (b *Buffer) grow(n int) int.
This function recognizes that the buffer is empty, because it assumes that an empty buffer has 0 values for its internal fields, which is actually how a bytes.Buffer structure is initialized by default, just like every structure in go.
Related
I'm currently teaching myself Go, and I'm having trouble understanding a certain behavior:
package main
import (
"fmt"
)
type List struct {
n int
}
func (l List) Increment() {
l.n += 1
l.LogState() // size: 1
}
func (l List) LogState() {
fmt.Printf("size: %v\n", l.n)
}
func main() {
list := List{}
list.Increment()
fmt.Println("----")
list.LogState() // size: 0
}
https://play.golang.org/p/-O24DiNPkxx
LogState is executed twice. The initial time, during the Increment call, it prints size: 1 but after Increment has returned it prints size: 0. Why are those values different?
The reason your nodes are not added to the original linkedList because you are not using pointer to the struct. So even if the Increment function in your example code changes the value. The copy of the struct is changed not the actual struct.
You can declare methods with pointer receivers. This means the
receiver type has the literal syntax *T for some type T. (Also, T
cannot itself be a pointer such as *int.)
If you want to change the linkedlistNode struct counter to show the nodes added to the list you should be using a pointer type receiver on both methdos working to modify the linked list as:
func (l *LinkedList) AddInitialValue(v interface{})
func (l *LinkedList) LogState()
And Inside the main pass an address to the linkedList to use those pointer type receivers as:
func main() {
list := &LinkedList{}
list.AddInitialValue(9)
fmt.Println("----")
list.LogState() // size: 0
}
Working Code Go playground
Note:-
There are two reasons to use a pointer receiver.
To modify the value that its receiver points to.
To avoid copying the value on each method call. This can be more efficient if the receiver is a large struct
For more information go through Method Sets
With Increment and LogState defined the way you've defined them, you are working only with the copy of the value of List. This means that if you make some changes inside Increment function, they are visible only inside Increment's function scope and only for the remainder of that particular scope's existence. To confirm you are always working with a copy of of the initial List value, you can log &list before executing Increment function and &l inside the same function.
If you want to make changes permanent, you should work with a pointer to a memory address. That means your your function should be defined like this:
func (l *List) Increment()
func (l *List) LogState()
This way, you are passing a memory reference (pointer to an address in memory) and every time you change a value of l, you are changing it on the passed memory reference and it reflects everywhere.
Consider the following code
package main
import (
"fmt"
)
func main() {
x := []byte("a")
fmt.Println(x)
fmt.Println(cap(x) == cap([]byte("a"))) // prints false
y := []byte("a")
fmt.Println(cap(y) == cap([]byte("a"))) // prints true
}
https://play.golang.org/p/zv8KQekaxH8
Calling simple Println with a slice variable, changes its capacity. I suspect calling any function with variadic parameters of ...interface{} produces the same effect. Is there any sane explanation for such behavior?
The explanation is, like bradfitz point in github, if you don't use make to create a slice, the compiler will use the cap it believes convenient. Creating multiple slices in different versions, or even the same, can result on slices of different capacities.
In short, if you need a concrete capacity, use make([]byte, len, cap). Otherwise you can't trust on a fixed capacity.
I expected the following code snippet either to produce a as {0x01, 0x02} (pass by value) or as {0x03, 0x02, 0x01} (pass by reference). Strangely, it produces the output as {0x03, 0x02}. Could you please help me understand why is that?
package main
import "fmt"
func test1(t []byte) {
t[0] = 0x03
t = append(t, 0x01 )
}
func main() {
a := []byte{0x01, 0x02 }
test1(a)
_ = a
fmt.Printf("%v", a)
}
In Go []byte is not an array, it's a byte slice.
What happened is that the size of a was 2, and because of that, after its first element was changed to 3, append allocated a new bigger slice - and t was set to the address of the new slice that was allocated, but that doesn't affect the a in main.
From A Tour of Go about append:
If the backing array of s is too small to fit all the given values a bigger array will be allocated. The returned slice will point to the newly allocated array.
The slice header is passed by value, but it references the same backing array. So when you change the first value it is changed in the original array as well. When you call append a new slice header is generated and written to the space occupied by the original argument value thus staying local to the method.
Is there an implementation of io.ReadWriteSeeker to use in Golang?
Since, bytes.Buffer does not implement Seek method, I need to find such an implementation to use as a buffer written by zipwriter and to be read with seeking.
In addition I wont go with Reader(buff.Bytes()) to covert with memory copy, because I can not afford double memory size for buffered data.
In addition, when using os.File as the option, if I wont call f.Sync, it will never touch file system, right? Thanks.
My simplified codes:
func process() {
buff := new(bytes.Buffer)
zipWriter := zip.NewWriter(buff)
// here to add data into zipWriter in sequence
zipWriter.Close()
upload(buff) // upload(io.ReadSeeker)
}
For example, using the same underlying array for (uBuf and zBuf) buffers,
package main
import (
"archive/zip"
"bytes"
"io"
)
func upload(io.ReadSeeker) {}
func process() {
zBuf := new(bytes.Buffer)
zipWriter := zip.NewWriter(zBuf)
// add data into zipWriter in sequence
zipWriter.Close()
uBuf, zBuf := zBuf.Bytes(), nil
// upload(io.ReadSeeker)
upload(bytes.NewReader(uBuf))
}
func main() {}
Playground: https://play.golang.org/p/8TKmnL_vRY9
Package bytes
import "bytes"
func (*Buffer) Bytes
func (b *Buffer) Bytes() []byte
Bytes returns a slice of length b.Len() holding the unread portion of
the buffer. The slice is valid for use only until the next buffer
modification (that is, only until the next call to a method like Read,
Write, Reset, or Truncate). The slice aliases the buffer content at
least until the next buffer modification, so immediate changes to the
slice will affect the result of future reads.
The tuple assignment statement
uBuf, zBuf := zBuf.Bytes(), nil
gets the slice descriptor for the zipped bytes (zBuf.Bytes()) and assigns it to the slice descriptor uBuf. A slice descriptor is a struct with a pointer to the underlying array, the slice length, and the slice capacity. For example,
type slice struct {
array unsafe.Pointer
len int
cap int
}
Then, for safety, we assign nil to zBuf to ensure that no further changes can be made to its underlying array, which is now used by uBuf.
I'm trying to manipulate processes on Windows using Go language,
and I'm starting off by reading other process' memory by using ReadProcessMemory.
However, for most of the addresses I get Error: Only part of a ReadProcessMemory or WriteProcessMemory request was completed. error. Maybe my list of arguments is wrong, but I can't find out why.
Can anyone point out what I am doing wrong here?
package main
import (
"fmt"
)
import (
windows "golang.org/x/sys/windows"
)
func main() {
handle, _ := windows.OpenProcess(0x0010, false, 6100) // 0x0010 PROCESS_VM_READ, PID 6100
procReadProcessMemory := windows.MustLoadDLL("kernel32.dll").MustFindProc("ReadProcessMemory")
var data uint = 0
var length uint = 0
for i := 0; i < 0xffffffff; i += 2 {
fmt.Printf("0x%x\n", i)
// BOOL ReadProcessMemory(HANDLE hProcess, LPCVOID lpBaseAddress, LPVOID lpBuffer, DWORD nSize, LPDWORD lpNumberOfBytesRead)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i), uintptr(data), 2, uintptr(length)) // read 2 bytes
if (ret == 0) {
fmt.Println(" Error:", e)
} else {
fmt.Println(" Length:", length)
fmt.Println(" Data:", data)
}
}
windows.CloseHandle(handle)
}
uintptr(data) is incorrect: it takes the value from data (0 of type uint) and converts that to unitptr type — yielding the same value converted to another type — producing, on x86, a null pointer.
Note that Go is not C, and you can't really play dirty games with pointers in it, or, rather, you can, but only through using the unsafe built-in package and its Pointer type which is like void* (pointing somewhere in a data memory block) in C.
What you need is something like
import "unsafe"
var (
data [2]byte
length uint32
)
ret, _, e := procReadProcessMemory.Call(uintptr(handle), uintptr(i),
uintptr(unsafe.Pointer(&data[0])),
2, uintptr(unsafe.Pointer(&length))) // read 2 bytes
Observe what was done here:
A variable of type "array of two bytes" is declared;
The address of the first element of this array is taken;
That address is type-converted to the type unsafe.Pointer;
The obtained value is then type-converted to uintptr.
The last two steps are needed because Go features garbage collection:
In Go, when you take an address of a value in memory and store it in a variable, the GC knows about this "implicit" pointer and the value which address was taken won't be garbage-collected even if it becomes unreachable with that value holding its address being the only reference left.
Even if you make that address value lose the type information it maintains — through type-converting it to unsafe.Pointer, the new value is still considered by GC and behaves like "normal" values containing addresses — as explained above.
By type-converting such value to uintptr you make GC stop considering it as a pointer. Hence this type is there only for FFI/interop.
In other words, in
var data [2]byte
a := &data[0]
p := unsafe.Pointer(a)
i := uintptr(p)
there are only three references to the value in data: that variable itself, a and p, but not i.
You should consider these rules when dealing with calling outside code because you should never ever pass around unitptr-typed values: they're only for marshaling data to the called functions and unmarshaling it back, and have to be used "on the spot" — in the same scope as the values they are type-converted from/to.
Also observe that in Go, you can't just take the address of a variable of an integer type and supply that address to a function which expects a pointer to a memory block of an appropriate size. You have to deal with byte arrays and after the data has been written by the called function, you need to explicitly convert it to a value of the type you need. That's why there's no "type casts" in Go but only "type conversions": you can't reinterpret the data type of a value through type-conversion, with the uintptr(unsafe.Pointer) (and back) being a notable exception for the purpose of FFI/interop, and even in this case you basically convert a pointer to a pointer, just transfer it through the GC boundary.
To "serialize" and "deserialize" a value of an integer type you might use the encoding/binary standard package or hand-roll no-brainer simple functions which do bitwise shifts and or-s and so on ;-)
2015-10-05, updated as per the suggestion of James Henstridge.
Note that after the function returns, and ret signalizes there's no error
you have to check the value of the length variable.