Go: read bytes into array - go

I'm trying to append bytes to an array in the following way:
Go
func readBytes() {
b := make([]byte, 1)
a := [][]byte{}
for i := 0, i < 4, i++ {
conn.Read(b)
a = append(a, b)
fmt.Println(b)
}
fmt.Println(a)
}
Result from fmt.Println(b):
[2]
[5]
[5]
[3]
Result from fmt.Println(a):
[[3], [3], [3], [3]]
Why does it only print out the last byte sent?? Am I missing something?

b is a slice - and you're therefore updating the same underlying array each time you pass it to conn.Read. You can look at this Golang blog post to understand how this works.
Once you call fmt.Println(a) .. each slice is looking at the same underlying array.
You could fix this by instantiating the buffer b in the loop, or using an array instead of a slice.
Here's a working example that re-allocates the b slice inside the loop: http://play.golang.org/p/cN1BE8WSFE
It is essentially (with an int slice):
for i := 0; i < 5; i++ {
b = []int{i, i + 1, i + 2, i + 3, i + 4}
a = append(a, b)
}

Related

Merge pointer to a slice

playground
package main
import (
"fmt"
"math/rand"
)
func randoms() *[]int {
var nums []int = make([]int, 5, 5) //Created slice with fixed Len, cap
fmt.Println(len(nums))
for i := range [5]int{} {//Added random numbers.
nums[i] = rand.Intn(10)
}
return &nums//Returning pointer to the slice
}
func main() {
fmt.Println("Hello, playground")
var nums []int = make([]int, 0, 25)
for _ = range [5]int{} {//Calling the functions 5 times
res := randoms()
fmt.Println(res)
//nums = append(nums, res)
for _, i := range *res {//Iterating and appending them
nums = append(nums, i)
}
}
fmt.Println(nums)
}
I am trying to mimic my problem. I have dynamic number of function calls i.e randoms and dynamic number of results. I need to append all of the results i.e numbers in this case.
I am able to do this with iteration and no issues with it. I am looking for a way to do something like nums = append(nums, res). Is there any way to do this/any built-in methods/did I misunderstand the pointers?
I think you're looking for append(nums, (*res)...):
nums = append(nums, (*res)...)
playground
See this answer for more about ..., but in short it expands the contents of a slice. Example:
x := []int{1, 2, 3}
y := []int{4, 5, 6}
x = append(x, y...) // Now x = []int{1, 2, 3, 4, 5, 6}
Further, since you have a pointer to a slice, you need to dereference the pointer with *.
x := []int{1, 2, 3}
y := &x
x = append(x, (*x)...) // x = []int{1, 2, 3, 1, 2, 3}

Working with maps in Golang

I am new to Go and doing a few exercises. One of them is to sort the numbers in an array by frequency, from most to least frequent.
Example:
Input: [2, 2, 5, 7, 4, 4, 4, 7, 2]
Output: [2, 4, 7, 5]
Note that [4, 2, 7, 5] would also be correct, since 4 and 2 have the same frequency.
For this purpose I am converting the array into a value value map, which here would look like this: [2:3][4:3][7:2][5:1] (2 and 3 have freq. of 3, 7 has the freq of 2,... )
Afterwards I would like to simply loop through the map and output the keys ordered by value. For that I use the following code, which apparently does not work. Why?
count := 0
max := -1
// break loop, if map is empty
for i := 0; i < 1; i-- {
if len(m) == 0 {
break
}
max = -1
// get key of biggest value
for k, v := range m {
if v > max {
max = k
}
}
// res (for result) is a slice of integers
res[count] = max
// remove key-value-pair from map
delete(m, max)
count++
}
return res
Please keep in mind that this is an exercise. I am very sure there are much better, build in ways to do this.
Your 'max' variable is meant to keep track of the maximum frequency seen so far. However when you do 'max = k' you're assigning a key.
You need to keep track of the maximum frequency and the key associated with that frequency in separate variables.
...
for k, v := range m {
if v > maxFreq {
maxFreq = v
mostFrequentKey = k
}
}
// res (for result) is a slice of integers
res[count] = mostFrequentKey
// remove key-value-pair from map
delete(m, mostFrequentKey)
count++
...
For sorted frequencies, use a map then a slice. For example,
package main
import (
"fmt"
"sort"
)
func main() {
Input := []int{2, 2, 5, 7, 4, 4, 4, 7, 2}
fmt.Println("Input: ", Input)
mFreq := make(map[int]int, len(Input))
for _, n := range Input {
mFreq[n]++
}
sFreq := make([][2]int, 0, len(mFreq))
for n, f := range mFreq {
sFreq = append(sFreq, [2]int{n, f})
}
sort.Slice(sFreq, func(i, j int) bool {
if sFreq[i][1] <= sFreq[j][1] {
if sFreq[i][1] < sFreq[j][1] {
return false
}
if sFreq[i][0] >= sFreq[j][0] {
return false
}
}
return true
},
)
Output := []int{2, 4, 7, 5}
fmt.Println("Output: ", Output)
fmt.Println("Frequencies:", sFreq)
}
Playground: https://play.golang.org/p/8tiSksz3S76
Output:
Input: [2 2 5 7 4 4 4 7 2]
Output: [2 4 7 5]
Frequencies: [[2 3] [4 3] [7 2] [5 1]]

Similar Go functions appending strings and arrays do not behave as expected

I have two Go functions:
func permutation(prefix, str []int) {
n := len(str)
if n == 0 {
fmt.Println(prefix)
} else {
for i := 0; i < n; i++ {
permutation(
append(prefix, str[i]),
append(str[0:i], str[i+1:]...),
)
}
}
}
func perms(prefix, str string) {
n := len(str)
if n == 0 {
fmt.Println(prefix)
} else {
for i := 0; i < n; i++ {
perms(
prefix+string(str[i]),
string(str[0:i])+string(str[i+1:]),
)
}
}
}
The first takes an array of ints, the second takes a string. They both then calculate all permutations of the array, or the string.
I can run them like so:
permutation([]int{}, []int{1, 2, 3})
perms("", "123")
Their output is not the same:
$ go run main.go
[1 2 3]
[1 3 3]
[3 3 3]
[3 3 3]
[3 3 3]
[3 3 3]
123
132
213
231
312
321
I guess there is some nuance to appending arrays that I am missing. I can't seem to figure it out. Any idea what's going on?
While str1+str2 does return new (unrelated in terms of memory) string, append doesn't behave this way. For example append(str[0:i], str[i+1:]...) will destroy original content of str, overwriting str[i:] with str[i+1:]. This is because str[0:i] will have capacity to append str[i+1:] without allocating new buffer.
The solution would be to create a completely new array in every iteration. At least for str, as append(prefix, str[i]) is immune to this problem. For example:
for i := 0; i < n; i++ {
var s []int
s = append(s, str[0:i]...)
s = append(s, str[i+1:]...)
permutation(append(prefix, str[i]), s)
}
https://play.golang.org/p/lXwu39AA0V
More about slices and mechanism of append:
http://blog.golang.org/go-slices-usage-and-internals
https://blog.golang.org/slices

How to remove items from a slice while ranging over it?

What is the best way to remove items from a slice while ranging over it?
For example:
type MultiDataPoint []*DataPoint
func (m MultiDataPoint) Json() ([]byte, error) {
for i, d := range m {
err := d.clean()
if ( err != nil ) {
//Remove the DP from m
}
}
return json.Marshal(m)
}
As you have mentioned elsewhere, you can allocate new memory block and copy only valid elements to it. However, if you want to avoid the allocation, you can rewrite your slice in-place:
i := 0 // output index
for _, x := range s {
if isValid(x) {
// copy and increment index
s[i] = x
i++
}
}
// Prevent memory leak by erasing truncated values
// (not needed if values don't contain pointers, directly or indirectly)
for j := i; j < len(s); j++ {
s[j] = nil
}
s = s[:i]
Full example: http://play.golang.org/p/FNDFswPeDJ
Note this will leave old values after index i in the underlying array, so this will leak memory until the slice itself is garbage collected, if values are or contain pointers. You can solve this by setting all values to nil or the zero value from i until the end of the slice before truncating it.
I know its answered long time ago but i use something like this in other languages, but i don't know if it is the golang way.
Just iterate from back to front so you don't have to worry about indexes that are deleted. I am using the same example as Adam.
m = []int{3, 7, 2, 9, 4, 5}
for i := len(m)-1; i >= 0; i-- {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
}
}
There might be better ways, but here's an example that deletes the even values from a slice:
m := []int{1,2,3,4,5,6}
deleted := 0
for i := range m {
j := i - deleted
if (m[j] & 1) == 0 {
m = m[:j+copy(m[j:], m[j+1:])]
deleted++
}
}
Note that I don't get the element using the i, d := range m syntax, since d would end up getting set to the wrong elements once you start deleting from the slice.
Here is a more idiomatic Go way to remove elements from slices.
temp := s[:0]
for _, x := range s {
if isValid(x) {
temp = append(temp, x)
}
}
s = temp
Playground link: https://play.golang.org/p/OH5Ymsat7s9
Note: The example and playground links are based upon #tomasz's answer https://stackoverflow.com/a/20551116/12003457
One other option is to use a normal for loop using the length of the slice and subtract 1 from the index each time a value is removed. See the following example:
m := []int{3, 7, 2, 9, 4, 5}
for i := 0; i < len(m); i++ {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
i-- // -1 as the slice just got shorter
}
}
I don't know if len() uses enough resources to make any difference but you could also run it just once and subtract from the length value too:
m := []int{3, 7, 2, 9, 4, 5}
for i, s := 0, len(m); i < s; i++ {
if m[i] < 5 {
m = append(m[:i], m[i+1:]...)
s--
i--
}
}
Something like:
m = append(m[:i], m[i+1:]...)
You don't even need to count backwards but you do need to check that you're at the end of the array where the suggested append() will fail. Here's an example of removing duplicate positive integers from a sorted list:
// Remove repeating numbers
numbers := []int{1, 2, 3, 3, 4, 5, 5}
log.Println(numbers)
for i, numbersCount, prevNum := 0, len(numbers), -1; i < numbersCount; numbersCount = len(numbers) {
if numbers[i] == prevNum {
if i == numbersCount-1 {
numbers = numbers[:i]
} else {
numbers = append(numbers[:i], numbers[i+1:]...)
}
continue
}
prevNum = numbers[i]
i++
}
log.Println(numbers)
Playground: https://play.golang.org/p/v93MgtCQsaN
I just implement a method which removes all nil elements in slice.
And I used it to solve a leetcode problems, it works perfectly.
/**
* Definition for singly-linked list.
* type ListNode struct {
* Val int
* Next *ListNode
* }
*/
func removeNil(lists *[]*ListNode) {
for i := 0; i < len(*lists); i++ {
if (*lists)[i] == nil {
*lists = append((*lists)[:i], (*lists)[i+1:]...)
i--
}
}
}
You can avoid memory leaks, as suggested in #tomasz's answer, controlling the capacity of the underlying array with a full slice expression. Look at the following function that remove duplicates from a slice of integers:
package main
import "fmt"
func removeDuplicates(a []int) []int {
for i, j := 0, 1; i < len(a) && j < len(a); i, j = i+1, j+1 {
if a[i] == a[j] {
copy(a[j:], a[j+1:])
// resize the capacity of the underlying array using the "full slice expression"
// a[low : high : max]
a = a[: len(a)-1 : len(a)-1]
i--
j--
}
}
return a
}
func main() {
a := []int{2, 3, 3, 3, 6, 9, 9}
fmt.Println(a)
a = removeDuplicates(a)
fmt.Println(a)
}
// [2 3 3 3 6 9 9]
// [2 3 6 9]
For reasons #tomasz has explained, there are issues with removing in place. That's why it is practice in golang not to do that, but to reconstruct the slice. So several answers go beyond the answer of #tomasz.
If elements should be unique, it's practice to use the keys of a map for this. I like to contribute an example of deletion by use of a map.
What's nice, the boolean values are available for a second purpose. In this example I calculate Set a minus Set b. As Golang doesn't have a real set, I make sure the output is unique. I use the boolean values as well for the algorithm.
The map gets close to O(n). I don't know the implementation. append() should be O(n). So the runtime is similar fast as deletion in place. Real deletion in place would cause a shifting of the upper end to clean up. If not done in batch, the runtime should be worse.
In this special case, I also use the map as a register, to avoid a nested loop over Set a and Set b to keep the runtime close to O(n).
type Set []int
func differenceOfSets(a, b Set) (difference Set) {
m := map[int]bool{}
for _, element := range a {
m[element] = true
}
for _, element := range b {
if _, registered := m[element]; registered {
m[element] = false
}
}
for element, present := range m {
if present {
difference = append(difference, element)
}
}
return difference
}
Try Sort and Binary search.
Example:
package main
import (
"fmt"
"sort"
)
func main() {
// Our slice.
s := []int{3, 7, 2, 9, 4, 5}
// 1. Iterate over it.
for i, v := range s {
func(i, v int) {}(i, v)
}
// 2. Sort it. (by whatever condition of yours)
sort.Slice(s, func(i, j int) bool {
return s[i] < s[j]
})
// 3. Cut it only once.
i := sort.Search(len(s), func(i int) bool { return s[i] >= 5 })
s = s[i:]
// That's it!
fmt.Println(s) // [5 7 9]
}
https://play.golang.org/p/LnF6o0yMJGT

Go variables being overwritten (bug?)

So bit of a weird one here. My question is, do people get the same results from running my code as I do? And if you do, is it a fault of my code (I'm a python programmer usually), or a bug in golang?
System info: Go version (1.1.2) linux x64 (fedora 19)
Background info on the code: What I'm doing is finding the highest cost route from the top of a triangle to the bottom, this is from project_euler 18 and 67
The bug: I set a variable called pathA, this is an integer list, plus a new int for the new value found from the triangle
e.g. 3, 7, 2 append 8 should equal 3, 2, 7, 8
and, it does! ... until I set pathB. pathB gets set correctly however suddenly pathA is the same value as pathB.
tl;dr one variable is being overwritten when I set another
My code is as follows:
package main
import (
"fmt"
)
func extendPaths(triangle, prePaths [][]int) [][]int {
nextLine := triangle[len(prePaths)]
fmt.Println("#####PrePaths: ", prePaths)
fmt.Println("#####nextLine: ", nextLine)
postPaths := [][]int{{}}
for i := 0; i < len(prePaths); i++ {
route := prePaths[i]
nextA := nextLine[i]
nextB := nextLine[i+1]
fmt.Println("Next A:", nextA, "Next B:", nextB, "\n")
pathA := append(route, nextA)
fmt.Println("pathA check#1:", pathA)
pathB := append(route, nextB)
fmt.Println("pathA check#2:", pathA, "\n")
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
postPaths = postPaths[1:]
prePaths = [][]int{postPaths[0]}
for i := 1; i < len(postPaths)-1; i += 2 {
if getSum(postPaths[i]) > getSum(postPaths[i+1]) {
prePaths = append(prePaths, postPaths[i])
} else {
prePaths = append(prePaths, postPaths[i+1])
}
}
prePaths = append(prePaths, postPaths[len(postPaths)-1])
return prePaths
}
func getSum(sumList []int) int {
total := 0
for i := 0; i < len(sumList); i++ {
total += sumList[i]
}
return total
}
func getPaths(triangle [][]int) {
prePaths := [][]int{{triangle[0][0]}}
for i := 0; i < len(triangle)-1; i++ {
prePaths = extendPaths(triangle, prePaths)
}
}
func main() {
triangle := [][]int{{3}, {7, 4}, {2, 4, 6}, {8, 5, 9, 3}}
getPaths(triangle)
}
This gives the output in my terminal shown below:
#####PrePaths: [[3]]
#####nextLine: [7 4]
Next A: 7 Next B: 4
pathA check#1: [3 7]
pathA check#2: [3 7]
#####PrePaths: [[3 7] [3 4]]
#####nextLine: [2 4 6]
Next A: 2 Next B: 4
pathA check#1: [3 7 2]
pathA check#2: [3 7 2]
Next A: 4 Next B: 6
pathA check#1: [3 4 4]
pathA check#2: [3 4 4]
#####PrePaths: [[3 7 2] [3 7 4] [3 4 6]]
#####nextLine: [8 5 9 3]
Next A: 8 Next B: 5
pathA check#1: [3 7 2 8]
pathA check#2: [3 7 2 5]
Next A: 5 Next B: 9
pathA check#1: [3 7 4 5]
pathA check#2: [3 7 4 9]
Next A: 9 Next B: 3
pathA check#1: [3 4 6 9]
pathA check#2: [3 4 6 3]
Here you can see that for the last 4 times that I set pathA, it is initially set correctly, but then gets overwritten by pathB.
Does anyone have any thoughts on this?
EDIT:
As pointed out by the comments below, what was needed was to make new slices and copy data from the originals. This was done using code from http://blog.golang.org/go-slices-usage-and-internals modified slightly:
func AppendInt(slice []int, data ...int) []int {
m := len(slice)
n := m + len(data)
if n > cap(slice) {
newSlice := make([]int, (n+1)*2)
copy(newSlice, slice)
slice = newSlice
}
slice = slice[0:n]
copy(slice[m:n], data)
return slice
}
I also changed the code on the other side, where I created the slices pathA and pathB. This changed to:
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := AppendInt(prePaths[i], nextA)
pathB := AppendInt(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
EDIT2:
It's quite early in the morning here, and I flat out made a mistake in my first edit, I did not fully understand your solution, after a bit of hacking I got there in the end:
This code does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(prePaths[i], nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
This code also does not work (pathA gets overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(newRoute, nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
However, if I mix the 2 scenarios above into the code below, it works fine (pathA does not get overwritten):
for i := 0; i < len(prePaths); i++ {
newRoute := make([]int, len(prePaths[i]), (cap(prePaths[i])+1)*2)
copy(newRoute, prePaths[i])
nextA := nextLine[i]
nextB := nextLine[i+1]
pathA := append(newRoute, nextA)
pathB := append(prePaths[i], nextB)
postPaths = append(postPaths, pathA)
postPaths = append(postPaths, pathB)
}
So, my solution was to make a copy of the array, and have them both use different ones.
A slice is basically a structure consisting of 3 things:
A pointer to an array of the elements in the slice
The length of that array (the "capacity")
The number of elements actually stored in the array (the "length")
When you run the following code:
append(x, element)
It does the following:
Check if extending the slice will exceed the capacity of the underlying array. If so, allocate a larger one and copy the existing elements to the new array, and update the capacity.
Write the new element (or elements) to the end of the array and update the length.
Return the new slice.
In your code, you have the following:
pathA := append(route, nextA)
pathB := append(route, nextB)
Now there are two possibilities here:
len(route) == cap(route), and a new backing array will be allocated, with pathA and pathB having independent values.
len(route) < cap(route), so pathA and pathB end up sharing the same backing array. The last element in the array will be nextB, since that operation was run second.
It seems that the first case is true for the first few iterations of your loop, after which you hit the second case. You could avoid this by manually making a copy for one of your paths (allocate a slice with make(), and then use copy() to copy the old data).

Resources