How to efficiently concatenate string array and string in golang - go

I have a bunch of strings and []strings in golang which I need to concatenate. But for some reason I am getting a lot of whitespaces along the way which I need to get rid of.
Here's the code
tests := strings.TrimSpace(s[0])
dep_string := make ([]string, len(tests) + len(sfinal))
dep_string = append (dep_string,tests)
for _,v := range sfinal {
dep_string = append(dep_string,v)
}
fmt.Println("dep_String is ",dep_string)
Input:
s[0] = "filename"
sfinal = [test test1]
expected output
[filename test test1]
actual output
[ filename test test1]
It's really weird; even after using TrimSpace I am not able to get rid of excess space. Is there any efficient way to concatenate them?

The whitespace is due to all of the empty elements in dep_string. When you use the make function, it creates a slice with the specified length and capacity, filled with a bunch of nothing. Then, when you use append, it sees that the slice has reached its maximum capacity, extends the slice, then adds your elements, after all of the nothing. The solution is to make a slice with the capacity to hold all of your elements, but with initial length zero:
dep_string := make ([]string, 0, len(tests) + len(sfinal))
strings.TrimSpace is unnecessary. You can read more at http://blog.golang.org/slices

Bill DeRose and Saposhiente are correct about how slices work.
As for a simpler way of solving your problem, you could also do (play):
fmt.Println("join is",strings.Join(append(s[:1],sfinal...)," "))

When you do the assignment dep_string := make ([]string, len(tests) + len(sfinal)), Go zeros out the allocated memory so dep_string then has len(tests) + len(sfinal) empty strings at the front of it. As it's written now you append to the end of the slice, after all those zeroed out strings.
Run this to see where those blanks are showing up in your code. You can fix it by making a slice of length 0 and capacity len(tests) + len(sfinal) instead. You can then concatenate them by using strings.Join.

Here's a simple and efficient solution to your problem.
package main
import "fmt"
func main() {
s := make([]string, 1, 4)
s[0] = "filename"
sfinal := []string{"test", "test1"}
dep_string := append(s[:1], sfinal...)
fmt.Println("dep_String is ", dep_string)
}
Output:
dep_String is [filename test test1]

When you do the assignment dep_string := make ([]string, len(tests) + len(sfinal)), it allocate len(tests) + len(sfinal) null strings ,it is 10 null strings in your case,so when you do the assignment fmt.Println("dep_String is ",dep_string) ,it will print 10 null strings, because fmt.Println(slice of string) will add blank between two elements,so it will print 9 blanks ,so it will prints [ filename test test1] after you append, the whitespaces is the blanks between the 10 null string.

Related

How to convert the string representation of a Terraform set of strings to a slice of strings

I've a terratest where I get an output from terraform like so s := "[a b]". The terraform output's value = toset([resource.name]), it's a set of strings.
Apparently fmt.Printf("%T", s) returns string. I need to iterate to perform further validation.
I tried the below approach but errors!
var v interface{}
if err := json.Unmarshal([]byte(s), &v); err != nil {
fmt.Println(err)
}
My current implementation to convert to a slice is:
s := "[a b]"
s1 := strings.Fields(strings.Trim(s, "[]"))
for _, v:= range s1 {
fmt.Println("v -> " + v)
}
Looking for suggestions to current approach or alternative ways to convert to arr/slice that I should be considering. Appreciate any inputs. Thanks.
Actually your current implementation seems just fine.
You can't use JSON unmarshaling because JSON strings must be enclosed in double quotes ".
Instead strings.Fields does just that, it splits a string on one or more characters that match unicode.IsSpace, which is \t, \n, \v. \f, \r and .
Moeover this works also if terraform sends an empty set as [], as stated in the documentation:
returning [...] an empty slice if s contains only white space.
...which includes the case of s being empty "" altogether.
In case you need additional control over this, you can use strings.FieldsFunc, which accepts a function of type func(rune) bool so you can determine yourself what constitutes a "space". But since your input string comes from terraform, I guess it's going to be well-behaved enough.
There may be third-party packages that already implement this functionality, but unless your program already imports them, I think the native solution based on the standard lib is always preferrable.
unicode.IsSpace actually includes also the higher runes 0x85 and 0xA0, in which case strings.Fields calls FieldsFunc(s, unicode.IsSpace)
package main
import (
"fmt"
"strings"
)
func main() {
src := "[a b]"
dst := strings.Split(src[1:len(src)-1], " ")
fmt.Println(dst)
}
https://play.golang.org/p/KVY4r_8RWv6

Golang for seems to be skipping some entities

So I have an out string like this:
out :=
"mobile" + "\n" +
"mobile/communicators" + "\n" +
"mobile/communicators/mock" + "\n" +
"mobile/handlers" + "\n" +
"mobile/mocks" + "\n" +
"mobile/models" + "\n" +
"mobile/requests"
Which I'm trying to transform into a slice and then remove the entries which have matching sub-strings in the pkgsToIgnore slice:
pkgs := strings.Split(strings.TrimSpace(string(out)), "\n")
pkgsToIgnore := []string{"mock", "models"}
for i, pkg := range pkgs {
for _, pkgToIgnore := range pkgsToIgnore {
if strings.Contains(pkg, pkgToIgnore){
pkgs = append(pkgs[:i], pkgs[i+1:]...)
}
}
}
for _, pkg := range pkgs {
fmt.Println(pkg)
}
And then this is the result I get from the Println iterations:
mobile
mobile/communicators
mobile/handlers
mobile/models
mobile/requests
My code seems to work fine for the lines containing mock as a sub-string, as they get removed. But I can't understand why mobile/models doesn't.
There are several ways to do this without accidentally skipping elements. One way is simply to copy the elements you want into a temporary slice and then replace the original once you've done everything (it's much harder to make mistakes that way). But if you don't like that, it helps to use two indices:
src, dst := 0, 0
for src < len(pkgs) {
var ignored bool
for _, pkgToIgnore := range pkgsToIgnore {
if strings.Contains(pkg, pkgToIgnore) {
ignored = true
break
}
}
if !ignored {
pkgs[dst] = pkgs[src]
dst += 1
}
src += 1
}
pkgs = pkgs[:dst]
src will range from 0 to the last index of pkgs; dst will always be less than or equal to src (so we never overwrite an element we haven't seen yet); each element is only copied once, to its final destination, instead of shifting all elements left by one each time an element is removed (which is potentially quadratic). The length of the slice is only adjusted at the end, to reflect the number of elements actually retained.
The easiest way to see what is happening is to output i and pkg during each iteration i.e.
for i, pkg := range pkgs {
fmt.Println(i, pkg)
for _, pkgToIgnore := range pkgsToIgnore {
if strings.Contains(pkg, pkgToIgnore) {
pkgs = append(pkgs[:i], pkgs[i+1:]...)
}
}
}
The output of this is:
0 mobile
1 mobile/communicators
2 mobile/communicators/mock
3 mobile/mocks
4 mobile/requests
5 mobile/requests
6 mobile/requests
This is probably not what you were expecting! The reason for that is that you are removing elements from pkgs whilst iterating over it. Lets work through this when i = 2; pkg == "mobile/communicators/mock" so the string will be removed from pkgs. Element 2 of pkgs is now "mobile/handlers". We loop around to the for and move to the next element (3) which is "mobile/mocks" (not "mobile/handler" - that is now element 2 so "mobile/handler" does not get checked).
The relevant section of the go spec may help in understanding this:
For an array, pointer to array, or slice value a, the index iteration
values are produced in increasing order, starting at element index 0.
If at most one iteration variable is present, the range loop produces
iteration values from 0 up to len(a)-1 and does not index into the
array or slice itself. For a nil slice, the number of iterations is 0.
It is possible to update a slice within a loop like this but doing so requires care; generally its simpler to copy the elements you want to keep into a new slice.

Tour of Go Exercise #23: my word counter doesn't work

I'm trying to resolve the puzzle from go tour #23 and I don't understand why my word counter doesn't work. print seems to print the expected value but the tests sees only 1 regardless the count.
package main
import (
"strings"
"unicode/utf8"
"golang.org/x/tour/wc"
)
func WordCount(s string) map[string]int {
// explode the string into a slice without whitespaces
ws := strings.Fields(s)
//make a new map
c := make(map[string]int)
//iterate over each word
for _, v := range ws {
c[v] = utf8.RuneCountInString(v)
}
print(c["am"])
return c
}
func main() {
wc.Test(WordCount)
}
The playground is available here
You're solving the wrong problem. It doesn't ask you for the length of each word, but for the number of times each word occurs. Change
c[v] = utf8.RuneCountInString(v)
for
c[v] += 1 // or c[v]++
The problem is c[v] = utf8.RuneCountInString(v). It has two problems:
You're resetting the counter for each word every time you re-encounter it. You should increment, not set.
You are setting the number of runes in the word to the counter. The puzzle is "how many times a word appears in the text". so just do something like c[v] = c[v] + 1 (if the entry is empty it will default to 0)
Also, I'd normalize the text - strip punctuation marks and lowercase everything.

How to ignore fields with sscanf (%* is rejected)

I wish to ignore a particular field whilst processing a string with sscanf.
Man page for sscanf says
An optional '*' assignment-suppression character: scanf() reads input as directed by the conversion specification, but discards the input. No corresponding pointer argument is required, and this specification is not included in the count of successful assignments returned by scanf().
Attempting to use this in Golang, to ignore the 3rd field:
if c, err := fmt.Sscanf(str, " %s %d %*d %d ", &iface.Name, &iface.BTx, &iface.BytesRx); err != nil || c != 3 {
compiles OK, but at runtime err is set to:
bad verb %* for integer
Golang doco doesn't specifically mention the %* conversion specification, but it does say,
Package fmt implements formatted I/O with functions analogous to C's printf and scanf.
It doesn't indicate that %* is not implemented, so... Am I doing it wrong? Or has it just been quietly omitted? ...but then, why does it compile?
To the best of my knowledge there is no such verb (as the format specifiers are called in the fmt package) for this task. What you can do however, is specifying some verb and ignoring its value. This is not particularly memory friendly, though. Ideally this would work:
fmt.Scan(&a, _, &b)
Sadly, it doesn't. So your next best option would be to declare the variables and ignore the one
you don't want:
var a,b,c int
fmt.Scanf("%d %v %d", &a, &b, &c)
fmt.Println(a,c)
%v would read a space separated token. Depending on what you're scanning on, you may fast forward the
stream to the position you need to scan on. See this answer
for details on seeking in buffers. If you're using stdio or you don't know which length your input may
have, you seem to be out of luck here.
It doesn't indicate that %* is not implemented, so... Am I doing it
wrong? Or has it just been quietly omitted? ...but then, why does it
compile?
It compiles because for the compiler a format string is just a string like any other. The content of that string is evaluated at run time by functions of the fmt package. Some C compilers may check format strings
for correctness, but this is a feature, not the norm. With go, the go vet command will try to warn you about format string errors with mismatched arguments.
Edit:
For the special case of needing to parse a row of integers and just caring for some of them, you
can use fmt.Scan in combination with a slice of integers. The following example reads 3 integers
from stdin and stores them in the slice named vals:
ints := make([]interface{}, 3)
vals := make([]int, len(ints))
for i, _ := range ints {
ints[i] = interface{}(&vals[i])
}
fmt.Scan(ints...)
fmt.Println(vals)
This is probably shorter than the conventional split/trim/strconv chain. It makes a slice of pointers
which each points to a value in vals. fmt.Scan then fills these pointers. With this you can even
ignore most of the values by assigning the same pointer over and over for the values you don't want:
ignored := 0
for i, _ := range ints {
if(i == 0 || i == 2) {
ints[i] = interface{}(&vals[i])
} else {
ints[i] = interface{}(&ignored)
}
}
The example above would assign the address of ignore to all values except the first and the second, thus
effectively ignoring them by overwriting.

Removing NUL characters from bytes

To teach myself Go I'm building a simple server that takes some input, does some processing, and sends output back to the client (that includes the original input).
The input can vary in length from around 5 - 13 characters + endlines and whatever other guff the client sends.
The input is read into a byte array and then converted to a string for some processing. Another string is appended to this string and the whole thing is converted back into a byte array to get sent back to the client.
The problem is that the input is padded with a bunch of NUL characters, and I'm not sure how to get rid of them.
So I could loop through the array and when I come to a nul character, note the length (n), create a new byte array of that length, and copy the first n characters over to the new byte array and use that. Is that the best way, or is there something to make this easier for me?
Some stripped down code:
data := make([]byte, 16)
c.Read(data)
s := strings.Replace(string(data[:]), "an", "", -1)
s = strings.Replace(s, "\r", "", -1)
s += "some other string"
response := []byte(s)
c.Write(response)
c.close()
Also if I'm doing anything else obviously stupid here it would be nice to know.
In package "bytes", func Trim(s []byte, cutset string) []byte is your friend:
Trim returns a subslice of s by slicing off all leading and trailing UTF-8-encoded Unicode code points contained in cutset.
// Remove any NULL characters from 'b'
b = bytes.Trim(b, "\x00")
Your approach sounds basically right. Some remarks:
When you have found the index of the first nul byte in data, you don't need to copy, just truncate the slice: data[:idx].
bytes.Index should be able to find that index for you.
There is also bytes.Replace so you don't need to convert to string.
The io.Reader documentation says:
Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered.
If the call to Read in the application does not read 16 bytes, then data will have trailing zero bytes. Use the number of bytes read to trim the zero bytes from the buffer.
data := make([]byte, 16)
n, err := c.Read(data)
if err != nil {
// handle error
}
data = data[:n]
There's another issue. There's no guarantee that Read slurps up all of the "message" sent by the peer. The application may need to call Read more than once to get the complete message.
You mention endlines in the question. If the message from the client is terminated but a newline, then use bufio.Scanner to read lines from the connection:
s := bufio.NewScanner(c)
if s.Scan() {
data = s.Bytes() // data is next line, not including end lines, etc.
}
if s.Err() != nil {
// handle error
}
You could utilize the return value of Read:
package main
import "strings"
func main() {
r, b := strings.NewReader("north east south west"), make([]byte, 16)
n, e := r.Read(b)
if e != nil {
panic(e)
}
b = b[:n]
println(string(b) == "north east south")
}
https://golang.org/pkg/io#Reader

Resources