I recently saw the following code in a Golang markdown parser:
blankLines := make([]lineStat, 0, 128)
isBlank := false
for { // process blocks separated by blank lines
_, lines, ok := reader.SkipBlankLines()
if !ok {
return
}
lineNum, _ := reader.Position()
if lines != 0 {
blankLines = blankLines[0:0]
l := len(pc.OpenedBlocks())
for i := 0; i < l; i++ {
blankLines = append(blankLines, lineStat{lineNum - 1, i, lines != 0})
}
}
I'm confused as to what blankLines = blankLines[0:0] does. Is this a way to prepend to an array?
This slicing [0:0] creates a slice that has the same backing array, but zero length. All it's really doing is "resetting" the len on the slice so that the underlying array can be re-used. It avoids the allocation that may be required if a completely new slice was created for each iteration.
Related
I am working on some coding exercises to better understand Go. A given exercise instructs me to create a program that will accept user input as follows:
The first line specifies how many strings will be provided as input on separate lines
The subsequent N lines will each be single strings
I am to output the characters corresponding to even and odd indices of each string separated by a space, and each string on it's separate line.
Example Input:
2
foo_bar
fizz_buzz
Should Output:
fobr o_a
fz_uz izbz
But in my program accessing a slice of strings returns an empty string:
package main
import (
"fmt"
)
func main() {
// read an integer describing how many strings will be input
var num_strings int
fmt.Scan(&num_strings)
// create a slice of strings to hold provided strings
strings := make([]string, num_strings)
// add provided strings to slice
for i := 0; i < num_strings; i++ {
var temp string
fmt.Scan(&temp)
strings = append(strings, temp)
}
// check that strings have been appended
fmt.Println("Strings:", strings)
// check that strings can be accessed
for i := 0; i < num_strings; i++ {
fmt.Println(i, strings[i]) // only i prints, not strings[i]
}
// loop over all strings
for i := 0; i < num_strings; i++ {
// if string index is even print the char
for index, val := range strings[i] {
if index%2 == 0 {
fmt.Print(val)
}
}
fmt.Print(" ")
// if string index is odd print the char
for index, val := range strings[i] {
if index%2 != 0 {
fmt.Print(val)
}
}
// newline for next string
fmt.Print("\n")
}
}
2
foo_bar
fizz_buzz
Strings: [ foo_bar fizz_buzz]
0
1
Because when you make your strings slice, you're creating a slice with both a capacity and length of n. So when you append to it, you're increasing the length of the slice:
Change this bit of code:
// create a slice of strings to hold provided strings
strings := make([]string, num_strings)
// add provided strings to slice
for i := 0; i < num_strings; i++ {
var temp string
fmt.Scan(&temp)
strings = append(strings, temp)
}
to either:
// create a slice of strings to hold provided strings
strings := []{}
// add provided strings to slice
for i := 0; i < num_strings; i++ {
var temp string
fmt.Scan(&temp)
strings = append(strings, temp)
}
Or
// create a slice of strings to hold provided strings
strings := make([]string, num_strings)
// add provided strings to slice
for i := 0; i < num_strings; i++ {
var temp string
fmt.Scan(&temp)
strings[i] = temp
}
And you should be good.
I catch Consider preallocating [to] (prealloc) this problen in golangci-lint
my code is:
var to []string
for _, t := range s.To {
to = append(to, t.String())
}
Do you have an idea to resolve this problem in lint?
Preallocate a slice with capacity so append() will have less (or no) copying to do:
to := make([]string, 0, len(s.To))
for _, t := range s.To {
to = append(to, t.String())
}
Or even better, don't use append() but assign to individual slice elements:
to := make([]string, len(s.To))
for i, t := range s.To {
to[i] = t.String()
}
I'm trying to delete multiple values from my map[string][]interface{}
I am using the strings.Split function to separate each value i wish to delete, and then looping through them.
I have managed to get it so i can delete index values 0 and 1, however, 1,2 would delete index value 1, but error on index 2.
I have also managed to get it to delete a single value
My thought process was that if I can get it to delete just one value (any index i enter, inc first and last index), then i could use a loop to loop through, and delete the rest.
Everything is stored in the below:
package db
var DataStore map[string][]interface{}
The function
func HandleDelete(w http.ResponseWriter, k, v string) {
It takes the value you wish to delete in as a parameter (and the key, but that's fully functional)
The block of code the issue resides in
The loop starts at the end of the map slice, so when you remove index value 5 for example, 4 is still 4. Whereas if I go the other way, if i delete 5, 6 then becomes index 5. So 5,6 being deleted would effectively mean 5,7 is being deleted.
for i := len(db.DataStore) - 1; i >= 0; i-- {
for _, idxvalue := range values {
val, err := strconv.Atoi(idxvalue)
if err != nil {
log.Fatal(err)
return
}
dbval := db.DataStore[k][val]
if i == val {
if len(db.DataStore[k])-1 == i { //the length goes 1,2,3,4... the index goes 0,1,2,3 - therefore length -1, would be 3 - deletes the last index value
db.DataStore[k] = db.DataStore[k][:i]
} else { //delete everything else
db.DataStore[k] = append(db.DataStore[k][:i], db.DataStore[k][i+1:]...)
}
//when you delete the last value in that key, delete the key.
/*if len(db.DataStore[k]) == 0 {
delete(db.DataStore, k)
}*/
fmt.Fprintf(w, "Key: %v, Value: %v was deleted successfully", k, dbval)
}
}
}
I have tried both loops as below:
for i := len(db.DataStore) - 1; i >= 0; i-- {
Of course the reason the below didn't work, is because you're getting the length, before the loop (in the func body) which won't change after each iteration.
idx := len(db.DataStore) - 1
for i := idx; i >= 0; i-- {
The below code is to delete the index entered (this works with a single value)
if len(db.DataStore[k])-1 == i { //the length goes 1,2,3,4... the index goes 0,1,2,3 - therefore length -1, would be 3 - deletes the last index value
db.DataStore[k] = db.DataStore[k][:i]
} else { //delete everything else
db.DataStore[k] = append(db.DataStore[k][:i], db.DataStore[k][i+1:]...)
}
I expect the out put of '2,1' to delete index 1 and 2, but the actual input is that it just deletes index 1.
For example,
package main
import "fmt"
// Delete m k v elements indexed by d.
func deleteMKVD(m map[string][]interface{}, k string, d []int) {
v, ok := m[k]
if !ok {
return
}
for _, i := range d {
if 0 <= i && i < len(v) {
v[i] = nil
}
}
lw := 0
for i := range v {
if v[i] != nil {
lw++
}
}
if lw == 0 {
delete(m, k)
return
}
w := make([]interface{}, 0, lw)
for i := range v {
if v[i] != nil {
w = append(w, v[i])
}
}
m[k] = w
}
func main() {
m := map[string][]interface{}{
"k0": {"v0", "v1", "v2", "v3"},
}
fmt.Println(m)
deleteMKVD(m, "k0", []int{0, 3})
fmt.Println(m)
deleteMKVD(m, "k0", []int{1, 0})
fmt.Println(m)
}
Playground: https://play.golang.org/p/biEAxthTaj8
Output:
map[k0:[v0 v1 v2 v3]]
map[k0:[v1 v2]]
map[]
I think your problem is actually to remove elements from an array with an array of indices.
The easy fix here would be:
1) Find all the indices with certain k, make it an array(vals []int).
2) Sort this array int descendent. and iterate this array to delete
3) Then iterate this array to delete the elements.
In this way, every time you delete an element, it won't touch other elements' indices.
It may not be most efficient, but it would be a quick fix.
BTW, I think for i := len(db.DataStore) - 1; i >= 0; i--is not what you want.
If I understand correctly, this code here seems to make sure the val is the largest index in those indices.
So instead of write i:=len(db.DataStore) - 1, you actually need i:=len(db.DataStore[k])-1
I have a bytes.Buffer type variable which I filled with Unicode characters:
var mbuff bytes.Buffer
unicodeSource := 'کیا حال ھے؟'
for i,r := range(unicodeSource) {
mbuff.WriteRune(r)
}
Note: I iterated over a Unicode literals here, but really the source is an infinite loop of user input characters.
Now, I want to remove a Unicode character from any position in the buffer mbuff. The problem is that characters may be of variable byte sizes. So I cannot just pick out the ith byte from mbuff.String() as it might be the beginning, middle, or end of a character. This is my trivial (and horrendous) solution:
// removing Unicode character at position n
var tempString string
currChar := 0
for _, ch := range(mbuff.String()) { // iterate over Unicode chars
if currChar != n { // skip concatenating nth char
tempString += ch
}
currChar++
}
mbuff.Reset() // empty buffer
mbuff.WriteString(tempString) // write new string
This is bad in many ways. For one, I convert buffer to string, remove ith element, and write a new string back into the buffer. Too many operations. Second, I use the += operator in the loop to concatenate Unicode characters into a new string. I am using buffers in the first place exactly to avoid concatenation using += which is slow as this answer points out.
What is an efficient method to remove the ith Unicode character in a bytes.Buffer?
Also what is an efficient way to insert a Unicode character after i-1 Unicode characters (i.e. in the ith place)?
To remove the ith rune from a slice of bytes, loop through the slice counting runes. When the ith rune is found, copy the bytes following the rune down to the position of the ith rune:
func removeAtBytes(p []byte, i int) []byte {
j := 0
k := 0
for k < len(p) {
_, n := utf8.DecodeRune(p[k:])
if i == j {
p = p[:k+copy(p[k:], p[k+n:])]
}
j++
k += n
}
return p
}
This function modifies the backing array of the argument slice, but it does not allocate memory.
Use this function to remove a rune from a bytes.Buffer.
p := removeAtBytes(mbuf.Bytes(), i)
mbuf.Truncate(len(p)) // backing bytes were updated, adjust length
playground example
To remove the ith rune from a string, loop through the string counting runes. When the ith rune is found, create a string by concatenating the segment of the string before the rune with the segment of the string after the rune.
func removeAt(s string, i int) string {
j := 0 // count of runes
k := 0 // index in string of current rune
for k < len(s) {
_, n := utf8.DecodeRuneInString(s[k:])
if i == j {
return s[:k] + s[k+n:]
}
j++
k += n
}
return s
}
This function allocates a single string, the result. DecodeRuneInString is a function in the standard library unicode/utf8 package.
Taking a step back, go often works on Readers and Writers, so an alternative solution would be to use the text/transform package. You create a Transformer, attach it to a Reader and use the new Reader to produce a transformed string. For example here's a skipper:
func main() {
src := strings.NewReader("کیا حال ھے؟")
skipped := transform.NewReader(src, NewSkipper(5))
var buf bytes.Buffer
io.Copy(&buf, skipped)
fmt.Println("RESULT:", buf.String())
}
And here's the implementation:
package main
import (
"bytes"
"fmt"
"io"
"strings"
"unicode/utf8"
"golang.org/x/text/transform"
)
type skipper struct {
pos int
cnt int
}
// NewSkipper creates a text transformer which will remove the rune at pos
func NewSkipper(pos int) transform.Transformer {
return &skipper{pos: pos}
}
func (s *skipper) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for utf8.FullRune(src) {
_, sz := utf8.DecodeRune(src)
// not enough space in the dst
if len(dst) < sz {
return nDst, nSrc, transform.ErrShortDst
}
if s.pos != s.cnt {
copy(dst[:sz], src[:sz])
// track that we stored in dst
dst = dst[sz:]
nDst += sz
}
// track that we read from src
src = src[sz:]
nSrc += sz
// on to the next rune
s.cnt++
}
if len(src) > 0 && !atEOF {
return nDst, nSrc, transform.ErrShortSrc
}
return nDst, nSrc, nil
}
func (s *skipper) Reset() {
s.cnt = 0
}
There may be bugs with this code, but hopefully you can see the idea.
The benefit of this approach is it could work on a potentially infinite amount of data without having to store all of it in memory. For example you could transform a file this way.
Edit:
Remove the ith rune in the buffer:
A: Shift all runes one location to the left (Here A is faster than B), try it on The Go Playground:
func removeRuneAt(s string, runePosition int) string {
if runePosition < 0 {
return s
}
r := []rune(s)
if runePosition >= len(r) {
return s
}
copy(r[runePosition:], r[runePosition+1:])
return string(r[:len(r)-1])
}
B: Copy to new buffer, try it on The Go Playground
func removeRuneAt(s string, runePosition int) string {
if runePosition < 0 {
return s // avoid allocation
}
r := []rune(s)
if runePosition >= len(r) {
return s // avoid allocation
}
t := make([]rune, len(r)-1) // Apply replacements to buffer.
w := copy(t, r[:runePosition])
w += copy(t[w:], r[runePosition+1:])
return string(t[:w])
}
C: Try it on The Go Playground:
package main
import (
"bytes"
"fmt"
)
func main() {
str := "hello"
fmt.Println(str)
fmt.Println(removeRuneAt(str, 1))
buf := bytes.NewBuffer([]byte(str))
fmt.Println(buf.Bytes())
buf = bytes.NewBuffer([]byte(removeRuneAt(buf.String(), 1)))
fmt.Println(buf.Bytes())
}
func removeRuneAt(s string, runePosition int) string {
if runePosition < 0 {
return s // avoid allocation
}
r := []rune(s)
if runePosition >= len(r) {
return s // avoid allocation
}
t := make([]rune, len(r)-1) // Apply replacements to buffer.
w := copy(t, r[0:runePosition])
w += copy(t[w:], r[runePosition+1:])
return string(t[0:w])
}
D: Benchmark:
A: 745.0426ms
B: 1.0160581s
for 2000000 iterations
1- Short Answer: to replace all (n) instances of a character (or even a string):
n := -1
newR := ""
old := "µ"
buf = bytes.NewBuffer([]byte(strings.Replace(buf.String(), old, newR, n)))
2- For replacing the character(string) in the ith instance in the buffer, you may use:
buf = bytes.NewBuffer([]byte(Replace(buf.String(), oldString, newOrEmptyString, ith)))
See:
// Replace returns a copy of the string s with the ith
// non-overlapping instance of old replaced by new.
func Replace(s, old, new string, ith int) string {
if len(old) == 0 || old == new || ith < 0 {
return s // avoid allocation
}
i, j := 0, 0
for ; ith >= 0; ith-- {
j = strings.Index(s[i:], old)
if j < 0 {
return s // avoid allocation
}
j += i
i = j + len(old)
}
t := make([]byte, len(s)+(len(new)-len(old))) // Apply replacements to buffer.
w := copy(t, s[0:j])
w += copy(t[w:], new)
w += copy(t[w:], s[j+len(old):])
return string(t[0:w])
}
Try it on The Go Playground:
package main
import (
"bytes"
"fmt"
"strings"
)
func main() {
str := `How are you?µ`
fmt.Println(str)
fmt.Println(Replace(str, "µ", "", 0))
buf := bytes.NewBuffer([]byte(str))
fmt.Println(buf.Bytes())
buf = bytes.NewBuffer([]byte(Replace(buf.String(), "µ", "", 0)))
fmt.Println(buf.Bytes())
}
func Replace(s, old, new string, ith int) string {
if len(old) == 0 || old == new || ith < 0 {
return s // avoid allocation
}
i, j := 0, 0
for ; ith >= 0; ith-- {
j = strings.Index(s[i:], old)
if j < 0 {
return s // avoid allocation
}
j += i
i = j + len(old)
}
t := make([]byte, len(s)+(len(new)-len(old))) // Apply replacements to buffer.
w := copy(t, s[0:j])
w += copy(t[w:], new)
w += copy(t[w:], s[j+len(old):])
return string(t[0:w])
}
3- If you want to remove all instances of Unicode character (old string) from any position in the string, you may use:
strings.Replace(str, old, "", -1)
4- Also this works fine for removing from bytes.buffer:
strings.Replace(buf.String(), old, newR, -1)
Like so:
buf = bytes.NewBuffer([]byte(strings.Replace(buf.String(), old, newR, -1)))
Here is the complete working code (try it on The Go Playground):
package main
import (
"bytes"
"fmt"
"strings"
)
func main() {
str := `کیا حال ھے؟` //How are you?
old := `ک`
newR := ""
fmt.Println(strings.Replace(str, old, newR, -1))
buf := bytes.NewBuffer([]byte(str))
// for _, r := range str {
// buf.WriteRune(r)
// }
fmt.Println(buf.Bytes())
bs := []byte(strings.Replace(buf.String(), old, newR, -1))
buf = bytes.NewBuffer(bs)
fmt.Println(" ", buf.Bytes())
}
output:
یا حال ھے؟
[218 169 219 140 216 167 32 216 173 216 167 217 132 32 218 190 219 146 216 159]
[219 140 216 167 32 216 173 216 167 217 132 32 218 190 219 146 216 159]
5- strings.Replace is very efficient, see inside:
// Replace returns a copy of the string s with the first n
// non-overlapping instances of old replaced by new.
// If old is empty, it matches at the beginning of the string
// and after each UTF-8 sequence, yielding up to k+1 replacements
// for a k-rune string.
// If n < 0, there is no limit on the number of replacements.
func Replace(s, old, new string, n int) string {
if old == new || n == 0 {
return s // avoid allocation
}
// Compute number of replacements.
if m := Count(s, old); m == 0 {
return s // avoid allocation
} else if n < 0 || m < n {
n = m
}
// Apply replacements to buffer.
t := make([]byte, len(s)+n*(len(new)-len(old)))
w := 0
start := 0
for i := 0; i < n; i++ {
j := start
if len(old) == 0 {
if i > 0 {
_, wid := utf8.DecodeRuneInString(s[start:])
j += wid
}
} else {
j += Index(s[start:], old)
}
w += copy(t[w:], s[start:j])
w += copy(t[w:], new)
start = j + len(old)
}
w += copy(t[w:], s[start:])
return string(t[0:w])
}
I have no idea why this code always slice bound out of range:
parts := make([]string, 0, len(encodedCode)/4)
for i := 0; i < len(encodedCode); i += 4 {
parts = append(parts, encodedCode[i:4])
}
encodedCode is string with length always multiply with 4. That mean encodedCode[i:4] never out of bound.
Slices are [idx_start:idx_end+1], not [idx_start:length]
Try this.
parts := make([]string, 0, len(encodedCode)/4)
for i := 0; i < len(encodedCode); i += 4 {
parts = append(parts, encodedCode[i:i+4])
}
Good examples # http://blog.golang.org/go-slices-usage-and-internals