In the Go Tour exercise #17, I don't understand this expression 1 << uint(i)
package main
import "fmt"
func main() {
pow := make([]int, 10)
for i := range pow {
pow[i] = 1 << uint(i)
}
for _, value := range pow {
fmt.Printf("%d\n", value)
}
}
What is the purpose of this operator ? <<
The program outputs:
1
2
4
8
16
32
64
128
256
512
Program exited.
Its a binary shift operator. Specifically, its a left shift (since they point to the left).
What it does, is move all bits in the binary representation of a number ... left.
For example. The binary representation of 1 is (with a safe assumption of 8 bits per byte): 00000001. Applying a left shift produces:
00000001
<<
00000010
Which is the binary representation of 2. Applying it again produces:
00000010
<<
00000100
..which is the binary representation of 4.. and so on.
Conversely, a right shift >> does the opposite, so applying >> to the value 4, produces:
00000100
>>
00000010
..2.
You can change the tour to count backwards by using the right shift operator:
pow[i] = 512 >> uint(i)
Related
How do you apply a mask to get only one bit after you shift right? Does it depend on how many positions you shifted right?
In a 32 bit structure I'm trying to get the value of the 9th bit and the 10th bit.
x := uint32(11537664)
0000 0000 1011 0000 0000 1101 0000 0000
^^
So for the 9th bit, if I shift right 23 bits I need to mask one byte? That seems to isolate the 9th bit because I'm getting a value of 1.
(x >> 23) & 0xff
9th bit...should be 1... looks ok.
00000000000000000000000000000001
0x1
So to get the 10th bit which should be 0 I am shifting one less bit which does make 0 all the way to the right. But there is a 1 after it which needs to be masked. I figured 1 byte plus 1 bit for the mask but I'm still seeing the the bit in position two so that can't be right.
(x >> 22) & 0x1ff
10th bit... should be 0, but this shift and mask does not look correct.
00000000000000000000000000000010
^ This bit I don't want.
0x2
Link to example:
https://play.golang.org/p/zqofCAAKDZz
package main
import (
"fmt"
)
func bin(i uint32) {
fmt.Printf("%032b\n", i)
}
func hex(i uint32) {
fmt.Printf("0x%x\n", i)
}
func show(i uint32) {
bin(i)
hex(i)
fmt.Println()
}
func main() {
x := uint32(11537664)
fmt.Println("Data")
show(x)
fmt.Println("First 8 bits.")
show(x >> 24)
fmt.Println("9th bit...should be 1")
show((x >> 23) & 0xff)
fmt.Println("10th bit... should be 0")
show((x >> 22) & 0x1ff)
}
After the shift you get a number being 0b10, and you only need the lowest bit. So why are you masking with 0x1ff? That has 9 one bits, that will leave the lowest 9 bits unchanged (unmasked).
Instead mask with 0b01 = 0x01. That only leaves the lowest bit, and zeroes all others:
show((x >> 22) & 0x01)
Try it on the Go Playground.
Also note that if you just want to test if a certain bit is one or zero, you don't neccessarily have to shift. Masking by a proper bitmask that contains a single one at the certain position is enough. You may compare the masking result with zero.
The proper bitmask for testing the nth bit is simply 1<<n (where bits are zero indexed). The 2 bits you want to test are the 22. and 23. bits.
See this example:
x := uint32(11537664)
fmt.Printf("x : %032b\n", x)
fmt.Println()
const mask22 = 1 << 22
fmt.Printf("mask22 : %032b\n", mask22)
fmt.Printf("22. bit: %032b %t\n", x&mask22, x&mask22 != 0)
fmt.Println()
const mask23 = 1 << 23
fmt.Printf("mask23 : %032b\n", mask23)
fmt.Printf("23. bit: %032b %t\n", x&mask23, x&mask23 != 0)
It outputs (try it on the Go Playground):
x : 00000000101100000000110100000000
mask22 : 00000000010000000000000000000000
22. bit: 00000000000000000000000000000000 false
mask23 : 00000000100000000000000000000000
23. bit: 00000000100000000000000000000000 true
type IntSet struct {
words []uint64
}
func (s *IntSet) Has(x int) bool {
word, bit := x/64, uint(x%64)
return word < len(s.words) && s.words[word]&(1<<bit) != 0
}
Lets go through what I think is going on:
A new type is declared called IntSet. Underneath its new type declaration it is unint64 slice.
A method is created called Has(). It can only receive IntSet types, after playing around with ints she returns a bool
Before she can play she needs two ints. She stores these babies on the stack.
Lost for words
This methods purpose is to report whether the set contains the non-negative value x. Here is a the go test:
func TestExample1(t *testing.T) {
//!+main
var x, y IntSet
fmt.Println(x.Has(9), x.Has(123)) // "true false"
//!-main
// Output:
// true false
}
Looking for some guidance understanding what this method is doing inside. And why the programmer did it in such complicated means (I feel like I am missing something).
The return statement:
return word < len(s.words) && s.words[word]&(1<<bit) != 0
Are the order of operations this?
return ( word < len(s.words) && ( s.words[word]&(1<<bit)!= 0 )
And what is the [words] and & doing within:
s.words[word]&(1<<bit)!= 0
edit: Am beginning to see slightly seeing that:
s.words[word]&(1<<bit)!= 0
Is just a slice but don't understand the &
As I read the code, I scribbled some notes:
package main
import "fmt"
// A set of bits
type IntSet struct {
// bits are grouped into 64 bit words
words []uint64
}
// x is the index for a bit
func (s *IntSet) Has(x int) bool {
// The word index for the bit
word := x / 64
// The bit index within a word for the bit
bit := uint(x % 64)
if word < 0 || word >= len(s.words) {
// error: word index out of range
return false
}
// the bit set within the word
mask := uint64(1 << bit)
// true if the bit in the word set
return s.words[word]&mask != 0
}
func main() {
nBits := 2*64 + 42
// round up to whole word
nWords := (nBits + (64 - 1)) / 64
bits := IntSet{words: make([]uint64, nWords)}
// bit 127 = 1 * 64 + 63
bits.words[1] = 1 << 63
fmt.Printf("%b\n", bits.words)
for i := 0; i < nWords*64; i++ {
has := bits.Has(i)
if has {
fmt.Println(i, has)
}
}
has := bits.Has(127)
fmt.Println(has)
}
Playground: https://play.golang.org/p/rxquNZ_23w1
Output:
[0 1000000000000000000000000000000000000000000000000000000000000000 0]
127 true
true
The Go Programming Language Specification
Arithmetic operators
& bitwise AND integers
peterSO's answer is spot on - read it. But I figured this might also help you understand.
Imagine I want to store some random numbers in the range 1 - 8. After I store these numbers I will be asked if the number n (also in the range of 1 - 8) appears in the numbers I recorded earlier. How would we store the numbers?
One, probably obvious, way would be to store them in a slice or maybe a map. Maybe we would choose a map since lookups will be constant time. So we create our map
seen := map[uint8]struct{}{}
Our code might look something like this
type IntSet struct {
seen: map[uint8]struct{}
}
func (i *IntSet) AddValue(v uint8) {
i.seen[v] = struct{}{}
}
func (i *IntSet) Has(v uint8) bool {
_, ok := i.seen[v]
return ok
}
For each number we store we take up (at least) 1 byte (8 bits) of memory. If we were to store all 8 numbers we would be using 64 bits / 8 bytes.
However, as the name implies, this is an int Set. We don't care about duplicates, we only care about membership (which Has provides for us).
But there is another way we could store these numbers, and we could do it all within a single byte. Since a byte provides 8 bits, we can use these 8 bits as markers for values we have seen. The initial value (in binary notation) would be
00000000 == uint8(0)
If we did an AddValue(3) we could change the 3rd bit and end up with
00000100 == uint8(3)
^
|______ 3rd bit
If we then called AddValue(8) we would have
10000100 == uint8(132)
^ ^
| |______ 3rd bit
|___________ 8th bit
So after adding 3 and 8 to our IntSet we have the internally stored integer value of 132. But how do we take 132 and figure out whether a particular bit is set? Easy, we use bitwise operators.
The & operator is a logical AND. It will return the value of the bits common between the numbers on each side of the operator. For example
10001100 01110111 11111111
& 01110100 & 01110000 & 00000001
-------- -------- --------
00000100 01110000 00000001
So to find out if n is in our set we simply do
our_set_value & (1 << (value_we_are_looking_for - 1))
which if we were searching for 4 would yield
10000100
& 00001000
----------
0 <-- so 4 is not present
or if we were searching for 8
10000100
& 10000000
----------
10000000 <-- so 8 is present
You may have noticed I subtracted 1 from our value_we_are_looking for. This is because I am fitting 1-8 into our 8bit number. If we only wanted to store seven numbers then we could just skip using the very first bit and assume our counting starts at bit #2 then we wouldn't have to subtract 1, like the code you posted does.
Assuming you understand all of that, here's where things get interesting. So far we have been storing our values in a uint8 (so we could only have 8 values, or 7 if you omit the first bit). But there are larger numbers that have more bits, like uint64. Instead of 8 values, we can store 64 values! But what happens if the range of values we want to track exceed 1-64? What if we want to store 65? This is where the slice of words comes from in the original code.
Since the code posted skips the first bit, from now on I will do so as well.
We can use the first uint64 to store the numbers 1 - 63. When we want to store the numbers 64-127 we need a new uint64. So our slice would be something like
[ uint64_of_1-63, uint64_of_64-127, uint64_of_128-192, etc]
Now, to answer the question about whether a number is in our set we need to first find the uint64 whose range would contain our number. If we were searching for 110 we would want to use the uint64 located at index 1 (uint64_of_64-128) because 110 would fall in that range.
To find the index of the word we need to look at, we take the whole number value of n / 64. In the case of 110 we would get 1, which is exactly what we want.
Now we need to examine the specific bit of that number. The bit that needs to be checked would be the remainder when dividing 110 by 64, or 46. So if the 46th bit of the word at index 1 is set, then we have seen 110 before.
This is how it might look in code
type IntSet struct {
words []uint64
}
func (s *IntSet) Has(x int) bool {
word, bit := x/64, uint(x%64)
return word < len(s.words) && s.words[word]&(1<<bit) != 0
}
func (s *IntSet) AddValue(x int) {
word := x / 64
bit := x % 64
if word < len(s.words) {
s.words[word] |= (1 << uint64(bit))
}
}
And here is some code to test it
func main() {
rangeUpper := 1000
bits := IntSet{words: make([]uint64, (rangeUpper/64)+1)}
bits.AddValue(127)
bits.AddValue(8)
bits.AddValue(63)
bits.AddValue(64)
bits.AddValue(998)
fmt.Printf("%b\n", bits.words)
for i := 0; i < rangeUpper; i++ {
if ok := bits.Has(i); ok {
fmt.Printf("Found %d\n", i)
}
}
}
OUTPUT
Found 8
Found 63
Found 64
Found 127
Found 998
Playground of above
Note
The |= is another bitwise operator OR. It means combine the two values keeping anywhere there is a 1 in either value
10000000 00000001 00000001
& 01000000 & 10000000 & 00000001
-------- -------- --------
11000000 10000001 00000001 <-- important that we
can set the value
multiple times
Using this method we can reduce the cost of storage for 65535 numbers from 131KB to just 1KB. This type of bit manipulation for set membership is very common in implementations of Bloom Filters
An IntSet represents a Set of integers. The presence in the set of any of a contiguous range of integers can be established by writing a single bit in the IntSet. Likewise, checking whether a specific integer is in the IntSet can be done by checking whether the particular integer corresponding to that bit is set.
So the code is finding the specific uint64 in the Intset corresponding to the integer:
word := x/64
and then the specific bit in that uint64:
bit := uint(x%64)
and then checking first that the integer being tested is in the range supported by the IntSet:
word < len(s.words)
and then whether the specific bit corresponding to the specific integer is set:
&& s.words[word]&(1<<bit) != 0
This part:
s.words[word]
pulls out the specific uint64 of the IntSet that tracks whether the integer in question is in the set.
&
is a bitwise AND.
(1<<bit)
means take a 1, shift it to the bit position representing the specific integer being tested.
Performing the bitwise AND between the integer in question, and the bit-shifted 1 will return a 0 if the bit corresponding to the integer is not set, and a 1 if the bit is set (meaning, the integer in question is a member of the IntSet).
Can someone please explain the left/right shift behaviour in Golang. Please refer the sample code here: https://play.golang.org/p/7vjwCbOEkw
package main
import (
"fmt"
)
func main() {
var lf int8 = -3
fmt.Printf("-3 : %08b\n", lf)
fmt.Printf("<<1: %08b\n", lf<<1)
fmt.Printf("<<2: %08b\n", lf<<2)
fmt.Printf("<<3: %08b\n", lf<<3)
fmt.Printf("<<4: %08b\n", lf<<4)
fmt.Printf("<<5: %08b, %d\n", lf<<5, lf<<5)
fmt.Printf("<<6: %08b, %d\n", lf<<6, lf<<6)
fmt.Printf("<<7: %08b, %d\n", lf<<7, lf<<7)
fmt.Printf("<<8: %08b, %d\n", lf<<8, lf<<8)
fmt.Printf("<<9: %08b, %d\n", lf<<9, lf<<9)
}
-3 : -0000011
<<1: -0000110
<<2: -0001100
<<3: -0011000
<<4: -0110000
<<5: -1100000, -96
<<6: 01000000, 64
<<7: -10000000, -128
<<8: 00000000, 0
<<9: 00000000, 0
-3 is, in two's complement, 11111101 and what you see when the program prints -0000011 is a - and the binary representation of the absolute value of the number. In two's complement, the highest bit is 0 for positive (including zero), and 1 for negative numbers. If you shift this number (11111101) left, the lower 7 bits move one to the left and a 0 comes in from the right, replacing the lowest bit. Shifting as you do in your example will result in:
11111101 -3
11111010 -6
11110100 -12
11101000 -24
11010000 -48
10100000 -96
01000000 64
10000000 -128
00000000 0
00000000 0
...
You just have to consider all the bit patterns as two's complement, once you know how that works, everything will make sense.
I am fiddling around with Go at the moment and have stumpled upon a problem where I want to get some feedback and help :)
My problem is that I have a string containing a hexadecimal value as input, such as this:
"60A100"
Now, I want to convert this to the binary representation of the number and be able to look at specific bits within.
My solution to this right now is:
i, err := strconv.ParseUint(rawHex, 16, 32)
if err != nil {
fmt.Printf("%s", err)
}
// Convert int to binary representation
// %024b indicates base 2, padding with 0, with 24 characters.
bin := fmt.Sprintf("%024b", i)
The variable bin now holds exactly what I want, except it is a string which I don't think is optimal. I would rather that I could have an array of the individual bits such that I could just choose index i to get bit number i :)
Because as far as I know right now, if I lookup index 8 like so; bin[8], I will get a decimal that corresponds to the binary number, in the ASCII table.
I have searched quite a bit, but I can't find a solution that fits perfectly, but maybe I am looking in the wrong spot.
I hope you guys can guide me to the correct / optimal solution in this case :)
Thanks in advance!
You could turn it into a slice representing bits
// This could also return []bool
func asBits(val uint64) []uint64 {
bits := []uint64{}
for i := 0; i < 24; i++ {
bits = append([]uint64{val & 0x1}, bits...)
// or
// bits = append(bits, val & 0x1)
// depending on the order you want
val = val >> 1
}
return bits
}
func main() {
rawHex := "60A100"
i, err := strconv.ParseUint(rawHex, 16, 32)
if err != nil {
fmt.Printf("%s", err)
}
fmt.Printf("%024b\n", i)
fmt.Println(asBits(i))
}
OUTPUT
011000001010000100000000
[0 1 1 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0]
https://play.golang.org/p/KK_AUPgbZu
As #jimb points out, you can also just check an individual bit
fmt.Printf("9th bit is set? %t\n", (i >> 8) & 1 == 1)
which is what #n-carter's answer does.
After parsing the value you can directly access each bit. You can use something like this:
func getNthBit(val, n uint32) int {
n = 32 - n
if 1 << n & val > 0 {
return 1
}
return 0
}
Following #n-carter answer, you can access each bit individually
There are two approaches:
Option 1: Shifting the value:
Shift the bin number to the right n possitions to get the n-th bit the first one. then mask it with 1
func getNthBit(val, n uint32) int {
// 1. reverse the golang endian
nthBit := 32-n
// 2. move the nth bit to the first position
movedVal := val >> nthBit
// 3. mask the value, selecting only this first bit
maskedValue := movedVal & 1
return maskedValue
// can be shortened like so
// return (val >> (32-n)) & 1
}
Explanation:
1. Get the right bit index according to the endian
01100000101000010000000001000101
^
(32-3)=29nth bit
2. Shift the bits to get n-th in the first possition
01100000101000010000000001000101 >> 29
^^^
00000000000000000000000000000011
^^^
3. Mask first bit. This picks(extracts) the value from this bit
00000000000000000000000000000011
& ^
00000000000000000000000000000001
1
Option 2: shifting 1 and masking with it
This can be done the way #n-carter does. Shift a 1 to the left
func getNthBit(val, n uint32) int {
// 1. reverse the golang endian
nthBit := 32-n
// 2. move the mask 1 bit to the nth position
mask := 1 << nthBit
// 3. mask the value, selecting only this nth bit
maskedValue := val & mask
if maskedValue == 0 {
return 0
}
return 1
// can be written shorter like:
//if val & (1 << (32-n)) == 0 {
// return 0
//}
//return 1
}
Explanation:
1. Get the right bit index according to the endian
01100000101000010000000001000101
^
(32-3)=29nth bit
2. Shift the 1 to the n-th position (1 << 29 == 2^(29-1))
00000000000000000000000000000001 << 29
00100000000000000000000000000000
3. Mask n-th bit. This picks(extracts) the value from this bit
01100000101000010000000001000101
&
00100000000000000000000000000000
1
Hope this helps. It takes some time to visualise bit operations in your head.
Could someone please explain to me the usage of << and >> in Go? I guess it is similar to some other languages.
The super (possibly over) simplified definition is just that << is used for "times 2" and >> is for "divided by 2" - and the number after it is how many times.
So n << x is "n times 2, x times". And y >> z is "y divided by 2, z times".
For example, 1 << 5 is "1 times 2, 5 times" or 32. And 32 >> 5 is "32 divided by 2, 5 times" or 1.
From the spec at http://golang.org/doc/go_spec.html, it seems that at least with integers, it's a binary shift. for example, binary 0b00001000 >> 1 would be 0b00000100, and 0b00001000 << 1 would be 0b00010000.
Go apparently doesn't accept the 0b notation for binary integers. I was just using it for the example. In decimal, 8 >> 1 is 4, and 8 << 1 is 16. Shifting left by one is the same as multiplication by 2, and shifting right by one is the same as dividing by two, discarding any remainder.
The << and >> operators are Go Arithmetic Operators.
<< left shift integer << unsigned integer
>> right shift integer >> unsigned integer
The shift operators shift the left
operand by the shift count specified
by the right operand. They implement
arithmetic shifts if the left operand
is a signed integer and logical shifts
if it is an unsigned integer. The
shift count must be an unsigned
integer. There is no upper limit on
the shift count. Shifts behave as if
the left operand is shifted n times by
1 for a shift count of n. As a result,
x << 1 is the same as x*2 and x >> 1
is the same as x/2 but truncated
towards negative infinity.
They are basically Arithmetic operators and its the same in other languages here is a basic PHP , C , Go Example
GO
package main
import (
"fmt"
)
func main() {
var t , i uint
t , i = 1 , 1
for i = 1 ; i < 10 ; i++ {
fmt.Printf("%d << %d = %d \n", t , i , t<<i)
}
fmt.Println()
t = 512
for i = 1 ; i < 10 ; i++ {
fmt.Printf("%d >> %d = %d \n", t , i , t>>i)
}
}
GO Demo
C
#include <stdio.h>
int main()
{
int t = 1 ;
int i = 1 ;
for(i = 1; i < 10; i++) {
printf("%d << %d = %d \n", t, i, t << i);
}
printf("\n");
t = 512;
for(i = 1; i < 10; i++) {
printf("%d >> %d = %d \n", t, i, t >> i);
}
return 0;
}
C Demo
PHP
$t = $i = 1;
for($i = 1; $i < 10; $i++) {
printf("%d << %d = %d \n", $t, $i, $t << $i);
}
print PHP_EOL;
$t = 512;
for($i = 1; $i < 10; $i++) {
printf("%d >> %d = %d \n", $t, $i, $t >> $i);
}
PHP Demo
They would all output
1 << 1 = 2
1 << 2 = 4
1 << 3 = 8
1 << 4 = 16
1 << 5 = 32
1 << 6 = 64
1 << 7 = 128
1 << 8 = 256
1 << 9 = 512
512 >> 1 = 256
512 >> 2 = 128
512 >> 3 = 64
512 >> 4 = 32
512 >> 5 = 16
512 >> 6 = 8
512 >> 7 = 4
512 >> 8 = 2
512 >> 9 = 1
n << x = n * 2^x Example: 3 << 5 = 3 * 2^5 = 96
y >> z = y / 2^z Example: 512 >> 4 = 512 / 2^4 = 32
<< is left shift. >> is sign-extending right shift when the left operand is a signed integer, and is zero-extending right shift when the left operand is an unsigned integer.
To better understand >> think of
var u uint32 = 0x80000000;
var i int32 = -2;
u >> 1; // Is 0x40000000 similar to >>> in Java
i >> 1; // Is -1 similar to >> in Java
So when applied to an unsigned integer, the bits at the left are filled with zero, whereas when applied to a signed integer, the bits at the left are filled with the leftmost bit (which is 1 when the signed integer is negative as per 2's complement).
Go's << and >> are similar to shifts (that is: division or multiplication by a power of 2) in other languages, but because Go is a safer language than C/C++ it does some extra work when the shift count is a number.
Shift instructions in x86 CPUs consider only 5 bits (6 bits on 64-bit x86 CPUs) of the shift count. In languages like C/C++, the shift operator translates into a single CPU instruction.
The following Go code
x := 10
y := uint(1025) // A big shift count
println(x >> y)
println(x << y)
prints
0
0
while a C/C++ program would print
5
20
In decimal math, when we multiply or divide by 10, we effect the zeros on the end of the number.
In binary, 2 has the same effect. So we are adding a zero to the end, or removing the last digit
<< is the bitwise left shift operator ,which shifts the bits of corresponding integer to the left….the rightmost bit being ‘0’ after the shift .
For example:
In gcc we have 4 bytes integer which means 32 bits .
like binary representation of 3 is
00000000 00000000 00000000 00000011
3<<1 would give
00000000 00000000 00000000 00000110 which is 6.
In general 1<<x would give you 2^x
In gcc
1<<20 would give 2^20 that is 1048576
but in tcc it would give you 0 as result because integer is of 2 bytes in tcc.
in simple terms we can take it like this in golang
So
n << x is "n times 2, x times". And y >> z is "y divided by 2, z times".
n << x = n * 2^x Example: 3<< 5 = 3 * 2^5 = 96
y >> z = y / 2^z Example: 512 >> 4 = 512 / 2^4 = 32
These are Right bitwise and left bitwise operators