I'm attempting to recreate a PKCS #5 Padding algorithm I've found written in python.
The main line I'm struggling to recreate is this
return data + (chr(pad_count) * pad_count).encode('utf-8')
which essentially repeats pad_count (an integer, between 1 and 16), as a char, between 1 and 16 times. I'm having trouble getting a similar result in Go.
For example, pad_count of 11 will return the string
\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b
The closeset I've come is this:
b := make([]byte, 2)
binary.LittleEndian.PutUint16(b, uint16(padCount))
fmt.Println("Pad: ", padCount, "Hex: ", hex.EncodeToString(b))
which will return:
Pad: 11 Hex: 0b00
This is pretty close, and obviously I could take a substring, and add the \x myself, but is there a better way to go about this? Also if I substring, I feel there is no guarantee that would work for all the combinations.
As James Henstridge already mentioned the formatting you want (\x0b...) is not something that's required but rather python's representation of non-printable characters. See for yourself:
>>> chr(3)
'\x03'
What you have to do is defined in RFC2898:
[...] where the padding string PS consists of 8-(||M|| mod 8) octets
each with value 8-(||M|| mod 8). The padding string PS will
satisfy one of the following statements:
PS = 01, if ||M|| mod 8 = 7 ;
PS = 02 02, if ||M|| mod 8 = 6 ;
...
PS = 08 08 08 08 08 08 08 08, if ||M|| mod 8 = 0.
This means that you do not need uint16 but uint8 (since an octet has only 8 bits) and you also do not need to format your bytes the way python does. So the only thing you have to do is to use bytes.Repeat:
bytes.Repeat(paddingChar, paddingCount)
func pad(input []byte, pad_count int) []byte {
out := make([]byte, len(input) + int(pad_count))
copy(out, input)
for i := 0; i < pad_count; i++ {
out[len(input) + i] = byte(pad_count)
}
return out
}
Related
A script I am making scans a 5-character code and assigns it a number based on the contents of characters within the code. The code is a randomly-generated number/letter combination. For example 7D3B5 or HH42B where any position can be any one of (26 + 10) characters.
Now, the issue I am having is I would like to figure out the number from 1-(36^5) based on the code. For example:
00000 = 0
00001 = 1
00002 = 2
0000A = 10
0000B = 11
0000Z = 36
00010 = 37
00011 = 38
So on and so forth until the final possible code which is:
ZZZZZ = 60466176 (36^5)
What I need to work out is a formula to figure out, let's say G47DU in its number form, using the examples below.
Something like this?
function getCount(s){
if (!isNaN(s))
return Number(s);
return s.charCodeAt(0) - 55;
}
function f(str){
let result = 0;
for (let i=0; i<str.length; i++)
result += Math.pow(36, str.length - i - 1) * getCount(str[i]);
return result;
}
var strs = [
'00000',
'00001',
'00002',
'0000A',
'0000B',
'0000Z',
'00010',
'00011',
'ZZZZZ'
];
for (str of strs)
console.log(str, f(str));
You are trying to create a base 36 numeric system. Since there are 5 'digits' each digit being 0 to Z, the value can go from 0 to 36^5. (If we are comparing this with hexadecimal system, in hexadecimal each 'digit' goes from 0 to F). Now to convert this to decimal, you could try use the same method used to convert from hex or binary etc... system to the decimal system.
It will be something like d4 * (36 ^ 4) + d3 * (36 ^ 3) + d2 * (36 ^ 2) + d1 * (36 ^ 1) + d0 * (36 ^ 0)
Note: Here 36 is the total number of symbols.
d0, d1, d2, d3, d4 can range from 0 to 35 in decimal (Important: Not 0 to 36).
Also, you can extend this for any number of digits or symbols and you can implement operations like addition, subtraction etc in this system itself as well. (It will be fun to implement that. :) ) But it will be easier to convert it to decimal do the operations and convert it back though.
When doing bitwise not, get a lot ffffffff. How to do correctly?
space := " "
str := "12345678999298765432179.170.184.81"
sp := len(str) % 4
if sp > 0 {
str = str + space[0:4-sp]
}
fmt.Println(str, len(str))
hx := hex.EncodeToString([]byte(str))
ln := len(hx)
a, _ := strconv.ParseUint(hx[0:8], 16, 0)
for i := 8; i < ln; i += 8 {
b, _ := strconv.ParseUint(hx[i:i+8], 16, 0)
a = a ^ b
}
xh := strconv.FormatUint(^a, 16)
fmt.Println(xh)
output
ffffffffc7c7dbcb
I need only
c7c7dbcb
You get a lot of leading ff because your a number in fact is only 32-bit "large" but is used "within" a 64-bit uint64 value. (You're processing numbers with 8 hex digits = 4 bytes data = 32 bit.) It has 4 leading 0 bytes, which when negated will turn into ff. You can verify this with:
fmt.Printf("a %#x\n",a)
Outputs:
a 0x38382434
To get rid of those leading ff, convert the result to uint32:
xh := strconv.FormatUint(uint64(uint32(^a)), 16)
fmt.Println(xh)
(Converting back to uint64 is because strconv.FormatUint() expects / requires uint64.)
This outputs:
c7c7dbcb
Another option is to apply a 0xffffffff bitmask:
xh = strconv.FormatUint(^a&0xffffffff, 16)
fmt.Println(xh)
Also note that you could print it using fmt.Printf() (or fmt.Sprintf() if you need it as a string) where you specify %08x verb which also adds leading zeros should the input has more than 3 leading 0 bits (and thus strconv.FormatUint() would not add leading hex zeros):
fmt.Printf("%08x", uint32(^a))
This outputs the same. Try the examples on the Go Playground.
type IntSet struct {
words []uint64
}
func (s *IntSet) Has(x int) bool {
word, bit := x/64, uint(x%64)
return word < len(s.words) && s.words[word]&(1<<bit) != 0
}
Lets go through what I think is going on:
A new type is declared called IntSet. Underneath its new type declaration it is unint64 slice.
A method is created called Has(). It can only receive IntSet types, after playing around with ints she returns a bool
Before she can play she needs two ints. She stores these babies on the stack.
Lost for words
This methods purpose is to report whether the set contains the non-negative value x. Here is a the go test:
func TestExample1(t *testing.T) {
//!+main
var x, y IntSet
fmt.Println(x.Has(9), x.Has(123)) // "true false"
//!-main
// Output:
// true false
}
Looking for some guidance understanding what this method is doing inside. And why the programmer did it in such complicated means (I feel like I am missing something).
The return statement:
return word < len(s.words) && s.words[word]&(1<<bit) != 0
Are the order of operations this?
return ( word < len(s.words) && ( s.words[word]&(1<<bit)!= 0 )
And what is the [words] and & doing within:
s.words[word]&(1<<bit)!= 0
edit: Am beginning to see slightly seeing that:
s.words[word]&(1<<bit)!= 0
Is just a slice but don't understand the &
As I read the code, I scribbled some notes:
package main
import "fmt"
// A set of bits
type IntSet struct {
// bits are grouped into 64 bit words
words []uint64
}
// x is the index for a bit
func (s *IntSet) Has(x int) bool {
// The word index for the bit
word := x / 64
// The bit index within a word for the bit
bit := uint(x % 64)
if word < 0 || word >= len(s.words) {
// error: word index out of range
return false
}
// the bit set within the word
mask := uint64(1 << bit)
// true if the bit in the word set
return s.words[word]&mask != 0
}
func main() {
nBits := 2*64 + 42
// round up to whole word
nWords := (nBits + (64 - 1)) / 64
bits := IntSet{words: make([]uint64, nWords)}
// bit 127 = 1 * 64 + 63
bits.words[1] = 1 << 63
fmt.Printf("%b\n", bits.words)
for i := 0; i < nWords*64; i++ {
has := bits.Has(i)
if has {
fmt.Println(i, has)
}
}
has := bits.Has(127)
fmt.Println(has)
}
Playground: https://play.golang.org/p/rxquNZ_23w1
Output:
[0 1000000000000000000000000000000000000000000000000000000000000000 0]
127 true
true
The Go Programming Language Specification
Arithmetic operators
& bitwise AND integers
peterSO's answer is spot on - read it. But I figured this might also help you understand.
Imagine I want to store some random numbers in the range 1 - 8. After I store these numbers I will be asked if the number n (also in the range of 1 - 8) appears in the numbers I recorded earlier. How would we store the numbers?
One, probably obvious, way would be to store them in a slice or maybe a map. Maybe we would choose a map since lookups will be constant time. So we create our map
seen := map[uint8]struct{}{}
Our code might look something like this
type IntSet struct {
seen: map[uint8]struct{}
}
func (i *IntSet) AddValue(v uint8) {
i.seen[v] = struct{}{}
}
func (i *IntSet) Has(v uint8) bool {
_, ok := i.seen[v]
return ok
}
For each number we store we take up (at least) 1 byte (8 bits) of memory. If we were to store all 8 numbers we would be using 64 bits / 8 bytes.
However, as the name implies, this is an int Set. We don't care about duplicates, we only care about membership (which Has provides for us).
But there is another way we could store these numbers, and we could do it all within a single byte. Since a byte provides 8 bits, we can use these 8 bits as markers for values we have seen. The initial value (in binary notation) would be
00000000 == uint8(0)
If we did an AddValue(3) we could change the 3rd bit and end up with
00000100 == uint8(3)
^
|______ 3rd bit
If we then called AddValue(8) we would have
10000100 == uint8(132)
^ ^
| |______ 3rd bit
|___________ 8th bit
So after adding 3 and 8 to our IntSet we have the internally stored integer value of 132. But how do we take 132 and figure out whether a particular bit is set? Easy, we use bitwise operators.
The & operator is a logical AND. It will return the value of the bits common between the numbers on each side of the operator. For example
10001100 01110111 11111111
& 01110100 & 01110000 & 00000001
-------- -------- --------
00000100 01110000 00000001
So to find out if n is in our set we simply do
our_set_value & (1 << (value_we_are_looking_for - 1))
which if we were searching for 4 would yield
10000100
& 00001000
----------
0 <-- so 4 is not present
or if we were searching for 8
10000100
& 10000000
----------
10000000 <-- so 8 is present
You may have noticed I subtracted 1 from our value_we_are_looking for. This is because I am fitting 1-8 into our 8bit number. If we only wanted to store seven numbers then we could just skip using the very first bit and assume our counting starts at bit #2 then we wouldn't have to subtract 1, like the code you posted does.
Assuming you understand all of that, here's where things get interesting. So far we have been storing our values in a uint8 (so we could only have 8 values, or 7 if you omit the first bit). But there are larger numbers that have more bits, like uint64. Instead of 8 values, we can store 64 values! But what happens if the range of values we want to track exceed 1-64? What if we want to store 65? This is where the slice of words comes from in the original code.
Since the code posted skips the first bit, from now on I will do so as well.
We can use the first uint64 to store the numbers 1 - 63. When we want to store the numbers 64-127 we need a new uint64. So our slice would be something like
[ uint64_of_1-63, uint64_of_64-127, uint64_of_128-192, etc]
Now, to answer the question about whether a number is in our set we need to first find the uint64 whose range would contain our number. If we were searching for 110 we would want to use the uint64 located at index 1 (uint64_of_64-128) because 110 would fall in that range.
To find the index of the word we need to look at, we take the whole number value of n / 64. In the case of 110 we would get 1, which is exactly what we want.
Now we need to examine the specific bit of that number. The bit that needs to be checked would be the remainder when dividing 110 by 64, or 46. So if the 46th bit of the word at index 1 is set, then we have seen 110 before.
This is how it might look in code
type IntSet struct {
words []uint64
}
func (s *IntSet) Has(x int) bool {
word, bit := x/64, uint(x%64)
return word < len(s.words) && s.words[word]&(1<<bit) != 0
}
func (s *IntSet) AddValue(x int) {
word := x / 64
bit := x % 64
if word < len(s.words) {
s.words[word] |= (1 << uint64(bit))
}
}
And here is some code to test it
func main() {
rangeUpper := 1000
bits := IntSet{words: make([]uint64, (rangeUpper/64)+1)}
bits.AddValue(127)
bits.AddValue(8)
bits.AddValue(63)
bits.AddValue(64)
bits.AddValue(998)
fmt.Printf("%b\n", bits.words)
for i := 0; i < rangeUpper; i++ {
if ok := bits.Has(i); ok {
fmt.Printf("Found %d\n", i)
}
}
}
OUTPUT
Found 8
Found 63
Found 64
Found 127
Found 998
Playground of above
Note
The |= is another bitwise operator OR. It means combine the two values keeping anywhere there is a 1 in either value
10000000 00000001 00000001
& 01000000 & 10000000 & 00000001
-------- -------- --------
11000000 10000001 00000001 <-- important that we
can set the value
multiple times
Using this method we can reduce the cost of storage for 65535 numbers from 131KB to just 1KB. This type of bit manipulation for set membership is very common in implementations of Bloom Filters
An IntSet represents a Set of integers. The presence in the set of any of a contiguous range of integers can be established by writing a single bit in the IntSet. Likewise, checking whether a specific integer is in the IntSet can be done by checking whether the particular integer corresponding to that bit is set.
So the code is finding the specific uint64 in the Intset corresponding to the integer:
word := x/64
and then the specific bit in that uint64:
bit := uint(x%64)
and then checking first that the integer being tested is in the range supported by the IntSet:
word < len(s.words)
and then whether the specific bit corresponding to the specific integer is set:
&& s.words[word]&(1<<bit) != 0
This part:
s.words[word]
pulls out the specific uint64 of the IntSet that tracks whether the integer in question is in the set.
&
is a bitwise AND.
(1<<bit)
means take a 1, shift it to the bit position representing the specific integer being tested.
Performing the bitwise AND between the integer in question, and the bit-shifted 1 will return a 0 if the bit corresponding to the integer is not set, and a 1 if the bit is set (meaning, the integer in question is a member of the IntSet).
How should I do base 10 to base 16 integer conversion in Squirrel? In Javascript I can do this with: parseInt("ff", 16).
I'm trying to do a HEX color code to RGB calculator for an Electric Imp. #ffaaccwould be split into 3 parts (ff, aa and cc). I would then calculate these to base 10 integers and achieve RGB(255, 170, 204). These numbers I will then use to control an RGB led with PWM.
Try String tointeger() function.
local s = "ff";
print (s.tointeger(16));
If you want convert conversely, try format() function.
local i = 255;
print (format("%x", i));
Here's one approach using array.find (and format for reversal):
local lookup = ['0','1','2','3','4','5','6','7','8','9',
'a','b','c','d','e','f']
local hex = "7f"
local dec = lookup.find(hex[0]) * 0x10 + lookup.find(hex[1])
server.log(format("%s -> %d -> %02x", hex, dec, dec))
Example :
a = 1
b = 2
c = 3
..
..
z = 26
aa = 27
ab = 28
how to convert another string into an integer? for example i want to convert 'lmao' to an integer. please help me :) thank you.
in pascal :)
To convert ordinary base-10 strings into numbers, you take each character from left to right, convert it to its numeric value (between 0 and 9) and add it to the total you already have (which you initialize to zero). If there are more characters following the one you just processed, then multiply the total by 10. Repeat until you run out of characters.
For example, the number 374 is 3×102 + 7×101 + 4×100. Another way of writing that, which more closely models the conversion algorithm I described above, is (((3)×10+7)×10+4.
You can adapt that to handle any string of characters, not just numeric characters. Instead of 10, the base is 26, so multiply by that. And instead of digits, the characters are a through z. Your example string would be evaluated like this: (((l)×26+m)×26+a)×26+o. Substitute numbers for those letters, and you get 219,742.
Here's some code to do it. It doesn't check for errors; it assumes that the string will only contain valid characters and that the string won't represent a number that's too big to fit in an Integer variable.
function SpecialStrToInt(const s: string): Integer;
var
i: Integer;
subtotal: Integer;
c: Char;
charval: Integer;
begin
subtotal := 0;
for i := 1 to Length(s) do begin
c := s[i];
charval := Ord(c) - Ord('a') + 1;
subtotal := subtotal * 26;
subtotal := subtotal + charval;
end;
SpecialStrToInt := subtotal;
end;
An oddity about your format is that there's no way to represent zero.