we are sending some data over a serial line, and i can do pretty much everything via a bash script (instead of code), except for the crc16 calculation. if i can do it all in scripts versus code, it would make configuration a heckofalot easier (especially while in the field).
i'm alright with commands, but i lose all ability when we get to the tricky stuff.
so my question is, can someone do a rewrite of this CRC16 for me, but within bash?
here is the algorithm grabbed from wikipedia, and it is the one in our code:
uint16_t Encoder::checksum(std::string thestring)
{
uint8_t d, e, f;
uint16_t c, r, crccalc;
c = 0xffff;
for (unsigned int i = 0; i < thestring.length(); i++)
{
d = thestring[i];
e = c ^ d;
f = e ^ (e << 4);
r = (c >> 8) ^ (f << 8) ^ (f << 3) ^ (f >> 4);
c = r;
}
c ^= 0xffff;
crccalc = c;
return crccalc;
}
i can easily create an executable out of the C++ code, and just feed it stdin, but i think it would be really neat to be able to have this within the bash.
the other thing i don't know is how to ensure that my variable sizes are correct. how can i ensure that i am getting a 16 bit integer?
any help would be great. i found a little script online, but i didn't trust it. thought it would be really cool to have answered here.
Bash have:
xor ($((5^2)) will be 7);
left shift ($(3<<2) will be 12);
right shift ($(8>>2) will be 2);
hexademical numbers support ($((0xFF)) will be 255).
Nothing comes to mind to convert from 32 (64) to 16 bit integer in pure Bash but you can do it with awk:
$ echo 65536 | awk '{printf("%hu\n",$1)}'
0
This should be enough to rewrite algorithm in Bash.
Related
I'm computing CRC32 in a rolling fashion on the contents of a file. If the file has 3 blocks ABC, CRC32 is computed linearly CRC(CRC(CRC(A, 0xffffffff), B), C). This is done with code that looks like:
uint32_t crc32(unsigned char const *buf, uint32_t buf_size, uint32_t crc32) {
for (int i = 0; i < buf_size; i++)
crc32 = (crc32 >> 8) ^ table[(crc32 ^ buf[i]) & 0xff];
return crc32;
}
Even though I write entire content ABC at once, computing the CRC as above(which gets verified at the server), read is normally done on a specific block. So, I would like to track CRC32 of each individual block as it is written.
Based on my limited understanding of how CRC32 polynomial works,
A mod G = CRC1
AB mod G = CRC2
If I want CRC32 of B, I'm thinking following should do the trick:
(CRC2 - CRC1) mod G
or
(CRC2 ^ CRC1) mod G
Of course, following code doesn't work.
uint32_t
crc32sw_diff(uint32_t crc1, uint32_t crc2)
{
uint32_t delta = crc1 ^ crc2;
return crc32(&delta, 4, 0xffffffff);
}
Other option is probably to compute CRC32 of individual blocks and combine it with something like zlib's crc32_combine() to get CRC32 of entire file.
See this answer for how CRC combination works. CRC(A) ^ CRC(B) is not equal to CRC(AB). However (for pure CRCs) using the notation that AB is the concatenated message of A followed by B, and 0 meaning an equal length message will all zeros, then CRC(A0) ^ CRC(0B) is equal to CRC(AB).
This also means that CRC(A0) ^ CRC(AB) == CRC(0B). Since CRC(0B) == CRC(B) (feeding zeros doesn't change a pure CRC), you can find it using crc32_combine() from zlib.
So, crc32_combine(crca, crcab, lenb) will return crcb.
I need to swap alternate bytes in a buffer as quickly as possible in an embedded system using ARM Cortex M4 processor. I use gcc. The amount of data is variable but the max is a little over 2K. it doesn't matter if a few extra bytes are converted because I can use an over-sized buffer.
I know that the ARM has the REV16 instruction, which I can use to swap alternate bytes in a 32-bit word. What I don't know is:
Is there a way of getting at this instruction in gcc without resorting to assembler? The __builtin_bswap16 intrinsic appears to operate on 16-bit words only. Converting 4 bytes at a time will surely be faster than converting 2 bytes.
Does the Cortex M4 have a reorder buffer and/or do register renaming? If not, what do I need to do to minimise pipeline stalls when I convert the dwords of the buffer in a partially-unrolled loop?
For example, is this code efficient, where REV16 is appropriately defined to resolve (1):
uint32_t *buf = ... ;
size_t n = ... ; // (number of bytes to convert + 15)/16
for (size_t i = 0; i < n; ++i)
{
uint32_t a = buf[0];
uint32_t b = buf[1];
uint32_t c = buf[2];
uint32_t d = buf[3];
REV16(a, a);
REV16(b, b);
REV16(c, c);
REV16(d, d);
buf[0] = a;
buf[1] = b;
buf[2] = c;
buf[3] = d;
buf += 4;
}
You can't use the __builtin_bswap16 function for the reason you stated, it works on 16 bit words so will 0 the other halfword. I guess the reason for this is to keep the intrinsic working the same on processors which don't have an instruction behaving similarly to REV16 on ARM.
The function
uint32_t swap(uint32_t in)
{
in = __builtin_bswap32(in);
in = (in >> 16) | (in << 16);
return in;
}
compiles to (ARM GCC 5.4.1 -O3 -std=c++11 -march=armv7-m -mtune=cortex-m4 -mthumb)
rev r0, r0
ror r0, r0, #16
bx lr
And you could probably ask the compiler to inline it, which would give you 2 instructions per 32bit word. I can't think of a way to get GCC to generate REV16 with a 32bit operand, without declaring your own function with inline assembly.
EDIT
As a follow up, and based on artless noise's comment about the non portability of the __builtin_bswap functions, the compiler recognizes
uint32_t swap(uint32_t in)
{
in = ((in & 0xff000000) >> 24) | ((in & 0x00FF0000) >> 8) | ((in & 0x0000FF00) << 8) | ((in & 0xFF) << 24);
in = (in >> 16) | (in << 16);
return in;
}
and creates the same 3 instruction function as above, so that is a more portable way to achieve it. Whether different compilers would produce the same output though...
EDIT EDIT
If inline assembler is allowed, the following function
inline uint32_t Rev16(uint32_t a)
{
asm ("rev16 %1,%0"
: "=r" (a)
: "r" (a));
return a;
}
gets inlined, and acts as a single instruction as can be seen here.
I have some c# code that performs some bitwise operations on a byte. I am trying to do the same in golang but am having difficulties.
Example in c#
byte a, c;
byte[] data;
int j;
c = data[j];
c = (byte)(c + j);
c ^= a;
c ^= 0xFF;
c += 0x48;
I have read that golang cannot perform bitwise operations on the byte type. Therefore will I have to modify my code to a type uint8 to perform these operations? If so is there a clean and correct/standard way to implement this?
Go certainly can do bitwise operations on the byte type, which is simply an alias of uint8. The only changes I had to make to your code were:
Syntax of the variable declarations
Convert j to byte before adding it to c, since Go lacks (by design) integer promotion conversions when doing arithmetic.
Removing the semicolons.
Here you go
var a, c byte
var data []byte
var j int
c = data[j]
c = c + byte(j)
c ^= a
c ^= 0xFF
c += 0x48
If you're planning to do bitwise-not in Go, note that the operator for that is ^, not the ~ that is used in most other contemporary programming languages. This is the same operator that is used for xor, but the two are not ambiguous, since the compiler can tell which is which by determining whether the ^ is used as a unary or binary operator.
Originally this post requested an inverse sheep-and-goats operation, but I realized that it was more than I really needed, so I edited the title, because I only need an expand-right algorithm, which is simpler. The example that I described below is still relevant.
Original Post:
I'm trying to figure out how to do either an inverse sheep-and-goats operation or, even better, an expand-right-flip.
According to Hacker's Delight, a sheeps-and-goats operation can be represented by:
SAG(x, m) = compress_left(x, m) | compress(x, ~m)
According to this site, the inverse can be found by:
INV_SAG(x, m, sw) = expand_left(x, ~m, sw) | expand_right(x, m, sw)
However, I can't find any code for the expand_left and expand_right functions. They are, of course, the inverse functions for compress, but compress is kind of hard to understand in itself.
Example:
To better explain what I'm looking for, consider a set of 8 bits like:
0000abcd
The variables a, b, c and d may be either ones or zeros. In addition, there is a mask which repositions the bits. So for example, if the mask were 01100101, the resulting bits would be repositioned as follows:
0ab00c0d
This can be done with an inverse sheeps-and-goats operation. However, according to this section of the site mentioned above, there is a more efficient way which he refers to as the expand-right-flip. Looking at his site, I was unable to figure out how that can be done.
Here's the expand_right from Hacker's Delight, it just says expand but it's the right version.
unsigned expand(unsigned x, unsigned m) {
unsigned m0, mk, mp, mv, t;
unsigned array[5];
int i;
m0 = m; // Save original mask.
mk = ~m << 1; // We will count 0's to right.
for (i = 0; i < 5; i++) {
mp = mk ^ (mk << 1); // Parallel suffix.
mp = mp ^ (mp << 2);
mp = mp ^ (mp << 4);
mp = mp ^ (mp << 8);
mp = mp ^ (mp << 16);
mv = mp & m; // Bits to move.
array[i] = mv;
m = (m ^ mv) | (mv >> (1 << i)); // Compress m.
mk = mk & ~mp;
}
for (i = 4; i >= 0; i--) {
mv = array[i];
t = x << (1 << i);
x = (x & ~mv) | (t & mv);
}
return x & m0; // Clear out extraneous bits.
}
You can use expand_left(x, m) == expand_right(x >> (32 - popcnt(m)), m) to make the left version, but that's probably not the best way.
Not sure if this is possible, but is there an automatic way, using mod or something similiar, to automatically correct bad input values? For example:
If r>255, then set r=255 and
if r<0, then set r=0
So basically what I'm asking is whats a clever mathematical way to set this rather than using
if(r>255)
r=255;
if(r<0)
r=0;
How about:
r = std:max(0, std::min(r, 255));
The following function will output what you are looking for:
f(x) = (510*(1 + Sign[-255 + x]) + x*(1 + Sign[255 - x])*(1 + Sign[x]))/4
As shown here:
Could you do something like --
R = MIN(r, 255);
R = MAX(R, 0);
Depending on how your hardware and possibly how your interpreter deal with ints, you can do this:
Assuming that an unsigned int is 16 bits (to keep my masks short):
r = r & 0000000011111111;
If an int was 32 bits, you'd need 16 more zeros at the start of the bit mask.
After that bitwise AND, the maximum value r can have is 255. Depending on the hardware, an unsigned int might do something odd given a value below zero. I believe that case is already handled by the bitmask (at least on the hardware that I've used). If not, you can do r = min(r, 0); first.
I had similar problem when dealing with images. For some special values (like these ones, 0 and 255) you can use this nonportable method:
static inline int trim_8bit(unsigned i){
return 0xff & ((i | -!!(i & ~0xff))) + (i >> 31);
// where "0xff &" can be omitted if you return unsigned char
};
In real cases the clamping have to be performed rarely, so that you could write
static inline unsigned char trim_8bit_v2(unsigned i){
if (__builtin_expect(i & ~0xFF, 0)) // it's for gcc, use __assume for MSVC
return (i >> 31) - 1;
return i;
};
And to be sure which is fastest, measure.