Offset in function in disassembler output - debugging

Cannot find (or formulate a question to Google to find) an answer on the simple (or noob) question.
I'm inspecting an application with objdump -d tool:
. . .
5212c0: 73 2e jae 5212f0 <rfb::SMsgReaderV3::readSetDesktopSize()+0x130>
5213e8: 73 2e jae 521418 <rfb::SMsgReaderV3::readSetDesktopSize()+0x258>
521462: 73 2c jae 521490 <rfb::SMsgReaderV3::readSetDesktopSize()+0x2d0>
. . .
What does it mean the +XXXX offset in the output? How can I relate it to the source code, if possible? (Postprocessed with c++filt)

It's the offset in bytes from the beginning of the function.
Here's an example from WinDbg, but it's the same everywhere:
This is the current call stack:
0:000> k L1
# Child-SP RetAddr Call Site
00 00000000`001afcb8 00000000`77b39ece USER32!NtUserGetMessage+0xa
This is how the function looks like:
0:000> uf USER32!NtUserGetMessage
USER32!NtUserGetMessage:
00000000`77b39e90 4c8bd1 mov r10,rcx
00000000`77b39e93 b806100000 mov eax,1006h
00000000`77b39e98 0f05 syscall
00000000`77b39e9a c3 ret
And this is what the current instruction is:
0:000> u USER32!NtUserGetMessage+a L1
USER32!NtUserGetMessage+0xa:
00000000`77b39e9a c3 ret
So, the offset 0x0A is 10 bytes from the function start. 3 bytes for the first mov, 5 bytes for the second mov and 2 bytes for the syscall.
If you want to relate it to your code, it heavily depends on whether or not it was optimized.
If the offset is very high, you might not have enough symbols. E.g. with export symbols only you may see offsets like +0x2AF4 and you can't tell anything about the real function any more.

Related

How to simplify algorithm of pasting addresses of data to immediate fields in machine code?

When developing my assembler I got one problem. In assembly language we can define data values (e. g. msg db 'Hi') and paste address of this data anywhere, even above this data. However when assembling code assembler don't know address of data until is not processed the line with those data.
Of course, assembler can remember addresses of machine code where we use address of defined data and after processing a code replace values in remembered addresses to address of our data. But if we define data in 2-byted address (e. g. at 01 F1) assembler would be to replace 1 bytes in remembered addresses to 2 bytes of address (01 F1) and so immediate field size will be changed (imm8 -> imm16) and assembler shall be rewrite instruction in same address (change bits w and s at opcode and maybe set prefix 0x66). If assembler will set prefix 0x66 and our data definition be after this instruction it shall be rewrite immediate field bytes (increment address value).
Illustration of this algoritm :
The following code:
mov dh, 09h
add dx, msg
;...
msg db 'Hello$'
will be assembled in the following principle:
preparing the code:
Comment : |===> Remember address of this byte (0x0004)
Comment : | ADD DX,MSG |
Address : 0000 0001 |0002 0003 0004| ... 01F1 01F2 01F3 01F4 01F5 01F6
Code : B4 09 | 83 C2 00 | ... 48 65 6C 6C 6F 24
Comment : ---------------- H e l l o $
rewriting code in remebered addresses:
Comment : |=============|-This address (msg)
Comment : | ADD DX,01F1 | v
Address : 0000 0001 |0002 0003 0004 0005| ... 01F2 01F3 01F4 01F5 01F6 01F7
Code : B4 09 | 83 C2 F1 01 | ... 48 65 6C 6C 6F 24
Comment : --------------------- H e l l o $
rewriting instruction's opcode 83h -> 81h (10000011b -> 10000001b: bit s=0):
Comment : |=============|-This address (msg)
Comment : | ADD DX,01F1 | v
Address : 0000 0001 |0002 0003 0004 0005| ... 01F2 01F3 01F4 01F5 01F6 01F7
Code : B4 09 | 81 C2 F1 01 | ... 48 65 6C 6C 6F 24
Comment : --------------------- H e l l o $
write to immediate field the new address of data (0x01F2):
Comment : |=============|-This address (msg)
Comment : | ADD DX,01F2 | v
Address : 0000 0001 |0002 0003 0004 0005| ... 01F2 01F3 01F4 01F5 01F6 01F7
Code : B4 09 | 81 C2 F2 01 | ... 48 65 6C 6C 6F 24
Comment : --------------------- H e l l o $
I think that this algorithm is difficult. Is it possible to simplify its?
If the assembler isn't emitting a flat binary (i.e. also being a linker), the assembler has to assume the symbol address might be 2 bytes, because the final absolute address won't be known until link time, after the assembler is done. (So it will just leave space for a 2-byte address and a relocation for the linker to fill it in).
But if you are assembling directly into a flat binary and want to do this optimization, presumably you'd treat it like branch displacements with a start-small algorithm and do multi-pass optimization until everything fits. In fact you'd do this as part of the same passes that look at jmp/jcc rel8 vs. jmp/jcc rel16. (Why is the "start small" algorithm for branch displacement not optimal? - it is optimal if you don't have stuff like align 8, otherwise there are corner cases where it does ok but not optimal.)
These optimization passes are just looping over internal data-structures that represent the code, not actually writing final machine-code at each step. There's no need to calculate or look up the actual opcodes and ModRM encodings until after the last optimization pass.
You just need your optimizer to know rules for instruction size, e.g. that add reg, imm8 is 3 bytes, add reg, imm16 is 4 bytes (except for AX where add ax, imm16 has a 3-byte special encoding, same as add ax, imm8, so add to AX doesn't need to be part of the multi-pass optimization at all, it can just choose an encoding when we reach it after all symbol addresses are known.)
Note that it's much more common to use addresses as immediates for mov which doesn't allow narrow immediates at all (mov reg, imm16 is always 3 bytes). But this optimization is also relevant for disp8 vs. disp16 in addressing modes, e.g. for xor cl, [di + msg] could use reg+disp8 for small addresses so it's worth having this optimization.
So again, your optimizer passes would know that [di + disp8] takes 1 byte after the ModRM, [di + disp16] takes 2 extra.
And [msg] always takes 2 bytes after ModRM, there is no [disp8] encoding. The asm source would need to have a zeroed register if it wanted to take advantage of disp8 for small addresses.
Of course a simplistic or one-pass assembler could always just assume addresses are 16-bit and encode the other parts of the machine code accordingly, only going back to fill in numeric addresses once unresolved symbols are seen. (Or emit relocation info at the end for a linker to do it.)

Write assembly language program to sort student names according to their grades

I want to write a simple assembly language program to sort student names according to their grades.
I am just using:
.data
.code
I try this bubble sort but this one is only for numbers. How can I add names for the students?
.data
array db 9,6,5,4,3,2,1
count dw 7
.code
mov cx,count
dec cx
nextscan:
mov bx,cx
mov si,0
nextcomp:
mov al,array[si]
mov dl,array[si+1]
cmp al,dl
jnc noswap
mov array[si],dl
mov array[si+1],al
noswap:
inc si
dec bx
jnz nextcomp
loop nextscan
Long ago, one of the most common way to represent data was with what was called fixed length fields. It wasn't uncommon to find all related data in one place like this;
Student: db 72, 'Marie '
db 91, 'Barry '
db 83, 'Constantine '
db 59, 'Wil-Alexander '
db 97, 'Jake '
db 89, 'Ceciel '
This is doable, as each of the fields is 16 bytes long and that is the way data used to be constructed in multiples of 2. So the data length was either 2, 4, 8, 16, 32, 64 and so on. Didn't have to be this way and a lot of times it wasn't, but multiples like that made the code simpler.
Problem is, each time we want to sort, all data has to be moved, so the relational database was born. Here we separate variable data from static.
Student: db 'Marie '
db 'Barry '
db 'Constantine '
db 'Wil-Alexander '
db 'Jake '
db 'Ceciel '
Grades: db 72, 0
db 91, 1
db 83, 2
db 59, 3
db 97, 4
db 89, 5
dw -1 ; Marks end of list
Not only will this be easier to manage in the program, but to add more grades and even grades for the same person is easier. Here is an example of how code would work to do comparisons.
mov si, Grades
mov bl, 0
push si
L0: lodsw
cmp ax, -1
jz .done
cmp [si-4], al
jae L0
.... Exchange Data Here ....
bts bx, 0
jmp L0
.done:
pop si
btc bx, 0
jc L0 - 1
ret
After routine has been executed the contents of grades is as follows;
61 04 5B 01 59 05 53 02 48 00 3B 00
I do have a working copy of this program tested in DOSBOX and because this is a homework assignment, I'm not going to hand it to you on a silver platter, but 95% of the work is done. All you need to do before handing in is make sure you can explain why BTS & BTC makes the bubble work and implement something that will exchange data.
If you needed to display this data, you'd need to device a conversion routine from binary -> decimal, but by simply multiplying the index number by 16 associated with each grade and adding the address of Student to it, that would give you a pointer to the appropriate name.
Sort pointers to name, grade structs, or indices into separate name and grade arrays.
That's one extra level of indirection in the compare, but not in the swap.

Why does the 0x55555556 divide by 3 hack work?

There is a (relatively) well known hack for dividing a 32-bit number by three. Instead of using actual expensive division, the number can be multiplied by the magic number 0x55555556, and the upper 32 bits of the result are what we're looking for. For example, the following C code:
int32_t div3(int32_t x)
{
return x / 3;
}
compiled with GCC and -O2, results in this:
08048460 <div3>:
8048460: 8b 4c 24 04 mov ecx,DWORD PTR [esp+0x4]
8048464: ba 56 55 55 55 mov edx,0x55555556
8048469: 89 c8 mov eax,ecx
804846b: c1 f9 1f sar ecx,0x1f
804846e: f7 ea imul edx
8048470: 89 d0 mov eax,edx
8048472: 29 c8 sub eax,ecx
8048474: c3 ret
I'm guessing the sub instruction is responsible for fixing negative numbers, because what it does is essentially add 1 if the argument is negative, and it's a NOP otherwise.
But why does this work? I've been trying to manually multiply smaller numbers by a 1-byte version of this mask, but I fail to see a pattern, and I can't really find any explanations anywhere. It seems to be a mystery magic number whose origin isn't clear to anyone, just like 0x5f3759df.
Can someone provide an explanation of the arithmetic behind this?
It's because 0x55555556 is really 0x100000000 / 3, rounded up.
The rounding is important. Since 0x100000000 doesn't divide evenly by 3, there will be an error in the full 64-bit result. If that error were negative, the result after truncation of the lower 32 bits would be too low. By rounding up, the error is positive, and it's all in the lower 32 bits so the truncation wipes it out.

Packing BCD to DPD: How to improve this amd64 assembly routine?

I'm writing a routine to convert between BCD (4 bits per decimal digit) and Densely Packed Decimal (DPD) (10 bits per 3 decimal digits). DPD is further documented (with the suggestion for software to use lookup-tables) on Mike Cowlishaw's web site.
This routine only ever requires the lower 16 bit of the registers it uses, yet for shorter instruction encoding I have used 32 bit instructions wherever possible. Is a speed penalty associated with code like:
mov data,%eax # high 16 bit of data are cleared
...
shl %al
shr %eax
or
and $0x888,%edi # = 0000 a000 e000 i000
imul $0x0490,%di # = aei0 0000 0000 0000
where the alternative to a 16 bit imul would be either a 32 bit imul and a subsequent and or a series of lea instructions and a final and.
The whole code in my routine can be found below. Is there anything in it where performance is worse than it could be due to me mixing word and dword instructions?
.section .text
.type bcd2dpd_mul,#function
.globl bcd2dpd_mul
# convert BCD to DPD with multiplication tricks
# input abcd efgh iklm in edi
.align 8
bcd2dpd_mul:
mov %edi,%eax # = 0000 abcd efgh iklm
shl %al # = 0000 abcd fghi klm0
shr %eax # = 0000 0abc dfgh iklm
test $0x880,%edi # fast path for a = e = 0
jz 1f
and $0x888,%edi # = 0000 a000 e000 i000
imul $0x0490,%di # = aei0 0000 0000 0000
mov %eax,%esi
and $0x66,%esi # q = 0000 0000 0fg0 0kl0
shr $13,%edi # u = 0000 0000 0000 0aei
imul tab-8(,%rdi,4),%si # v = q * tab[u-2][0]
and $0x397,%eax # r = 0000 00bc d00h 0klm
xor %esi,%eax # w = r ^ v
or tab-6(,%rdi,4),%ax # x = w | tab[u-2][1]
and $0x3ff,%eax # = 0000 00xx xxxx xxxx
1: ret
.size bcd2dpd_mul,.-bcd2dpd_mul
.section .rodata
.align 4
tab:
.short 0x0011 ; .short 0x000a
.short 0x0000 ; .short 0x004e
.short 0x0081 ; .short 0x000c
.short 0x0008 ; .short 0x002e
.short 0x0081 ; .short 0x000e
.short 0x0000 ; .short 0x006e
.size tab,.-tab
Improved Code
After applying some suggestions from the answer and comments and some other trickery, here is my improved code.
.section .text
.type bcd2dpd_mul,#function
.globl bcd2dpd_mul
# convert BCD to DPD with multiplication tricks
# input abcd efgh iklm in edi
.align 8
bcd2dpd_mul:
mov %edi,%eax # = 0000 abcd efgh iklm
shl %al # = 0000 abcd fghi klm0
shr %eax # = 0000 0abc dfgh iklm
test $0x880,%edi # fast path for a = e = 0
jnz 1f
ret
.align 8
1: and $0x888,%edi # = 0000 a000 e000 i000
imul $0x49,%edi # = 0ae0 aei0 ei00 i000
mov %eax,%esi
and $0x66,%esi # q = 0000 0000 0fg0 0kl0
shr $8,%edi # = 0000 0000 0ae0 aei0
and $0xe,%edi # = 0000 0000 0000 aei0
movzwl lookup-4(%rdi),%edx
movzbl %dl,%edi
imul %edi,%esi # v = q * tab[u-2][0]
and $0x397,%eax # r = 0000 00bc d00h 0klm
xor %esi,%eax # w = r ^ v
or %dh,%al # = w | tab[u-2][1]
and $0x3ff,%eax # = 0000 00xx xxxx xxxx
ret
.size bcd2dpd_mul,.-bcd2dpd_mul
.section .rodata
.align 4
lookup:
.byte 0x11
.byte 0x0a
.byte 0x00
.byte 0x4e
.byte 0x81
.byte 0x0c
.byte 0x08
.byte 0x2e
.byte 0x81
.byte 0x0e
.byte 0x00
.byte 0x6e
.size lookup,.-lookup
TYVM for commenting the code clearly and well, BTW. It made is super easy to figure out what was going on, and where the bits were going. I'd never heard of DPD before, so puzzling it out from uncommented code and the wikipedia article would have sucked.
The relevant gotchas are:
Avoid 16bit operand size for instructions with immediate constants, on Intel CPUs. (LCP stalls)
avoid reading the full 32 or 64bit register after writing only the low 8 or 16, on Intel pre-IvyBridge. (partial-register extra uop). (IvB still has that slowdown if you modify an upper8 reg like AH, but Haswell removes that too). It's not just an extra uop: the penalty on Core2 is 2 to 3 cycles, according to Agner Fog. I might be measuring it wrong, but it seems a lot less bad on SnB.
See http://agner.org/optimize/ for full details.
Other than that, there's no general problem with mixing in some instructions using the operand-size prefix to make them 16-bit.
You should maybe write this as inline asm, rather than as a called function. You only use a couple registers, and the fast-path case is very few instructions.
I had a look at the code. I didn't look into achieving the same result with significantly different logic, just at optimizing the logic you do have.
Possible code suggestions: Switch the branching so the fast-path has the not-taken branch. Actually, it might make no diff either way in this case, or might improve the alignment of the slow-path code.
.p2align 4,,10 # align to 16, unless we're already in the first 6 bytes of a block of 16
bcd2dpd_mul:
mov %edi,%eax # = 0000 abcd efgh iklm
shl %al # = 0000 abcd fghi klm0
shr %eax # = 0000 0abc dfgh iklm
test $0x880,%edi # fast path for a = e = 0
jnz .Lslow_path
ret
.p2align 4 # Maybe fine-tune this alignment based on how the rest of the code assembles.
.Lslow_path:
...
ret
It's sometimes better to duplicate return instructions than to absolutely minimize code-size. The compare-and-branch in this case is the 4th uop of the function, though, so a taken branch wouldn't have prevented 4 uops from issuing in the first clock cycle, and a correctly-predicted branch would still issue the return on the 2nd clock cycle.
You should use a 32bit imul for the one with the table source. (see next section about aligning the table so reading an extra 2B is ok). 32bit imul is one uop instead of two on Intel SnB-family microarches. The result in the low16 should be the same, since the sign bit can't be set. The upper16 gets zeroed by the final and before ret, and doesn't get used in any way where garbage in the upper16 matters while it's there.
Your imul with an immediate operand is problematic, though.
It causes an LCP stall when decoding on Intel, and it writes the the low16 of a register that is later read at full width. Its upper16 would be a problem if not masked off (since it's used as a table index). Its operands are large enough that they will put garbage into the upper16, so it does need to be discarded.
I thought your way of doing it would be optimal for some architectures, but it turns out imul r16,r16,imm16 itself is slower than imul r32,r32,imm32 on every architecture except VIA Nano, AMD K7 (where it's faster than imul32), and Intel P6 (where using it from 32bit / 64bit mode will LCP-stall, and where partial-reg slowdowns are a problem).
On Intel SnB-family CPUs, where imul r16,r16,imm16 is two uops, imul32/movzx would be strictly better, with no downside except code size. On P6-family CPUs (i.e. PPro to Nehalem), imul r16,r16,imm16 is one uop, but those CPUs don't have a uop cache, so the LCP stall is probably critical (except maybe Nehalem calling this in a tight loop, fitting in the 28 uop loop buffer). And for those CPUs, the explicit movzx is probably better from the perspective of the partial-reg stall. Agner Fog says something about there being an extra cycle while the CPU inserts the merging uop, which might mean a cycle where that extra uop is issued alone.
On AMD K8-Steamroller, imul imm16 is 2 m-ops instead of 1 for imul imm32, so imul32/movzx is about equal to imul16 there. They don't suffer from LCP stalls, or from partial-reg problems.
On Intel Silvermont, imul imm16 is 2 uops (with one per 4 clocks throughput), vs. imul imm32 being 1 uops (with one per 1 clock throughput). Same thing on Atom (the in-order predecessor to Silvermont): imul16 is an extra uop and much slower. On most other microarchitectures, throughput isn't worse, just latency.
So if you're willing to increase the code-size in bytes where it will give a speedup, you should use a 32bit imul and a movzwl %di, %edi. On some architectures, this will be about the same speed as the imul imm16, while on others it will be much faster. It might be slightly worse on AMD bulldozer-family, which isn't very good at using both integer execution units at once, apparently, so a 2 m-op instruction for EX1 might be better than two 1 m-op instructions where one of them is still an EX1-only instruction. Benchmark this if you care.
Align tab to at least a 32B boundary, so your 32bit imul and or can do a 4B load from any 2B-aligned entry in it without crossing a cache-line boundary. Unaligned accesses have no penalty on all recent CPUs (Nehalem and later, and recent AMD), as long as they don't span two cache lines.
Making the operations that read from the table 32bit avoids the partial-register penalty that Intel CPUs have. AMD CPUs, and Silvermont, don't track partial-registers separately, so even instructions that write-only to the low16 have to wait for the result in the rest of the reg. This stops 16bit insns from breaking dependency chains. Intel P6 and SnB microarch families track partial regs. Haswell does full dual bookkeeping or something, because there's no penalty when merging is needed, like after you shift al, then shift eax. SnB will insert an extra uop there, and there may be a penalty of a cycle or two while it does this. I'm not sure, and haven't tested. However, I don't see a nice way to avoid this.
The shl %al could be replaced with a add %al, %al. That can run on more ports. Probably no difference, since port0/5 (or port0/6 on Haswell and later) probably aren't saturated. They have the same effect on the bits, but set flags differently. Otherwise they could be decoded to the same uop.
changes: split the pext/pdep / vectorize version into a separate answer, partly so it can have its own comment thread.
(I split the BMI2 version into a separate answer, since it could end up totally different)
After seeing what you're doing with that imul/shr to get a table index, I can see where you could use BMI2 pextr to replace and/imul/shr, or BMI1 bextr to replace just the shr (allowing use of imul32, instead of imul16, since you'd just extract the bits you want, rather than needing to shift zeros from the upper16). There are AMD CPUs with BMI1, but even steamroller lacks BMI2. Intel introduced BMI1 and BMI2 at the same time with Haswell.
You could maybe process two or four 16bit words at once, with 64bit pextr. But not for the whole algorithm: you can't do 4 parallel table lookups. (AVX2 VPGATHERDD is not worth using here.) Actually, you can use pshufb to implement a LUT with indices up to 4bits, see below.
Minor improvement version:
.section .rodata
# This won't won't assemble, written this way for humans to line up with comments.
extmask_lobits: .long 0b0000 0111 0111 0111
extmask_hibits: .long 0b0000 1000 1000 1000
# pext doesn't have an immediate-operand form, but it can take the mask from a memory operand.
# Load these into regs if running in a tight loop.
#### TOTALLY UNTESTED #####
.text
.p2align 4,,10
bcd2dpd_bmi2:
# mov %edi,%eax # = 0000 abcd efgh iklm
# shl %al # = 0000 abcd fghi klm0
# shr %eax # = 0000 0abc dfgh iklm
pext extmask_lobits, %edi, %eax
# = 0000 0abc dfgh iklm
mov %eax, %esi # insn scheduling for 4-issue front-end: Fast-path is 4 fused-domain uops
# And doesn't waste issue capacity when we're taking the slow path. CPUs with mov-elimination won't waste execution units from issuing an extra mov
test $0x880, %edi # fast path for a = e = 0
jnz .Lslow_path
ret
.p2align 4
.Lslow_path:
# 8 uops, including the `ret`: can issue in 2 clocks.
# replaces and/imul/shr
pext extmask_hibits, %edi, %edi #u= 0000 0000 0000 0aei
and $0x66, %esi # q = 0000 0000 0fg0 0kl0
imul tab-8(,%rdi,4), %esi # v = q * tab[u-2][0]
and $0x397, %eax # r = 0000 00bc d00h 0klm
xor %esi, %eax # w = r ^ v
or tab-6(,%rdi,4), %eax # x = w | tab[u-2][1]
and $0x3ff, %eax # = 0000 00xx xxxx xxxx
ret
Of course, if making this an inline-asm, rather than a stand-alone function, you'd change back to the fast path branching to the end, and the slow-path falling through. And you wouldn't waste space with alignment padding mid-function either.
There might be more scope for using pextr and/or pdep for more of the rest of the function.
I was thinking about how to do even better with BMI2. I think we could get multiple aei selectors from four shorts packed into 64b, then use pdep to deposit them in the low bits of different bytes. Then movq that to a vector register, where you use it as a shuffle control-mask for pshufb to do multiple 4bit LUT lookups.
So we could go from 60 BCD bits to 50 DPD bits at a time. (Use shrd to shift bits between registers to handle loads/stores to byte-addressable memory.)
Actually, 48 BCD bits (4 groups of 12bits each) -> 40 DPD bits is probably a lot easier, because you can unpack that to 4 groups of 16bits in a 64b integer register, using pdep. Dealing with the selectors for 5 groups is fine, you can unpack with pmovzx, but dealing with the rest of the data would require bit-shuffling in vector registers. Not even the slow AVX2 variable-shift insns would make that easy to do. (Although it might be interesting to consider how to implement this with BMI2 at all, for big speedups on CPUs with just SSSE3 (i.e. every relevant CPU) or maybe SSE4.1.)
This also means we can put two clusters of 4 groups into the low and high halves of a 128b register, to get even more parallelism.
As a bonus, 48bits is a whole number of bytes, so reading from a buffer of BCD digits wouldn't require any shrd insns to get the leftover 4 bits from the last 64b into the low 4 for the next. Or two offset pextr masks to work when the 4 ignored bits were the low or high 4 of the 64b.... Anyway, I think doing 5 groups at once just isn't worth considering.
Full BMI2 / AVX pshufb LUT version (vectorizable)
The data movement could be:
ignored | group 3 | group 2 | group 1 | group 0
16bits | abcd efgh iklm | abcd efgh iklm | abcd efgh iklm | abcd efgh iklm
3 2 1 | 0
pext -> aei|aei|aei|aei # packed together in the low bits
2 | 1 | 0
pdep -> ... |0000 0000 0000 0aei|0000 0000 0000 0aei # each in a separate 16b word
movq -> xmm vector register.
(Then pinsrq another group of 4 selectors into the upper 64b of the vector reg). So the vector part can handle 2 (or AVX2: 4) of this at once
vpshufb xmm2 -> map each byte to another byte (IMUL table)
vpshufb xmm3 -> map each byte to another byte (OR table)
Get the bits other than `aei` from each group of 3 BCD digits unpacked from 48b to 64b, into separate 16b words:
group 3 | group 2 | group 1 | group 0
pdep(src)-> 0000 abcd efgh iklm | 0000 abcd efgh iklm | 0000 abcd efgh iklm | 0000 abcd efgh iklm
movq this into a vector reg (xmm1). (And repeat for the next 48b and pinsrq that to the upper64)
VPAND xmm1, mask (to zero aei in each group)
Then use the vector-LUT results:
VPMULLW xmm1, xmm2 -> packed 16b multiply, keeping only the low16 of the result
VPAND xmm1, mask
VPXOR xmm1, something
VPOR xmm1, xmm3
movq / pextrq back to integer regs
pext to pack the bits back together
You don't need the AND 0x3ff or equivalent:
Those bits go away when you pext to pack each 16b down to 10b
shrd or something to pack the 40b results of this into 64b chunks for store to memory.
Or: 32b store, then shift and store the last 8b, but that seems lame
Or: just do 64b stores, overlapping with the previous. So you write 24b of garbage every time. Take care at the very end of the buffer.
Use AVX 3-operand versions of the 128b SSE instructions to avoid needing movdqa to not overwrite the table for pshufb. As long as you never run a 256b AVX instruction, you don't need to mess with vzeroupper. You might as well use the v (VEX) versions of all vector instructions, though, if you use any. Inside a VM, you might be running on a virtual CPU with BMI2 but not AVX support, so it's prob. still a good idea to check both CPU feature flags, rather than assuming AVX if you see BMI2 (even though that's safe for all physical hardware that currently exists).
This is starting to look really efficient. It might be worth doing the mul/xor/and stuff in vector regs, even if you don't have BMI2 pext/pdep to do the bit packing/unpacking. I guess you could use code like the existing non-BMI scalar routing to get selectors, and mask/shift/or could build up the non-selector data into 16b chunks. Or maybe shrd for shifting data from one reg into another?

gcc compiled binary in x86 assembly

gcc compile binary has following assembly:
8049264: 8d 44 24 3e lea 0x3e(%esp),%eax
8049268: 89 c2 mov %eax,%edx
804926a: bb ff 00 00 00 mov $0xff,%ebx
804926f: b8 00 00 00 00 mov $0x0,%eax
8049274: 89 d1 mov %edx,%ecx
8049276: 83 e1 02 and $0x2,%ecx
8049279: 85 c9 test %ecx,%ecx
804927b: 74 09 je 0x8049286
At first glance, I had no idea what it is doing at all. My best guess is some sort of memory alignment and clearing up local variable (because rep stos is filling 0 at local variable location). If you take a look at first few lines, load address into eax and move to ecx and test if it is even address or not, but I'm lost why this is happening. I want to know what exactly happen in here.
It looks like initialising a local variable located at [ESP + 0x03e] to zeroes. At first, EDX is initialised to hold the address and EBX is initialised to hold the size in bytes. Then, it's checked whether EDX & 2 is nonzero; in other words, whether EDX as a pointer is wyde-aligned but not tetra-aligned. (Assuming ESP is tetrabyte aligned, as it generally should, EDX, which was initialised at 0x3E bytes above ESP, would not be tetrabyte aligned. But this is slightly besides the point.) If this is the case, the wyde from AX, which is zero, is stored at [EDX], EDX is incremented by two, and the counter EBX is decremented by two. Now, assuming ESP was at least wyde-aligned, EDX is guaranteed to be tetra-aligned. ECX is calculated to hold the number of tetrabytes remaining by shifting EBX right two bits, EDI is loaded from EDX, and the REP STOS stores that many zero tetrabytes at [EDI], incrementing EDI in the process. Then, EDX is loaded from EDI to get the pointer-past-space-initialised-so-far. Finally, if there were at least two bytes remaining uninitialised, a zero wyde is stored at [EDX] and EDX is incremented by two, and if there was at least one byte remaining uninitialised, a zero byte is stored at [EDX] and EDX is incremented by one. The point of this extra complexity is apparently to store most of the zeroes as four-byte values rather than single-byte values, which may, under certain circumstances and in certain CPU architectures, be slightly faster.

Resources