I have ran into a curious problem. An algorithm I am working on consists of lots of computations like this
q = x(0)*y(0)*z(0) + x(1)*y(1)*z(1) + ...
where the length of summation is between 4 and 7.
The original computations are all done using 64-bit precision. For experimentation, I tried using 32-bit precision for x,y,z input values (so that computations are performed using 32-bit), and storing final result as 64-bit value (straightforward cast).
I expected 32-bit performance to be better (cache size, SIMD size, etc.), but to my surprise there was no difference in performance, maybe even decrease.
The architecture in question is Intel 64, Linux, and GCC. Both codes do seem to use SSE and arrays in both cases are aligned to 16 byte boundary.
Why would it be so? My guess so far is that 32-bit precision can use SSE only on the first four elements, with the rest being done serially compounded by cast overhead.
On x87 at least, everything is really done in 80-bit precision internally. The precision really just determines how many of those bits are stored in memory. This is part of the reason why different optimization settings can change results slightly: They change the amount of rounding from 80-bit to 32- or 64-bit.
In practice, using 80-bit floating point (long double in C and C++, real in D) is usually slow because there's no efficient way to load and store 80 bits from memory. 32- and 64-bit are usually equally fast provided that memory bandwidth isn't the bottleneck, i.e. if everything is in cache anyhow. 64-bit can be slower if either of the following happens:
Memory bandwidth is the bottleneck.
The 64-bit numbers aren't properly aligned on 8-byte boundaries. 32-bit numbers only require 4-byte alignment for optimal efficiency, so they're less finicky. Some compilers (the Digital Mars D compiler comes to mind) don't always get this right for 64-bit doubles stored on the stack. This causes twice the amount of memory operations to be necessary to load one, in practice resulting in about a 2x performance hit compared to properly aligned 64-bit floats or 32-bit floats.
As far as SIMD optimizations go, it should be noted that most compilers are horrible at auto-vectorizing code. If you don't want to write directly in assembly language, the best way to take advantage of these instructions is to use things like array-wise operations, which are available, for example, in D, and implemented in terms of SSE instructions. Similarly, in C or C++, you would probably want to use a high level library of functions that are SSE-optimized, though I don't know of a good one off the top of my head because I mostly program in D.
It's probably because your processor still makes the 64bit counting and then trimms the number. There was some CPU flag you could change, but I can't remember...
First check the ASM that gets produced. It may not be what you expect.
Also try writing it as a loop:
typedef float fp;
fp q = 0
for(int i = 0; i < N; i++)
q += x[i]*y[i]*z[i]
Some compiler might notice the loop and not the unrolled form.
Lastly, your code used () rather than []. If your code is making lots of function calls (12 to 21), that will swamp the FP cost and even removing the fp computation all together won't make much difference. Inlineing OTOH might.
Related
Why don't we see twice better performance when executing a 64-bit operations (e.g. double precision operation) on a 64-bit machine, compared to executing on a 32-bit machine?
In a 32-bit machine, don't we need to fetch from memory twice as much? More importantly, don't we need twice as much cycles to execute a 64-bit operation?
“64-bit machine” is an ambiguous term but usually means that the processor's General-Purpose Registers are 64-bit wide. Compare 8086 and 8088, which have the same instruction set and can both be called 16-bit processors in this sense.
When the phrase is used in this sense, it has nothing to do with the width of the memory bus, the width of the internal buses inside the CPU, and the ability of the ALU to operate efficiently on 32- or 64-bit wide data.
Your question also assumes that the hardest part of a multiplication is moving the operands to the unit that takes care of multiplication inside the processor, which wouldn't be quite true even if the operands came from memory and the bus was 32-bit wide, because latency != throughput. Also, regarding the mathematics of floating-point multiplication, a 64-bit multiplication is not twice as hard as a 32-bit one, it is roughly (53/24)2 times as hard (but, again, the transistors can be there to compute the double-precision multiplication efficiently regardless of the width of the General-Purpose Registers).
In a 32-bit machine, don't we need to fetch from memory twice as much?
No. In most modern CPUs, memory bus width is at least 64 bits. Newer microarchitectures may have wider bus. Quad-channel memory will have a minimum 256-bit bus. Many contemporary CPUs even support 6 or 8-channel memory. So you need only 1 fetch to get a double. Besides most of the time the value has already been in cache, so loading it won't take much time. CPUs don't load a single value but the whole cache line each time
more importantly, don't we need twice as much cycles to execute a 64-bit operation?
First you should know that the actual number of significant bits in double is 53 so it's not "twice as much" harder. It's more than twice the number in float (24 significant bits). When the number of bits is doubled, addition and subtraction is twice as hard while multiplication is 4 times as hard. Many other more complex operations will need even more effort
But despite that harder math work, non-memory operations on both float and double are usually the same on most modern architectures because both will be done in the same set of registers with the same ALU/FPU. Those powerful FPUs can add two doubles in a single cycle, so obviously even if you can add two floats faster then it still consumes 1 cycle. In the old Intel x87 the internal registers are 80 bits in length and both single and double precision must be extended to 80 bits, hence their performance will also be the same. There's no way to do math in narrower types than 80-bit extended
With SIMD support like SSE2/AVX/AVX-512 you'll be able to process 2/4/8 doubles at a time (or even more in other SIMD ISAs), so you can see that adding two doubles like that is only a tiny task for modern FPUs. However with SIMD we can fit twice the number of floats in a register compared to double, so float operations will be faster if you need to do a lot of math in parallel. In a cycle if you can work on 4 doubles at a time then you'll be able to do the same on 8 floats
Another case where float is faster than double would be when you work on a huge array, because more floats fit in the same cache line than double. As a result using double incurs more cache misses when you traverse through the array
A colleague and I recently were arguing about whether or not it is a good idea to use 64-bit variables in 32-bit code. I took a side saying "It might be dangerous and slow us down somewhere", he said "Nah. Nothing bad will happen." So who is right?
The bitness of parts of our ecosystem is as follows:
Windows: either 32-bit or 64-bit
Compiler: 32-bit only (Delphi...)
Processor: modern Intel ones... that's 64-bit, isn't it?
Let's say we have some variables which use some 64-bit type built into the language, with Delphi that would be Int64. Is it safe (performance-wise) to scatter the code with calculations based on these?
It sounds like you are doing some premature optimization.
Even if it would cause a very slight performance hit, the hardware of your users are more likely to have 64-bit machines as the software ages. So it would be a self correcting problem, if it even is a problem.
If the compiler supports them, go ahead and use them. If you find a slowdown in a tight loop with lots of calculations, then perhaps consider changing it.
It's bad. The data takes twice the memory, so it's like running on a processor with half the cache. It also requires 2 instructions to load, 2 instructions to add, and 2 instructions to store a pair of numbers. You can test this with simple programs like a prime number sieve. These will generally run measurably slower using the longer types on a 32bit machine, and even on a 64bit machine when you get larger than the cache. The worst part is that once you sprinkle this stuff all over your code, you'll have a systemic performance problem that will be hard to correct later. And yes, similar is even true of using 32 bit integers where shorts could be used, but only with regard to memory performance - a 32bit CPU can do 32bit math just as fast as 16bit - so don't worry about that (or your spreadsheet will only support 65K rows for example).
GCC (and i doubt your Delphi-compiler is much better) compiles/assembles its long long data-datatype so that it uses the EAX and ECX registers, loading them already takes longer than only loading one of them (if it is actually double the time depends on the cache). It then assembles multiple instructions where you would only need one for e.g. ADD/SUB/IDIV if you were only operating on a single 32-bit value. Then follow the stores for EAX and ECX which again take longer than storing only one of them.
On x64 architecture, if you only need 32 bit wide integers, then using 32 bit wide integers will result in faster code, even when running 64 bit code.
Ask yourself this:
Does this variable ever need to store a value that can be bigger than 32-bit, and then choose from the following:
If this answer is "no", use an unsigned int.
If the answer is "yes", use an unsigned 64-bit int. 64-bit ints aren't very memory expensive and are cheap to add or subtract (and reasonably cheap to multiply divide).
If the answer is "yes, but only on 64-bit systems" (for example the length of a buffer might be > 4GB on 64-bit but never on 32-bit), use a "size_t". This is defined to be 32-bit on 32-bit systems and 64-bit on 64-bit systems.
Write your program to be correct, and let the optimiser / performance tests help you get it fast afterwards. It's more expensive for you to write it fast first and fix it later than for you to write it correct first and make it fast later.
While HDDs evolve and offer more and more space on less room, why are we "sticking with" 32-bit or 64-bit?
Why can't there be a e.g.: 128-bit processor?
(This is not my homework; I'm just a student interested beyond the things they teach us in informatics)
Because the difference between 32-bit and 64-bit is astronomical - it's really the difference between 232 (a ten-digit number in the billions) and 264 (a twenty-digit number in the squillions :-).
64 bits will be more than enough for decades to come.
There's very little need for this, when do you deal with numbers that large? The current addressable memory space available to 64-bit is well beyond what any machine can handle for at least a few years...and beyond that it's probably more than any desktop will hold for quite a while.
Yes, desktop memory will continue to increase, but 4 billion times what it is now? That's going to take a while...sure we'll get to 128-bit, if the whole current model isn't thrown out before then, which I see equally as likely.
Also, it's worth noting that upgrading something from 32-bit to 64-bit puts you in a performance hole immediately in most scenarios (this is a major reason Visual Studio 2010 remains 32-bit only). The same will happen with 64-bit to 128-bit. The more small objects you have, the more pointers, which are now twice as large, that's more data to pass around to do the same thing, especially if you don't need that much addressable memory space.
When we talk about an n-bit architecture we are often conflating two rather different things:
(1) n-bit addressing, e.g. a CPU with 32-bit address registers and a 32-bit address bus can address 4 GB of physical memory
(2) size of CPU internal data paths and general purpose registers, e.g. a CPU with 32-bit internal architecture has 32-bit registers, 32-bit integer ALUs, 32-bit internal data paths, etc
In many cases (1) and (2) are the same, but there are plenty of exceptions and this may become increasingly the case, e.g. we may not need more than 64-bit addressing for the forseeable future, but we may want > 64 bits for registers and data paths (this is already the case with many CPUs with SIMD support).
So, in short, you need to be careful when you talk about, e.g. a "64-bit CPU" - it can mean different things in different contexts.
Cost. Also, what do you think the 128-bit architecture will get you? Memory addressing and such, but to handle it effectively, you need higher bandwidth buses and basically some new instruction languages that handle it. 64-bit is more than enough for addressing (18446744073709551616 bytes).
HDDs still have a bit of ground to catchup to RAM and such. They're still going to be the IO bottleneck I think. Plus, newer chips are just supporting more cores rather than making a massive change to the language.
Well, I happen to be a professional computer architect (my inventions are probably in the computer you are reading this on), and although I have not yet been paid to work on any processor with more than 64 bits of address, I know some of my friends who have been.
And I have been playing around with 128 bit architectures for fun for a few decades.
I.e. its already happening.
Actually, it has already happened to a limited extent. The HP Precision Architecture, Intel Itanium, and the higher end versions of the IBM Power line, have what I call a folded virtual memory. I have described these elsewhere, e.g. in comp.arch posts in some details, http://groups.google.com/group/comp.arch/browse_thread/thread/53a7396f56860e17/f62404dd5782f309?lnk=gst&q=folded+virtual+memory#f62404dd5782f309
I need to create a comp-arch.net wiki post for these.
But you can get the manuals for these processors and read them yourself.
E.g. you might start with a 64 bit user virtual address.
The upper 8 bits may be used to index a region table, that returns an upper 24 bits that is concatenated with the remaining 64-8=56 bits to produce an 80 bit expanded virtual address. Which is then translated by TLBs and page tables and hash lookups, as usual,
to whatever your physical address is.
Why go from 64->80?
One reason is shared libraries. You may want to have the shared libraries to stay at the same expanded virtual address in all processors, so that you cam share TLB entries. But you may be required, by your language tools, to relocate them to different user virtual addresses. Folded virtual addresses allow this.
Folded virtual addresses are not true >64 bit virtual addresses usable by the user.
For that matter, there are many proposals for >64 bit pointers: e.g. I worked on one where a pointer consisted of a 64bit address, and 64 bit lower and upper bounds, and metadata, for a total of 128 bits. Bounds checking. But, although these have >64 bit pointers or capabilities, they are not truly >64 bit virtual addresses.
Linus posts about 128 bit virtual addresses at http://www.realworldtech.com/beta/forums/index.cfm?action=detail&id=103574&threadid=103545&roomid=2
I'd also like to offer a computer architect's view of why 128bit is impractical at the moment:
Energy cost. See Bill Dally's presentations on how today, most energy in processors is spent moving data around (dissipated in the wires). However, since the most significant bits of a 128bit computation should change little, it should mitigate this problem.
Most arithmetic operations have a non-linear cost w.r.t operand size:
a. A tree multiplier has space complexity n^2, w.r.t. number of bits.
b. The delay of a hierarchical carry look ahead adder is Log[n] w.r.t number of bits (I think). So a 128bit adder will be slower than a 64bit add. Can anyone give some hard numbers (Log[n] seems very cheap) ?
Few programs use 128bit integers or quad precision floating point, and when they do, there are efficient ways to compose them from 32 or 64bit ops.
The next big thing in processor's architecture will be quantum computing. Instead of beeing just 0 or 1, a qbit has a probability of being 0 or 1.
This will lead to huge improvements in the performance of algorithm (for instance, it will be very easy to crack down any RSA private/public key).
Check http://en.wikipedia.org/wiki/Quantum_computer for more information and see you in 15 years ;-)
The main need for a 64 bit processor is to address more memory - and that is the driving force to switch to 64 bit. On 32 bit systems, you can really only address 4Gb of RAM, at least per process. 4Gb is not much.
64 bits give you an address space of several petabytes.(though, a lot of current 64 bit hardware can address "only" 48 bits - thats still enough to support 256 terrabytes of ram though).
Upping the natural integer sizes for a processor does not automatically make it "better" though. There are tradeoffs. With 128bit you'd need twice as much storage(registers/ram/caches/etc.) compared to 64 bit for common data types - with all the drawback that might have - more ram needed to store data, more data to transmit = slower, wider buses might requires more physical space/perhaps more power, etc.
I suppose I am focussing on x86, but I am generally interested in the move from 32 to 64 bit.
Logically, I can see that constants and pointers, in some cases, will be larger so programs are likely to be larger. And the desire to allocate memory on word boundaries for efficiency would mean more white-space between allocations.
I have also heard that 32 bit mode on the x86 has to flush its cache when context switching due to possible overlapping 4G address spaces.
So, what are the real benefits of 64 bit?
And as a supplementary question, would 128 bit be even better?
Edit:
I have just written my first 32/64 bit program. It makes linked lists/trees of 16 byte (32b version) or 32 byte (64b version) objects and does a lot of printing to stderr - not a really useful program, and not something typical, but it is my first.
Size: 81128(32b) v 83672(64b) - so not much difference
Speed: 17s(32b) v 24s(64b) - running on 32 bit OS (OS-X 10.5.8)
Update:
I note that a new hybrid x32 ABI (Application Binary Interface) is being developed that is 64b but uses 32b pointers. For some tests it results in smaller code and faster execution than either 32b or 64b.
https://sites.google.com/site/x32abi/
I typically see a 30% speed improvement for compute-intensive code on x86-64 compared to x86. This is most likely due to the fact that we have 16 x 64 bit general purpose registers and 16 x SSE registers instead of 8 x 32 bit general purpose registers and 8 x SSE registers. This is with the Intel ICC compiler (11.1) on an x86-64 Linux - results with other compilers (e.g. gcc), or with other operating systems (e.g. Windows), may be different of course.
Unless you need to access more memory that 32b addressing will allow you, the benefits will be small, if any.
When running on 64b CPU, you get the same memory interface no matter if you are running 32b or 64b code (you are using the same cache and same BUS).
While x64 architecture has a few more registers which allows easier optimizations, this is often counteracted by the fact pointers are now larger and using any structures with pointers results in a higher memory traffic. I would estimate the increase in the overall memory usage for a 64b application compared to a 32b one to be around 15-30 %.
Regardless of the benefits, I would suggest that you always compile your program for the system's default word size (32-bit or 64-bit), since if you compile a library as a 32-bit binary and provide it on a 64-bit system, you will force anyone who wants to link with your library to provide their library (and any other library dependencies) as a 32-bit binary, when the 64-bit version is the default available. This can be quite a nuisance for everyone. When in doubt, provide both versions of your library.
As to the practical benefits of 64-bit... the most obvious is that you get a bigger address space, so if mmap a file, you can address more of it at once (and load larger files into memory). Another benefit is that, assuming the compiler does a good job of optimizing, many of your arithmetic operations can be parallelized (for example, placing two pairs of 32-bit numbers in two registers and performing two adds in single add operation), and big number computations will run more quickly. That said, the whole 64-bit vs 32-bit thing won't help you with asymptotic complexity at all, so if you are looking to optimize your code, you should probably be looking at the algorithms rather than the constant factors like this.
EDIT:
Please disregard my statement about the parallelized addition. This is not performed by an ordinary add statement... I was confusing that with some of the vectorized/SSE instructions. A more accurate benefit, aside from the larger address space, is that there are more general purpose registers, which means more local variables can be maintained in the CPU register file, which is much faster to access, than if you place the variables in the program stack (which usually means going out to the L1 cache).
I'm coding a chess engine named foolsmate. The best move extraction using a minimax-based tree search to depth 9 (from a certain position) took:
on Win32 configuration: ~17.0s;
after switching to x64 configuration: ~10.3s;
This is 41% of acceleration!
In addition to having more registers, 64-bit has SSE2 by default. This means that you can indeed perform some calculations in parallel. The SSE extensions had other goodies too. But I guess the main benefit is not having to check for the presence of the extensions. If it's x64, it has SSE2 available. ...If my memory serves me correctly.
In the specific case of x68 to x68_64, the 64 bit program will be about the same size, if not slightly smaller, use a bit more memory, and run faster. Mostly this is because x86_64 doesn't just have 64 bit registers, it also has twice as many. x86 does not have enough registers to make compiled languages as efficient as they could be, so x86 code spends a lot of instructions and memory bandwidth shifting data back and forth between registers and memory. x86_64 has much less of that, and so it takes a little less space and runs faster. Floating point and bit-twiddling vector instructions are also much more efficient in x86_64.
In general, though, 64 bit code is not necessarily any faster, and is usually larger, both for code and memory usage at runtime.
Only justification for moving your application to 64 bit is need for more memory in applications like large databases or ERP applications with at least 100s of concurrent users where 2 GB limit will be exceeded fairly quickly when applications cache for better performance. This is case specially on Windows OS where integer and long is still 32 bit (they have new variable _int64. Only pointers are 64 bit. In fact WOW64 is highly optimised on Windows x64 so that 32 bit applications run with low penalty on 64 bit Windows OS. My experience on Windows x64 is 32 bit application version run 10-15% faster than 64 bit since in former case at least for proprietary memory databases you can use pointer arithmatic for maintaining b-tree (most processor intensive part of database systems). Compuatation intensive applications which require large decimals for highest accuracy not afforded by double on 32-64 bit operating system. These applications can use _int64 in natively instead of software emulation. Of course large disk based databases will also show improvement over 32 bit simply due to ability to use large memory for caching query plans and so on.
Any applications that require CPU usage such as transcoding, display performance and media rendering, whether it be audio or visual, will certainly require (at this point) and benefit from using 64 bit versus 32 bit due to the CPU's ability to deal with the sheer amount of data being thrown at it. It's not so much a question of address space as it is the way the data is being dealt with. A 64 bit processor, given 64 bit code, is going to perform better, especially with mathematically difficult things like transcoding and VoIP data - in fact, any sort of 'math' applications should benefit by the usage of 64 bit CPUs and operating systems. Prove me wrong.
On my machine, same h265 encode works almost twice as fast using virtulDub_x64 (with x64 h265 library) vs virtulDub_x32 (regular x32 h265 library). That's probably because longint (64bits) numbers operations (ie: add) can be done on a single instruction on x64, but on 32bit needs two: add lower part, and then add (with carry) the higher part. So unless integer maths are limited to 32bit integers, most of it will take more time under x32.
Mason asked about the advantages of a 64-bit processor.
Well, an obvious disadvantage is that you have to move more bits around. And given that memory accesses are a serious issue these days[1], moving around twice as much memory for a fair number of operations can't be a good thing.
But how bad is the effect of this, really? And what makes up for it? Or should I be running all my small apps on 32-bit machines?
I should mention that I'm considering, in particular, the case where one has a choice of running 32- or 64-bit on the same machine, so in either mode the bandwidth to main memory is the same.
[1]: And even fifteen years ago, for that matter. I remember talk as far back as that about good cache behaviour, and also particularly that the Alpha CPUs that won all the benchmarks had a giant, for the time, 8 MB of L2 cache.
Whether your app should be 64-bit depends a lot on what kind of computation it does. If you need to process very large data sets, you obviously need 64-bit pointers. If not, you need to know whether your app spends relatively more time doing arithmetic or memory accesses. On x86-64, the general purpose registers are not only twice as wide, there are twice as many and they are more "general purpose". This means that 64-bit code can have much better integer op performance. However, if your code doesn't need the extra register space, you'll probably see better performance by using smaller pointers and data, due to increased cache effectiveness. If your app is dominated by floating point operations, there probably isn't much point in making it 32-bit, because most of the memory accesses will be for wide vectors anyways, and having the extra SSE registers will help.
Most 64-bit programming environments use the "LP64" model, meaning that only pointers and long int variables (if you're a C/C++ programmer) are 64 bits. Integers (ints) remain 32-bits unless you're in the "ILP64" model, which is fairly uncommon.
I only bring it up because most int variables aren't being used for size_t-like purposes--that is, they stay within ranges comfortably held by 32 bits. For variables of that nature, you'll never be able to tell the difference.
If you're doing numerical or data-heavy work with > 4GB of data, you'll need 64 bits anyways. If you're not, you won't notice the difference, unless you're in the habit of using longs where most would use ints.
I think you're starting off with a bad assumption here. You say:
moving around twice as much memory
for a fair number of operations can't
be a good thing
and the first question is ask is "why not"? In a true 64 bit machine, the data path is 64 bits wide, and so moving 64 bits takes exactly (to a first approximation) as many cycles as moving 32 bits on a 32 bit machine. So, if you need to move 128 bytes, it takes half as many cycles as it would take on a 32 bit machine.