A 32 bit host Windows application setups shared memory (using memory mapped file / CreateFileMapping() API), and then other 32 bit client processes use this shared memory to communicate with each other.
I am planning to port the host application to 64 bit platform and once it is ready, I intend that both 32 bit and 64 bit client processes should be able to use the shared memory setup by the main 64 bit host application.
The original code written for host x32 application uses "size_t" almost everywhere, since this differs from 4 bytes to 8 bytes as we move from x32 to x64, I am looking for replacing it.
I intend to replace "size_t" by "unsigned long long", so that its size will be same on 32 bit & 64 bit.
Can you please suggest me better alternative?
Also, will the use of "unsigned long long" have performance impact on x32 app .. i guess yes?
Research Done -
Found very useful articles -
a) 20 issue in porting from 32 bit to 64 bit (www.viva64.com)
b) No way to restrict/change "size_t" on x64 platform to 4 bytes using compiler flags or any hooks/crooks since it is typedef
The use of 64-bit variables will typically slow down the 32-bit application.
However: since it typically isn't possible to locate the shared memory at the same virtual address in all of your processes, you are presumably using addresses relative to the beginning of the shared memory block; also, since your application is going to support 32-bit processes the shared memory block is presumably going to be less than 4GB in size. So why not use unsigned int?
Whatever type you choose, it would be sensible to use typedef to give it a meaningful name, e.g., shared_memory_address, and use that name consistently. That way you can change the underlying type later on if you need to.
Related
So this document says that running 64 bit Windows gives you 64 bit atomicity: http://msdn.microsoft.com/en-us/library/windows/desktop/ms684122%28v=vs.85%29.aspx
This post indicates that you have to run a 64 bit app to gain 64 bit atomicity: atomicity in 32/64 bit
I'm developing a Win32 console app. So, if I understand correctly, I have to use 32 bit types to get atomicity, right? I cannot assume a 64 bit type has atomic writes/reads?
In a 64-bit app, 64-bit read/write operations can be automatic because the compiler can take advantage of the x64 extended instruction set which has atomic 64-bit read/write operations.
In 32-bit code on a 64-bit OS & hardware, on the other hand, if the app needs to read/write 64-bit data, there are no 64-bit read/write instructions available so the compiler has to generate (at least) two read/write operations. Because the OS could pre-empt the process in between the two reads/writes, you should employ the Interlockedxxx API's.
Note: You can build 64-bit Win32 console apps if you want. In this case, the compiler can generate code that uses the 64-bit read/write op's.
Of course, since your code might want to read/write data types that are larger than 64-bit (e.g. SSE2/3, AVX, etc), thus requiring multiple read/write op's, you should employ the appropriate intrinsics to ensure that the required operations are made atomic.
Rather than assume that you can rely on the compiler to do the right thing for your specific scenario, I'd recommend employing the necessary API's and intrinsics to explicitly declare which of your operations MUST be atomic. This way you won't care if your code is compiled for x64, x64, ARM, MIPS, etc.
I just got a new 64 bit computer and I am still trying to understand the differences between 32 bit and 64 bit. I understand that applications built using 64-bit dependencies can only run on a 64 bit, but applications built with 32-bit dependencies can run on both 32 and 64 bit systems.
However, is there any other differences? I know some programs have two different windows versions you can download, one for 64 bit and one for 32 bit. Why do they provide the two different types? Is there a speed increase for compiling a program with 64 bit dependencies for a program to run on a 64 bit system?
Besides from speed some programs do not have a choice. They need to match the bitness of the host operating system exactly.
A good example is TortoiseSVN which installs a shell extension. Shell extensions are DLLs loaded into other processes. So TortoiseSVN must provide a 64 bit DLL on x64 system if it wants to provide a shell extension.
Drivers are a second example of this.
When it comes to speed there is a difference of course. It depends very much on what the program does. If a program does not require high performance providing an x64 executable does not help and is a waste of time for both developers just as for users.
Here is what causes a performance difference for the case 32 bit on 64 bit OS:
Higher kernel-call cost because of the mode switch
More registers, higher code size: For calculation intensive programs this works vastly in favor of native 64 bit apps. For other types of app, or for business style web applications the bigger code and pointer sizes might cause a net loss (so 64 bit slower than 32 bit!)
Access to more than 3GB of memory. This obviously comes into play rarely as of 2012. Think of databases, Photoshop, ...
The speed increase depends on what the application does, as usr said. But fundamentally a 64-bit CPU is able to process twice as much data as a 32-bit one in the same amount of time; but the machine instructions to do so are specific to the 64-bit processor, so existing native programs have to be recompiled (at the very least) to show the benefit.
In the case of x86_64, memory pointers are also now 64 bits wide instead of 32, meaning much more memory can be addressed by programs. However, the code takes up roughly twice as much space in disk and on memory; if you're on a 64-bit system that's starved for memory, swapping can potentially be a lot worse.
Performance will roughly the same for most languages.
The main reason to go for 64 bit is the addressable memory. A32 bit process has only access to 2 GB of data, but a 64 bit process can access 18 quintillion Bytes (that is way more than the amount of RAM you can put in your machine).
I hope someone with a bit of knowledge can clear this up. There's many discussions about the reasons to run a 64-bit OS (e.g. Windows 7 x64), but many people seem to think that their old x86 apps will be able to take advantage of any RAM greater than 3.5GB.
As I understand it, though, x86 apps cannot address memory that high... unless they've been specifically programmed to (which very few will have).
Can someone knowledgeable clear this up for me, once and for all? Can 32-bit apps take advantage of a system running 8GB of RAM?
E.g. If a user decided (for whatever reason) to run several x86 apps at once, filling the RAM as much as possible, would the extra addressable memory available in Windows 7 x64 be used?
Thanks!
On a 64 bit system, 32 bit applications are able to use the full 4GB virtual address space, minus about 64K. A default 32 bit windows system will only allow a 32 bit process to use 2 GB of virtual address space. By specially configuring the OS it's possible to push that limit up to 3 GB, but it's still not as good as what you would get on a 64 bit version of windows.
If you have 8GB of ram, that 8 GB can be divided up between multiple 32 bit processes, and the entire 8 GB will be utilized if necessary. However, no single 32 bit process will be allocated more than 4 GB of memory.
Although i don't have sources to cite, but from my knowledge: 32bit app will not be able to address more than 4GB of memory itself, unless it uses some tricks(that is very unlikely), but if you have some 32bit apps running at the same time, they can all have 4GB each, and thus two 32bit apps should be able to use all 8GB of memory. Though I'm not 100% sure.
Yes. x86 apps cannot use more than 2GB of memory at once without special tricks, but they can use any memory available.
Adding to the other (correct) answers:
Instead of the term "application" the word "process" should be used. Applications often consist of multiple processes whereas the limits discussed here apply to single processes.
Thus applications benefit from x64 that either are linked with the LARGEADDRESSAWARE flag (they can use 4 GB instead of 2 GB) or that share the load between multiple processes.
32-bit processes can work with more than 4 GB RAM even on 32-bit systems by using AWE. But a 32-bit process can only ever use 2 GB at once (4 GB with LARGEADDRESSAWARE on 64 bit respectively). AWE is primarily used by databases where it is essential for performance that the entire database fit into RAM. It works by providing a 2 GB window into a larger chunk of memory.
Here are some articles for further reading:
Windows x64 – All the Same Yet Very Different, Part 1: Virtual Memory
Windows x64 – All the Same Yet Very Different, Part 2: Kernel Memory, /3GB, PTEs, (Non-) Paged Pool
x64? My Terminal Servers Run Just Fine With 32 Bits and 8/12/16 GB RAM!
E.g. If a user decided (for whatever
reason) to run several x86 apps at
once, filling the RAM as much as
possible, would the extra addressable
memory available in Windows 7 x64 be
used?
The answer is yes. That's one of the benefits a virtual address space gives us--the ability for each process to appear (to the process) as though it's executing in a linear address space that starts at 0 and goes up from there.
As far as each of the 32-bit applications is concerned, it has its own address space from 0 to 2 gigabytes (without special tricks). The operating system handles the virtual-to-physical address translation.
I have some matlab code that will only run on 32 bit windows, but I need atleast 6 gb of ram to run it. In my lab the only machine that has 6gb ram is running 64 bit windows, is there some way to run this code on here?
I am thinking of emulating a 32 bit windows and running it on that, will that work?
Your 32 bit application will only be able to address 4 GB when running as a 32 bit process on Windows 64 bit. In order to address more memory you need a 64 bit application.
I've recently been engaged in moving several of our large Matlab codes from 32-bit Windows XP to 64-bit Windows XP. Unlike one or two of the other answerers here, I think 6GB is a perfectly reasonable memory requirement these days; the largest desktop we use here has dual quad-core processors and 32GB RAM. Some of my scientist colleagues want more, but then they always want more.
I'd go further and advise against trying to rewrite your application to use less RAM; that would be heading in the wrong direction. It will be cheaper and quicker and generate fewer errors if you port your code to the 64-bit machine. To those who would argue for refactoring to use less RAM I would say At what point in the near future will you face up to the realities of the market and recognise that 64-bit computing is here to stay and that RAM is quite cheap, especially compared to the expense of developers time ?
Forget trying to use 6GB RAM on a 32-bit machine; as others have told you, Matlab and Windows won't do it. But, as others have remarked, Matlab is indifferent to the number of bits, so a pure Matlab code which works on a 32-bit release will work on the 64-bit version of the same release. If you move from an older 32-bit release to a newer 64-bit release you may find some Matlab functions have been retired, but you'd have the same issues moving to the newer 32-bit release anyway.
If you have a code which will only run on 32-bit Windows then I suspect you are using MEX files. These will have to be recompiled for execution on a 64-bit machine. Again, be careful to watch out for functionality which has been retired if you are moving between Matlab releases as you port. The other chief gotcha, in my experience, is false assumptions that you might make about the sizes of certain fundamental data types. Neither the Fortran nor the C standards (no C++ experience to speak of) actually specify the size in bytes of default INTEGER (for Fortran) or int (for C -- I think, not much of a C programmer). Pointer sizes are probably only vaguely specified too. What this means is that you may have to either (a) change a lot of variable declarations in your code or (b) use compiler directives/flags to specify default sizes for INTEGERS, REALS, ints and possibly others. (b) is easier, (a) is probably preferable for future maintainability.
You cannot access 6GB of RAM from a 32-bit application. This leaves you two options:
Either you optimize your code to use less RAM, or you make your Matlab program 64-bit compatible.
For the first option, the fine folks at stackoverflow can help you.
For the second option: No part of Matlab that I'm aware of is 32-bit only - unless you have a 32-bit Matlab version. However, if you have an active maintenance contract, you can download and install the 64-bit version of Matlab.
If your code calls mex-functions that are compiled for 32-bit Windows, you can try and recompile them for 64-bit Windows.
64 bit Windows will run 32 bit Windows programs just fine. But the 32 bit programs won't be able to access 6 Gb or memory, so I'm not sure that help you.
I'm pretty sure most versions of 32 bit Windows(as in, everything but Server editions) limit a single process to 2GB by default and to 3GB by changing a setting... though it is technically possible to access 16G of RAM from a Pentium Pro(and higher) CPU in 32 bit mode, I don't think anything but Windows Server takes advantage of it
and for your Matlab program to take advantage of it, you'd have to do some interesting multi-process work to manually page between 3G memory banks
You can not use 6 GB with 32Bit addresses. A 32 Bit address can only contain values from
0 to 0xffffffff (== 4294967295 == 4GB)
Such addresses are used by and are essential for every 32 Bit program, regardless if you explicitly use pointers or not.
(Although there were/are some very exotic exceptions - but those dont apply here).
From what I understand, a 32-bit process can only access 2 GB of memory on 32-bit Windows without the /3GB switch, and that some of that memory is taken up by the OS for its own diabolical reasons. This seems to mesh with my experience as we have an app that crashes when it reaches around 1.2 - 1.5 GB of RAM without memory exceptions, even though there is still plenty of memory available.
Would moving this 32-bit application to 64-bit Windows allowing it accesses more than 1.5 GB it can now? Would the application itself have to be upgraded to 64-bit?
Newer versions of Visual Studio have a new flag which make 32-bit apps "big address space aware". Basically what it does is say that if it's loaded on a 64-bit version of windows, then it will get 4GB (the limit of 32-bit pointers). This is certainly better than the 2 or 3 GB you get on 32-bit versions of windows. See http://msdn.microsoft.com/en-us/library/aa366778.aspx:
Most notably it says:
Limits on memory and address space
vary by platform, operating system,
and by whether the
IMAGE_FILE_LARGE_ADDRESS_AWARE value
of the LOADED_IMAGE structure and
4-gigabyte tuning (4GT) are in use.
IMAGE_FILE_LARGE_ADDRESS_AWARE is set
or cleared by using the
/LARGEADDRESSAWARE linker option.
Also see: http://msdn.microsoft.com/en-us/library/wz223b1z.aspx
Yes, under the right circumstances, a 32-bit process on Windows can access a full 4GB of memory, rather than the 2Gb it's normally limited to.
For this to work, you need the following:
The app must be running on a 64-bit OS
The app must be compiled with the /LARGEADDRESSAWARE flag.
The app should be tested to make sure it actually works properly in this case. ;) (specifically, code that relies on all pointers pointing to addresses below the 2GB boundary will obviously not work here)
Your app will be limited by the pointer size, in your example 32 bits.
If your app was to access more memory then you would need some sort of segmented memory architecture like we had in the 16 bit days where apps used 16bit pointers and offsets to access the full 32bit memory space.
WOW64 allows using 32-bit Windows application on 64-bit Windows, translating 32-bit pointers to real 64-bit pointers. And actually 32-bit addressing should allow accessing 4GB of memory.