Shared memory containers between a 32 and 64 bit processes - shared-memory

Is it possible to share a stl container (for e.g std::map) via shared memory between a 32 and 64 bit process ?

Related

Which Kernel file is responsible for setting up 32 bit or 64 bit mode

I'm going through Linux Kernel code and interested in knowing which file will be responsible for setting long mode to support 64 bit or which will be telling the OS going to boot would be 32 bit or 64 bit??
Is it bootloader code responsible to set this mode up??
The boot loader (usually GRUB) is booting some Linux kernel. This kernel may be a 32 bits kernel (then able only to run 32 bits programs) or a 64 bits kernel.
64 bits Linux kernels are able to run thru execve(2) syscalls either 32 bits binary executable program (in a 32 bits process) or 64 bits program
So most kernels are 64 bits and can run both 32 or 64 bits ELF executables.
I guess that, depending how they are booted, Linux kernels start in 32 bits mode (at least when booted by GRUB).

Porting - Shared Memory x32 & x64 processes

A 32 bit host Windows application setups shared memory (using memory mapped file / CreateFileMapping() API), and then other 32 bit client processes use this shared memory to communicate with each other.
I am planning to port the host application to 64 bit platform and once it is ready, I intend that both 32 bit and 64 bit client processes should be able to use the shared memory setup by the main 64 bit host application.
The original code written for host x32 application uses "size_t" almost everywhere, since this differs from 4 bytes to 8 bytes as we move from x32 to x64, I am looking for replacing it.
I intend to replace "size_t" by "unsigned long long", so that its size will be same on 32 bit & 64 bit.
Can you please suggest me better alternative?
Also, will the use of "unsigned long long" have performance impact on x32 app .. i guess yes?
Research Done -
Found very useful articles -
a) 20 issue in porting from 32 bit to 64 bit (www.viva64.com)
b) No way to restrict/change "size_t" on x64 platform to 4 bytes using compiler flags or any hooks/crooks since it is typedef
The use of 64-bit variables will typically slow down the 32-bit application.
However: since it typically isn't possible to locate the shared memory at the same virtual address in all of your processes, you are presumably using addresses relative to the beginning of the shared memory block; also, since your application is going to support 32-bit processes the shared memory block is presumably going to be less than 4GB in size. So why not use unsigned int?
Whatever type you choose, it would be sensible to use typedef to give it a meaningful name, e.g., shared_memory_address, and use that name consistently. That way you can change the underlying type later on if you need to.

Behavior of 32 bit applications in 64 bit Windows (memory)

I am developing a 32 bit application in .NET that for various reasons cannot be compiled as a 64 bit application.
I need to run many of these concurrently and they use a lot of memory. I want to load up a Windows 7 box with tonnes of memory and consequently would like to use the 64 bit version of Windows 7 so that we can put many gigabytes of RAM on those boxes.
My question is this: The maximum memory used by each instance of my app is ~500mb. In Windows 7 64bit, these 32-bit applications will run (I assume) using the WOW64 emulation layer in Windows. As I begin to run more and more of these instances concurrently, will they all be stuck running in the bottom 2gb of ram, or will Windows allocate memory for them using all of the higher-address range of memory possible within 64-bit Windows? Is the addressable-memory limitation of 32 bit software only a per-instance limitation in this case, or will all the instances be limited to the bottom 2gb of ram?
You are confusing memory (physical address space) with virtual address space. You can put more than 4GB of memory into a 32-bit system; you don't need to move to 64-bit to gain physical address space. Each process gets its own virtual address space, so each one will get its own 2GB of user-mode address space to play with. (Or 3GB if /3GB or 4GB if running on WOW64 with /LARGEADDRESSAWARE.)

32 bit applications on 64 bit OS ( windows )

I need some help understanding how 32 bit applications use memory on a 64 bit OS.
A 32 bit application can use 2 gb of memory on 64 bit OS, correct?
Does this mean that 3 32 bit applications running in parrallel could address 6 gb of memory...
Or do the 3 32 bit applications have to share the 2-4 gb of 32 bit memory that the os has?
Likewise, If I have a webservice that is compiled as 32 bits, running under IIS on a 64 bit machine. As long as a single request to that webservice always stays under 2gb of memory usage, is there any point in recompiling to 64 bit? My theory is that IIS creates a new process for each request, so the whole pool of processes will be able to make use of all the memory the 64bit machine has , 8 or 15 or 20 gig or whatever.
Let me know your thoughts, thanks
Yes, the total usage of all the 32-bit programs can exceed 2 GB. So yes you can have a bunch of 32-bit processes using all the memory in a 64-bit machine.
Actually, there's a compiler option that lets 32-bit programs use up to 3GB in Windows.
If performance isn't important, then there isn't much of a reason to use 64-bit.

How much memory can a 32 bit process access on a 64 bit operating system?

On Windows, under normal circumstances a 32 bit process can only access 2GB of RAM (or 3GB with a special switch in the boot.ini file). When running a 32 bit process on a 64 bit operating system, how much memory is available? Are there any special switches or settings that can change this?
2 GB by default. If the application is large address space aware (linked with /LARGEADDRESSAWARE), it gets 4 GB (not 3 GB, see http://msdn.microsoft.com/en-us/library/aa366778.aspx)
They're still limited to 2 GB since many application depends on the top bit of pointers to be zero.
4 GB minus what is in use by the system if you link with /LARGEADDRESSAWARE.
Of course, you should be even more careful with pointer arithmetic if you set that flag.
Nobody seems to touch upon the fact that if you have many different 32-bit applications, the wow64 subsystem can map them anywhere in memory above 4G, so on a 64-bit windows with sufficient memory, you can run many more 32-bit applications than on a native 32-bit system.
A 32-bit process is still limited to the same constraints in a 64-bit OS. The issue is that memory pointers are only 32-bits wide, so the program can't assign/resolve any memory address larger than 32 bits.
An single 32-bit process under a 64-bit OS is limited to 2Gb. But if it is compiled to an EXE file with IMAGE_FILE_LARGE_ADDRESS_AWARE bit set, it then has a limit of 4 GB, not 2Gb - see https://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx
The things you hear about special boot flags, 3 GB, /3GB switches, or /userva are all about 32-bit operating systems and do not apply on 64-bit Windows.
See https://msdn.microsoft.com/en-us/library/aa366778(v=vs.85).aspx for more details.
As about the 32-bit operating systems, contrary to the belief, there is no physical limit of 4GB for 32-bit operating systems. For example, 32-bit Server Operating Systems like Microsoft Windows Server 2008 32-bit can access up to 64 GB (Windows Server 2008 Enterprise and Datacenter editions) – by means of Physical Address Extension (PAE), which was first introduced by Intel in the Pentium Pro, and later by AMD in the Athlon processor - it defines a page table hierarchy of three levels, with table entries of 64 bits each instead of 32, allowing these CPUs to directly access a physical address space larger than 4 gigabytes – so theoretically, a 32-bit OS can access 2^64 bytes theoretically, or 17,179,869,184 gigabytes, but the segment is limited by 4GB. However, due to marketing reasons, Microsoft have limited maximum accessible memory on non-server operating systems to just 4GB, or, even, 3GB effectively. Thus, a single process can access more than 4GB on a 32-bit OS - and Microsoft SQL server is an example.
32-bit processes under 64-bit Windows do not have any disadvantage comparing to 64-bit processes in using shared kernel's virtual address space (also called system space). All processes, be it 64-bit or 32-bit, under 64-bit Windows share the same 64-bit system space.
Given the fact that the system space is shared across all processes, on 32-bit Windows, processes that create large amount of handles (like threads, semaphores, files, etc.) consume system space by kernel objects and can run out of memory even if you have lot of memory available in total. In contrast, on 64-bit Windows, the kernel space is 64-bit and is not limited by 4 GB. All system calls made by 32-bit applications are converted to native 64-bit calls in the user mode.
You've got the same basic restriction when running a 32bit process under Win64. Your app runs in a 32 but subsystem which does its best to look like Win32, and this will include the memory restrictions for your process (lower 2GB for you, upper 2GB for the OS)
The limit is not 2g or 3gb its 4gb for 32bit.
The reason people think its 3gb is that the OS shows 3gb free when they really have 4gb of system ram.
Its total RAM of 4gb. So if you have a 1 gb video card that counts as part of the total ram viewed by the 32bit OS.
4Gig not 3 not 2 got it?

Resources