Is Windows Portable Executables are really portable across machine architectures? If so how it works? If not then what does "Portable Executable" mean or which part of executable is portable?
Thanks, Siva Chandran
The executables aren't themselves portable. PE format is "portable" in the sense that executables for different architectures use the same PE format, but the executable code within a PE file is specific to a single processor architecture.
In practice this means that a lot of the same compiler and linker code can be reused for different architectures, and that tools for examining executables can (to some extent) work for "foreign" executables.
(I'm talking about native executables here - .NET assemblies also use PE format and can be truly portable.)
From Wikipedia:
"The term "portable" refers to the format's versatility in numerous environments of operating system software architecture."
http://en.wikipedia.org/wiki/Portable_Executable
Well, it is portable in means of that the format can be used for executables on various platforms (SkyOS used it before they switched to ELF). It is not portable in means of platform independent code or that if you produce a PE file on one OS that it runs on another.
PE-files are just containers for binary application data.
They allow to target different CPU architectures (or even non-CPU software architectures like .Net). That is why they are called "portable".
Each binary application image they contain, however is suited for exactly one architecture.
Related
I have a .go file and produced the binary file using go build command from Mac. Is there a way to build a binary file which runs in windows,linux,IOS ?
I am aware we can build binary file for each of them by changing the GOOS,GOARCH params but i would like to have a single go binary file which should run in all the platforms . Please help me out of this.
Thanks in advance
No, it is not at all possible in Go or any other programming language (the executable is necessarily tailored to individual platforms and architectures).
However, to cross-compile, some tools do exist which do the cross compiling for you.
This post helps explain how to cross compile with Golang (which is pretty easy at this point).
There's also a Unix StackExchange question, https://unix.stackexchange.com/a/298283/177527, which explains why different architectures require different binaries:
The reason is because the code is compiled to machine code for a specific architecture, and machine code is very different between most processor families (ARM and x86 for instance are very different).
The binary also depends on the OS, as explained here https://softwareengineering.stackexchange.com/a/251255:
Binary Format: The executable has to conform to a certain binary format, which allows the operating system to correctly load, initialize, and start the program. Windows mainly uses the Portable Executable format, while Linux uses ELF.
System APIs: The program may be using libraries, which have to be present on the executing system. If a program uses functions from Windows APIs, it can't be run on Linux. In the Unix world, the central operating system APIs have been standardized to POSIX: a program using only the POSIX functions will be able to run on any conformant Unix system, such as Mac OS X and Solaris.
For Mac (not Windows), you can associate cross-compilation with a tool like randall77/makefat to generate a "universal binary", which will run on any architecture supported by one of the input executables.
This is currently implemented in goreleaser/goreleaser PR 2572, which means the process would be completely automated.
I'm very intrigued by the fact that Go (since v1.5) has in-built cross compilation options.
But how is it possible to compile for a different OS and architecture?
I mean that would require knowing (and probably behaving like) the target machine language and platform.
I mean that would require knowing (and probably behaving like) the target machine language and platform.
Yes, the Go compiler has to know how the target operating system works, but it doesn't need to behave like the target OS, as the Go compiler will not run the compiled executable binary, it just needs to produce it.
All the Go tools need to know is the binary formats of the different Operating Systems, and OS and architectural details (such as the instruction set, word size, endianness, alignment, available registers etc.; more info on this). And this knowledge is built into the Go tools.
I understand that each CPU/architecture has it's own instruction set, therefore a program(binary) written for a specific CPU cannot run on another. But what i don't really understand is why an executable file (binary like .exe for instance) cannot run on Linux but can run on windows even on the very same machine.
This is a basic question, and the answer i'm expecting is that .exe and other binary formats are probably not Raw machine instructions but they contain some data that is operating system dependent. If this is true, then what this OS dependent data is like? and as an example what is the format of an .exe file and the difference between it and Linux executables?
Is there a source i can get brief and detailed information about this?
In order to do something meaningful, applications will need to interface with the OS. Since system calls and user-space infrastructure look fundamentally different on Windows and Unix/Linux, having different formats for executable programs is the smallest trouble. It's the program logic that would need to be changed.
(You might argue that this is meaningless if you have a program that solely depends on standardized components, for example the C runtime library. This is theoretically true - but irrelevant for most applications since they are forced to use OS-dependent stuff).
The other differences between Windows PE (EXE,DLL,..) files and Linux ELF binaries are related to the different image loaders and some design characteristics of both OSs. For example on Linux a separate program is used to resolve external library imports while this functionality is built-in on Windows. Another example: Linux shared libraries function differently than DLLs on Windows. Not to mention that both formats are optimized to enable the respective OS kernels to load programs as quick as possible.
Emulators like Wine try to fill the gap (and actually prove that the biggest problem is not the binary format but rather the OS interface!).
.exe and other binary formats are [definitely] not Raw machine instructions but they contain some data that is operating system dependent.
what this OS dependent data is like? and as an example what is the format of an .exe file and the difference between it and Linux executables?
Well, I guess Google failed you utterly. .EXE formats are very well-defined by Windows documentation.
http://support.microsoft.com/kb/65122
The Linux ld application loads an executable into memory prior to "exec" to that file. You could read up on ld format or even the famous a.out file.
http://linux.die.net/man/1/ld
http://en.wikipedia.org/wiki/A.out
http://en.wikipedia.org/wiki/Executable
Apart from the executable format that must be recognized by the system loader (i.e. that part of an OS that brings the executable into memory) the real problem is the interface to the OS. You can think of an OS as a kind of API that provides entry points one must call for doing specific things, like for example, writing a character to the console.
These details are usually more or less hidden from the end user, so that you can achieve writing a character to the screen with the same source code in higher level languages. But often, things are more different, like for example the Windowing environment. Not all high level languages provide a windowing layer that abstracts even over those differences.
I can't comment too much on *nix but yes, the code part of the binary is typically happy to run on either environment, but it is the OS that places certain demands on the binary. In windows you should read up on PE Headers.
The second part is simply up to the developer, many times the code part will reference libaries that are OS specific - which is why you can have both portable and non-portable C++ code before being compiled into a binary.
A very naive answer:
Their structure are different because of different process loaders;
The use os-dependent features like syscalls, which vary from OS to OS.
Programs need to know how to invoke operating system services. How this is done depends on the operating system: some use interrupts, some use the x86 lcall instruction, some (notably Windows) have distinguished shared libraries and don't document how to directly invoke services. Old 680x0 Macs and some other 680x0 operating systems used a reserved instruction set area and trapped the resulting "invalid CPU opcode" exception. Moreover, even when the mechanism is the same, the order and argument format of system calls differs between operating systems (and sometimes different versions of the same operating system; see stat() in the Linux kernel for an example of an interface that has changed several times).
There is some ability to deal with other operating systems' conventions: FreeBSD has the "linuxulator" which handles the Linux-specific kernel interface, NetBSD similarly has emulators for the system call formats of other operating systems using the same hardware (say, Ultrix on MIPS or OSF/1 on Alpha), Linux used to have iBCS2 to handle the UnixWare/SCO Unix kernel interface, Wine provides replacement shared libraries and a binary loader for PE-style Windows executables. (I don't recall if Wine also supports OS/2-style LX .exes; it probably does handle original format .exe; and then there's .com which is a raw memory dump with a header slapped on.) Even so, there is always some format that uses different conventions, and sometimes the conventions are similar enough to require hints to the OS as to how to deal with it. (See bless on FreeBSD, for example.)
i am looking for some files like DLLs that can be used on windows as well as Mac OS is there any way of doing this will c++ or any other language that must be higher level than c++
need help regards.
Java seems like an obvious choice. Java's variant of DLL files are JAR files, which can be used on any system that has a Java Runtime Environment. This includes Windows, Mac OS, Linux and various other operating systems.
And, Java is percieved as being "higher level" than C++, although language wars might arise once in a while about topics such as this one.
I know that on Mac OS X, you can combine multiple binaries that target different architectures into a single binary using lipo. I am wondering if there is a similar solution on the Windows side. Thank you.
Unfortunately all the Windows architectures use the same structures in the PE header, so no. It is possible to fuse a Windows executable and a DOS executable though, since DOS uses the MZ header instead.
There's a real fat PE format in Windows 11 on ARM called Arm64X which contains both Arm64 code and Arm64EC code
Fundamentally, an Arm64X binary contains all of the content that would be in separate x64/Arm64EC and Arm64 binaries, but merged into one more efficient file on disk. The built Arm64X binary has two sets of code, entry points, etc., while eliminating redundant parts to save space on disk.
How do Arm64X binaries work?
Arm64EC is a completely new ABI that's incompatible with ARM64, designed for mixing x86-64 and ARM64 code
I think you are reffering to the ADS, that has been added to the NTFS to provide similar functionality as on Mac. Although NTFS supports multiple streams, the PE loader should choose the right one, and I'm not sure if it's implemented.