In the PE format we have Import Table Directory (accessed by IMAGE_DIRECTORY_ENTRY_IMPORT) and IAT Directory (accessed by IMAGE_DIRECTORY_ENTRY_IAT)
both are part of the Optional Header Data Directory.
Using the Import Table, the loader dynamically loads and resolves necessary libraries and functions.
This is done by iterating through the Import Address Table RVA (Thunk Table) which is part of the Import Table.
So, if we use the import directory for import resolution what do we need IAT Directory for ?
I've been reading the Microsoft PE specification but couldn't find an answer. Also, there are some questions in SO but most of them use IAT to refer to the Thunk Table and not the IAT Directory.
Thanks
EDIT
I think that there is a confusion between Import Address Table which is a field in the Import Table Directory and the Import Address Table which is called IAT Directory.
My question is regarding the IAT Directory.
Thanks again
It is described well in the PE specification you linked, chapter 5.4.4. They are the same tables:
The structure and content of the import address table are identical to those of the import lookup table, until the file is bound. During binding, the entries in the import address table are overwritten with the 32-bit (for PE32) or 64-bit (for PE32+) addresses of the symbols that are being imported. These addresses are the actual memory addresses of the symbols, although technically they are still called “virtual addresses.” The loader typically processes the binding
Perhaps it is important to explain why it is done this ways. A PE file is loaded into a process by mapping it directly to memory. The underlying operating system primitive is a memory mapped file. This provides several important optimizations:
the memory used by the executable doesn't have to be backed by the paging file. If the operating system needs RAM for another process then the pages mapped to the executable can simply be discarded. To be reloaded again from the PE file when the process generates a page fault.
the RAM used by a process for its executable code can be shared by any instance of the process. In other words, when you start Notepad.exe multiple times then there's only one copy of the code in RAM. Every process shares the same pages. This is most of all important for DLLs, particularly the operating system DLLs that are used in every process, like ntdll.dll, kernel32.dll and user32.dll (etcetera).
When the loader fills in the IAT with the actual addresses of the imported functions then the operating system remaps the pages for the IAT and has them backed by the paging file. So every process can have its own set of imported addresses. The rest of the pages, containing code and the import table, are still shared.
According to the documentation for PE the IAT / IMAGE_DIRECTORY_ENTRY_IAT seems to be used for delayed loading of DLL
https://learn.microsoft.com/en-us/windows/desktop/Debug/pe-format#delay-import-address-table
Nobody answers your question here. The reason is that IMAGE_DIRECTORY_ENTRY_IAT is practially undocumented.
I studied the code of ReactOS where they are using this directory to understand how it works. Then I wrote my own code to confirm my theory. Here are my results.
I will explain it based on the example of the 32 bit Calc.exe from Windows XP SP3.
When you list all directories of Calc.exe you get:
0 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_EXPORT
1 VirtAddr: 00012B80 Size: 0000008C IMAGE_DIRECTORY_ENTRY_IMPORT
2 VirtAddr: 00016000 Size: 00008A5C IMAGE_DIRECTORY_ENTRY_RESOURCE
3 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_EXCEPTION
4 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_SECURITY
5 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_BASERELOC
6 VirtAddr: 00001240 Size: 0000001C IMAGE_DIRECTORY_ENTRY_DEBUG
7 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_ARCHITECTURE
8 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_GLOBALPTR
9 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_TLS
10 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG
11 VirtAddr: 00000260 Size: 00000080 IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT
12 VirtAddr: 00001000 Size: 00000228 IMAGE_DIRECTORY_ENTRY_IAT
13 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT
14 VirtAddr: 00000000 Size: 00000000 IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR
etc..
You see that there are 3 directories related to import:
IMAGE_DIRECTORY_ENTRY_IAT
IMAGE_DIRECTORY_ENTRY_IMPORT
IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT
As you see above the IAT directory starts at offset 0x1000 and has 0x228 bytes.
It merely consists of function pointers which point into the imported DLLs.
Each pointer has 4 Byte in a 32 bit process so there are (0x228 = 552) / 4 = 138 entries. I wrote a loop to list them:
Addr 01001000 --> function 77DA22EA
Addr 01001004 --> function 77DA23D7
Addr 01001008 --> function 77DA189A
Addr 0100100C --> function 00000000
Addr 01001010 --> function 77C41E2E
Addr 01001014 --> function 77C41D83
Addr 01001018 --> function 77C41EFF
Addr 0100101C --> function 00000000
Addr 01001020 --> function 77E59F93
Addr 01001024 --> function 77E605D8
Addr 01001028 --> function 77E5A5FD
Addr 0100102C --> function 77E7A9AD
Addr 01001030 --> function 77E536A3
Addr 01001034 --> function 77E53803
Addr 01001038 --> function 77E4E341
Addr 0100103C --> function 77E58D60
Addr 01001040 --> function 77E41BE6
Addr 01001044 --> function 77E52A2B
Addr 01001048 --> function 77E4177A
Addr 0100104C --> function 77E4C879
Addr 01001050 --> function 77E51B14
Addr 01001054 --> function 77E530C1
Addr 01001058 --> function 77E5AC37
Addr 0100105C --> function 77E54A69
etc...
This is the so called Import Address Table where the code in the module looks up the calls to external functions. If your code calls GetLastError() it looks here where in Kernel32.dll this function is.
The 00000000 in this list mark that another DLL is following. This list alone is useless because you don't know what is the meaning of these addresses.
IMPORTANT: Every executable has an IAT. But not every executable exposes it with IMAGE_DIRECTORY_ENTRY_IAT.
In case that the executable has additionally an IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT directory, the IAT comes with preloaded values. The bound directory consists of a chain of IMAGE_BOUND_IMPORT_DESCRIPTOR and IMAGE_BOUND_FORWARDER_REF entries. This is the content:
Bound DLL: SHELL32.dll Timestamp: 3B842039
Bound DLL: msvcrt.dll Timestamp: 3B842039
Bound DLL: ADVAPI32.dll Timestamp: 3B842038
Bound DLL: KERNEL32.dll Timestamp: 3B842038
Bound DLL: GDI32.dll Timestamp: 3B842039
Bound DLL: USER32.dll Timestamp: 3B842038
The Windows loader checks if the timestamps in this directory when Calc.exe was compiled is identical to the timestamp of the DLL on disk. In that specific case the calls to GetProcAddress() are not necessary to resolve the imports and the preloaded entry points in the IAT can be used. This is for speed optimization.
Some DLLs forward calls to another DLL. For example Kernel32.dll has some calls which go directly into NtDll.dll. In this case there is a forwarder entry (IMAGE_BOUND_FORWARDER_REF) which allows to check also the timestamp of the forwarded DLL.
In case the DLL has been loaded to another base address the delta must be added to the addresses in the IAT. But I don't know where the original base address is stored in the image? When Microsoft introduced ASLR (Address Space Layout Randomization) in 2004 it became the rule that all DLLs are loaded to a random base address.
In my case the preloaded IAT is completely useless because my Calc.exe on XP is from 2004 while the imported DLLs are from 2008. So all entries must be resolved anew. Once you install a Windows Update which updates some DLLs on your system the bound imports do not work anymore.
And the IMAGE_DIRECTORY_ENTRY_IMPORT points to the exactly same addresses in the IAT (via IMAGE_IMPORT_DESCRIPTOR->FirstThunk). I printed the preloaded IAT and the values which overwrite them with the result from GetProcAddress():
LoadLibrary(ADVAPI32.dll) --> HMODULE 77DA0000, Timestamp 4802BE8C
IAT 01001000 'RegOpenKeyExA' --> Value 77DA22EA updated 77DA7842
IAT 01001004 'RegQueryValueExA' --> Value 77DA23D7 updated 77DA7AAB
IAT 01001008 'RegCloseKey' --> Value 77DA189A updated 77DA6C17
LoadLibrary(GDI32.dll) --> HMODULE 77EF0000, Timestamp 4802BE8A
IAT 01001010 'SetBkColor' --> Value 77C41E2E updated 77EF5E29
IAT 01001014 'SetTextColor' --> Value 77C41D83 updated 77EF5D77
IAT 01001018 'SetBkMode' --> Value 77C41EFF updated 77EF5EDB
etc...
As you see the bound imports are quite useless in all these cases. The new values (at the right) are not just a constant offset from the preloaded values. They point to a different DLL. As Advapi32.dll is not the same anymore as when Calc.exe has been compiled they must all be resolved anew.
In the Calc.exe from Windows 7 I found the same scheme. But on Windows 10 it is different. It seems they store an offset into the DLL rather than the entry point address?
SUMMARY: If you write your own DLL loader you can completely ignore IMAGE_DIRECTORY_ENTRY_IAT and IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT. The tiny speed optimization by not calling GetProcAddress() can be neglected on todays fast CPU's.
The following article and it's first part is a good source for information on PE executable files:
From the March 2002 issue of MSDN Magazine: Inside Windows
An In-Depth Look into the Win32 Portable Executable File Format, Part 2
IMAGE_DIRECTORY_ENTRY_IMPORT eventually leads to multiple IAT thunks, which are stored in a memory region, which starts at [IMAGE_DIRECTORY_ENTRY_IAT].VirtualAddress, and has size [IMAGE_DIRECTORY_ENTRY_IAT].Size.
I guess it is useful when all the sections are loaded by default as read-only, and you can use IMAGE_DIRECTORY_ENTRY_IAT to make the IAT (but not the ILT) thunks writable.
Related
The question is about loading portable executable images to a random address.
Let's take kernel32.dll as an example, loaded at 0x75A00000.
I can see that at offset 0x10e15 from the image, there is an assembler instruction, which depends on where the image is located.
address:
75A10E13
bytes:
8B 35 18 03 AE 75
command:
MOV ESI,DWORD PTR DS:[75AE0318]
It turns out that by launching the executable file, we must tell the system that we need to relocation to this address.
The system looks at the relocation table, which is in the executable file, and sees the following:
base relocation table
To get the absolute address of the first element to be moved, I do the following: add the virtual address to the address of the image, and then I add the first element of the block to the resulting number.
0x75A00000 + 0x10000 + 0x3E15 = 75A10E15
it's a good number, but always 0x3000 more than I expect. i just subtract 0x3000 and it works. Please, help me find the answer, where does 0x3000 for x86 come from?
Relocation in Portable Executables were resolved when the file was linked. The base relocation table, which you are referring, has a different function: it is used by Windows loader when the PE could not be loaded at the prefered ImageBase address specified by the linker, usually 0x0040_0000.
Dynamically Loaded Libraries shipped with MS Windows are linked to ImageBase addresses different for each core DLL and chosen not to colide with one another, so an executable which imports usual combination of libraries doesn't have to relocate them.
You misinterpreted the format of base relocation section .reloc.
Those 16bit words TypeOrOffset which follow PageRVA and BlockSize have their Base Relocation Type encoded in four most significant bits.
For instance the first TypeOrOffset entry in you dump 0x3E15 has type IMAGE_REL_BASED_HIGHLOW (3) and offset 0x0E15, which is the number to be added to PageRVA.
I noticed a potential bug in some code i'm writing.
I though that if I used mov ax, seg segment_name, the program might be non-portable and only work on one machine in a specific configuration since the load location can vary from machine to machine.
So I decided to disassemble a program containing just that one instruction on two different machines running DOS and I found that the problem was magically solved.
Output of debug on machine one: 0C7A:014C B8BB0C MOV AX,0CBB
Output of debug on machine two: 06CA:014C B80B07 MOV AX,070B
After hex dumping the program I found that the unaltered bytes are actually B84200.
Manually inserting those bytes back into the program results in mov ax, 0042
So does the PE format store references to those instructions and update them at runtime?
As Peter Cordes noted, MS-DOS doesn't use the PECOFF executable format that Windows uses. It has it's own "MZ" executable format, named after the first two bytes of the executable that identify as being in this format.
The MZ format supports the use of multiple segments through a relocation table containing relocations. These relocations are just simple segment:offset values that indicate the location of 16-bit segment values that need to be adjusted based on where the executable was loaded in memory. MS-DOS performs these adjustments by simply adding the actual load segment of the program to the value contained in the executable. This means that without relocations applied the executable would only work if loaded at segment 0, which happens to be impossible.
Note this isn't just necessary for a program to work on multiple machines, it's also necessary for the same program to work reliably on the same machine. The load address can change based on what various configuration details, was well as other programs and drivers that have already been loaded in memory, so the load address of an MS-DOS executable is essentially unpredictable.
Working backwards from your example, we can tell where your example program was loaded into memory on both machines. Since 0042h was relocated into 0CBBh on the first machine and into 070Bh on the second machine, we know MS-DOS loaded your program on the two machines at segments 0C79h and 06C9h respectively:
0CBB - 0042 = 0C79
070B - 0042 = 06C9
From that we can determine that your example executable has the entry 0001:014D, or equivalent segment:offset value, in it's relocation table:
0C7A:014D - 0C79:0000 = 0001:014D
06CA:014D - 06C9:0000 = 0001:014D
This entry indicates the unrelocated location of the 16-bit immediate operand of the mov ax, seg segname instruction that needs adjusting.
Trying to understand how the relocation happens in the situation below
I have a shared library libbigshr.so which uses another shared library libfunlib.so. In the latter I have declared this global variable foo. To compile the former I had to do forward declaration as extern int foo.
.rel.dyn at libbigshr.so
Offset Info Type Sym.Value Sym. Name
000005a5 00000401 R_386_32 00000000 foo
000005ab 00000401 R_386_32 00000000 foo
000005c7 00000401 R_386_32 00000000 foo
.rel.dyn at libfunlib.so
000005a5 00000901 R_386_32 00002010 foo
In the libfunlib the translation offset is proper value (0x2010), therefore I don't have problem. But wanted to know the how the correct addresses were inserted in libbigshr. I can understand, that once a variable has been allocated the memory and its location is identified the same can be used everywhere else. But I am interested in the procedure of doing so.
Due to my ignorance my question may not have sufficient data to answer it - so please let me know and I will furnish more details.
The correct address is generated by relocation processing at run time, by the dynamic linker. For example, in glibc, R_386_32 is processed in the i386 version of the elf_machine_rel function in the file sysdeps/i386/dl-machine.h. The glibc wiki has an overview of process startup. For details, see the references in the Linux Standard Base, particular those concerning the System V Application Binary Interface. For machine-specific information, H.J. Lu maintains a set of x86 ABI documents.
I believe that the MODULE_VERSION does not work if the driver is statically compiled into the kernel. The version number was no where to be seen in the sysfs. the modinfo does not work as its not a loaded module.
So Whats the best way for to wither get the MODULE_VERSION of this driver or encode version number in the driver. Is there a standard way of doing this or should I simply use sysfs?
First of all, there is no much sense to have a module version for in tree modules. Otherwise it is kept is special section called __modver.
$ objdump -h ~/prj/TMP/out/mfld/vmlinux -j __modver
/home/andy/prj/TMP/out/mfld/vmlinux: file format elf32-i386
Sections:
Idx Name Size VMA LMA File off Algn
12 __modver 00000c40 c1a003c0 01a003c0 00a013c0 2**2
CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
It contains pointers to corresponding structures defined in include/linux/module.h in macro MODULE_VERSION.
I have attempted the following test to see if the .data section gets loaded into memory when the program is executed:
global _start
section .data
arr times 99999999 DB 0xAF
section .text
_start:
jmp _start ; prevent process from terminating
Assemble and link:
nasm -f win32 D:\file.asm
link D:\file.obj /OUT:D:\file.exe /ENTRY:start /SUBSYSTEM:CONSOLE
I have executed the program, and the result was the following:
As you can see the program only occupied 276 KB of memory while it has an array with a size of 99999999 bytes!
The paging model on most systems will cause the pages comprising the sections of the binary not requiring some kind of dynamic linking to only be loaded when they are accessed - Windows is no exception. So, the .data section is memory-mapped as a binary file to your process memory space, but is not actually swapped in until you need it. The process monitor only reports the memory actually in by default, although you can configure the columns to show all of the memory in the image, also. There may also be compiler options you can use to change the paging behavior, and you can always remap the memory manually (perhaps locking it in) if you need.