In the linux kernel 3.16.56, there is a source code including linux/netfilter_ipv4/ipt_ECN.h ( https://elixir.bootlin.com/linux/v3.16.56/source/net/ipv4/netfilter/ipt_ECN.c#L20 )
But there is no ipt_ECN.h in that location. Then, where is ipt_ECN.h?
How the kernel can be compiled?
Thanks in advance.
Given header exists under include/uapi: include/uapi/linux/netfilter_ipv4/ipt_ECN.h.
Directory include/uapi contains header files common both for kernel code and for user programs/libraries. Normally, there are only types definitions, without functions.
Related
I'm using NVRTC to compile a kernel. The relevant API call is:
nvrtcResult nvrtcCreateProgram (
nvrtcProgram* prog,
const char* src,
const char* name,
int numHeaders,
const char** headers,
const char** includeNames )
As you can see, the source is a raw string, and not associated with a file. That means that when you --generate-line-info, you get line numbers, but no related filename. And that means that if you then use, say, NSight Compute - you won't be able to see your kernel source code.
Obviously, neither NSight Compute itself, nor NVRTC itself, can figure out that the raw source is mirrored in some file. But there has to be some way to get around this:
Perhaps I'm missing something in the NVRTC API which can make the source <-> file association?
Perhaps we can manipulate the resulting compiled program (reasonably, not manually, or write-my-own-new-API) to make the association?
Perhaps we can shove the source code into the compiled program somehow?
Here's my initial workaround:
Place your source in a file, say my_kernel.cuh.
Create the string:
#include "my_kernel.cuh"
Compile just this string using NVRTC
Now, NVRTC is able to associate included files' sources with the files, so it's only a stub that will be missing in terms of source<->file association.
Caveat: You will need to be careful about paths - NVRTC's include paths, the working directory from which you invoke your program vs the directory of the source file etc.
It seems NVRTC does provides a default filename, such that if you place your source in the file with that name - NSight Compute may be able to find it.
The name is the one you passed to nvrtcCreateProgram() as the name argument.
So, if your kernel function (i.e. your __global__ function) is in my_kernel.cuh, and you place this file in the working directory of the profile program (which you tell NSight Compute about), or in one of the include directories you built your program with, you'll be able to read your source. If the original file's own directory is also one of the include directories, then you're in luck and you don't even have to make a copy.
First of - Hello and thank you for reading this,
I have one DLL which I do not have the source code but need to add some functionalities into it.
I made up another DLL implementing all these needed functionalities in C - using Visual Studio.
Now I need to insert the generated code from this new DLL into the target DLL (it has to be done at the file level {not at runtime}).
I am probably creating a new PE section on the target DLL and put there all the code/data/rdata from the dll I made up. The problem is that I need somehow to fix the IAT and the relocs relative to this new inserted code on the target DLL.
My question is:
What is the best way to do it?
It would be nice if Visual Studio came up with an option to build using only (mostly) relative addressing - This would save me a lot when dealing with the relocs.
I guess I could encapsulate all my vars and constants into a struct, hopefully MSVC would then only need to relocate the address of this "container" struct and use relative addressing to access its members. But don't know if this is a good idea.
I could even go further and get rid of the IAT by making a function pointer which would dynamically load the needed function module (kind of the Delay Load Module). And again, put this function pointer inside the "container" struct I said before.
The last option I have is to make it all by hand, manually editing the binary in hex... which I really didn`t want to do, because it would take some good time to do it for every single IAT entry and reloc entry. I have already written a PE file encryptor some time ago so I know most of the inner workings and know it can be done, just want to know your thoughts and maybe a tool already exists to help me out?
Any suggestions is highly appreciated!
Thanks again for your time for reading this!
Since you are asking for suggestions, take a look at the very good PORTABLE EXECUTABLE FILE FORMAT – A REVERSE ENGINEER VIEW PDF Document. The Section "Adding Code to a PE File" describes some techniques (and presents Tools) to add code to an existing PE image without having the code of the target image (your scenario) by manipulation the IAT table and Sections tables.
Following the post ,
if I have header file,which has some functions implementations in it and should be included in several kernels(I mean these function are auxilary in all kernels and I don`t want to duplicate the code)
How I make this inclusion - can I keep the functions in header?Will the kernels and the header functions be compiled?
Can you specify (maybe by example) how I use the "-I" option in these case?
I am using VS2010(if its matter at all)
Note:Each kernel runs in different program
Yes, you can use headers in OpenCL for exactly what you are suggesting. Each kernel file will include the header and compile it.
The "-I" option is only used to specify the path for includes. If your includes are in your working directory it is not necessary. Here is an example:
/////////////////////////////////////////////////////////////////
// Load CL file, build CL program object, create CL kernel object
/////////////////////////////////////////////////////////////////
std::string sourceStr = FileToString(params.kernelFile);
cl::Program::Sources sources(1, std::make_pair(sourceStr.c_str(), sourceStr.length()));
cl::Program program = cl::Program(oclHandles.context, sources);
program.build(oclHandles.devices,"-I c:/Includes/");
I'm confusing little in terminology.
A file that is given as input to the linker is called Object File.
The linker produces an Image file, which in turn is used as input by the loader.
I got this from "MS PE & COFF Specification"
Q1. Image file is also referred to as Binary Image, Binary File or just Binary. Right?
Q2. So, according to the above stated terminology, the PE/ELF/COFF are the formats of Image File & not the Object File. right? But http://www.sco.com/developers/gabi/latest/ch4.intro.html says
This chapter describes the object file format, called ELF (Executable and Linking Format). There are three main types of object files.
A relocatable file holds code and data suitable for linking with other
object files to create an executable
or a shared object file.
An executable file holds a program suitable for execution; the
file specifies how exec(BA_OS) creates
a program's process image.
A shared object file holds code and data suitable for linking in two
contexts. First, the link editor [see
ld(BA_OS)] processes the shared object
file with other relocatable and shared
object files to create another object
file. Second, the dynamic linker
combines it with an executable file
and other shared objects to create a
process image.
contradictorily he is saying that both Object File & Image File are ELF formats & He is not at all differentiating between object & image files but referring them commonly as Object files. Isn't it wrong?
Q3. I know that PE is derived from COFF. But why does the Microsoft specifications of PE format is named Microsoft Portable Executable "and Common Object File Format Specification". Do they still support COFF? If they, in which OS? I thought PE completely replaced COFF long ago.
I'm the OP. Every one's answer is a partial answer. So, I'm combining all the other answers with what I learnt to complete the answer.
This is the "Generally" used terminology.
A file that is given as input to the linker (output of assembler) is called Object File or Relocatable File.
The linker produces an Image file, which in turn is used as input by the loader. Now, an Image file can either be an Executable file or Library file. These 'Library files' are of two kinds:
Static Library (*.lib files for windows. *.a for linux)
Shared/Dynamic libraries : DLL ( *.dll on windows) & Shared Object file( *.so in Linux)
The term Binary File / Binary can be used to refer to either an ObjectFile or an ImageFile. Undestand depending up on the context. It is a very general term.
Loader when loads the image file into memory. Then it is called Module (I'm not sure about Linux guys, but windows guys call it Module
http://www.gliffy.com/pubdoc/1978433/L.jpg
alt text http://www.gliffy.com/pubdoc/1978433/L.jpg
As I said, these are "Generally" used terminology. There are no strict definitions for the terms 'binary file', 'image file', or 'object file'.
Particularly the term 'object file' might sometimes be used to mean an intermediate file output by the compiler for use by the linker, but in another context might mean an executable file.
Especially on different platforms they might be used for refer to different or similar things. Even when discussing issues on a single platform, one writer might use the terms somewhat differently than another.
Both ObjectFile & ImageFile are in PE Format in windows & ELF format in linux.
ELF is not only the format of the image file but is also the format of the object file.
Every ELF file starts with an ELF Header. The second field of an ELF Header is e_type; this fields lets us know whether the file is an object file (aka a relocatable in ELF parlance), or an image (which can be either an executable or a shared object) or something else (core file's are also ELF files).
I don't know if there is any bit in header that differentiates an Object file from Image file. It needs to be checked.
I know that PE is derived from COFF.
But why does the Microsoft
specifications of PE format is named
Microsoft Portable Executable "and
Common Object File Format
Specification". Do they still support
COFF? If they, in which OS? I thought
PE completely replaced COFF long ago.
As far as "PE" vs "COFF", my recollection is that Microsoft use the "COFF" specification as the starting point for the "PE" specification but extended it for their needs. So strictly speaking a "PE" file isn't a "COFF" file, but it's very similar in many ways.
There are no strict definitions for the terms 'binary file', 'image file', or 'object file'.
Particularly the term 'object file' might sometimes be used to mean an intermediate file output by the compiler for use by the linker, but in another context might mean an executable file.
Especially on different platforms they might be used for refer to different or similar things. Even when discussing issues on a single platform, one writer might use the terms somewhat differently than another.
As far as "PE" vs "COFF", my recollection is that Microsoft use the "COFF" specification as the starting point for the "PE" specification but extended it for their needs. So strictly speaking a "PE" file isn't a "COFF" file, but it's very similar in many ways.
gcc -c will produce a .o file, which is an elf format object file, on a Linux system. "ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV)" is how a .o file is described by the file command on my machine.
In regards to Q2 for ELF, ELF is not only the format of the image file but is also the format of the object file.
Every ELF file starts with an ELF Header. The second field of an ELF Header is e_type; this fields lets us know whether the file is an object file (aka a relocatable in ELF parlance), an image (which can be either an executable or a shared object) or something else (core file's are also ELF files).
As an aside, I know core dumps on Solaris (and I am guessing other Unix flavors) can be in ELF format.
I have a static library *.lib created using MSVC on windows. The size of library is say 70KB. Then I have an application which links this library. But now the size of the final executable (*.exe) is 29KB, less than the library. What i want to know is :
Since the library is statically linked, I was thinking it should add directly to the executable size and the final exe size should be more than that? Does windows exe format also do some compression of the binary data?
How is it for linux systems, that is how do sizes of library on linux (*.a/*.la file) relate with size of linux executable (*.out) ?
-AD
A static library on both Windows and Unix is a collection of .obj/.o files. The linker looks at each of these object files and determines if it is needed for the program to link. If it isn't needed, then the object file won't get included in the final executable. This can lead to executables that are smaller then the library.
EDIT: As MSalters points out, on Windows the VC++ compiler now supports generating object files that enable function-level linking, e.g., see here. In fact, edit-and-continue requires this, since the edit-and-continue needs to be able to replace the smallest possible part of the executable.
There is additional bookkeeping information in the .lib file that is not needed for the final executable. This information helps the linker find the code to actually link. Also, debug information may be stored in the .lib file but not in the .exe file (I don't recall where debug info is stored for objs in a lib file, it might be somewhere else).
The static library probably contains several functions which are never used. When the linker links the library with the main executable, it sees that certain functions are never used (and that their addresses are never taken and stored in function pointers), it just throws away the code. It can also do this recursively: if function A() is never called, and A() calls B(), but B() is never otherwise called, it can remove the code for both A() and B(). On Linux, the same thing happens.
A static library has to contain every symbol defined in its source code, because it might get linked into an executable which needs just that specific symbol. But once it is linked into an executable, we know exactly which symbols end up being used, and which ones don't. So the linker can trivially remove unused code, trimming the file size by a lot. Similarly, any duplicate symbols (anything that's defined in both the static library and the executable it's linked into gets merged into a single instance.
Disclaimer: It's been a long time since I dealt with static linking, so take my answer with a grain of salt.
You wrote: I was thinking it should add directly to the executable size and final exe size should be more than that?
Naive linkers work exactly this way - back when I was doing hobby development for CP/M systems (a LONG time ago), this was a real problem.
Modern linkers are smarter, however - they only link in the functions referenced by the original code, or as required.
Additionally to the current answers, the linker is allowed to remove function definitions if they have identical object code - this is intended to help reduce the bloating effects of templated code.
#All: Thanks for the pointers.
#Greg Hewgill - Your answer was a good pointer. Thanks.
The answer i found out was as follows:
1.)During Library building what happens is if the option "Keep Program debug databse" in MSVC (or something alike ) is ON, then library will have this debug info bloating its size.
but when i statically include that library and create a executable, the linker strips all that debug info from the library before geenrating the exe and hence the exe size is less than that of the library.
2.) When i disabled the option "Keep Program debug databse", i got an library whose size was smaller than the final executable, which was what i thought is nromal in most situations.
-AD