I am reading the Linux kernel code for copy_fom_user, which is architecture dependent and I focus on x86 architectures.
But I got two pieces of implementation for it.
One is here (in arch/x86/lib/usercopy_32.c), while the other is here (in include/asm-generic/uaccess.h).
Which one will be finally compiled into the kernel. I guess the former is the real one, but I am not sure. What is more strange is that the former has the function name _copy_from_user instead of copy_from_user
I always have this kind of confusions when reading the kernel code. For example, due to the conditional compiling, the same function may have multiple implementation and I cannot determine which one will be used in general. Is there any tool that, given a complied kernel and a function of interest, tells you the corresponding binary code, so that you can disassemble it? Or it would be even better if it can tell you the source code that the binary code corresponds to.
Generally, if there is a module present in the architecture-specific subdirectory, that is the one being used. Otherwise, the generic one is it.
For the modules given, the .c is the correct one. Rarely is there any executable code in a .h. I have 2.6.27.8's uaccess.h handy:
#ifndef _ASM_GENERIC_UACCESS_H_
#define _ASM_GENERIC_UACCESS_H_
/*
* This macro should be used instead of __get_user() when accessing
* values at locations that are not known to be aligned.
*/
#define __get_user_unaligned(x, ptr) \
({ \
__typeof__ (*(ptr)) __x; \
__copy_from_user(&__x, (ptr), sizeof(*(ptr))) ? -EFAULT : 0; \
(x) = __x; \
})
/*
* This macro should be used instead of __put_user() when accessing
* values at locations that are not known to be aligned.
*/
#define __put_user_unaligned(x, ptr) \
({ \
__typeof__ (*(ptr)) __x = (x); \
__copy_to_user((ptr), &__x, sizeof(*(ptr))) ? -EFAULT : 0; \
})
#endif /* _ASM_GENERIC_UACCESS_H */
Look at that carefully. These are macro wrappers to call the underlying __copy_from_user() and __copy_to_user() functions, which are implemented differently on each architecture.
Related
I'm studying Operating Systems and, of course, Minix 3.
Can someone tell me which file and line of code in Minix 3 are the entry points of:
Kernel;
Process Manager;
Memory Manager;
Virtual Memory Manager;
Any Device Driver.
Any of above will help me a lot.
Thanks.
I'm using the main github source:
https://github.com/Stichting-MINIX-Research-Foundation/minix
Try first searching for the device drivers since they are more "self-contained". Device drivers are found inhttps://github.com/Stichting-MINIX-Research-Foundation/minix/tree/master/minix/drivers
Instead of just pointing the entry points, I will show how I usually do. I will use the minix/drivers/video/fb/fb.h as an example. The fb is a driver responsible for controlling video displayed on a monitor. I could open any file, but I chose the fb.h since it is a header file and will contain the signatures of public functions.
I am on Linux so I can use the grep -r command. The -r flag search recursively for the given pattern. So by inspecting fb.h I found the following functions:
int arch_fb_init(int minor, struct edid_info *info);
int arch_get_device(int minor, struct device *dev);
int arch_get_varscreeninfo(int minor, struct fb_var_screeninfo *fbvsp);
int arch_put_varscreeninfo(int minor, struct fb_var_screeninfo *fbvs_copy);
int arch_get_fixscreeninfo(int minor, struct fb_fix_screeninfo *fbfsp);
int arch_pan_display(int minor, struct fb_var_screeninfo *fbvs_copy);
Well, arch_fb_init seems to be an entry point. To verify if arch_fb_init is indeed an entry point, I use
grep -r "arch_fb_init"
Which in turn returns the following
$ grep -r "arch_fb_init"
drivers/video/fb/arch/earm/fb_arch.c:arch_fb_init(int minor, struct edid_info *info)
drivers/video/fb/fb.h:int arch_fb_init(int minor, struct edid_info *info);
drivers/video/fb/fb.c: if (arch_fb_init(minor, infop) == OK) {
The second match is the fb.h itself. The other two are in fb_arch.c and fb.c. Then I go to those files and inspect each of them. Opening fb_arch.c and searching for arch_fb_init I find on line 315 an implementation of this function. By now, the implementation of the function could be more interesting because we want to figure out the caller/callee hierarchy.
Opening fb.c I find an occurrence of arch_fb_init on line 74, which is a call to arch_fb_init. This call to arch_fb_init is performed by another function named fb_open (line 60 on fb.c).
Now I repeat the process but now searching for fb_open. Performing another grep now returns the following.
$ grep -r "fb_open"
drivers/video/fb/fb.c:static int fb_open(devminor_t minor, int access, endpoint_t user_endpt);
drivers/video/fb/fb.c: .cdr_open = fb_open,
drivers/video/fb/fb.c:fb_open(devminor_t minor, int UNUSED(access), endpoint_t UNUSED(user_endpt))
All the occurrences are also in fb.c. By inspecting fb.c, one can see that the first grep match is a declaration of the fb_open function, and the third one is the implementation itself. The second one is using fb_open to construct a table (line 46 on file fb.c) that you can see below
/* Entry points to the fb driver. */
static struct chardriver fb_tab =
{
.cdr_open = fb_open,
.cdr_close = fb_close,
.cdr_read = fb_read,
.cdr_write = fb_write,
.cdr_ioctl = fb_ioctl
};
The commentary "Entry points to the fb driver." indicates that we are getting closer. If you have an IDE capable of parsing the Minix project, it will be easier to find all the callers/callee. If you don't have an IDE, you can use the grep command with the -r flag. If you know object orientation, this fb_tab is kind of a virtual table. Probably is setting callbacks that the kernel can use when calling this driver. I'm not sure because I never worked with minix before, but I have some general knowledge in OS, and the names used imply that I may be right.
Now searching for fb_tab by using grep, I get the following:
$ grep -r "fb_tab"
drivers/video/fb/fb.c:static struct chardriver fb_tab =
drivers/video/fb/fb.c: chardriver_task(&fb_tab);
All the occurrences once again in fb.c. The first match is the definition of fb_tab itself. The second one is the following
int
main(int argc, char *argv[])
{
env_setargs(argc, argv);
fb_edid_args_parse();
sef_local_startup();
chardriver_task(&fb_tab);
return OK;
}
By inspecting this main function, env_setargs and fb_edid_args_parse are probably just parsing arguments and setting flags. Now, the sef_local_startup and chardriver_task functions may be worth performing a grep.
Performing a grep on chardriver_task returns the following:
$ grep -r "chardriver_task"
drivers/examples/hello/hello.c: chardriver_task(&hello_tab);
drivers/sensors/tsl2550/tsl2550.c: chardriver_task(&tsl2550_tab);
drivers/sensors/bmp085/bmp085.c: chardriver_task(&bmp085_tab);
drivers/sensors/sht21/sht21.c: chardriver_task(&sht21_tab);
drivers/printer/printer/printer.c: chardriver_task(&printer_tab);
drivers/system/log/log.c: chardriver_task(&log_dtab);
drivers/system/random/main.c: chardriver_task(&r_dtab);
drivers/bus/i2c/i2c.c: chardriver_task(&i2c_tab);
drivers/bus/pci/main.c: chardriver_task(&driver);
drivers/video/fb/fb.c: chardriver_task(&fb_tab);
include/minix/chardriver.h:void chardriver_task(const struct chardriver *cdp);
servers/input/input.c: chardriver_task(&input_tab);
lib/libchardriver/chardriver.c: * chardriver_task *
lib/libchardriver/chardriver.c:void chardriver_task(const struct chardriver *cdp)
lib/libaudiodriver/audio_fw.c: chardriver_task(&audio_tab);
Many of the matches are calling the chardriver_task similar to the one found in fb.c but the following probably is the declaration
include/minix/chardriver.h:void chardriver_task(const struct chardriver *cdp);
And this one probably is the implementation
lib/libchardriver/chardriver.c:void chardriver_task(const struct chardriver *cdp)
As you can see, there isn't a clear entry point. It depends on what you're searching for. In the case of the fb driver, if you're searching for its initialization, it is probably the main function. However, if you're searching for how the kernel will be calling the driver probably is by the callbacks set on fb_tab.
If you can edit your question, someone will probably give you a better answer.
I'm aware of many similar questions on this site. I really like the solution mention in the following link:
https://stackoverflow.com/a/25021520/884553
with some modification, you can include text file at compile time, for example:
constexpr const char* s =
#include "file.txt"
BUT to make this work you have to add string literal prefix and suffix to your original file, for example
R"(
This is the original content,
and I don't want this file to be modified. but i
don't know how to do it.
)";
My question is: is there a way to make this work but not modifying file.txt?
(I know I can use command line tools to make a copy, prepend and append to the copy, remove the copy after compile. I'm looking for a more elegant solution than this. hopefully no need of other tools)
Here's what I've tried (but not working):
#include <iostream>
int main() {
constexpr const char* s =
#include "bra.txt" // R"(
#include "file.txt" //original file without R"( and )";
#include "ket.txt" // )";
std::cout << s << "\n";
return 0;
}
/opt/gcc8/bin/g++ -std=c++1z a.cpp
In file included from a.cpp:5:
bra.txt:1:1: error: unterminated raw string
R"(
^
a.cpp: In function ‘int main()’:
a.cpp:4:27: error: expected primary-expression at end of input
constexpr const char* s =
^
a.cpp:4:27: error: expected ‘}’ at end of input
a.cpp:3:12: note: to match this ‘{’
int main() {
^
No, this cannot be done.
There is a c++2a proposal to allow inclusion of such resources at compile time called std::embed.
The motivation part of ths p1040r1 proposal:
Motivation
Every C and C++ programmer -- at some point -- attempts to #include large chunks of non-C++ data into their code. Of course, #include expects the format of the data to be source code, and thusly the program fails with spectacular lexer errors. Thusly, many different tools and practices were adapted to handle this, as far back as 1995 with the xxd tool. Many industries need such functionality, including (but hardly limited to):
Financial Development
representing coefficients and numeric constants for performance-critical algorithms;
Game Development
assets that do not change at runtime, such as icons, fixed textures and other data
Shader and scripting code;
Embedded Development
storing large chunks of binary, such as firmware, in a well-compressed format
placing data in memory on chips and systems that do not have an operating system or file system;
Application Development
compressed binary blobs representing data
non-C++ script code that is not changed at runtime; and
Server Development
configuration parameters which are known at build-time and are baked in to set limits and give compile-time information to tweak performance under certain loads
SSL/TLS Certificates hard-coded into your executable (requiring a rebuild and potential authorization before deploying new certificates).
In the pursuit of this goal, these tools have proven to have inadequacies and contribute poorly to the C++ development cycle as it continues to scale up for larger and better low-end devices and high-performance machines, bogging developers down with menial build tasks and trying to cover-up disappointing differences between platforms.
MongoDB has been kind enough to share some of their code below. Other companies have had their example code anonymized or simply not included directly out of shame for the things they need to do to support their workflows. The author thanks MongoDB for their courage and their support for std::embed.
The request for some form of #include_string or similar dates back quite a long time, with one of the oldest stack overflow questions asked-and-answered about it dating back nearly 10 years. Predating even that is a plethora of mailing list posts and forum posts asking how to get script code and other things that are not likely to change into the binary.
This paper proposes <embed> to make this process much more efficient, portable, and streamlined. Here’s an example of the ideal:
#include <embed>
int main (int, char*[]) {
constexpr std::span<const std::byte> fxaa_binary = std::embed( "fxaa.spirv" );
// assert this is a SPIRV file, compile-time
static_assert( fxaa_binary[0] == 0x03 && fxaa_binary[1] == 0x02
&& fxaa_binary[2] == 0x23 && fxaa_binary[3] == 0x07
, "given wrong SPIRV data, check rebuild or check the binaries!" )
auto context = make_vulkan_context();
// data kept around and made available for binary
// to use at runtime
auto fxaa_shader = make_shader( context, fxaa_binary );
for (;;) {
// ...
// and we’re off!
// ...
}
return 0;
}
Related to but not equivalent to DLL Get Symbols From Its Parent (Loader)
Is there a way to convince the Windows loader to resolve a particular symbol referenced by A.dll from either the loading executable or an intermediate dll without specifying the file to resolve symbols from in A.dll?
It's pretty obvious how to do it if the loading .exe has a known name, but if it isn't ...
Here's a good reason why you'd actually want to do this: https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html
If this can be done, a good answer would say how to do it some way or another.
I'm half-expecting the answer is it can't be done. In that case, a good answer would show why this is impossible. "The build tools don't support this." is a bad answer.
when we use import we need exactly indicate module name and function name. and we can not use complex algorithms. also for exe not exist well known alias which we can use in place exactly exe name. for compare: in case get GetModuleHandle we can use NULL for get handle to the file used to create the calling process (.exe file). but in case LoadLibraryExW we can not use 0 or empty string (L"") or some another alias for say - we want handle to exe. when loader load our module - he read dll name from IMAGE_IMPORT_DESCRIPTOR and try found or load module with this name first by low level, private, core of LoadLibraryExW. here need exactly name. or load fail. as result use import - not a solution here, if we dont know exe name at build time
possible variant - resolve functions pointers yourself at runtime. here we can get exe HMODULE by GetModuleHandle(0). also if need we can search function not only in exe but somewhere else. can implement any search algorithm.
here exist several ways. for concrete example let we need get pointer to function with signature:
void WINAPI fn(int i);
we can declare pointer to this function and resolve it in runtime
void (WINAPI *fn)(int);
*(void**)&fn = GetProcAddress(GetModuleHandleW(0), "fn");
say on DLL_PROCESS_ATTACH
a slightly different solution (although at the binary level it is completely equivalent) declare function with __declspec(dllimport) attribute. this is for CL.EXE (more known as MSVC) compiler only. so
__declspec(dllimport) void fn(int i);
in this case CL yourself generate pointer to function with name __imp_ ## __FUNCDNAME__ name. so by fact the same as in first variant, when we declare pointer yourself. only difference in syntax and.. symbol name. it will be look like __imp_?fn2##YAXH#Z. problem here that __imp_?fn2##YAXH#Z not valid name for c/c++ - we can not direct assign value to it from c/c++. even if we declare function with extern "C" - function name will be containing # symbol (illegal for c++) for __stdcall and __fastcall functions, for x86. also name will be different for different platforms (x86, x64, etc). for access such names - need or use external asm file (for asm ? and # symbols valid in name) or use /alternatename linker option - for set alias for such name and access symbol via it. say like
__pragma(comment(linker, "/alternatename:__imp_?fn##YAXH#Z=__imp_fn"))
and init via
*(void**)&__imp_fn = GetProcAddress(GetModuleHandle(0), "fn");
another option use __declspec(dllimport) in function declarations + add import library, where all __imp___FUNCDNAME__ (such __imp_?fn2##YAXH#Z) is defined. (even if we have not such library we can easy create it yourself - all what need - correct function declarations with empty implementation). and after we add such import lib to linker input - add /DELAYLOAD:dllname where dllname - exactly name from import lib. sense that this dllname will(can) be not match to exe - all what need - it must be unique. and we need yourself handle delayload (called when we first time call fn). for implement delayload we need implement
extern "C" FARPROC WINAPI __delayLoadHelper2(
PCImgDelayDescr pidd,
FARPROC * ppfnIATEntry
);
we can implement it yourself, or add delayimp.lib to our project. here (in delayimp.lib) the delayLoadHelper2 and implemented. however we must customize this process (default implementation(look in /include/DelayHlp.cpp) will be use LoadLibraryExA with dllname which is not excepted in our case - otherwise we can be simply use import as is). so we need mandatory implement __pfnDliNotifyHook2:
for example:
FARPROC WINAPI MyDliHook(
unsigned dliNotify,
PDelayLoadInfo pdli
)
{
switch (dliNotify)
{
case dliNotePreLoadLibrary:
if (!strcmp(pdli->szDll, "unique_exe_alias"))
{
return (FARPROC)GetModuleHandle(0);
}
}
return 0;
}
const PfnDliHook __pfnDliNotifyHook2 = MyDliHook;
we can look for dliNotePreLoadLibrary notification and instead default LoadLibraryEx(dli.szDll, NULL, 0); use GetModuleHandle(0); for get base of exe.
the "unique_exe_alias" (which linker got from import library) here play role not real exe name, which is unknown, but unique tag(alias) for exe
I'm using gcc toolchain for ARM and I need a linker to place a structure on a 256B boundary in memory. I've tried the aligned attribute:
volatile struct ohci_hcca hcca __attribute__ ((aligned (256)));
but with no luck:
nm out.elf | grep hcca
20011dc0 B hcca
I remember using this in past for fields that had to be on 512B boundary so I know it's possible but I seem to be missing something this time.
Thanks for any guidance.
EDIT:
The mystery's been solved. The BSP library is composed from tens of drivers and one of their header files contaned this:
#if defined __ICCARM__ || defined __CC_ARM || defined __GNUC__
//#pragma data_alignment=8 /* IAR */
#pragma pack(1) /* IAR */
#define __attribute__(...) /* IAR */
#endif /* IAR */
For __GNUC__ it doesn't make any sense unless the author of this chunk is a member of guerrilla force against GCC users. Ruined my entire evening :)
while trying to create an object oriented OpenACC implementation I stumbled upon this question.
From there I took the code provided by #mat-colgrove at the GTC15 (code available at http://www.pgroup.com/lit/samples/gtc15_S5233.tar).
Since I am interested how to use objects to manage data on with OpenACC I posted another question.
I was quite impressed by the ease of the OpenACCArray::swap function, so I created a small example to test it (see gist).
First I tried to just swap and hope that it is sufficient to swap the pointers on the host, but this ends in a fatal memory error. (presumably because the size and capacity members are not updated on the device)
A safer approach, that I assumed to work is to update the host, swap arrays and update device. This runs but creates wrong results.
I am compiling for nvidia accelerators.
Looks like this is my fault since I didn't test the swap routine.
The problem here is while the code is swapping the data on the host, the device copy of the objects still point to the old array. The fix is to re-attach (i.e. set the object's device pointers to the correct arrays) the lists.
void swap(OpenACCArray<type>& x)
{
type* tmp_list = list;
int tmp_size = _size;
int tmp_capacity = _capacity;
list = x.list;
_size = x._size;
_capacity = x._capacity;
x.list = tmp_list;
x._size = tmp_size;
x._capacity = tmp_capacity;
#ifdef _OPENACC
#pragma acc update device(_size,_capacity,x._size,x._capacity)
acc_attach((void**)&list);
acc_attach((void**)&x.list);
#endif
}
"acc_attach" is a PGI extension that hopefully will be adopted in the OpenACC 3.0 standard.
Thanks for trying things out and let me know if you encounter other issues.
- Mat