I have some firmware built with GCC that runs on an ARM Cortex M0 based microcontroller. The build currently generates a single binary image that can be written into the program memory of the microcontroller.
For reasons to do with field update, I need to split this image into two parts that can be updated separately. I'll call these Core and App.
Core: contains the interrupt vector table, main() routine, and various drivers and library routines. It will be located in the first half of the program memory.
App: contains application-specific code. It will be located in the second half of the program memory. It will have a single entry point, at a known address, which is called by the core to start the application. It will access functions and data in the core via known addresses.
There are some obvious limitations here, which I'm well aware of:
When building the app, the addresses of symbols in the core will need to be known. So the core must be built first, and must be available when linking the app.
An app image will only be compatible with the specific core image it was built against.
It will be possible to update the app without updating the core, but not vice versa.
All of that is OK.
My question is simply, how can I build these images using GCC and the GNU binutils?
Essentially I want to build the core like a normal firmware image, and then build the app image, with the app treating the core like a library. But neither shared linking (which would require a dynamic linking mechanism) or static linking (which would copy the core functions used into the app binary) are applicable here. What I'm trying to do is actually a lot simpler: link against an existing binary using its known, fixed addresses. It's just not clear to me how to do so with the tools.
We have this working now so I am going to answer my own question. Here is what was necessary to do this, starting from a normal single image build, turning that into the "core" and then setting up the build for the "app".
Decide how to split up both the flash and the RAM into separate areas for the core and the app. Define the start address and size of each area.
Create a linker script for the core. This will be the same as the standard linker script for the platform except that it must only use the areas reserved for the core. This can be done by changing the ORIGIN and LENGTH of the flash & RAM entries in the MEMORY section of the linker script.
Create a header file declaring the entry point for the app. This just needs a prototype e.g.:
void app_init(void);.
Include this header from the core C code and have the core call app_init() to start the app.
Create a symbol file declaring the address of the entry point, which will be the start address of the flash area for the app. I'll call this app.sym. It can just be one line in the following format:
app_init = 0x00010000;
Build the core, using the core linker script and adding --just-symbols=app.sym to the linker parameters to give the address of app_init. Retain the ELF file from the build, which I'll call core.elf.
Create a linker script for the app. This will again be based on the standard linker script for the platform, but with the flash & RAM memory ranges changed to those reserved for the app. Additionally, it will need a special section to ensure that app_init is placed at the start of the app flash area, before the rest of the code in the .text section:
SECTIONS
{
.text :
{
KEEP(*(.app_init))
*(.text*)
Write the app_init function. This will need to be in assembly, as it must do some low level work before any C code in the app can be called. It will need to be marked with .section .app_init so that the linker puts it in the correct place at the start of the app flash area. The app_init function needs to:
Populate variables in the app's .data section with initial values from flash.
Set variables in the app's .bss section to zero.
Call the C entry point for the app, which I'll call app_start().
Write the app_start() function that starts the app.
Build the app, using the app linker script. This link step should be passed the object files containing app_init, app_start, and any code called by app_start that is not already in the core. The linker parameter --just-symbols=core.elf should be passed to link functions in the core by their addresses. Additionally, -nostartfiles should be passed to leave out the normal C runtime startup code.
It took a while to figure all this out but it is now working nicely.
First of all... if this is just for field updating, you don't need to rely on the interrupt vector table in the core space for the app. I think ARM M0 parts always have the ability to move it. I know it can be done on some (all?) the STM32Fx stuff, but I believe this is an ARM M-x thing, not an ST thing. Look into this before committing yourself to the decision to make your application ISRs all be hooks called from the core.
If you plan on having a lot of interaction with your core (btw, I always call the piece that does self-updating a "bootloader" on MCUs), here's an alternate suggestion:
Have the Core pass a pointer to a struct / table of functions that describes its capabilities into the App entry point?
This would allow complete separation of the code for the app vs core except for a shared header (assuming your ABI doesn't change) and prevent name collisions.
It also provides a reasonable way to prevent GCC from optimizing away any functions that you might call only from the App without messing up your optimization settings or screwing around with pragmas.
core.h:
struct core_functions
{
int (*pcore_func1)(int a, int b);
void (*pcore_func2)(void);
};
core.c:
int core_func1(int a, int b){ return a + b; }
void core_func2(void){ // do something here }
static const struct core_functions cfuncs=
{
core_func1,
core_func2
};
void core_main()
{
// do setup here
void (app_entry*)(const struct core_functions *) = ENTRY_POINT;
app_entry( &cfuncs );
}
app.c
void app_main(const struct core_functions * core)
{
int res;
res = core->pcore_func1(20, 30);
}
The downside / cost is a slight runtime & memory overhead and more code.
Related
I've read this tutorial
I could follow the guide and run the code. but I have questions.
1) Why do we need both load-address and run-time address. As I understand it is because we have put .data at flash too; so why we don't run app there, but need start-up code to copy it into RAM?
http://www.bravegnu.org/gnu-eprog/c-startup.html
2) Why we need linker script and start-up code here. Can I not just build C source as below and run it with qemu?
arm-none-eabi-gcc -nostdlib -o sum_array.elf sum_array.c
Many thanks
Your first question was answered in the guide.
When you load a program on an operating system your .data section, basically non-zero globals, are loaded from the "binary" into the right offset in memory for you, so that when your program starts those memory locations that represent your variables have those values.
unsigned int x=5;
unsigned int y;
As a C programmer you write the above code and you expect x to be 5 when you first start using it yes? Well, if are booting from flash, bare metal, you dont have an operating system to copy that value into ram for you, somebody has to do it. Further all of the .data stuff has to be in flash, that number 5 has to be somewhere in flash so that it can be copied to ram. So you need a flash address for it and a ram address for it. Two addresses for the same thing.
And that begins to answer your second question, for every line of C code you write you assume things like for example that any function can call any other function. You would like to be able to call functions yes? And you would like to be able to have local variables, and you would like the variable x above to be 5 and you might assume that y will be zero, although, thankfully, compilers are starting to warn about that. The startup code at a minimum for generic C sets up the stack pointer, which allows you to call other functions and have local variables and have functions more than one or two lines of code long, it zeros the .bss so that the y variable above is zero and it copies the value 5 over to ram so that x is ready to go when the code your entry point C function is run.
If you dont have an operating system then you have to have code to do this, and yes, there are many many many sandboxes and toolchains that are setup for various platforms that already have the startup and linker script so that you can just
gcc -O myprog.elf myprog.c
Now that doesnt mean you can make system calls without a...system...printf, fopen, etc. But if you download one of these toolchains it does mean that you dont actually have to write the linker script nor the bootstrap.
But it is still valuable information, note that the startup code and linker script are required for operating system based programs too, it is just that native compilers for your operating system assume you are going to mostly write programs for that operating system, and as a result they provide a linker script and startup code in that toolchain.
1) The .data section contains variables. Variables are, well, variable -- they change at run time. The variables need to be in RAM so that they can be easily changed at run time. Flash, unlike RAM, is not easily changed at run time. The flash contains the initial values of the variables in the .data section. The startup code copies the .data section from flash to RAM to initialize the run-time variables in RAM.
2) Linker-script: The object code created by your compiler has not been located into the microcontroller's memory map. This is the job of the linker and that is why you need a linker script. The linker script is input to the linker and provides some instructions on the location and extent of the system's memory.
Startup code: Your C program that begins at main does not run in a vacuum but makes some assumptions about the environment. For example, it assumes that the initialized variables are already initialized before main executes. The startup code is necessary to put in place all the things that are assumed to be in place when main executes (i.e., the "run-time environment"). The stack pointer is another example of something that gets initialized in the startup code, before main executes. And if you are using C++ then the constructors of static objects are called from the startup code, before main executes.
1) Why do we need both load-address and run-time address.
While it is in most cases possible to run code from memory mapped ROM, often code will execute faster from RAM. In some cases also there may be a much larger RAM that ROM and application code may compressed in ROM, so the executable code may not simply be copied from ROM also decompressed - allowing a much larger application than the available ROM.
In situations where the code is stored on non-memory mapped mass-storage media such as NAND flash, it cannot be executed directly in any case and must be loaded into RAM by some sort of bootloader.
2) Why we need linker script and start-up code here. Can I not just build C source as below and run it with qemu?
The linker script defines the memory layout of you target and application. Since this tutorial is for bare-metal programming, there is no OS to handle that for you. Similarly the start-up code is required to at least set an initial stack-pointer, initialise static data, and jump to main. On an embedded system it is also necessary to initialise various hardware such as the PLL, memory controllers etc.
ELF executables have a fixed load address (0x804800 on 32-bit x86 Linux binaries, and 0x40000 on 64-bit x86_64 binaries).
I read the SO answers (e.g., this one) about the historical reasons for those specific addresses. What I still don't understand is why to use a fixed load address and not a randomized one (given some range to be randomized within)?
why to use a fixed load address and not a randomized one
Traditionally that's how executables worked. If you want a randomized load address, build a PIE binary (which is really a special case of shared library that has startup code in it) with -fPIE and link with -pie flags.
Building with -fPIE introduces runtime overhead, in some cases as bad as 10% performance degradation, which may not be tolerable if you have a large cluster or you need every last bit of performance.
not sure if I understood your question correct, but saying I did, that's sort-off a "legacy" / historical issue, ELF is the file format used by UNIX derived operating systems, both POSIX (IOS) and Unix-like (Linux).
and the elf format simply states that there must be some resolved and absolute virtual address that the code is loaded into and begins running from...
and simply that's how the file format is, and during to historical reasons that cant be changed... you couldn't just "throw" the executable in any memory address and have it run successfully, back in the 90's when the ELF format was introduced problems such as calling functions with virtual tables we're raised and it was decided that the elf format would have absolute addresses within it.
Also think about it, take a look at the elf format -https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
how would you design an OS executable-loader that would be able to handle an executable load it to ANY desired virtual address and have the code run successfully without actually having to change the binary itself... if you would like to do something like that you'd either need to vastly change the output compilers generate or the format itself, which again isn't possible
As time passed the requirement of position independent executing (PIE/PIC) has raised and shared objects we're introduced in order to allow that and ASLR
(Address Space Layout Randomization) - which means that the code could be thrown in any memory address and still be able to execute, that is simply implemented by making sure that all calls within the code itself are relative to the current address of the executed instruction, AND that when the shared object is loaded the OS loader would have to change some data within the binary given that the data changed is not executable instructions (R E) but actual data (RW, e.g .data segment), which also is implemented by calling functions from some "Jump tables" ( which would be changed at load time ) for example PLT / GOT.... those shared objects allow absolute randomization of the addresses the code is loaded to and if you want to execute some more "secure" code you'd have to compile it as a shared object and and dynamically link it and load time or run time..
( hope I've cleared some things out :) )
I am working on a boot loader for TMS320DM6437. The idea is to create 2 independent firmware that one will update another. In firmware1 I will download firmware2 file and write it to NOR flash in a specified address. Both firmware are stored in NOR flash in ais format. Now I have two applications in flash. One is my custom boot loader and the second one is my main project. I want to know how I can jump from the first program to the second program located at a specified address. I also expect information about documents which may help me to create custom bootloader
Any recommendations?
You can jump to the entry point. I'm using this approach on TMS320 2802x and 2803x, but it should be the same.
The symbol of the entry point is c_int00.
To get to know the address of c_int00 in the second application, you have to fix the Run-Time Support (RTS) library at a specific address, by modifying the linker command file.
Otherwise you can leave the RTS unconstrained, and create a C variable (at a fixed address) that is initialized with the value of cint_00. Using this method your memory map is more flexible and you can add, togheter with the C variable, a comprehensive data structure with other information for your bootloader, e.g. CRC, version number, etc.
Be carefull with the (re)initialization of the peripherals in the second application, since you are not starting from a hardware reset, and you may need to explicity reset some more registers, or clear interrupt requests.
I've heard the Google talk (http://www.youtube.com/watch?v=_gZK0tW8EhQ) by Ron Garret and read the paper (http://www.flownet.com/gat/jpl-lisp.html), but I'm not understanding how it worked to "correct" supposedly running code with a REPL. Was the DS-1's Lisp code running is some sort of virtual machine? Or was it "live" in the REPL's actual world? Or was the Lisp code an executable that got replaced? What exactly happened/happens when you dynamically change running Lisp code through a REPL?
Whereas most programs are built and distributed as an executable that contains only the necessary components to run the program, Lisp can be distributed as an image that contains not just the components for the specific program, but also much or all of the Lisp runtime and development environment.
The REPL is the quintessential mechanism for providing interactive access to a running Lisp environment. The two key components of the REPL, Read, and Eval, expose much of the Lisp runtime system. For example, many Lisp systems today implement Eval by compiling the provided form (that is read by the Reader), compiling the form to machine code, and then executing the result. This is in contrast to interpreting the form. Some systems, especially in the past, contained both an interpreter that executes quickly and is suitable for interactive access, and a compiler that produces better code. But modern systems are fast enough that the compiler phase isn't noticeable and simply forgo the interpreter.
Of course, you can do very similar things today. A simple example is running SSH to your Linux box that's hosting PHP. Your PHP server is up and running and live, serving pages and requests. But you login through SSH, go over and fix a PHP file, and as soon as you save that file, all of your users see the new result in real time -- the system updated itself on the fly.
The fact that PHP is running on a Linux runtime vs Lisp running on a Lisp runtime, is a detail. The effect is the same. The fact that PHP isn't compiled is a detail also. For example, you can do the same thing on a Java server: modify a JSP, save it, and the JSP is converted in to a Servlet as Java source code, then compiled on the fly by the Java runtime, then loaded in to the executing container, replacing the old code.
Lisps capability to do this is very nice, and it was very interesting far back in the day. Today, it's less so, as there are different system providing similar capabilities.
Addenda:
No, Lisp is not a virtual machine, there's no need for it to be that complicated.
The key to the concept is dynamic dispatch. With dynamic dispatch there is some lookup involved before a function is invoked.
In a static language like C, locations of things are pretty much set in stone once the linker and loader have finished processing the executable in preparation to start executing.
So, in C if you have something simple like:
int add(int i) {
return i + 1;
}
void main() {
add(1);
}
After compiling and linking and loading of the program, the address of the add function will be set in stone, and thus thing referring to that function will know exactly where to find it.
So, in assembly language: (note this is a contrived assembly language)
add: pop r1 ; pop R1 from the stack, loading the i parameter
add r1, 1; Add 1 to the parameter.
push r1 ; push result of function call
rts ; return from subroutine
main: push 1 ; Push parameter to function
call add ; call function
pop r1 ; gather (and ignore) the result
So, you can see here that add is fixed in place.
In something like Lisp, function are referred to indirectly.
int add(int i) {
return i + 1;
}
int *add_ptr() = &add;
void main() {
*(add_ptr)(1);
}
In assembly you get:
add: pop r1 ; pop R1 from the stack, loading the i parameter
add r1, 1; Add 1 to the parameter.
push r1 ; push result of function call
rts ; return from subroutine
add_ptr: dw add ; put the address of the add routine in add_ptr
main: push 1 ; Push parameter to function
mov r1, add_ptr ; Put the contents of add_ptr into R1
call (r1) ; call function indirectly through R1
pop r1 ; gather (and ignore) the result
Now, you can see here that rather than calling add directly, it is called indirectly through the add_ptr. In a Lisp runtime, it has the capability of compiling new code, and when that happens, add_ptr would be overwritten to point to the newly compiled code. You can see how the code in main never has to change, it will call whatever function add_ptr is pointing to.
Since most all of the functions in Lisp are indirectly referenced through their symbols, a lot can change "behind the back" of a running system, and the system, will continue to run.
When a function is recompiled, the old function code (assuming no other references) become eligible for garbage collection, and will, typically, eventually go away.
You can also see that when the system is garbage collected, any code that is moved (such as the code for the add function) can be moved by the runtime, and it's new location updated in the add_ptr so the system will continue operating even after code and been relocated by the garbage collector.
So, the key to it all, is to have your functions invoked through some lookup mechanism. Having this gives you a lot of flexibility.
Note, you can also do this is a running C system, for example. You can put code in a dynamic library, load the library, execute the code, and if you want you can build a new dynamic library, close the old one, open the new one, and invoke the new code -- all in a "running" system. The dynamic library interface provides the lookup mechanism that isolates the code.
I'm developing a launcher for a game.
Want to intercept game's call for a function that prints text.
I don't know whether the code that contains this function is dynamically linked or statically. So I dont even know the function name.
I did intercepted some windows-api calls of this game through microsoft Detours, Ninject and some others.
But this one is not in import table either.
What should I do to catch this function call? What profiler should be used? IDA? How this could be done?
EDIT:
Finally found function address. Thanks, Skino!
Tried to hook it with Detours, injected dll. Injected DllMain:
typedef int (WINAPI *PrintTextType)(char *, int, float , int);
static PrintTextType PrintText_Origin = NULL;
int WINAPI PrintText_Hooked(char * a, int b, float c, int d)
{
return PrintText_Origin(a, b, c , d);
}
HMODULE game_dll_base;
/* game_dll_base initialization goes here */
BOOL APIENTRY DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
if(fdwReason==DLL_PROCESS_ATTACH)
{
DisableThreadLibraryCalls(hinstDLL);
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
PrintText_Origin = (PrintTextType)((DWORD)game_dll_base + 0x6049B0);
DetourAttach((PVOID *)&PrintText_Origin , PrintText_Hooked);
DetourTransactionCommit();
}
}
It hooks as expected. Parameter a has text that should be displayed. But when calling original function return PrintText_Origin (a, b, c , d); application crashes(http://i46.tinypic.com/ohabm.png, http://i46.tinypic.com/dfeh4.png)
Original function disassembly:
http://pastebin.com/1Ydg7NED
After Detours:
http://pastebin.com/eM3L8EJh
EDIT2:
After Detours:
http://pastebin.com/GuJXtyad
PrintText_Hooked disassembly http://pastebin.com/FPRMK5qt w3_loader.dll is the injected dll
Im bad at ASM, please tell what can be wrong ?
Want to intercept game's call for a function that prints text.
You can use a debugger for the investigative phase. Either IDA, or even Visual Studio (in combination with e.g. HxD), should do. It should be relatively easy to identify the function using the steps below:
Identify a particular fragment of text whose printing you want to trace (e.g. Hello World!)
Break the game execution at any point before the game normally prints the fragment you identified above
Search for that fragment of text† (look for either Unicode or ANSI) in the game's memory. IDA will allow you to do that IIRC, as will the free HxD (Extras > Open RAM...)
Once the address of the fragment has been identified, set a break-on-access/read data breakpoint so the debugger will give you control the moment the game attempts to read said fragment (while or immediately prior to displaying it)
Resume execution, wait for the data breakpoint to trigger
Inspect the stack trace and look for a suitable candidate for hooking
Step through from the moment the fragment is read from memory until it is printed if you want to explore additional potential hook points
†provided text is not kept compressed (or, for whatever reason, encrypted) until the very last moment
Once you are done with the investigative phase and you have identified where you'd like to inject your hook, you have two options when writing your launcher:
If, based on the above exercise, you were able to identify an export/import after all, then use any API hooking techniques
EDIT Use Microsoft Detours, making sure that you first correctly identify the calling convention (cdecl, fastcall, stdcall) of the function you are trying to detour, and use that calling convention for both the prototype of the original as well as for the implementation of the dummy. See examples.
If not, you will have to
use the Debugging API to programatically load the game
compute the hook address based on your investigative phase (either as a hard-coded offset from the module base, or by looking for the instruction bytes around the hook site‡)
set a breakpoint
resume the process
wait for the breakpoint to trigger, do whatever you have to do
resume execution, wait for the next trigger etc. again, all done programatically by your launcher via the Debugging API.
‡to be able to continue to work with eventual patch releases of the game
At this stage it sounds like you don't have a notion of what library function you're trying to hook, and you've stated it's not (obviously at least) an imported external function in the import table which probably means that the function responsible for generating the text is likely located inside the .text of the application you are disassembling directly or loaded dynamically, the text generation (especially in a game) is likely a part of the application.
In my experience, this simplest way to find code that is difficult to trace such as this is by stopping the application shortly during or before/after text is displayed and using IDA's fabulous call-graph functionality to establish what is responsible for writing it out (use watches and breakpoints liberally!)
Look carefully to calls to CreateRemoteThread or any other commonly used dynamic loading mechanism if you have reason to believe this functionality might be provided by an exported function that isn't showing up in the import table.
I strongly advice against it but for the sake of completeness, you could also hook NtSetInformationThread in the system service dispatch table. here's a good dump of the table for different Windows versions here. If you want to get the index in the table yourself you can just disassemble the NtSetInformationThread export from ntdll.dll.