I'm writing the startup code for an embedded system -- the code that loads the initial stack pointer before jumping to the main() function -- and I need to tell it how many bytes of stack my application will use (or some larger, conservative estimate).
I've been told the gcc compiler now has a -fstack-usage option and -fcallgraph-info option that can somehow be used to statically calculates the exact "Maximum Stack Usage" for me.
( "Compile-time stack requirements analysis with GCC" by Botcazou, Comar, and Hainque ).
Nigel Jones says that recursion is a really bad idea in embedded systems ("Computing your stack size" 2009), so I've been careful not to make any mutually recursive functions in this code.
Also, I make sure that none of my interrupt handlers ever re-enable interrupts until their final return-from-interrupt instruction, so I don't need to worry about re-entrant interrupt handlers.
Without recursion or re-entrant interrupt handlers, it should possible to statically determine the maximum stack usage. (And so most of the answers to How to determine maximum stack usage? do not apply).
My understanding is I (or preferably, some bit of code on my PC that is automatically run every time I rebuild the executable) first find the maximum stack depth for each interrupt handler when it's not interrupted by a higher-priority interrupt, and the maximum stack depth of the main() function when it is not interrupted.
Then I add them all up to find the total (worst-case) maximum stack depth. That occurs (in my embedded system) when the main() background task is at its maximum depth when it is interrupted by the lowest-priority interrupt, and that interrupt is at its maximum depth when it is interrupted by the next-lowest-priority interrupt, and so on.
I'm using YAGARTO with gcc 4.6.0 to compile code for the LM3S1968 ARM Cortex-M3.
So how do I use the -fstack-usage option and -fcallgraph-info option with gcc to calculate the maximum stack depth? Or is there some better approach to determine maximum stack usage?
(See How to determine maximum stack usage in embedded system? for almost the same question targeted to the Keil compiler .)
GCC docs :
-fstack-usage
Makes the compiler output stack usage information for the program, on a per-function basis. The filename for the dump is made by appending .su to the auxname. auxname is generated from the name of the output file, if explicitly specified and it is not an executable, otherwise it is the basename of the source file. An entry is made up of three fields:
The name of the function.
A number of bytes.
One or more qualifiers: static, dynamic, bounded.
The qualifier static means that the function manipulates the stack statically: a fixed number of bytes are allocated for the frame on function entry and released on function exit; no stack adjustments are otherwise made in the function. The second field is this fixed number of bytes.
The qualifier dynamic means that the function manipulates the stack dynamically: in addition to the static allocation described above, stack adjustments are made in the body of the function, for example to push/pop arguments around function calls. If the qualifier bounded is also present, the amount of these adjustments is bounded at compile-time and the second field is an upper bound of the total amount of stack used by the function. If it is not present, the amount of these adjustments is not bounded at compile-time and the second field only represents the bounded part.
I can't find any references to -fcallgraph-info
You could potentially create the information you need from -fstack-usage and -fdump-tree-optimized
For each leaf in -fdump-tree-optimized, get its parents and sum their stack size number (keeping in mind that this number lies for any function with "dynamic" but not "bounded") from -fstack-usage, find the max of these values and this should be your maximum stack usage.
Just in case no one comes up with a better answer, I'll post what I had in the comment to your other question, even though I have no experience using these options and tools:
GCC 4.6 adds the -fstack-usage option which gives the stack usage statistics on a function-by-function basis.
If you combine this information with a call graph produced by cflow or a similar tool you can get the kind of stack depth analysis you're looking for (a script could probably be written pretty easily to do this). Have the script read the stack-usage info and load up a map of function names with the stack used by the function. Then have the script walk the cflow graph (which can be an easy-to-parse text tree), adding up the stack usage associated with each line for each branch in the call graph.
So, it looks like this can be done with GCC, but you might have to cobble together the right set of tools.
Quite late, but for anyone looking at this, the answers given involving combining the outputs from fstack-usage and call graph tools like cflow can end up being wildly incorrect for any dynamic allocation, even bounded, because there's no information about when that dynamic stack allocation occurs. It's therefore not possible to know to what functions you should apply the value towards. As a contrived example, if (simplified) fstack-usage output is:
main 1024 dynamic,bounded
functionA 512 static
functionB 16 static
and a very simple call tree is:
main
functionA
functionB
The naive approach to combine these may result in main -> functionA being chosen as the path of maximum stack usage, at 1536 bytes. But, if the largest dynamic stack allocation in main() is to push a large argument like a record to functionB() directly on the stack in a conditional block that calls functionB (I already said this was contrived), then really main -> functionB is the path of maximum stack usage, at 1040 bytes. Depending on existing software design, and also for other more restricted targets that pass everything on the stack, cumulative errors may quickly lead you toward looking at entirely wrong paths claiming significantly overstated maximum stack sizes.
Also, depending on your classification of "reentrant" when talking about interrupts, it's possible to miss some stack allocations entirely. For instance, many Coldfire processors' level 7 interrupt is edge-sensitive and therefore ignores the interrupt disable mask, so if a semaphore is used to leave the instruction early, you may not consider it reentrant, but the initial stack allocation will still happen before the semaphore is checked.
In short, you have to be extremely careful about using this approach.
I ended up writing a python script to implement τεκ's answer. It's too much code to post here, but can be found on github
I am not familiar with the -fstack-usage and -fcallgraph-info options. However, it is always possible to figure out actual stack usage by:
Allocate adequate stack space (for this experiment), and initialize it to something easily identifiable. I like 0xee.
Run the application and test all its internal paths (by all combinations of input and parameters). Let it run for more than "long enough".
Examine the stack area and see how much of the stack was used.
Make that the stack size, plus 10% or 20% to tolerate software updates and rare conditions.
There are generally two approaches - static and runtime.
Static: compile your project with -fdump-rtl-expand -fstack-usage and from the *.expand script get the call tree and stack usage of each function. Then iterate over all leaves in the call tree and calculate stack usage in each leaf and get the highest stack usage. Then compare that value with available memory on the target. This works statically and doesn't require running the program. This does not work with recursive functions. Does not work with VLA arrays. In case sbrk() operates on a linker section not on a statically preallocated buffer, it does not take dynamic allocation into account, which may grow on itself from the other side. I have a script in my tree ,stacklyze.sh that I explored this option with.
Runtime: before and after each function call check the current stack usage. Compile the code with -finstrument-functions. Then define two functions in your code that roughly should get the current stack usage and operate on them:
static unsigned long long max_stack_usage = 0;
void __cyg_profile_func_enter(void * this, void * call) __attribute__((no_instrument_function)) {
// get current stack usage using some hardware facility or intrisic function
// like __get_SP() on ARM with CMSIS
unsigned cur_stack_usage = __GET_CURRENT_STACK_POINTER() - __GET_BEGINNING_OF_STACK();
// use debugger to output current stack pointer
// for example semihosting on ARM
__DEBUGGER_TRANSFER_STACK_POINTER(cur_stack_usage);
// or you could store the max somewhere
// then just run the program
if (max_stack_usage < cur_stack_usage) {
max_stack_usage = max_stack_usage;
}
// you could also manually inspect with debugger
unsigned long long somelimit = 0x2000;
if (cur_stack_usage > somelimit) {
__BREAKPOINT();
}
}
void __cyg_profile_func_exit(void * this, void * call) __attribute__((no_instrument_function)) {
// well, nothing
}
Before and after each function is made - you can check the current stack usage. Because function is called before stack is used within the function, this method does not take the current function stack usage - which is only one function and doesn't do much, and can be somehow mitigated by getting which function is it and then getting stack usage with -fstack-usage and adding it to result.
In general you need to combine call-graph information with the .su files generated by -fstack-usage to find the deepest stack usage starting from a specific function. Starting at main() or a thread entry-point will then give you the worst-case usage for that thread.
Helpfully the work to create such a tool has been done for you as discussed here, using a Perl script from here.
Related
Based on my earlier question, I understand the benefit of using stack allocation. Suppose I have an array of arrays. For example, A is a list of matrices and each element A[i] is a 1x3 matrix. The length of A and the dimension of A[i] are known at run time (given by the user). Each A[i] is a matrix of Float64 and this is also known at run time. However, through out the program, I will be modifying the values of A[i] element by element. What data structure can also allow me to use stack allocation? I tried StaticArrays but it doesn't allow me to modify a static array.
StaticArrays defines MArray (MVector, MMatrix) types that are fixed-size and mutable. If you use these there's a higher chance of the compiler determining that they can be stack-allocated, but it's not guaranteed. Moreover, since the pattern you're using is that you're passing the mutable state vector into a function which presumably modifies it, it's not going to be valid or helpful to stack allocate that anyway. If you're going to allocate state once and modify it throughout the program, it doesn't really matter if it is heap or stack allocated—stack allocation is only a big win for objects that are allocated, used locally and then don't escape the local scope, so they can be “freed” simply by popping the stack.
From the code snippet you showed in the linked question, the state vector is allocated in the outer function, test_for_loop, which shouldn't be a big deal since it's done once at the beginning of execution. Using a variably sized state vector to index into an array with a splat (...) might be an issue, however, and that's done in test_function. Using something with fixed size like MVector might be better for that. It might, however, be better still, to use a state tuple and return a new rather than mutated state tuple at the end. The compiler is very good at turning that kind of thing into very efficient code because of immutability.
Note that by convention test_function should be called test_function! since it modifies its M argument and even more so if it modifies the state vector.
I would also note that this isn't a great question/answer pair since it's not standalone at all and really just a continuation of your other question. StackOverflow isn't very good for this kind of iterative question/discussion interaction, I'm afraid.
I was doing a graded programming assignment — an implementation of Rope data structure. The grader fed it an initial string and a series of edit operations. I did my development in C++ on a Linux machine. After testing my solution locally with small inputs (a string of ca 10 chars) I posted it to the grader, but got Segmentation Fault on one of the test cases.
I have generated a random input data with the maximum size given in the assignment specs (the string of 300k characters). I also got the Segmentation Fault locally. After a short debugging I found out that the leaves of my tree had random left and right pointers instead of NULL. After replacing the new Vertex calls with new Vertex() (the latter calls the default constructor, unlike the former which leaves the memory as-is) the code worked fine and got accepted by the grader.
This however makes me wonder — why did my code work correctly with a small input, both locally and on the grader’s machine? Is some amount of heap guaranteed to be zeroed when I run a process? Is this an artifact of some previously run program? What exactly is happening here?
Uninitialised objects can have any value. Uninitialised pointers can contain null, they can contain valid pointers by coincidence, or contain invalid pointers. It is completely undefined. Your program will behave accordingly. And it’s quite possible that memory is filled with some amount of zeroes followed by some amount of rubbish.
There may be a compiler option that will fill uninitialised variables with data that is likely to lead to a crash. More likely, there may be compiler options warning you when you use an uninitialised variable.
Since the stack grows torwards to smaller addesses, examining it with gdb is strenuous to me. So far I use
x/64xw 0xffffd0e8-64*4
if I want to see a value located at 0xffffd0e8 (on the stack) and the values following it on the stack (in this case the following 64 words on a 32 bit machine).
Is there an easier way?
Also, is there any way to automatically label the content on the stack with the corresponding variable names? Or to display only one word per line, not four?
If you don't have debug informations to help you, there is nothing else to do other than knowing the ABI and reading it by hand, with the help of GDB as you are doing, which can already do a good set of things only based on the ABI (like the backtrace, but without naming the callers).
If you do have debug informations of your binary, you can use info locals to list local variables of the selected stack frame, and navigate in the stack using frame, bt, info frame, info frame <address>, up, down, etc.
You can't really "annotate" the memory, but what you could do is create convenience variables to dynamically create GDB variables.
Regarding how to conveniently read large arrays of memory, I find very useful simply using print and casting addresses. For example: print (char(*)[]) 0xdeadbeef. And also using artifical arrays to print large regions. GDB will aggregate successive identical values, making it very clear and easy to read homogeneous memory regions (which is not really the case of the stack).
I wonder what assumptions compilers make about the relative locations of memory objects.
For example if we allocate two stack variables of size 1 byte each, right after another and initialize them both with zero, can a compiler optimize this case by only emitting one single instruction that overwrites both bytes in memory with zeros, because the compiler knows the relative position of both variables?
I am interested specifically in the more well known compilers like gcc, g++, clang, the Windows C/C++ compiler etc.
A compiler can optimize multiple assignments into one.
a = 0;
b = 0;
might become something like
*(short*)&a = 0;
The subtle part is "if we allocate two stack variables of size 1 byte each, right after another" since you cannot really do that. A compiler can shuffle stack positions around at will. Also, simply declaring variables will not necessarily mean any stack allocation. Variables might just be in registers. In C you would have to use alloca and even that does not provide "right after another".
Even more general, the C standard does not allow you to compare the memory positions of different objects. This is undefined behavior.
From http://www.golfscript.com/golfscript/syntax.html ,
Ruby is slow to start with so GolfScript is even slower. There is
nothing innately slow about GolfScript. Except for the string evaluate
method, everything could be statically compiled into C, and analysis
could be done to remove most if not all stack use. I do not plan on
making a more efficient interpreter, as the purpose of language is not
numerical analysis, however if any feels like creating one, I would be
delighted to use it.
Could someone illustrate with simple examples what are stacks, what does it mean to eliminate all stack use and how that could be done?
GolfScript is a stack-based language. Its behavior is similar to an RPN calculator. Each builtin consumes some number of the topmost stack values and pushes its results back onto the stack for future operations. If you want to test if a number is less than a constant, you'd use code like .5< where the . duplicates the value (because otherwise it would be consumed and lost) and then the constant is pushed. Finally < pops the copy and the constant and pushes back the result. A compiler could easily see a pattern like .X< and generate code which skips the intermediate steps (just "peek" at the top of the stack and compare). This would be in the category of "peephole" optmizations, which look for small output patterns and replace them with more efficient patterns.
Sometimes it would not be possible, if the values on the top of the stack came from complex (unpredictable) calculations.