I'm using profiler in Visual Studio 2008 like this, but when I profiling these codes I can only find the methods written by myself in "Call Tree" view. How can I track the inner/private methods defined in .NET Framework?
I do have to ask what is your purpose. Are you trying to find and remove performance problems? If so, any fixes you make can only be in your code. A simple way to find them is to run the program under the IDE and, while it is being slow, pause it and record the call stack. Do this several times. If there is any line of code that appears on multiple samples, those samples are occurring within work being requested by it, so if you can find a way to avoid doing that line of code, you will save a large fraction of time. The call tree may show such a line, but to see how much time it saves, you have to sum over all the branches in the tree where it occurs. You don't have that problem if you just sample the stack.
Here's a more complete explanation.
Here's a blow-by-blow example.
There are some myths re. performance tuning.
Related
This question is not about finding out who retained a particular object but rather looking at a section of code that appears from the profiler to have excessive retain/release calls and figuring out which objects are responsible.
I have a Swift application that after initial porting was spending 90% of its time in retain/release code. After a great deal of restructuring to avoid referencing objects I have gotten that down to about 25% - but this remaining bit is very hard to attribute. I can see that a given chunk of it is coming from a given section of code using the profiler, but sometimes I cannot see anything in that code that should (to my understanding) be causing a retain/release. I have spent time viewing the assembly code in both Instruments (with the side-by-side view when it's working) and also the output of otool -tvV and sometimes the proximity of the retain/release calls to a recognizable section give me a hint as to what is going on. I have even inserted dummy method calls at places just to give me a better handle on where I am in the code and turned off optimization to limit code reordering, etc. But in many cases it seems like I would have to trace the code back to follow branches and figure out what is on the stack in order to understand the calls and I am not familiar enough with x86 to know know if that is practical. (I will add a couple of screenshots of the assembly view in Instruments and some otool output for reference below).
My question is - what else can I be doing to debug / examine / attribute these seemingly excessive retain/release calls to particular code? Is there something else I can do in Instruments to count these calls? I have played around with the allocation view and turned on the reference counting option but it didn't seem to give me any new information (I'm not actually sure what it did). Alternately, if I just try harder to interpret the assembly should I be able to figure out what objects are being retained by it? Are there any other tools or tricks I should know on that front?
EDIT: Rob's info below about single stepping into the assembly was what I was looking for. I also found it useful to set a symbolic breakpoint in XCode on the lib retain/release calls and log the item on the stack (using Rob's suggested "p (id)$rdi") to the console in order to get a rough count of how many calls are being made rather than inspect each one.
You should definitely focus on the assembly output. There are two views I find most useful: the Instruments view, and the Assembly assistant editor. The problem is that Swift doesn't support the Assembly assistant editor currently (I typically do this kind of thing in ObjC), so we come around to your complaint.
It looks like you're already working with the debug assembly view, which gives somewhat decent symbols and is useful because you can step through the code and hopefully see how it maps to the assembly. I also find Hopper useful, because it can give more symbols. Once you have enough "unique-ish" function calls in an area, you can usually start narrowing down how the assembly maps back to the source.
The other tool I use is to step into the retain bridge and see what object is being passed. To do this, instruction-step (^F7) into the call to swift_bridgeObjectRetain. At that point, you can call:
p (id)$rdi
And it should print out at least some type information about the what's being passed ($rdi is correct on x86_64 which is what you seem to be working with). I don't always have great luck extracting more information. It depends on exactly is in there. For example, sometimes it's a ContiguousArrayStorage<Swift.CVarArgType>, and I happen to have learned that usually means it's an NSArray. I'm sure better experts in LLDB could dig deeper, but this usually gets me at least in the right ballpark.
(BTW, I don't know why I can't call p (id)$rdi before jumping inside bridgeObjectRetain, but it gives strange type errors for me. I have to go into the function call.)
Wish I had more. The Swift tool chain just hasn't caught up to where the ObjC tool chain is for tracing this kind of stuff IMO.
I am doing memory profiling for my application using Visual Studio diagnostic tool. I find that there is Node takes up a lot of memory (based on Inclusive Size Diff. (bytes). (see below #1). And when I click on the first instance of Node, 'Referenced Objects', I see Node is referencing other Node. And I see something like 'Overlapped data' in the attribute.
How can I find out where is creating these Node as they are from mscorlib.ni.dll.
The weapon of choice when you are rooting through these .NET Framework objects is a good decompiler. I use Reflector, there are others.
You see an opaque Node<T> object back. Just type it into the Search box, out pops but a few types that use it. Most are in the System.Collections.Concurrent namespace. Well, look no further, the profiler already told you about that one. Clearly it is the Stack<T> class in the System.Collections.Concurrent namespace that's storing Nodes.
Your profiler told you there is just one Stack<> class object that owns these objects. Well good, that narrows it down to just a single object. It just happens to have 208 elements. Hmm, well, not that much, is it?
That's not where you have to stop, the Stack<> class is a pretty useless class, nobody ever actually uses it in their code. Keep using the decompiler and let it search for usages of that class.
Ah, nice, that's a very short list as well. You see System.Data.ProviderBase show up several times, hmm, this question probably isn't related to querying dbases. Only other set of references is System.PinnableBufferCache.
"Pinnable buffers", whoa, that's a match. Pinning a buffer is important when you ask native code to get a job done to fill a managed array. With BeginRead(), the universal asynchronous I/O call. The driver needs a stable reference to the array while it is working on the async I/O request. Getting a stable buffer requires pinning in .NET. And big bingo in the profiler data, you see OverlappedData, the core data-structure in Windows to do async I/O.
Long story short, you found this guy's project. Programmers noticed, not very often.
Knowing when the stop profiling is very important. You cannot change code written by another programmer. And nobody at Microsoft thinks that guy did anything wrong. He didn't, caches are Good Things.
You are most definitely done. Congratulations.
Is this possible in Visual Studio to generate a text list of the methods that are being called, and possibly execution time [of returned methods]? I know about a lot of approaches to profile an application, but I think that having a clear - even if long - callstack would be helpful in improving launch performances.
Here's a code project article about this
It basically boils down to using the GetThreadContext() to capture the context of the current thread and then using StackWalk64() to walk the stack. Alternatively you can also use CaptureStackBackTrace().
These functions will only get you the list of addresses that make the stack. To get the names of the functions and line numbers you'll need to use functions from dbghelp.dll like
SymGetModuleInfo64()
I am currently developing software for the web using visual studios and in the future I will be writing other thing using C# (among other languages). My question, is there a way to see the step by step execution of the code you wrote, outlining all the changes and procedures.
as an example, can i somehow see something that reads: "the function was executed with parameter value of 5. the value of y changed from 4 to 8. the string 'wording' now contains 20 characters. the function ABC executed for a second time with parameter 47." well you get the gist of it. I want to "read" my code after it executes. I feel like this would be the best debugger. Which brings me to my underlying goal of better debugging. So if you have any other 'techniques' for debugging, they would come a long way for a newbie.
Runtime Flow can show all executed functions with parameters, though configuring it for web projects is complicated.
You can look and learn how to use a debugger for instance the gdb debugger that allows you to step thru the instructions one at a time, and there is even reverse debugging now where you can step backwards thru the program ad execute the program backwards. Look at what gdb can do, and it will tell you what is possible.
I've been testing out the performance and memory profiler AQTime to see if it's worthwhile spending those big $$$ for it for my Delphi application.
What amazes me is how it can give you source line level performance tracing (which includes the number of times each line was executed and the amount of time that line took) without modifying the application's source code and without adding an inordinate amount of time to the debug run.
The way that they do this so efficiently makes me think there might be some techniques/technologies used here that I don't know about that would be useful to know about.
Do you know what kind of methods they use to capture the execution line-by-line without code changes?
Are there other profiling tools that also do non-invasive line-by-line checking and if so, do they use the same techniques?
I've made an open source profiler for Delphi which does the same:
http://code.google.com/p/asmprofiler/
It's not perfect, but it's free :-). Is also uses the Detour technique.
It stores every call (you must manual set which functions you want to profile),
so it can make an exact call history tree, including a time chart (!).
This is just speculation, but perhaps AQtime is based on a technology that is similar to Microsoft Detours?
Detours is a library for instrumenting
arbitrary Win32 functions on x86, x64,
and IA64 machines. Detours intercepts
Win32 functions by re-writing the
in-memory code for target functions.
I don't know about Delphi in particular, but a C application debugger can do line-by-line profiling relatively easily - it can load the code and associate every code path with a block of code. Then it can break on all the conditional jump instructions and just watch and see what code path is taken. Debuggers like gdb can operate relatively efficiently because they work through the kernel and don't modify the code, they just get informed when each line is executed. If something causes the block to be exited early (longjmp), the debugger can hook that and figure out how far it got into the blocks when it happened and increment only those lines.
Of course, it would still be tough to code, but when I say easily I mean that you could do it without wasting time breaking on each and every instruction to update a counter.
The long-since-defunct TurboPower also had a great profiling/analysis tool for Delphi called Sleuth QA Suite. I found it a lot simpler than AQTime, but also far easier to get meaningful result. Might be worth trying to track down - eBay, maybe?