Using the Windows Performance Recorder, is it possible to generate an ETL file based on the tracing of a single process? The ETL files generated for all of the processes in the system result in ETL files measured in GBs for intervals as small as a couple of minutes.
ETW (kernel event) tracing is system wide and captures all processes.
I don't think it is possible to record ETW traces that record just one process (at least not with xperf or wpr). If your traces are too big then the best tactic is to make sure that the rest of the system is as quiet as possible so that it doesn't contribute too much data.
If the rest of the system is already quiet then the traces are probably big because ETW traces tend to be big. You can use trace compression to make them smaller on disk - see UIforETW for how this works - https://randomascii.wordpress.com/2015/09/24/etw-central/.
If the rest of the system is not already quiet then yes, it probably is contributing to bloat in the traces. Note that it may also be affecting performance, so that data is not irrelevant.
And, if you really do need single-process profiling consider using a different profiler. The Visual Studio profiler does per-process profiling.
Related
I have an application of my own that increases the CPU usage of a process on Windows (in this case: audiodg.exe, which handles audio in 'some' way). I want to measure the overall CPU performance of this process over a minute or so.
It is possible to attach Visual Studio to this process (run as administrator and just do Attach to Process...), so I can view the CPU and memory performance. However, this isn't very useful. The process in constantly around 1-2% of the total CPU, so the graph doesn't give any interesting visual information.
Moreover, I'm interested in an average over ~1 minute, which the Performance Profiler can't do (I think?).
What is the best way to get accurate average CPU performance data out of a Windows process? Are there any tools for this or API that can get me this data?
I think Visual Studio is not the best tool for that usage as it's very heavy on its own.
I would use Windows Performance Recorder where you can select what you want to record:
Then you start the logging, replicate the issue and then stop the recording.
After that, you can open the log in Windows Performance Analyzer and try to understand why the issue is occurring.
Alternatively, you could take the process memory dump with ProcDump.exe when the CPU spikes above certain thresholds and then from the dump try to investigate the problem.
I am running Windows 7 on which I want to do kernel debugging and I do not want to mess with boot loader. So I've downloaded LiveKd as suggested here and make it run and seems it is working. If I understand correct it is some kind of read only debugging. Here is mentioned that it is very limited and even breakpoint cannot be used. I would like to ask if is possible in this mode to periodically dump all the instructions that are being executed or basically all events which are happening on current OS? I would like to have some system wide strace (Linux users know) and to do some statistical analysis on this. I suppose it depends on more factors like installed debug symbols to begin able resolve addresses etc.
I'm not sure if debugger is the best tool you can use for tracing live system calls. As you've mentioned LiveKd session is quite limited and you are not allowed to place breakpoints in it (otherwise you would hang your own system). However, you still can create memory dumps using the .dump command (check windbg help: .hh .dump). Keep in mind though that getting a full dump (/f) of a running system might take a lot of time.
Moving back to the subject of your question, by using the "dump approach" you will miss many system calls as you will have only snapshots of a system at given points in time. So if you are looking for something similar to Linux strace I would recommend checking those tools:
Process Monitor (procmon) - it's a tool which will show you all I/O requests in the system, as well as operations performed on the registry or process activity events
Windows Performance Toolkit - it contains tools for collecting (WPR) and analysing (WPA) system and application tracing events. It might be a lot of events and it's really important to filter them accordingly to your needs. ETW (Event Tracing for Windows) is a huge subject and you probably will need to read some tutorials or books before you will be able to use it effectively (but it's really worth it!).
API Monitor - it's one of many (I consider it as one of the best) tracing applications - this tool will allow you to trace method calls in any of the running processes. It has a nice interface and even allows you to place breakpoints on methods you'd like to intercept.
There are many other tools which might be used for tracing on Windows, but I would start with the ones I listed above. You may also check a great book on this subject: Inside Windows Debugging. Good luck! :)
I'm looking for an equivalent of java -verbose:gc or any tool or code snippet to get the same information. Ideally, this would be something I could run in an unattended fashion on a service and log everything to a file. My use case is profiling GC-induced latency in a long-running service.
For noninvasive .NET GC profiling you have few options. You can either use CLR Memory Performance Counters or CLR Memory Event Tracing or some profiler (SciTech memory profiler has a nice command line tool that allows you to collect CLR profiling data in the production environment - other .NET profiles probably also expose such a feature).
I suppose that Performance Counters are the least invasive method, but they don't give you a detailed information on GC working - though you can see how many collections were performed (in each generation) as well as how much time your process spent in GC. To collect this information you may use perfmon, typeperf or Powershell (I once described different ways of using perf counters so you may have a look: http://lowleveldesign.wordpress.com/2012/04/19/diagnosing-applications-using-performance-counters/)
ETW events provide much more details on GC inner workings. You can configure the ETW provider manually (using logman or xperf for example) or use an excellent tool PerfView (as #Marc pointed in a comment). If you are only interested in GC events, check GC Only checkbox in the Collect windows:
There is a great episode of Defrag Tools dedicated to CLR GC profiling (part 4): http://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-36-CLR-GC-Part-4, I also recommend you checking the other parts as well as reading the PerfView documentation. PerfView is a really powerful tool and it even allows you to analyse .NET Heap and compare memory snapshots.
The last option (that is using a memory profiler) is probably the most invasive of the three methods, but sometimes might give you even more details on GC heaps (especially when you would like to analyse the objects graphs). I can't think of any good free GC Memory Profiler so probably you will need to pay to get one of those tools. I have some experience with SciTech Memory Profiler (it's pretty good and, as I mentioned earlier, they have a command line client that allows you to collect data on production). I also tried Visual Studio Memory profile - it's not bad but less powerfull than the SciTech one - and finally JetBrains and RedGate also sell memory profilers which are well know among .NET developers and probably comparable to SciTech.
Currently, I am searching for a solution that would allow me to monitor a .NET Windows Service Application, in the production environment, and collect memory dumps. I'd like to collect them based on some specific thresholds, at will and on application crashes. I am aware of various methods to achieve this, such as:
DebugDiag
Procdump
Through WER
ADPlus
WinDbg etc.
Some of the methods facilitate the collection during a crash, such as option #3, while others can trigger based on performance counters, such as #1 and #2. Any non-invasive debugger can help me achieve the collection, but I am not sure what's the performance impact of having one attached. For example, if I use Procdump with the -e switch to collect memory dumps on unhandled exceptions, what would be the overhead for the monitored application? Bear in mind that I am referring to a production environment.
I'd be grateful if I you could point me a source or method that explains the performance impact of attaching a non-invasive debugger for memory dump collection. Ideally, that'd be a quantitative measure, although it exceeds my expectations.
P.S: I am not referring to the time needed for the memory dump to be written in the disk, where the application is completely frozen. That's another thing.
Is anybody know a good testing tool that can produce a graph containing the CPU cycle and RAM usage?
What I will do for ex. is I will run an application and while the application is running the testing tool will record CPU cycle and RAM Usage and it will make a graph as an output.
Basically what I'm trying to test is how much heavy load an application put on RAM and CPU.
Thanks in advance.
In case this is Windows the easiest way is probably Performance Monitor (perfmon.exe).
You can configure the counters you are interested in (Such as Processor Time/Commited Bytes/et) and create a Data Collector Set that measures these counters at the desired interval. There are even templates for basic System Performance Report or you can add counters for the particular process you are interested in.
You can schedule the time where you want to execute the sampling and you will be able to see the result using PerfMon or export to a file for further processing.
Video tutorial for the basics: http://www.youtube.com/watch?v=591kfPROYbs
Good Sample where it shows how to monitor SQL:
http://www.brentozar.com/archive/2006/12/dba-101-using-perfmon-for-sql-performance-tuning/
Loadrunner is the best I can think of ; but its very expensive too ! Depending on what you are trying to do, there might be cheaper alternatives.
Any tool which can either hook to the standard Windows or 'NIX system utilities can do this. This has been a defacto feature set on just about every commercial tool for the past 15 years (HP, IBM, Microfocus, etc). Some of the web only commercial tools (but not all) and the hosted services offer this as wekll. For the hosted services you will generally need to punch a hole through your firewall for them to get access to the hosts for monitoring purposes.
On the open source fron this is a totally mixed bag. Some have it, some don't. Some support one platform, but not others (i.e. support Windows, but not 'NIX or vice-versa).
What tools are you using? It is unfortunately common for people to have performance tools in use and not be aware of their existing toolset's monitoring capabilities.
All of the major commercial performance testing tools have this capability, as well as a fair number of the open source ones. The ability to integrate monitor data with response time data is key to the identification of bottlenecks in the system.
If you have a commercial tool and your staff is telling you that it cannot be done then what they are really telling you is that they don't know how to do this with the tool that you have.
It can be done using jmeter, once you install the agent in the target machine you just need to add the perfmon monitor to your test plan.
It will produce 2 result files, the pefmon file and the requests log.
You could also build a plot that compares the resource compsumtion to the load, and througput. The throughput stops increasing when some resource capacity is exceeded. As you can see in the image CPU time increases as the load increases.
JMeter perfmon plugin: http://jmeter-plugins.org/wiki/PerfMon/
I know this is an old thread but I was looking for the same thing today and as I did not found something that was simple to use and produced graphs I made this helper program for apachebench:
https://github.com/juanluisbaptiste/apachebench-graphs
It will run apachebench and plot the results and percentile files using gnuplot.
I hope it helps someone.