I'm trying to evaluate a particular cache design on my own simulator. I need traces of read and write operations. I found out that TPC workloads are memory stressful and I'd like to use them for my design evaluation. I was wondering if the traces are available or I need to run them on particular system and configuration and generate the traces myself?
I noticed that the original TPC website has links to download tools and scripts to run. But I'm looking to see if the traces with memory operation details are already available for architectural research purposes. If not what is the best way to generate them?
Any information would be appreciated.
Related
I understand why cold starts happen (Byte code needs to be turned into machine code through JIT compilation). However with all the generated meta data available for binaries these days I do not understand why there isn't a simple tool that automatically takes the byte code and turns ALL PATHS THROUGH THE CODE (auto discovered) into machine code specific for that target platform. That would mean the first request through any path (assume a rest api) would be fast and not require any further Just In Time Compilation.
We can create an automation test suite or load test to JIT all the paths before allowing the machine into the load balancer rotation (good best practice anyway). We can also flip the "always on" setting in cloud hosting providers to keep the warmed application from getting evicted from memory (requiring the entire process over again). However, it seems like such an archaic process to still be having in 2020.
Why isn't there a tool that does this? What is the limitation that prevents us from using meta data, debug symbols and/or other means to understand how to generate machine code that is already warm and ready for users from the start?
So I have been asking some sharp minds around my professional network and no one seems to be able to point out exactly what limitation makes this so hard to do. However, I did get a few tools on my radar that do what i'm looking for to some level.
Crossgen appears to be the most promising but it's far from widely used among the many peers I've spoken to. Will have to take a closer look.
Also several do some sort of startup task that runs some class initialization and also register them as singletons. I wouldn't consider those much different then just running integration or load tests on the application.
Most programming languages have some form of native image compiler tool. It's up to you to use them if that is what you are looking to do.
Providers are supposed to give you a platform for your application and there is a certain amount of isolation and privacy you should expect from your provider. They should not go digging into your application to figure out all its "paths". That would be very invasive.
Plus "warming up" all paths would be an overly resource intensive process for a provider to be obligated to perform for every application they host.
I'm slowly assembling the picture of how to use FreeRTOS in a real world application.
I've read a lot of partial features (stack supervision, memory, malloc etc...).
But haven't anywhere found a good instruction, what "supervision" to use to be able to follow the performance of tasks, system also after debugger is not connected anymore...
Can anyone help with some pointers, advices?
What features do you activate when a FreeRTOS app is designed?
How do you supervise, what is going on with tasks?
I'd rather read something short, to try feature by feature and see how it works. Something more for beginners. I understand, I have the documentation, but what I'm after is gradual introduction in FreeRTOS with examples. Maybe I overlooked a good info to read...
Let me illustrate it with few questions that I don't have the answers for:
Should I have a separate supervision task, that gathers the info about other tasks (state,memory,..) ?
What features should be used to supervise FreeRTOS based app in an "professional" way?
Should I use ITM/SWO, or maybe RTT?
Do you leave serial console on the system to supervise it?
Thanks in advance,
regards.
I'm slowly assembling the picture of how to use FreeRTOS in a real
world application. I've read a lot of partial features (stack
supervision, memory, malloc etc...). [...]
Can anyone help with some pointers, advices?
On the freeRTOS website, you find a lot of documentation for introduction as well as to understand detail features in depth.
I'd rather read something short, to try feature by feature and see how
it works. Something more for beginners. I understand, I have the
documenation, but what I'm after is gradual introduction in FreeRTOS
with examples. Maybe I overlooked a good info to read...
There is also a lot of third-party documentation. You may want to read general literature about RTOSes and how to use them: First, because many of them refer to one of the most well-known OSS implementation - freeRTOS. Second, because when working with RTOS, one has to take care of virtually the same aspects independent from which RTOS implementation is used.
How do you supervise, what is going on with tasks?
This depends on the purpose of supervision:
If the system that runs the RTOS is critical in some meaning
(e. g., it implements functional safety or
security requirements),
you'll probably need certain supervision measures at runtime that depend on the type and level of criticality.
Violating the expectations of such supervision usually triggers the system to switch off and fall into some kind of safe/secure operation mode.
More usually, you need supervision to debug or trace the application during development and testing to gain insights why certain errors appear in system behaviour, or how long the tasks/ISRs in the system need to execute and how long they are suppressing other contexts in doing so.
This will often allow you to attach a debug/trace adapter to the system all the time.
Violating the expectations here means guiding the developer to a remaining error in the system under development/test.
For many kinds of applications, you may have to measure (and log) the task timings over larger periods in order to get reliable statistics under controlled laboratory (or real-life) conditions.
Then you usually cannot keep a debug/trace adapter dongle at the embedded system because this would disturb the procedures under test. So, a logging concept/implementation is needed.
You have to evaluate the purpose of supervision. Then you can look up this board and others for more specific help and re-post further questions you may have.
But haven't anywhere found a good instruction, what "supervision" to use to be able to follow the performance of tasks, system also after debugger is not connected anymore...
What features do you activate when a FreeRTOS app is designed?
All your application requires (see above). One by one!
Let me illustrate with few questions, that I don't have the answers
for:
Should I have a separate supervision task, that gathers the info about other tasks (state,memory,..)?
What features should be used to supervise FreeRTOS based app in an "professional" way ?
Should I use ITM/SWO, or maybe RTT?
Do you leave serial console on the system to supervise it?
This all depends on the answers you find about the purpose of supervision.
The professional way to deal with it is a top-down approach to focus on the system requirements (and development needs), and to design/implement everything that is necessary to fulfill them.
If you are looking for a way to get a first insight how to activate ITM/SWO trace of freeRTOS for educational purposes, I can recommend the beautiful tutorial in the Atollic blog, a beginners' intro spread over several free articles, step-by-step.
For RTOS architecture hints, you may also like youtube introductions like the channel of beningo engineering, for example.
Is anybody know a good testing tool that can produce a graph containing the CPU cycle and RAM usage?
What I will do for ex. is I will run an application and while the application is running the testing tool will record CPU cycle and RAM Usage and it will make a graph as an output.
Basically what I'm trying to test is how much heavy load an application put on RAM and CPU.
Thanks in advance.
In case this is Windows the easiest way is probably Performance Monitor (perfmon.exe).
You can configure the counters you are interested in (Such as Processor Time/Commited Bytes/et) and create a Data Collector Set that measures these counters at the desired interval. There are even templates for basic System Performance Report or you can add counters for the particular process you are interested in.
You can schedule the time where you want to execute the sampling and you will be able to see the result using PerfMon or export to a file for further processing.
Video tutorial for the basics: http://www.youtube.com/watch?v=591kfPROYbs
Good Sample where it shows how to monitor SQL:
http://www.brentozar.com/archive/2006/12/dba-101-using-perfmon-for-sql-performance-tuning/
Loadrunner is the best I can think of ; but its very expensive too ! Depending on what you are trying to do, there might be cheaper alternatives.
Any tool which can either hook to the standard Windows or 'NIX system utilities can do this. This has been a defacto feature set on just about every commercial tool for the past 15 years (HP, IBM, Microfocus, etc). Some of the web only commercial tools (but not all) and the hosted services offer this as wekll. For the hosted services you will generally need to punch a hole through your firewall for them to get access to the hosts for monitoring purposes.
On the open source fron this is a totally mixed bag. Some have it, some don't. Some support one platform, but not others (i.e. support Windows, but not 'NIX or vice-versa).
What tools are you using? It is unfortunately common for people to have performance tools in use and not be aware of their existing toolset's monitoring capabilities.
All of the major commercial performance testing tools have this capability, as well as a fair number of the open source ones. The ability to integrate monitor data with response time data is key to the identification of bottlenecks in the system.
If you have a commercial tool and your staff is telling you that it cannot be done then what they are really telling you is that they don't know how to do this with the tool that you have.
It can be done using jmeter, once you install the agent in the target machine you just need to add the perfmon monitor to your test plan.
It will produce 2 result files, the pefmon file and the requests log.
You could also build a plot that compares the resource compsumtion to the load, and througput. The throughput stops increasing when some resource capacity is exceeded. As you can see in the image CPU time increases as the load increases.
JMeter perfmon plugin: http://jmeter-plugins.org/wiki/PerfMon/
I know this is an old thread but I was looking for the same thing today and as I did not found something that was simple to use and produced graphs I made this helper program for apachebench:
https://github.com/juanluisbaptiste/apachebench-graphs
It will run apachebench and plot the results and percentile files using gnuplot.
I hope it helps someone.
I am trying to implement some basic anti debugging functionality in my application. One area that I wanted to focus on in particular, is attempting to prevent people from easily taking a usable memory dump from my application. I read the article at:
http://www.codeproject.com/KB/security/AntiReverseEngineering.aspx
and that gave me a lot of tips for how to detect if a debugger is present, as well as some information on how I might prevent memory dumps. But the author notes that one should be careful about using these techniques, such as removing the executable header in memory. He mentions that there might be times when the OS or other programs may want to use this information, but I cannot see for what purpose.
Has anyone got some other tips as to how I could stop reverse engineers from dumping my program?
I am on Windows.
Kind regards,
Philip Bennefall
There is no reasonable way to prevent someone from capturing a memory dump of your process. For example, I could attach a kernel debugger to the system, break all execution, and extract your process' dump from the debugger. Therefore, I would focus on making analysis more difficult.
Here are some ideas:
Obfuscate and encrypt your executable code. Decrypt in-memory only, and do not keep decrypted code around for longer than you need it.
Do not store sensitive information in memory for longer than necessary. Use RtlZeroMemory or a similar API to clear out buffers that you are no longer using. This also applies to the stack (local variables and parameters).
I am facing a severe problem with my program, which gets reproduced only in the customer place. Putting logs, are not helping as I doubt the failure is happening in a third party dll. For some reasons, I couldn't get help from the library provider. I am thinking of producing a dump at the point of failure, so that to analyze it offline. Is this a recommended practice? Or any alternatives?
Yes, this is something that every program should have and utilize as often as possible.
I suggest that you don't use third party libraries. Create your own dumps instead. It's very simple and straight forward. You basically need to do the following:
Your program needs to access dbghelp.dll. It's a windows dll that allows you to create human readable call stacks etc. The debugger uses this dll to display data in your process. It also handles post mortem debugging, i.e. dumps of some sort. This dll can safely be distributed with your software. I suggest that you download and install Debugging Tools for Windows. This will give you access to all sorts of tools and the best tool WinDbg.exe and the latest dbghelp.dll is also in that distribution.
In dbghelp.dll you call e.g. MiniDumpWriteDump(), which will create the dump file and that's more or less it. You're done. As soon as you have that file in your hands, you can start using it. Either in the Visual Studio Debugger, which probably even might be associated with the .dmp file extension, or in WinDbg.
Now, there are a few things to think of while you're at it. When checking dump files like this, you need to generate .pdb files when you compile and link your executable. Otherwise there's no chance of mapping the dump data to human readable data, e.g. to get good callstacks and values of variables etc. This also means that you have to save these .pdb files. You need to be able to match them exactly against that very release. Since the dump files are date stamped with the date stamp of the executable, the debugger needs the exact pdb files. It doesn't matter if your code hasn't changed a single bit, if the .pdb files belong to another compilation session, you're toast.
I encourage every windows win32 developer to check out Oleg Starodumov's site DebugInfo.com. It contains a lot of samples and tutorials and how you can configure and tune your dump file generation. There are of course a myriad of ways to exclude certain data, create your custom debug message to attach to the dump etc.
Keep in mind that minidumps will contain very limited information about the application state at exception time. The trade off is a small file (around 50-100 kB depending on your settings). But if you want, you can create a full dump, which will contain the state of the whole application, i.e. globals and even kernel objects. These files can be HUGE and should only be used at extreme cases.
If there are legal aspects, just make sure your customers are aware of what you're doing. I bet you already have some contract where you aren't supposed to reveal business secrets or other legal aspects. If customers complain, convince them how important it is to find bugs and that this will improve the quality of the software drastically. More or less higher quality at the cost of nothing. If it doesn't cost them anything, that's also a good argument :)
Finally, here's another great site if you want to read up more on crash dump analysis: dumpanalysis.org
Hope this helps. Please comment if you want me to explain more.
Cheers !
Edit:
Just wanted to add that MiniDumpWriteDump() requires that you have a pointer to a MINIDUMP-EXCEPTION-INFORMATION (with underscores) struct. But the GetExceptionInformation() macro provides this for you at time of exception in your exception handler (structured exception handling or SEH):
__try {
}
__except (YourHandlerFunction(GetExceptionInformation())) {
}
YourHandlerFunction() will be the one taking care of generating the minidump (or some other function down the call chain). Also, if you have custom errors in your program, e.g. something happens that should not happen but technically is not an exception, you can use RaiseException() to create your own.
GetExceptionInformation() can only be used in this context and nowhere else during program execution.
Crash dumps are a pretty common troubleshooting method and can be very effective, especially for problems that only reproduce at the customer's site.
Just make sure the customer/client understands what you're doing and that you have permission. It's possible that a crash dump can have sensitive information that a customer may not want (or be permitted) to let walk out the door or over the wire.
Better than that there are libraries that will upload crash data back you.
BugDump and BugSplat
And there's the Microsoft way:
http://msdn.microsoft.com/en-us/library/aa936273.aspx
Disclaimer: I am not a lawyer, nor do I pretend to be one, this is not legal advice.
The data you can include in logs and crash dumps also depend on what domain you are working in. For example, medical equipment and patient information systems often contain sensitive data about patients that should not be visible to unauthorized persons.
The HIPAA Privacy Rule regulates
the use and disclosure of certain
information held by "covered entities"
(...) It establishes regulations for
the use and disclosure of Protected
Health Information (PHI). PHI is any
information held by a covered entity
which concerns health status,
provision of health care, or payment
for health care that can be linked to
an individual.[10] This is interpreted
rather broadly and includes any part
of an individual's medical record or
payment history. --Wikipedia
It should not be possible to link health information to an individual. The crash dumps and logs should be anonymized and stripped of any sensitive information, or not sent at all.
Maybe this does not apply to your specific case, so this is more of a general note. I think it applies to other domains that handle sensitive information, such as military and financial, and so on.
Basically the easiest way to produce a dumpfile is by using adplus. You don't need to change your code.
Adplus is part of the debugging tools for windows, as mentioned in the article above.
Adplus is basically a huge vbscript automation of windbg.
What you have to do to use adplus:
Download and install Debugging tools for windows to c:\debuggers
start your application
open a commandline and navigate to c:\debuggers
run this line "adplus -crash your_exe.exe"
reproduce the crash
you'll get a minidump with all the information you need.
you can open the crash dump in your favorite debugger.
within windbg, the command "analyze -v" helped me in at least 40% of all the crashes that only happened at customer site and were not reproducible in house.