I am going to program a little application for check any bottlenecks which might be in my Windows systems. So that, I think to use Performance Monitor to collect data from Disk, memory, network and CPU counters and save these data in a file .csv
Now I am wondering: what is the best choice for collect data?
My idea is to collect anything in one file, with all data log of counters, every 15 seconds, becouse a shorter detection might loads uselessly the system.
At work I have used Performance Monitor by Hexagora (its freeware) - don't know if you can record the result however as I have only ever used the graphical representations.
I have also used the Windows Performance Monitor, outputting .csv files (as you are doing), but if you are collecting alot of data I find it very messy.
A guide for best practice when monitoring performance from Microsoft is here. I often use this as a guide, hope this is helpful.
Related
I have been working on aggregation of streaming data, I found 2 tools to achieve the same. They are druid and pipelinedb. I have understood the implementation and architecture of the both. But couldn't figure out a way to benchmark these two. Is there any existing benchmark test that has been done? Or if I want to do a benchmarking of my own apart from the speed and scalability what are all the factors that I need to consider. Any ideas, links and help would be really appreciable. Also do share your own experience with pipelinedb and druid
Thanks
UPD:
After reading PipelineDB pages, I only wonder why do you need to compare such different things?
Druid is quite complex to install and maintain, it requires several external dependencies (such as zookeeper and hdfs/amazon, which must be maintained too).
And for that price you buy the key features of druid: column-oriented and distributed storage and processing. That also implies horizontal scalabitily out-of-the box, and it is completely automatic, you don't have even to think about it.
So if you don't need its distributed nature, I'd say you don't need druid at all.
FIRST VERSION:
I have no experience with pipelinedb (what is it? google shows nothing, pls share some link), but I have much experience with druid. So I would consider (apart from [query] speed and scalability):
ingesting performance (how many rows per sec/min/hour/... can be
inserted?)
RAM consumption of ingesting (how much RAM it needs to ingest with target speed?)
compression level (how many disk space needs one
hour/day/month/... of data?)
fault-tolerance (what happens when some
of the components fail? it is critical for my business?)
Caching (just keep in mind)
I am using Oracle as a DBMS and Tuxedo for application server.
Customer has the need to export data from Oracle to SAMFILE for interface purpose.
Unfortunately, the total number of records size is huge (over 10 million) so
I was wondering what is the best practice to extract big amounts of data to a file on the database server.
I am used to creating a cursor and fetching a record then writing to file.
Is there a better i.e. faster way to handle this? It is a recurring task.
I suggest you read Adrian Billington's article on tuning UTL_FILE. It covers all the bases. Find it here.
The important thing is buffering records, so reducing the number of file I/O calls. You will need to benchmark the different implementations, to see which works best in your situation.
Pay attention to his advice on query performance. Optimising file I/O is pointless if most of the time is spent on data acquisition.
there are many files need to process with two computers real-timely,I want to distribute them to the two computers and these tasks need to be completed as soon as possibile(means real-time processing),I am thinking about the below plan:
(1) distributed queue like Gearman
(2）distributed computing platform like hadoop/spark/storm/s4 and so on
I have two questions
（1）what is the advantage and disadvantage between (1) and (2)?
（2) How to choose in (2),hadoop?spark?storm?s4?or other?
thanks!
Maybe I have not described the question clearly. In most case,there are 1000-3000 files with the same format , these files are independent,you do not need to care their order,the size of one file maybe tens to hundreds of KB and in the future, the number of files and size of single file will rise. I have wrote a program , it can process the file and pick up the data and then store the data in mongodb. Now there are only two computers, I just want a solution that can process these files with the program quickly（as soon as possibile） and is easy to extend and maintain
distributed queue is easy to use in my case bur maybe hard to extend and maintain , hadoop/spark is to "big" in the two computers but easy to extend and maintain, which is better, i am confused.
It depends a lot on the nature of your "processing". Some dimensions that apply here are:
Are records independent from each other or you need some form of aggregation? i.e: do you need some pieces of data to go together? Say, all transactions from a single user account.
Is you processing CPU bound? Memory bound? FileSystem bound?
What will be persisted? How will you persist it?
Whenever you see new data, do you need to recompute any of the old?
Can you discard data?
Is the data somewhat ordered?
What is the expected load?
A good solution will depend on answers to these (and possibly others I'm forgetting). For instance:
If computation is simple but storage and retrieval is the main concern, you should maybe look into a distributed DB rather than either of your choices.
It could be that you are best served by just logging things into a distributed filesystem like HDFS and then run batch computations with Spark (should be generally better than plain hadoop).
Maybe not, and you can use Spark Streaming to process as you receive the data.
If order and consistency are important, you might be better served by a publish/subscribe architecture, especially if your load could be more than what your two servers can handle, but there are peak and slow hours where your workers can catch up.
etc. So the answer to "how you choose?" is "by carefully looking at the constraints of your particular problem, estimate the load demands to your system and picking the solution that better matches those". All of these solutions and frameworks dominate the others, that's why they are all alive and kicking. The choice is all in the tradeoffs you are willing/able to make.
Hope it helps.
First of all, dannyhow is right - this is not what real-time processing is about. There is a great book http://www.manning.com/marz/ which says a lot about lambda archtecture.
The two ways you mentioned serves completly different purposes and are connected to the definition of word "task". For example, Spark will take a whole job you got for him and divide it into "tasks", but the outcome of one task is useless for you, you still need to wait for whole job to finish. You can create small jobs working on the same dataset and use spark's caching to speed it up. But then you won't get much advantage from distribution (if they have to be run one after another).
Are the files big? Are there connected somehow to each other? If yes, I'd go with Spark. If no, distributed queue.
I want to do load testing for 10 million users for my site. The site is a Java based web-app. My approach is to create a Jmeter test plan for all the links and then take a report for the 10 million users. Then use jvisualVM to do profiling and check if there are any bottlenecks.
Is there any better way to do this? Is there any existing demo for doing this? I am doing this for the first time, so any assistance will be very helpful.
You are on the correct path, but your load limit is of with a high factor.
Why I'm saying this is cause your site probably will need more machine to handle 10Milj Concurrent users. A process alone would probably struggle to handle concurrent 32K TCP-streams. Also do some math of the bandwidth it would take to actually handle 10Milj users.
Now I do not know what kind of service you thinking of providing on your site, but when thinking of that JVisualVM slows down processing by a factor 10 (or more for method tracing), you would not actually measure the "real world" if you got JMeter and JVisualVM to work at the same time.
JVisualVM is more useful when you run on lower loads.
To create a good measurement first make sure your have a good baseline.
Make a test with 10 concurrent users, connect up JVisuamVM and let it run for a while, not down all interesting values.
After you have your baseline, then you can start adding more load.
Add 10times the load (ea: 100 users), look at the changes in JVisualVM. Continue this until it becomes obvious that JVisualVM slows you down, for every time to add extra load, make sure you have written down the numbers your are interested in. Plot down the numbers in a graph.
Now... Interpolate the graph (by hand) for the number of users you want. This works for memory usage, disc access etc, but not for used CPU time, cause JVisualVM will eat CPU and give you invalid numbers on that (especially if you have method tracing turned on).
If you really want to go as high as 10Milj users, I would not trust JMeter either, I would write a little test program of my own that performs the test you want. This would be okey, since the the setting up the site to handle 10Milj will also take time, so spending a little extra time of the test tools are not a waste.
Just because you have 10 million users in the database, doesn't mean that you need to load test using that many users. Think about it - is your site really going to have 10 million simultaneous users? For web applications, a ratio of 1:100 registered users is common i.e. you are unlikely to have more than 100K users at any moment.
Can JMeter handle that kind of load? I doubt it. Please try faban instead. It is very light-weight and can support thousands of users on a single VM. You also have much better flexibility in creating your workload and can also automate monitoring of your entire test infrastructure.
Now to the analysis part. You didn't say what server you were using. Any Java appserver will provide sufficient monitoring support. Commercial servers provide nice GUI tools while Tomcat provides extensive monitoring via JMX. You may want to start here before getting down to the JVM level.
For the JVM, you really don't want to use VisualVM while running such a large performance test. Besides to support such a load, I assume you are using multiple appserver/JVM instances. The major performance issue is usually GC, so use the JVM options to collect and log GC information. You will have to post-process the data.
This is a non-trivial exercise - good luck!
There are two types of load testing - bottleneck identification and throughput. The question leads me to believe this is about bottlenecks, so number of users is a something of a red herring, instead the goal being for a given configuration finding areas that can be improved to increase concurrency.
Application bottlenecks usually fall into three categories: database, memory leak, or slow algorithm. Finding them involves putting the application in question under stress (i.e. load) for an extended period of time - at least an hour, perhaps up to several days. Jmeter is a good tool for this purpose. One of the things to consider is running the same test with cookie handling enabled (i.e. Jmeter retains cookies and sends with each subsequent request) and disabled - sometimes you get very different results and this is important because the latter is effectively a simulation of what some crawlers do to your site. Details for bottleneck detection follow:
Database
Tables without indices or SQL statements involving multiple joins are frequent app bottlenecks. Every database server I've dealt with, MySQL, SQL Server, and Oracle has some way of logging or identifying slow running SQL statements. MySQL has the slow query log, whereas SQL Server has dynamic management views that track the slowest running SQL. Once you've got your hands on the slow statements use explain plan to see what the database engine is trying to do, use any features that suggest indices, and consider other strategies - such as denormalization - if those two options do not solve the bottleneck.
Memory Leak
Turn on verbose garbage collection logging and a JMX monitoring port. Then use jConsole, which provides much better graphs, to observe trends. In particular leaks usually show up as filling the Old Gen or Perm Gen spaces. Leaks are a bottleneck with the JVM spends increasing amounts of time attempting garbage collection unsuccessfully until an OOM Error is thrown.
Perm Gen implies the need to increase the space as a command line parameter to the JVM. While Old Gen implies a leak where you should stop the load test, generate a heap dump, and then use Eclipse Memory Analysis Tool to identify the leak.
Slow Algorithm
This is more difficult to track down. The most frequent offenders are synchronization, inter process communication (e.g. RMI, web services), and disk I/O. Another common issue is code using nested loops (look mom O(n^2) performance!).
Best way I've found to find these issues absent some deeper knowledge is generating stack traces. These will tell what all threads are doing at a given point in time. What you're looking for are BLOCKED threads or several threads all accessing the same code. This usually points at some slowness within the codebase.
I blogged, the way I proceeded with the performance test:
Make sure that the server (hardware can be as per the staging/production requirements) has no other installations that can affect the performance.
For setting up the users in DB, a procedure can be used and can be called as a part of jmeter test plan.
Install jmeter on a separate machine, so that jmeter won't affect the performance.
Create a test plan in jmeter (as shown in the figure 1) for all the uri's, with response checking and timer based requests.
Take the initial benchmark, using jmeter.
Check for the low performance uri's. These are the points to expect for bottlenecks.
Try different options for performance improvement, but focus on only one bottleneck at a time.
Try any one fix from step 6 and then take an benchmark. If there is any improvement commit the changes and repeat from step 5. Otherwise revert and try for any other options from step 6.
The next step would be to use load balancing, hardware scaling, clustering, etc. This may include some physical setup and hardware/software cost. Give the results with the scalability options.
For detailed explanation: http://www.daemonthread.com/2011/06/site-performance-tuning-using-jmeter.html
I started using JMeter plugins.
This allows me to gather application metrics available over JMX to use in my Load Test.
Calling all database guys...
The situation is this:
I have a DB2 database that is being written to and read from. I need to do some performance testing on programmatically executed read/writes.
I know how to write a program to read/write to this database, but I am not sure as to what factors I should consider in my performance test.
Do I need to worry about the difference between one session reading/writing vs multiple sessions?
What is the best way to interact with DB2 itself to get the amount of time these executions take?
The process I am testing is basically like a continuous batch proccess, constantly taking messages and persisting them. There will probably only be one or two sessions max on the DB at any given time.
Is time it takes to read/write really the best metric?
I am sure there are plenty of tools for this sort of testing. Any advice is appreciated.
Further info:
One thing I am considering is to try is to run X number of reads/writes with my database API (homebrew) and try to "time" how long it takes. Unfortuneately DB2 will buffer these messages. Is there any way to get DB2 to do a callback when it is done with a read/write? Or some way to externally measure the time these operations take? (tool, etc)
What is the goal for your performance testing?. Is it to test the performance for concurrent users or is it to test the load for batch process. Based on this there are tools available to test this. You may want to look jmeter from Apache.
In that case, you may want to trigger couple of concurrent processes to simaltaneously CRUD the data and monitor the activity using performance expert or something similar to that. While you do that you may want to use larger output so that you would be able to find any bottlenecks with larger sets of data. search for performance tuning in IBM redbooks site and you will find some case studies for this.
One huge factor in DB2 performance is how Buffer Pools are configured. e.g. http://www.ibm.com/developerworks/data/library/techarticle/0212wieser/0212wieser.html