What are the tools and approaches you suggest to use for monitoring performance? My app is running on Tomcat, Spring 3, JPA, Hibernate and solr. I've noticed some lag/slow activity on a certain page.
The app does not have any code of cache setup. Or even connection pooling.
Pardon my beginner-style questions, I'm only entering the "performance monitoring" world just now...
I know couple of things to look at is- dabatase calls, connection pooling, indexed tables, caching etc.
You absolutely need connection pooling. It's a no-brainer, there are libraries like Bone CP, Commons DBCP or c3p0 that do this transparently.
You must implemented automated load/stress test. JMeter is pretty easy to use, other tools like that are Gatling and Grinder.
Enable SQL logging and statistics. Most likely too many or too complex queries are slowing down your page.
Use a profiler. Either commercial (JProfiler, YourKit) or the one included in JVisualVM.
You can use this application to monitor your application:
http://newrelic.com/
He has a free plan!
Related
I am pretty sure WebLogic, Websphere, Jboss EAP and other J2EE application server provides their own service to optimize database connection pooling. At this point is there any benefit to still integrate a Spring application with HikariCp? May be HikariCp performs better (in question) but there will be unavoidable issues for configuration and compatibility.
Better question is: what are the benefits to use HikariCp in a Spring application if it will deploy on major commercial application servers other than Tomcat etc..
There is an opinion that HikariCp is the fastest connection pool in java world, this graphics prove this fact. In this case criteria is statement Cycle in ops/ms. HikariCp is faster in two times than the closest rival. It's a source code of this benchmarks, if i were you, i would launch them on own PC with specific to my environment. Last link saves your time, you have already written benchmarks (it's benefits of open source). HTH
It looks like the performance of Spring (Boot) is relatively low compare to other frameworks at "Web Framework Benchmarks" website Web Framework Benchmarks. I looked at the source code (JSON serialization) and could not find anything odd. So I am wondering whether the low performance is caused by Tomcat or by the framework itself?
Slow? It seems to be more or less in the middle of the ranks. Which doesn't seem too bad considering that it is actually a framework. The definition of framework for that set of benchmarks is very liberal, given that it includes benchmarks for a pretty raw Undertow application and another for a raw Servlet.
But given that the Spring benchmark itself is running on Undertow, it wouldn't be Tomcat causing the issue. :)
Looking at the source for the various benchmarks, they are really not comparing like for like. For example, the Spring benchmark project builds a reasonably standard multi-tiered application with ORM entities mapped with JPA and Hibernate, re-usable repository classes, etc.
On the other hand the Wicket application just has a controller with a JDBC call. It even uses a hand-cranked template for generating JSON. You could write a Spring application that way if you wanted to, but nobody would be impressed.
The Undertow benchmark again does little more than query a database with JDBC and spit out a response. It's not surprising that it would be faster than itself with a full stack framework running on it.
Of course, they hopefully get you to think about your priorities in writing an application. Do you want a full stack framework, which lets you write less code, provides various security features, helps you to achieve code re-use, and make your application more testable and maintainable? Or are you willing to sacrifice all that for speed?
I am putting together a regular Java EE application on jboss7 that will use JPA in the data tier. I would like to make this application such that it scales up with load. While it is pretty clear how to scale up the web tier: create more machines and throw them behind a load balancer, scaling up the data tier is less so.
I can probably cluster my database (MySQL). Stil, that leaves the JPA layer unclustered. Ideally, JPA will scale up by using in (clustered) memory caching backed by MySQL.
When I look around, all information around JPA scaling seems to be 3-4 years old. People talk about ehcache, memcached and infinispan. I am not sure if this is still current.
Can someone tell me the state of the art in Java EE clustering and scaling, especially in the data tier.
Various caching strategies are still the way to scale JPA/Hibernate (you basically named the most popular options in your question). Nothing extraordinary happend since 4-5 years in this field, as far as I know. One more option you haven't mentioned is JBoss Cache. So the Second Level Cache for JPA/Hibernate still rules in this area.
Why no progress here? My wild guess is that first of all people, who need scalable application tend to ignore JPA and Hibernate in areas where high performance is needed. Usually people go with SQL dressed in Spring Framework JDBCTemplate helpers and transaction management. Then scalability is the matter of database capabilities in this area.
The other trend is to use No-SQL databases. There is plany of solutions: MongoDB, CouchoDB, Cassandra, Redis, to name a few. These are usually Google BigTable like key-value storages (this is oversimplification, but it is more or less the idea behind that approach) and they scale as hell, if you accept their limitations (relations are no longer managed easily, etc.).
There are many solutions, the two main categories of solutions are:
scaling the database
using a clustered cache to reduce database load
EclipseLink supports data partitioning for sharding data across a set of database instances,
see:
http://java-persistence-performance.blogspot.com/2011/05/data-partitioning-scaling-database.html
You can also use MySQL Cluster,
see:
http://www.mysql.com/products/cluster/
Oracle TopLink Grid provides EclipseLink JPA support for integration with Oracle Coherence as a distributed cache,
see:
http://www.oracle.com/technetwork/middleware/ias/tl-grid-097210.html
EclipseLink's cache supports clustering through cache coordination,
see:
http://wiki.eclipse.org/EclipseLink/Examples/JPA/CacheCoordination
Which JDBC connection pool library should I use for a new application project (not web application)?
Apache DBCP has enough unresolved issues which are pushed until 2.0 I think.
C3P0 development seems to be stopped.
And both of them looks relatively slow.
Proxool is dead.
DBPool has almost no community (at least I've found no public one - no forums, no mailing lists...)
Apache Tomcat Pool looks to be unusable without Tomcat
I've found SQL Datasources article at Oracle website, but it seems, that it can be applied only to applets running in containers like servlets and web services.
Should I choose BoneCP may be? I don't have any huge requirements. I just need a good and easy to use database connection pool, that is in active development. Library, whose author can respond to bug reports, answer some specific question etc.
BTW, actually, I'm using MySQL only. I've found, that MySQL driver supports DriverManager interface, but I'm unsure if it actually pools connections or not.
You've done good research. I'd say, go ahead with BoneCP. Some years ago I would have suggested c3p0, but it has currently some open and unresolved issues as well. Too bad, actually. The BoneCP author is by the way also active here at SO.
It is possible use Tomcat Connection Pool without Tomcat, it is just a couple of .jars that you can include to any software. I have successfully used Tomcat Connection Pool with Resin and Jetty application servers.
Another alternative is Vibur DBCP. This is a new, concurrent JDBC connection pool based on Java dynamic proxies. It is fast, has concise source code, supports fairness parameter, JDBC Statement caching, long running SQL queries logging, and many other features.
Vibur DBCP is available in the central Maven repo, the website shows the needed Maven dependency, various configuration examples (with Spring, Hibernate 3.x/4.x/5.x), and all configuration options.
Vibur DBCP requires Java 1.7.
Disclaimer: I'm the author of Vibur DBCP.
I know its too late for answer but I just found an interesting link which has given a broader scope to answer most your queries. But at the end HikariCP and BoneCP are much better than other options.
Does anybody has an experience with Spring Integration project as embedded ESB?
I'm highly interesting in such use cases as:
Reading files from directory on schedule basis
Getting data from JDBC data source
Modularity and possibility to start/stop/redeploy module on the fly (e.g. one module can scan directory on schedule basis, another call query from jdbc data source etc.)
repeat/retry policy
UPDATE:
I found answers on all my questions except "Getting data from JDBC data source". Is it technically possible?
Remember, "ESB" is just a marketing term designed to sell more expensive software, it's not a magic bullet. You need to consider the specific jobs you need your software to do, and pick accordingly. If Spring Integration seems to fit the bill, I wouldn't be too concerned if it doesn't look much like an uber-expensive server installation.
The Spring Integration JDBC adapters are available in 2.0, and we just released GA last week. Here's the relevant section from the reference manual: http://static.springsource.org/spring-integration/docs/latest-ga/reference/htmlsingle/#jdbc
This link describes the FileSucker with Spring Integration. Read up on your Enterprise Integration patterns for more info I think.
I kinda think you need to do a bit more investigation your self, or do a couple of tries on some of your usecases. Then we can discuss whats good and bad
JDBC Adapters appear to be a work in progress.
Even if there is no specific adapter available, remember that Spring Integration is a thin wrapper around POJOs. You'll be able to access JDBC in any component e.g. your service activators.
See here for a solution based on a polling inbound channel adapter too.