Trying to manually commit during interceptor managed transaction - spring

This is a weird situation and I normally would never do it but our system has unfortunately now required this kind of scenario.
The System
We are running a Spring/Hibernate applications that is using OpenSessionInView and TransactionInterceptor to manage our transactions. For the most part it works great. However, we have recently required the need to spawn a number of threads to make some concurrent HTTP requests to providers.
The Problem
We need the entity that is passed into the thread to have all of the data that we have updated in our current transaction. The problem is we spawn the thread deep down in the guts of our service layer and it's very difficult to make a smaller transaction to allow this work. We tried originally just passing the entity to the thread and just calling:
leadDao.update(lead);
The problem is that we than get the error about the entity living in two sessions. Next we try to commit the original transaction and reopen as soon as the threads are complete.
This is what I have listed here.
try {
logger.info("------- BEGIN MULTITHREAD PING for leadId:" + lead.getId());
start = new Date();
leadDao.commitTransaction();
List<Future<T>> futures = pool.invokeAll(buyerClientThreads, lead.getAffiliate().getPingTimeout(), TimeUnit.SECONDS);
for (int i = 0; i < futures.size(); i++) {
Future<T> future = futures.get(i);
T leadStatus = null;
try {
leadStatus = future.get();
if (logger.isDebugEnabled())
logger.debug("Retrieved results from thread buyer" + leadStatus.getLeadBuyer().getName() + " leadId:" + leadStatus.getLead().getId() + " time:" + DateUtils.formatDate(start, "HH:mm:ss"));
} catch (CancellationException e) {
leadStatus = extractErrorPingLeadStatus(lead, "Timeout - CancellationException", buyerClientThreads.get(i).getBuyerClient().getLeadBuyer(), buyerClientThreads.get(i).getBuyerClient().constructPingLeadStatusInstance());
leadStatus.setTimeout(true);
leadStatus.setResponseTime(new Date().getTime() - start.getTime());
logger.debug("We had a ping that didn't make it in time");
}
if (leadStatus != null) {
completed.add(leadStatus);
}
}
} catch (InterruptedException e) {
logger.debug("There was a problem calling the pool of pings", e);
} catch (ExecutionException e) {
logger.error("There was a problem calling the pool of pings", e);
}
leadDao.beginNewTransaction();
The begin transaction looks like this:
public void beginNewTransaction() {
if (getCurrentSession().isConnected()) {
logger.info("Session is not connected");
getCurrentSession().reconnect();
if (getCurrentSession().isConnected()) {
logger.info("Now connected!");
} else {
logger.info("STill not connected---------------");
}
} else if (getCurrentSession().isOpen()) {
logger.info("Session is not open");
}
getCurrentSession().beginTransaction();
logger.info("BEGINNING TRANSAACTION - " + getCurrentSession().getTransaction().isActive());
}
The threads are using TransactionTemplates since my buyerClient object is not managed by spring (long involved requirements).
Here is that code:
#SuppressWarnings("unchecked")
private T processPing(Lead lead) {
Date now = new Date();
if (logger.isDebugEnabled()) {
logger.debug("BEGIN PINGING BUYER " + getLeadBuyer().getName() + " for leadId:" + lead.getId() + " time:" + DateUtils.formatDate(now, "HH:mm:ss:Z"));
}
Object leadStatus = transaction(lead);
if (logger.isDebugEnabled()) {
logger.debug("PING COMPLETE FOR BUYER " + getLeadBuyer().getName() + " for leadId:" + lead.getId() + " time:" + DateUtils.formatDate(now, "HH:mm:ss:Z"));
}
return (T) leadStatus;
}
public T transaction(final Lead incomingLead) {
final T pingLeadStatus = this.constructPingLeadStatusInstance();
Lead lead = leadDao.fetchLeadById(incomingLead.getId());
T object = transactionTemplate.execute(new TransactionCallback<T>() {
#Override
public T doInTransaction(TransactionStatus status) {
Date startTime = null, endTime = null;
logger.info("incomingLead obfid:" + incomingLead.getObfuscatedAffiliateId() + " affiliateId:" + incomingLead.getAffiliate().getId());
T leadStatus = null;
if (leadStatus == null) {
leadStatus = filterLead(incomingLead);
}
if (leadStatus == null) {
leadStatus = pingLeadStatus;
leadStatus.setLead(incomingLead);
...LOTS OF CODE
}
if (logger.isDebugEnabled())
logger.debug("RETURNING LEADSTATUS FOR BUYER " + getLeadBuyer().getName() + " for leadId:" + incomingLead.getId() + " time:" + DateUtils.formatDate(new Date(), "HH:mm:ss:Z"));
return leadStatus;
}
});
if (logger.isDebugEnabled()) {
logger.debug("Transaction complete for buyer:" + getLeadBuyer().getName() + " leadId:" + incomingLead.getId() + " time:" + DateUtils.formatDate(new Date(), "HH:mm:ss:Z"));
}
return object;
}
However, when we begin our new transaction we get this error:
org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is org.hibernate.TransactionException: Transaction not successfully started
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:660)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
My Goal
My goal is to be able to have that entity fully initalized on the other side or Does anyone have any ideas on how I can commit the data to the database so the thread can have a fully populated object. Or, have a way to query for a full object?
Thanks I know this is really involved. I apologize if I haven't been clear enough.
I have tried
Hibernate.initialize()
saveWithFlush()
update(lead)

I didn't follow everything - you can try one of this to workaround the issue that you get about the same object being associated with two sessions.
// do this in the main thread to detach the object
// from the current session
// if it has associations that also need to be handled the cascade=evict should
// be specified. Other option is to do flush & clear on the session.
session.evict(object);
// pass the object to the other thread
// in the other thread - use merge
session.merge(object)
Second approach - create a deep copy of the object and pass the copy. This can be easily achieved if your entity classes are serializable - just serialize the object and deserialize.

Thanks #gkamal for your help.
For everyone living in posterity. The answer to my dilemma was a left over call to hibernateTemplate instead of getCurrentSession(). I made the move about a year and a half ago and for some reason missed a few key places. This was generating a second transaction. After that I was able to use #gkamal suggestion and evict the object and grab it again.
This post helped me figure it out:
http://forum.springsource.org/showthread.php?26782-Illegal-attempt-to-associate-a-collection-with-two-open-sessions

Related

How to make Hibernate Transaction object in recursive call

I am getting below error in my console when I call recursively a method. The update query is running fine but it will not be updating record in the database.
org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is org.hibernate.TransactionException: Transaction not successfully started
public boolean abcMethod() {
Transaction txn = session.beginTransaction();
String querySqlSold = "UPDATE abc_table SET inventory_type='SOLD', status='ACTIVE' where set_id="
+ setId + " and game_num=" + gameMaster.getGameNum() + " and priceScheme=" + prizeSchemeId;
SQLQuery querySold = session.createSQLQuery(querySqlSold);
querySold.executeUpdate();
String querySqlSelect = "SELECT set_id FROM abc_table where inventory_type='UPCOMING' and `status`='ACTIVE' and game_num="
+ gameMaster.getGameNum() + " and priceScheme=" + prizeSchemeId;
List list = session.createSQLQuery(querySqlSelect).list();
int newSetId = Integer.valueOf(list.get(0).toString());
if (newSetId != 0) {
String querySqlCurrent = "UPDATE abc_table SET inventory_type='CURRENT' where game_num="
+ gameMaster.getGameNum() + " and priceScheme=" + prizeSchemeId + " and set_id=" + newSetId;
SQLQuery queryCurrent = session.createSQLQuery(querySqlCurrent);
queryCurrent.executeUpdate();
txn.commit();
return true;
} else {
JSONObject jsonObject = new JSONObject();
jsonObject.put("errorCode", "809");
jsonObject.put("errorMsg", "finished");
throw new CustomException(jsonObject.toString());
}
public void xyzMethod() {
abcMethod();
abcMethod();
}
You might try something like this
try {
tx = session.beginTransaction();
if (!tx.wasCommitted()){
tx.commit();
}
} catch (Exception exp) {
tx.rollback();
}
It should help you understand the problem better.

Is it possible to change the frequency in which spring actuator performs a health pulse?

I am attempting to look around to see how I can modify my actuator end points (specifically health) to limit its frequency. I want to see if I can set it up to being set to trigger once a minute for a specific dataset (ex mail) but leave it for others?
So far I can't seem to find that logic anywhere. The only known way I can think of is creating your own health server:
#Component
#RefreshScope
public class HealthCheckService implements HealthIndicator, Closeable {
#Override
public Health health() {
// check if things are stale
if (System.currentTimeMillis() - this.lastUpdate.get() > this.serviceProperties.getMonitorFailedThreshold()) {
String errMsg = '[' + this.serviceName + "] health status has not been updated in over ["
+ this.serviceProperties.getMonitorFailedThreshold() + "] milliseconds. Last updated: ["
+ this.lastUpdate.get() + ']';
log.error(errMsg);
return Health.down().withDetail(this.serviceName, errMsg).build();
}
// trace level since this could be called a lot.
if (this.detailMsg != null) {
Health.status(this.status);
}
Health.Builder health = Health.status(this.status);
return health.build();
}
/**
* Scheduled, low latency health check.
*/
#Scheduled(fixedDelayString = "${health.update-delay:60000}")
public void healthUpdate() {
if (this.isRunning.get()) {
if (log.isDebugEnabled()) {
log.debug("Updating Health Status of [" + this.serviceName + "]. Last Status = ["
+ this.status.getCode() + ']');
}
// do some sort of checking and update the value appropriately.
this.status = Status.UP;
this.lastUpdate.set(System.currentTimeMillis());
if (log.isDebugEnabled()) {
log.debug("Health Status of [" + this.serviceName + "] updated to [" + this.status.getCode() + ']');
}
}
}
I am not sure if there is a way to set this specifically in spring as a configuration or is the only way around this is to build a custom HealthIndicator?

How to obtain a specific person's tweets in a specific time interval (Twitter4J)?

I want to collect some specific people's tweets in recent one year. I'm using Twitter4J, like this:
Paging paging = new Paging(i, 200);
try {
statuses = twitter.getUserTimeline("martinsuchan",paging);
} catch (TwitterException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
But how can I filter the Tweets of that user for a certain time interval?
Any answer appreciated
You can filter the statuses locally, based on status.getCreatedAt(). Example:
try {
int statusesPerPage = 200;
int page = 1;
String username = "username";
Calendar cal = Calendar.getInstance();
cal.add(Calendar.YEAR, -1);
Twitter twitter = new TwitterFactory().getInstance();
Paging paging = new Paging(page, statusesPerPage);
List<Status> statuses = twitter.getUserTimeline(username, paging);
page_loop:
while (statuses.size() > 0) {
System.out.println("Showing #" + username + "'s home timeline, page " + page);
for (Status status : statuses) {
if (status.getCreatedAt().before(cal.getTime())) {
break page_loop;
}
System.out.println(status.getCreatedAt() + " - " + status.getText());
}
paging = new Paging(++page, statusesPerPage);
statuses = twitter.getUserTimeline(username, paging);
}
} catch (TwitterException te) {
te.printStackTrace();
}

How to get spark job status from program?

I am aware that hadoop REST API provides access to job status via program.
Similarly is there any way to get the spark job status in a program?
It is not similar to a REST API, but you can track the status of jobs from inside the application by registering a SparkListener with SparkContext.addSparkListener. It goes something like this:
sc.addSparkListener(new SparkListener {
override def onStageCompleted(event: SparkListenerStageCompleted) = {
if (event.stageInfo.stageId == myStage) {
println(s"Stage $myStage is done.")
}
}
})
Providing the answer for Java. In Scala would be almost similar just using SparkContext instead of JavaSparkContext.
Assume you have a JavaSparkContext:
private final JavaSparkContext sc;
Following code allow to get all info available from Jobs and Stages tabs:
JavaSparkStatusTracker statusTracker = sc.statusTracker();
for(int jobId: statusTracker.getActiveJobIds()) {
SparkJobInfo jobInfo = statusTracker.getJobInfo(jobId);
log.info("Job " + jobId + " status is " + jobInfo.status().name());
log.info("Stages status:");
for(int stageId: jobInfo.stageIds()) {
SparkStageInfo stageInfo = statusTracker.getStageInfo(stageId);
log.info("Stage id=" + stageId + "; name = " + stageInfo.name()
+ "; completed tasks:" + stageInfo.numCompletedTasks()
+ "; active tasks: " + stageInfo.numActiveTasks()
+ "; all tasks: " + stageInfo.numTasks()
+ "; submission time: " + stageInfo.submissionTime());
}
}
Unfortunately everything else is accessible only from scala Spark Context, so could be some difficulties to work with provided structures from Java.
Pools list: sc.sc().getAllPools()
Executor Memory Status: sc.sc().getExecutorMemoryStatus()
Executor ids: sc.sc().getExecutorIds()
Storage info: sc.sc().getRddStorageInfo()
... you can try to find there more useful info.
There's a (n)(almost) undocumented REST API feature that delivers almost everything you can see on the Spark UI:
http://<sparkMasterHost>:<uiPort>/api/v1/...
For local installation you can start from here:
http://localhost:8080/api/v1/applications
Possible end points you can find here: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala
There's a (n)(almost) undocumented REST API feature on the Spark UI that delivers metrics about the job and performance.
You can access it with:
http://<driverHost>:<uiPort>/metrics/json/
(UIPort is 4040 by default)
You can get Spark job status without using Spark Job History server too. You can use SparkLauncher 2.0.1 (even Spark 1.6 version will work too) for launching your Spark job from Java program:
SparkAppHandle appHandle = sparkLauncher.startApplication();
You can also add listener to startApplication() method:
SparkAppHandle appHandle = sparkLauncher.startApplication(sparkAppListener);
Where listener has 2 methods which will inform you about job state change and info change.
I implemented using CountDownLatch, and it works as expected. This is for SparkLauncher version 2.0.1 and it works in Yarn-cluster mode too.
...
final CountDownLatch countDownLatch = new CountDownLatch(1);
SparkAppListener sparkAppListener = new SparkAppListener(countDownLatch);
SparkAppHandle appHandle = sparkLauncher.startApplication(sparkAppListener);
Thread sparkAppListenerThread = new Thread(sparkAppListener);
sparkAppListenerThread.start();
long timeout = 120;
countDownLatch.await(timeout, TimeUnit.SECONDS);
...
private static class SparkAppListener implements SparkAppHandle.Listener, Runnable {
private static final Log log = LogFactory.getLog(SparkAppListener.class);
private final CountDownLatch countDownLatch;
public SparkAppListener(CountDownLatch countDownLatch) {
this.countDownLatch = countDownLatch;
}
#Override
public void stateChanged(SparkAppHandle handle) {
String sparkAppId = handle.getAppId();
State appState = handle.getState();
if (sparkAppId != null) {
log.info("Spark job with app id: " + sparkAppId + ",\t State changed to: " + appState + " - "
+ SPARK_STATE_MSG.get(appState));
} else {
log.info("Spark job's state changed to: " + appState + " - " + SPARK_STATE_MSG.get(appState));
}
if (appState != null && appState.isFinal()) {
countDownLatch.countDown();
}
}
#Override
public void infoChanged(SparkAppHandle handle) {}
#Override
public void run() {}
}

What is the use of "AsyncPattern" property of "OperationContractAttribute" + wcf?

Thus for used ajax enabled wcf services to get records from DB and display it in client without using AsyncPattern property of OperationContractAttribute....
When should i consider AsyncPattern property?
Sample of my operationcontract methods,
[OperationContract]
public string GetDesignationData()
{
DataSet dt = GetDesignationViewData();
return GetJSONString(dt.Tables[0]);
}
public string GetJSONString(DataTable Dt)
{
string[] StrDc = new string[Dt.Columns.Count];
string HeadStr = string.Empty;
for (int i = 0; i < Dt.Columns.Count; i++)
{
StrDc[i] = Dt.Columns[i].Caption;
HeadStr += "\"" + StrDc[i] + "\" : \"" + StrDc[i] + i.ToString() + "¾" + "\",";
}
HeadStr = HeadStr.Substring(0, HeadStr.Length - 1);
StringBuilder Sb = new StringBuilder();
Sb.Append("{\"" + Dt.TableName + "\" : [");
for (int i = 0; i < Dt.Rows.Count; i++)
{
string TempStr = HeadStr;
Sb.Append("{");
for (int j = 0; j < Dt.Columns.Count; j++)
{
if (Dt.Rows[i][j].ToString().Contains("'") == true)
{
Dt.Rows[i][j] = Dt.Rows[i][j].ToString().Replace("'", "");
}
TempStr = TempStr.Replace(Dt.Columns[j] + j.ToString() + "¾", Dt.Rows[i][j].ToString());
}
Sb.Append(TempStr + "},");
}
Sb = new StringBuilder(Sb.ToString().Substring(0, Sb.ToString().Length - 1));
Sb.Append("]}");
return Sb.ToString();
}
public DataSet GetDesignationViewData()
{
try
{
string connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["connectionString"].ConnectionString;
return SqlHelper.ExecuteDataset(connectionString, CommandType.StoredProcedure, DataTemplate.spDesignation_View);
}
catch (Exception err)
{
throw err;
}
}
AsyncPattern has a few uses- it's mainly a server performance optimization that allows you to free up worker pool request threads on blocking operations. For example, when a long-running blocking operation like DB access occurs, if you're using an async DB API on the server with AsyncPattern, the worker thread can return to the pool and service other requests. The original request is "awakened" later on another worker thread when the DB access completes, and the rest of the work is done (the service client just patiently waits- this is all transparent to it unless you're using an AsyncPattern-aware client and binding). This CAN allow your service to process more requests, if done carefully. To take advantage, you need to be using APIs on the server that have native async implementations. The only one I see that might be a candidate is the DB call that's happening in your SQLHelper.ExecuteDataset method- you'd have to read up on the underlying API to make sure a TRUE asynchronous option is available (presence of BeginXXX/EndXXX methods doesn't necessarily mean it's a TRUE async impl). The System.SqlClient stuff is truly async.
A word of caution: you have to be processing a lot of requests to make this worthwhile- there's a significant cost to code complexity and readability to split things up this way. You also need to understand multi-threaded programming very well- there are numerous pitfalls around locking, error handling, etc, that are well outside the scope of a SO post.
Good luck!

Resources