In a Quarkus application, I want to consume a Kafka message and persist its information in the database using an Entity Manager.
This is what I got so far:
#ApplicationScoped
public class ClientEventConsumer {
#Inject ClientRepository repository;
private final BlockingQueue<Message<ClientEvent>> messages = new LinkedBlockingQueue<>();
void startup(#Observes StartupEvent startupEvent) {
ScheduledExecutorService executor = Executors.newScheduledThreadPool(5);
executor.scheduleAtFixedRate(() -> {
if (messages.size() > 0) {
try {
Message<ClientEvent> message = messages.take();
ClientEvent clientEvent = message.getPayload();
ClientEntity clientEntity = new ClientEntity();
clientEntity.setId(clientEvent.getId());
clientEntity.setName(clientEvent.getName());
repository.merge(clientEntity);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}, 1000, 500, TimeUnit.MILLISECONDS);
}
#Incoming("qualificationCheck")
public CompletionStage<Void> consume(Message<ClientEvent> msg) {
messages.add(msg);
return msg.ack();
}
}
But with this approach the message gets acknowledged before the record is actually persisted in the database. Is there a way to only acknowledge the message if the JPA transaction succeeded?
Related
We have an order managament system in which after every order state update we make an api call to our client to keep them updated. We do this by first sending a message to a sqs queue and inside a consumer we hit our clients api. The processing on consumer side usually takes about 300-350ms but The approximate age of oldest message in sqs dashboard is showing spikes that reach upto 50-60 secs.
Seeing this I thought that maybe one consumer is not enough for our load and I created multiple DMLC beans and multiple copies of our consumer class. I attached these consumer classes as listeners in these DMLCs. But I have not seen any improvement in approximate age of oldest message.
I am guessing that maybe only one of the DMLC is processing these messages and others are just sitting idle.
I added multiple DMLCs because there are other places in pur codebase where the same thing is used, But now I am not sure if this is the correct way to solve the problem.
My Consumer class looks like this:
#Component
#Slf4j
#RequiredArgsConstructor
public class HOAEventsOMSConsumer extends ConsumerCommon implements MessageListener {
private static final int MAX_RETRY_LIMIT = 3;
private final OMSEventsWrapper omsEventsWrapper;
#Override
public void onMessage(Message message) {
try {
TextMessage textMessage = (TextMessage) message;
String jmsMessageId = textMessage.getJMSMessageID();
ConsumerLogging.logStart(jmsMessageId);
String text = textMessage.getText();
log.info(
"Inside HOA Events consumer Request jmsMessageId:- " + jmsMessageId + " Text:- "
+ text);
processAndAcknowledge(message, text, textMessage);
} catch (JMSException e) {
log.error("JMS Exception while processing surge message", e);
}
}
private void processAndAcknowledge(Message message, String text, TextMessage textMessage) throws JMSException {
try {
TrimmedHOAEvent hoaEvent = JsonHelper.convertFromJsonPro(text, TrimmedHOAEvent.class);
if (hoaEvent == null) {
throw new OMSValidationException("Empty message in hoa events queue");
}
EventType event = EventType.fromString(textMessage.getStringProperty("eventType"));
omsEventsWrapper.handleOmsEvent(event,hoaEvent);
acknowledgeMessage(message);
} catch (Exception e) {
int retryCount = message.getIntProperty("JMSXDeliveryCount");
log.info("Retrying... retryCount: {}, HOAEventsOMSConsumer: {}", retryCount, text);
if (retryCount > MAX_RETRY_LIMIT) {
log.info("about to acknowledge the message since it has exceeded maximum retry limit");
acknowledgeMessage(message);
}
}
}
}
And my DMLC configuration class looks like this:
#Configuration
#SuppressWarnings("unused")
public class HOAEventsOMSJMSConfig extends JMSConfigCommon{
private Boolean isSQSQueueEnabled;
#Autowired
private HOAEventsOMSConsumer hoaEventsOMSConsumer;
#Autowired
private HOAEventsOMSConsumer2 hoaEventsOMSConsumer2;
#Autowired
private HOAEventsOMSConsumer3 hoaEventsOMSConsumer3;
#Autowired
private HOAEventsOMSConsumer4 hoaEventsOMSConsumer4;
#Autowired
private HOAEventsOMSConsumer5 hoaEventsOMSConsumer5;
#Autowired
private HOAEventsOMSConsumer6 hoaEventsOMSConsumer6;
#Autowired
private HOAEventsOMSConsumer7 hoaEventsOMSConsumer7;
#Autowired
private HOAEventsOMSConsumer8 hoaEventsOMSConsumer8;
#Autowired
private HOAEventsOMSConsumer9 hoaEventsOMSConsumer9;
#Autowired
private HOAEventsOMSConsumer10 hoaEventsOMSConsumer10;
public HOAEventsOMSJMSConfig(IPropertyService propertyService, Environment env) {
queueName = env.getProperty("aws.sqs.queue.oms.hoa.events.queue");
endpoint = env.getProperty("aws.sqs.queue.endpoint") + queueName;
JMSConfigCommon.accessId = env.getProperty("aws.sqs.access.id");
JMSConfigCommon.accessKey = env.getProperty("aws.sqs.access.key");
try {
ServerNameCache serverNameCache = CacheManager.getInstance().getCache(ServerNameCache.class);
if (serverNameCache == null) {
serverNameCache = new ServerNameCache();
serverNameCache.set(InetAddress.getLocalHost().getHostName());
CacheManager.getInstance().setCache(serverNameCache);
}
this.isSQSQueueEnabled = propertyService.isConsumerEnabled(serverNameCache.get(), false);
} catch (Exception e) {
this.isSQSQueueEnabled = false;
}
}
#Bean
public JmsTemplate omsHOAEventsJMSTemplate(){
SQSConnectionFactory sqsConnectionFactory;
if (endpoint.toLowerCase().contains("localhost")) {
sqsConnectionFactory =
SQSConnectionFactory.builder().withEndpoint(getEndpoint("sqs")).build();
} else {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withNumberOfMessagesToPrefetch(10)
.withEndpoint(endpoint)
.build();
}
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(sqsConnectionFactory);
JmsTemplate jmsTemplate = new JmsTemplate(cachingConnectionFactory);
jmsTemplate.setDefaultDestinationName(queueName);
jmsTemplate.setDeliveryPersistent(false);
jmsTemplate.setSessionTransacted(false);
jmsTemplate.setSessionAcknowledgeMode(SQSSession.UNORDERED_ACKNOWLEDGE);
return jmsTemplate;
}
#Bean
public DefaultMessageListenerContainer jmsListenerHOAEventsListenerContainer() {
SQSConnectionFactory sqsConnectionFactory;
if (endpoint.toLowerCase().contains("localhost")) {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withEndpoint(getEndpoint("sqs"))
.build();
} else {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withNumberOfMessagesToPrefetch(10)
.withEndpoint(endpoint)
.build();
}
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(sqsConnectionFactory);
dmlc.setDestinationName(queueName);
dmlc.setAutoStartup(isSQSQueueEnabled);
dmlc.setMessageListener(hoaEventsOMSConsumer);
dmlc.setSessionTransacted(false);
dmlc.setSessionAcknowledgeMode(SQSSession.UNORDERED_ACKNOWLEDGE);
return dmlc;
}
#Bean
public DefaultMessageListenerContainer jmsListenerHOAEventsListenerContainerNo2() {
SQSConnectionFactory sqsConnectionFactory;
if (endpoint.toLowerCase().contains("localhost")) {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withEndpoint(getEndpoint("sqs"))
.build();
} else {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withNumberOfMessagesToPrefetch(10)
.withEndpoint(endpoint)
.build();
}
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(sqsConnectionFactory);
dmlc.setDestinationName(queueName);
dmlc.setAutoStartup(isSQSQueueEnabled);
dmlc.setMessageListener(hoaEventsOMSConsumer2);
dmlc.setSessionTransacted(false);
dmlc.setSessionAcknowledgeMode(SQSSession.UNORDERED_ACKNOWLEDGE);
return dmlc;
}
#Bean
public DefaultMessageListenerContainer jmsListenerHOAEventsListenerContainerNo3() {
SQSConnectionFactory sqsConnectionFactory;
if (endpoint.toLowerCase().contains("localhost")) {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withEndpoint(getEndpoint("sqs"))
.build();
} else {
sqsConnectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withNumberOfMessagesToPrefetch(10)
.withEndpoint(endpoint)
.build();
}
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(sqsConnectionFactory);
dmlc.setDestinationName(queueName);
dmlc.setAutoStartup(isSQSQueueEnabled);
dmlc.setMessageListener(hoaEventsOMSConsumer3);
dmlc.setSessionTransacted(false);
dmlc.setSessionAcknowledgeMode(SQSSession.UNORDERED_ACKNOWLEDGE);
return dmlc;
}
}
If this question is already answered somehwere else, then please point me towards that.
We have Spring JMS application ( deployed on K8s) which processes about 100 - 400 messages/sec. The application consumes messages from IBM MQ and processes them. Off late we have started noticing messages getting dropped whenever K8s pod restarts or deployments are done even though we have message ack in place. I am looking for a solution here to resolve this issue.
Software
Version
Spring Boot
2.1.7.RELEASE
IBM MQ Client
9.1.0.5
JMS
2.0.1
Java
11
#Configuration
#EnableJms
public class MqConfiguration {
#Bean
public MQConnectionFactory mqConnectionFactory(Servers configProperties) {
MQConnectionFactory mqConnectionFactory = new MQConnectionFactory();
try {
mqConnectionFactory.setHostName(configProperties.getHost());
mqConnectionFactory.setQueueManager(configProperties.getQueueManager());
mqConnectionFactory.setPort(Integer.valueOf(configProperties.getPort()));
mqConnectionFactory.setChannel(configProperties.getChannel());
mqConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
mqConnectionFactory.setCCSID(1208);
mqConnectionFactory.setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT);
} catch (Exception e) {
logger.logError(mqConnectionFactory, ,
"Failed to create MQ ConnectionFactory", String.valueOf(HttpStatus.SC_BAD_REQUEST), e);
}
return mqConnectionFactory;
}
#Bean(name = "messageListenerContainerFactory")
public DefaultJmsListenerContainerFactory provideJmsListenerContainerFactory(
MQConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setErrorHandler(new ErrorHandler() {
#Override
public void handleError(Throwable t) {
ServiceMetrics metrics = new ServiceMetrics();
metrics.setCorrelationId(UUID.getUUID());
logger.logError(factory, "Exception occured at JMS Factory Container Listener", String.valueOf(HttpStatus.SC_BAD_REQUEST), t);
}
});
return factory;
}
#Bean(name = "jmsQueueTemplate")
public JmsTemplate provideJmsQueueTemplate(MQConnectionFactory connectionFactory) {
return new JmsTemplate(connectionFactory);
}
}
#Configuration
public class AsyncConfiguration {
#Autowired
private Servers configProperties;
#Bean(name = "asyncTaskExecutor")
public ExecutorService getAsyncTaskExecutor() {
String THREAD_POOL = "th-pool-";
return getExecutor(THREAD_POOL, 70,true);
}
private ExecutorService getExecutor(String threadName, int maxPoolSize, boolean cached) {
final ThreadFactory threadFactory = new CustomizableThreadFactory(threadName);
if (cached) {
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(0, maxPoolSize,
60L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(), threadFactory);
threadPoolExecutor.setRejectedExecutionHandler((r, executor) -> {
if (!executor.isShutdown()) {
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
throw new RejectedExecutionException(e);
}
}
});
return threadPoolExecutor;
} else {
return new ThreadPoolExecutor(maxPoolSize, maxPoolSize,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>(),
threadFactory);
}
}
#Component
public class InputQueueListener {
#Autowired
private ExecutorService asyncTaskExecutor;
#JmsListener(destination = "${mqserver.queue}", containerFactory = "messageListenerContainerFactory", concurrency = "1-16")
public void processXMLMessage(Message message) {
CompletableFuture.runAsync(() -> processMessage(message), asyncTaskExecutor);
}
private void processMessage(Message message) {
String inputXmlMessage = null;
boolean isSuccess = false;
try {
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
inputXmlMessage = textMessage.getText();
} else if (message instanceof BytesMessage) {
BytesMessage byteMessage = (BytesMessage) message;
inputXmlMessage = CommonHelperUtil.getMessageFromBytes(byteMessage);
} else {
logger.logError(null, "Invalid message type received while converting Message to XML", String.valueOf(HttpStatus.SC_BAD_REQUEST));
errorQueuePublisher.publishErrorMessage(message);
try {
message.acknowledge();
} catch (JMSException jmsException) {
logger.logError(null, null, "Failed to Acknowledge XML message.",
String.valueOf(HttpStatus.SC_BAD_REQUEST), jmsException);
}
}
-
-
if (isSuccessProcessed) {
message.acknowledge();
} else {
message.acknowledge();
// Publishing back to the same queue
publishForRetry.publishMessageForRetry(message);
}
} catch (Exception e) {
if (StringUtils.isBlank(serviceMetrics.getCorrelationId())) {
serviceMetrics.setCorrelationId(UUID.getUUID());
}
logger.logError(null, null, "Exception while Converting Processing Message. Retrying to publish.",
String.valueOf(HttpStatus.SC_BAD_REQUEST), e);
// Publishing back to the same queue
publishForRetry.publishMessageForRetry(message);
try {
message.acknowledge();
} catch (JMSException jmsException) {
logger.logError(null, null,
"Failed to Acknowledge the Message when publishing" + "to Error Queue",
String.valueOf(HttpStatus.SC_BAD_REQUEST), jmsException);
}
}
}
}
}
I am using JMS to send receive message from IBM MQ message broker. I am currently working on listener service throwing unhandled excepion and message sent
back to queue without acknowledgement.
I want the service to retry a configurable number of time and throw meaning full exception message that listener service is unavailable.
My listener and container factory looks like below.
#JmsListener(destination = "testqueue", containerFactory = "queuejmsfactory")
public void consumer(String message) throws JMSException
{ handle(message); }
#Bean(name = "queuejmsfactory") public JmsListenerContainerFactory getQueueTopicFactory(ConnectionFactory con ,
DefaultJmsListenerContainerFactoryConfigurer config)
{ DefaultJmsListenerContainerFactory d = new DefaultJmsListenerContainerFactory();
d.setSessionTransacted(true);
d.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
config.configure(d,con);
return d; }
I short, I have an existing code using the SessionawareMessageListener onMessage which i am trying to
replicate to #JmsListener. How do i handle the session commit and rollback automatically and
how do i get the session in JmsListener if have to handle manually similar to onMessage.
#Override
public void onMessage(Mesage mes, Session ses) throws JMSException
{ try
{ TestMessage txtMessage = (TextMessage)message;
handle(txtMessage); ses.commit();
} catch (Exception exp)
{ if (shouldRollback(message))
{ ses.rollback();}
else{logger,warn("moved to dlq");
ses.commit();
}
} }
private boolean shouldRollback(Message mes) throws JMSException
{ int rollbackcount = mes.getIntProperty("JMSXDeliveryCount");
return (rollbackcount <= maxRollBackCountFromApplication.properties)
}
Updated code:
#JmsListener(destination = "testqueue", containerFactory = "queuejmsfactory")
public void consumer(Message message) throws JMSException
{
try {TestMessage txtMessage = (TextMessage)message;
handle(txtMessage);}
catch(Excepton ex) {
if shouldRollback(message)
{throw ex;}
else {logger.warn("moved to dlq")}
}}
private boolean shouldRollback(Message mes) throws JMSException
{ int rollbackcount = mes.getIntProperty("JMSXDeliveryCount");
return (rollbackcount <= maxRollBackCountFromApplication.properties)
}
#Bean(name = "queuejmsfactory") public JmsListenerContainerFactory getQueueTopicFactory(ConnectionFactory con ,
DefaultJmsListenerContainerFactoryConfigurer config)
{ DefaultJmsListenerContainerFactory d = new DefaultJmsListenerContainerFactory();
d.setSessionTransacted(true);
d.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
config.configure(d,con);
return d; }
I have also tried to access the JMSXDeliveryCount from Headers, but couldnt get the exact object to access delivery count. Can you clarify plz.
#JmsListener(destination = "testqueue", containerFactory = "queuejmsfactory")
public void consumer(Message message,
#Header(JmsHeaders.CORRELATION_ID) String correlationId,
#Header(name = "jms-header-not-exists") String nonExistingHeader,
#Headers Map<String, Object> headers,
MessageHeaders messageHeaders,
JmsMessageHeaderAccessor jmsMessageHeaderAccessor) {}
You can add the Session as another parameter to the JmsListener method.
I have a factory class that I use to create a mq consumer and a DefaultMessageListenerContainer bean from that consumer that will be used for consuming messages from a topic. It goes like this -
public class MQMessageFactory {
public static DemoMessageConsumer createMessageConumer(Map<String, String> queueDetails, Map<String, String> sslDetails) throws Exception {
MQConnectionFactory mqConnectionFactory = createMQConnectionFactory(queueDetails, sslDetails);
return new DemoMessageConsumer(queueDetails, mqConnectionFactory);
}
public static MQConnectionFactory createMQConnectionFactory(Map<String, String> queue, Map<String, String> sslDetails) throws Exception {
MQConnectionFactory cf = new MQConnectionFactory();
try {
cf.setHostName(queue.get("hostname"));
cf.setPort(queue.get("port"));
cf.setQueueManager(queue.get("queueManager"));
cf.setChannel(queue.get("channel"));
cf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
cf.setStringProperty(WMQConstants.USERID, queue.get("username"));
cf.setStringProperty(WMQConstants.PASSWORD, queue.get("password"));
cf.setSSLCipherSuite(queue.get("sslCipherSuite"));
cf.setSSLSocketFactory(someMethodToCreateSSLContextFactory(sslDetails));
return cf;
} catch (JMSException e) {
throw new RuntimeException("Unable to establish connection with host: " + queue.get("hostname"), e);
}
}
}
public class DemoMessageConsumer implements SessionAwareMessageListener {
private static final Logger LOGGER = LogManager.getLogger(DemoMessageConsumer.class);
private SingleConnectionFactory singleCf;
private Map<String, String> properties;
private DefaultMessageListenerContainer container;
private Consumer<Message> messageProcessor;
public DemoMessageConsumer(Map<String, String> properties, MQConnectionFactory connectionFactory) {
this.singleCf = new SingleConnectionFactory(connectionFactory);
this.singleCf.setReconnectOnException(true);
this.singleCf.afterPropertiesSet();
this.properties = properties;
}
public DefaultMessageListenerContainer listen(String queueName, Executor taskExecutor, Consumer<Message> messageProcessor) {
this.messageProcessor = messageProcessor;
this.container = new DefaultMessageListenerContainer();
this.container.setConnectionFactory(singleCf);
this.container.setDestinationName(queueName);
// this.container.setAcceptMessagesWhileStopping(true);
this.container.setSessionTransacted(true);
this.container.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
this.container.setMessageListener(this);
this.container.setConcurrentConsumers(5);
this.container.setTaskExecutor(taskExecutor);
this.container.afterPropertiesSet();
this.container.start();
LOGGER.info("Consumer started");
return this.container;
}
#Override
public void onMessage(Message message, Session session) {
try {
LOGGER.info("Message received with MessageID: {}", message.getJMSMessageID());
this.messageProcessor.accept(message);
} catch (JMSException e) {
LOGGER.error("Error while processing the message", e);
}
}
public void triggerShutdown() {
LOGGER.info("Shutdown called");
this.container.stop();
while (this.container.isRunning()) ;
this.container.shutdown();
while (this.container.isActive()) ;
this.singleCf.destroy();
LOGGER.info("Listener is shutdown");
}
}
Further, I have a Spring boot project class where I actually create the beans and use them to listen to the queue -
#Getter
#Setter
#Configuration
#ConfigurationProperties(prefix = "ibmmq")
public class MqConsumerImpl implements ApplicationListener<ContextClosedEvent> {
private static final Logger LOGGER = LogManager.getLogger(MqConsumerImpl.class);
public Map<String, String> ssl;
public Map<String, String> queue;
#Lazy
#Autowired
#Qualifier("mqConsumer")
private DemoMessageConsumer consumer;
#Bean("mqConsumer")
public DemoMessageConsumer createConsumer() throws Exception {
return MQMessageFactory.createMessageConumer(queue, ssl);
}
#Bean("mqListener")
public DefaultMessageListenerContainer listen() {
return this.consumer.listen(queue.get("name"), Executors.newFixedThreadPool(3), message -> {
try {
LOGGER.info("{} Message reading started: {} ", Thread.currentThread().getName(), message.getBody(String.class));
// My business logic goes here
Thread.sleep(1000);
LOGGER.info("{} Message reading completed: {} ", Thread.currentThread().getName(), message.getBody(String.class));
} catch (Exception e) {
LOGGER.error(e);
}
});
}
#Override
public void onApplicationEvent(ContextClosedEvent e) {
this.consumer.triggerShutdown();
}
}
Now, I run the application and the messages are being consumer properly and everything goes well. I have a taskExecutor of three threads and all of them are being used for message consumption and task execution.
Then, I trigger a command to stop the application and one or all of the threads on which the business task were running MAY/MAY NOT throw this warning -
2020-01-14 16:29:15.110 WARN 68468 --- [pool-2-thread-3] o.s.j.l.DefaultMessageListenerContainer : Rejecting received message because of the listener container having been stopped in the meantime:
JMSMessage class: jms_text
JMSType: null
JMSDeliveryMode: 2
JMSDeliveryDelay: 0
JMSDeliveryTime: 0
JMSExpiration: 0
JMSPriority: 4
JMSMessageID: ID:414d5120484b49473033533120202020a2e4f05dce410b24
JMSTimestamp: 1578982170556
JMSCorrelationID: null
JMSDestination: queue:///QUEUE.NAME
JMSReplyTo: null
JMSRedelivered: false
JMSXAppID: demo.Application
JMSXDeliveryCount: 1
JMSXUserID: username
JMS_IBM_Character_Set: UTF-8
JMS_IBM_Encoding: 273
JMS_IBM_Format: MQSTR
JMS_IBM_MsgType: 8
JMS_IBM_PutApplType: 28
JMS_IBM_PutDate: 20200114
JMS_IBM_PutTime: 06093062
hello everyone new 1578982170555
2020-01-14 16:29:15.129 INFO 68468 --- [pool-2-thread-2] c.s.l.i.c.MqConsumerImpl : pool-2-thread-2 Message reading completed: hello everyone new 1578982170344
2020-01-14 16:29:16.180 INFO 68468 --- [ Thread-9] c.s.l.n.c.MqConsumerImpl : Listener for queue: XXXXXXXXX is shutdown
Process finished with exit code 1
Now, It is completely fine for me to see this. According to spring boot jms classes, this happens and the session rollback should be called in this case, so that when I restart my consumer, the message is redelivered. THIS IS NOT HAPPENING. I am not getting the message - hello everyone new 1578982170555 the next time I started the consumer and I got the next one. A message is hence lost without being processed. How can I safeguard it?
Note: As you can see in the logs, when this warning was raised the onMessage() method for this method was not called.
I know this question has been asked but I am not able to send email using configuration. I don't know what I am doing wrong and why I am not getting the email. Here is my Spring configuration.
#Configuration
#PropertySource(value = {
"classpath:autoalert.properties"
})
#EnableAsync
#Import({PersistenceConfig.class, EmailConfig.class, VelocityConfig.class})
#ComponentScan(basePackageClasses = {
ServiceMarker.class,
RepositoryMarker.class }
)
public class AutoAlertAppConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertyConfigInDev() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Here is my email config
#Configuration
#PropertySources({
#PropertySource("classpath:email/email.properties")
})
public class EmailConfig {
#Autowired
private Environment env;
#Bean
public JavaMailSender mailSender() {
JavaMailSenderImpl mailSender = new JavaMailSenderImpl();
mailSender.setSession(getEmailSession());
return mailSender;
}
#Bean
public MailMessage mailSettings() {
SimpleMailMessage mailMessage = new SimpleMailMessage();
mailMessage.setFrom(env.getProperty("mail.from"));
...
mailMessage.setText(env.getProperty("mail.body"));
return mailMessage;
}
private Session getEmailSession() {
Properties emailProperties = SpringUtil.loadPropertiesFileFromClassPath("email" + File.separator + "general-mail-settings.properties");
final String userName = emailProperties.getProperty("user");
final String password = emailProperties.getProperty("password");
Session session = null;
try {
session = Session.getInstance(emailProperties, new javax.mail.Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(userName, password);
}
}); //end of anonymous class
} catch (Exception e) {
e.printStackTrace();
}
return session;
} //end of getEmailSession()
}
Here is my class
public static void main(String[] args) {
try (GenericApplicationContext springContext = new AnnotationConfigApplicationContext(AutoAlertAppConfig.class)) {
AutoAlertService autoAlertService = springContext.getBean(AutoAlertServiceImpl.class);
try {
autoAlertService.handleProcess(fromDate, toDate);
} catch (Exception e) {
logger.error("Exception occurs", e);
autoAlertService.handleException(fromDate, toDate, e);
}
} catch (Exception e) {
logger.error("Exception occurs in loading Spring context: ", e);
}
#Service
public class AutoAlertServiceImpl implements AutoAlertService {
#Inject
private AsyncEmailService asyncEmailService;
#Override
public void handleProcess(String fromDate, String toDate) {
logger.info("Start process");
try {
..
//Sending email
asyncEmailService.sendMailWithFileAttachment(fromDate, toDate, file);
} catch (Exception e) {
handleException(fromDate, toDate, e);
}
logger.info("Finish process");
}
}
here is my email service
#Component
public class AsyncEmailServiceImpl implements AsyncEmailService {
#Resource(name="mailSender")
private JavaMailSender mailSender;
#Resource(name="mailSettings")
private SimpleMailMessage simpleMailMessage;
#Async
#Override
public void sendMailWithFileAttachment(String from, String to, String attachFile) {
logger.info("Start execution of async. Sending email with file attachment");
MimeMessage message = mailSender.createMimeMessage();
try{
MimeMessageHelper helper = new MimeMessageHelper(message, true);
....
helper.setText(String.format(simpleMailMessage.getText(), from, to));
FileSystemResource file = new FileSystemResource(attachFile);
helper.addAttachment(file.getFilename(), file);
mailSender.send(message);
} catch (MessagingException e) {
logger.error("Exception occurs in sending email with file attachment: " + attachFile, e);
throw new MailParseException(e);
}
logger.info("Complete execution of async. Email with file attachment " + attachFile + " send successfully.");
}
}
When I run the code it comes to method. This is printed on the console
13:59:43.004 [main] INFO com.softech.vu360.autoalert.service.impl.AutoAlertServiceImpl - Finish process
13:59:43.005 [SimpleAsyncTaskExecutor-1] INFO com.softech.vu360.autoalert.service.impl.AsyncEmailServiceImpl - Start execution of async. Sending email with file attachment
13:59:43.007 [main] INFO com.softech.vu360.autoalert.AutoAlert - Exiting application.
But I get no email. In case of Synchronous calling I get the email. Why I am not getting email ? Am I doing something wrong ?
Thanks
I think this is better approach. Create a file AsyncConfig.java
#Configuration
#EnableAsync(proxyTargetClass = true)
#EnableScheduling
public class AsyncConfig implements SchedulingConfigurer, AsyncConfigurer {
private static final Logger log = LogManager.getLogger();
private static final Logger schedulingLogger = LogManager.getLogger(log.getName() + ".[scheduling]");
#Bean
public ThreadPoolTaskScheduler taskScheduler() {
log.info("Setting up thread pool task scheduler with 20 threads.");
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.setPoolSize(2);
scheduler.setThreadNamePrefix("task-");
scheduler.setAwaitTerminationSeconds(1200); // 20 minutes
scheduler.setWaitForTasksToCompleteOnShutdown(true);
scheduler.setErrorHandler(t -> schedulingLogger.error("Unknown error occurred while executing task.", t));
scheduler.setRejectedExecutionHandler((r, e) -> schedulingLogger.error("Execution of task {} was rejected for unknown reasons.", r));
return scheduler;
}
#Override
public Executor getAsyncExecutor() {
Executor executor = this.taskScheduler();
log.info("Configuring asynchronous method executor {}.", executor);
return executor;
}
#Override
public void configureTasks(ScheduledTaskRegistrar registrar) {
TaskScheduler scheduler = this.taskScheduler();
log.info("Configuring scheduled method executor {}.", scheduler);
registrar.setTaskScheduler(scheduler);
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
return new SimpleAsyncUncaughtExceptionHandler();
}
}
Now import it into your main configuration like this
#Configuration
#PropertySource(value = {
"classpath:autoalert.properties"
})
#Import({AsyncConfig.class, PersistenceConfig.class, EmailConfig.class, VelocityConfig.class})
#ComponentScan(basePackageClasses = {
ServiceMarker.class,
RepositoryMarker.class }
)
public class AutoAlertAppConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertyConfigInDev() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Change return type from void to Future
#Service
public class AsyncEmailServiceImpl implements AsyncEmailService {
#Resource(name="mailSender")
private JavaMailSender mailSender;
#Resource(name="mailSettings")
private SimpleMailMessage simpleMailMessage;
#Async
#Override
public Future<String> sendMailWithFileAttachment(String from, String to, String attachFile) {
....
return new AsyncResult<String>("Attachment File successfully send: " + attachFile);
}
#Async
#Override
public Future<String> sendMail(String from, String to, String emailBody) {
....
return new AsyncResult<String>("Email send successfully");
}
}
and in my service class just do this
logger.info("Start process");
try {
....
//Sending email
Future<String> result = asyncEmailService.sendMailWithFileAttachment(fromDate, toDate, file);
} catch (Exception e) {
handleException(fromDate, toDate, e);
}
logger.info("Finish process");
See no need to check result.get(). Now when new Thread starts and application starts to finish. I configured the scheduler.setAwaitTerminationSeconds(1200); // 20 minutes in AsyncConfig.java. This will ensure that all the pending threads must complete before the application shuts down. Ofcourse this can be change according to any ones need.
Now when I run the application it prints these on the console
12:55:33.879 [main] INFO com.softech.vu360.autoalert.service.impl.AutoAlertServiceImpl - Finish process
12:55:33.895 [task-1] INFO com.softech.vu360.autoalert.service.impl.AsyncEmailServiceImpl - Start execution of async. Sending email with file attachment
12:58:09.030 [task-1] INFO com.softech.vu360.autoalert.service.impl.AsyncEmailServiceImpl - Complete execution of async. Email with file attachment D:\projects\AutoAlerts\marketo\autoalert 2015-08-24 to 2015-08-30.csv send successfully.
12:58:09.033 [main] INFO com.softech.vu360.autoalert.AutoAlert - Exiting application.
See starts new thread, but before application shuts down make sure, the thread completes and then exits the application. I configured 20 minutes for email, but as soon as thread completes, application get shut down. This is happy ending :)