Spring Integration | TCP Connections dropping after idle wait of 350 seconds - spring

We have a java spring integration application running on aws (multiple pods within a Kubernetes cluster). We use TCP Outbound gateways to communicate with third party systems and cache these connections using a CachingClientConnectionFactory factory. On the factory we have set the sokeepalive as true however we still see that after 350 seconds the connection is dropped. Do we need anythign else in the configuration to keep pinging the server a little before 350 seconds of idle waiting time ? AWS talks about the 350s restriction here -
https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-troubleshooting.html#nat-gateway-troubleshooting-timeout
Configuration of our connection factory and gateway is as follows
#Bean
public AbstractClientConnectionFactory primeClientConnectionFactory() {
TcpNetClientConnectionFactory tcpNetClientConnectionFactory = new TcpNetClientConnectionFactory(host, port);
tcpNetClientConnectionFactory.setDeserializer(new PrimeCustomStxHeaderLengthSerializer());
tcpNetClientConnectionFactory.setSerializer(new PrimeCustomStxHeaderLengthSerializer());
tcpNetClientConnectionFactory.setSingleUse(false);
tcpNetClientConnectionFactory.setSoKeepAlive(true);
return tcpNetClientConnectionFactory;
}
#Bean
public AbstractClientConnectionFactory primeTcpCachedClientConnectionFactory() {
CachingClientConnectionFactory cachingConnFactory = new CachingClientConnectionFactory(primeClientConnectionFactory(), connectionPoolSize);
//cachingConnFactory.setSingleUse(false);
cachingConnFactory.setLeaveOpen(true);
cachingConnFactory.setSoKeepAlive(true);
return cachingConnFactory;
}
#Bean
public MessageChannel primeOutboundChannel() {
return new DirectChannel();
}
#Bean
public RequestHandlerRetryAdvice retryAdvice() {
RequestHandlerRetryAdvice retryAdvice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(500);
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(3);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(retryPolicy);
retryAdvice.setRetryTemplate(retryTemplate);
return retryAdvice;
}
#Bean
#ServiceActivator(inputChannel = "primeOutboundChannel")
public MessageHandler primeOutbound(AbstractClientConnectionFactory primeTcpCachedClientConnectionFactory) {
TcpOutboundGateway tcpOutboundGateway = new TcpOutboundGateway();
List<Advice> list = new ArrayList<>();
list.add(retryAdvice());
tcpOutboundGateway.setAdviceChain(list);
tcpOutboundGateway.setRemoteTimeout(timeOut);
tcpOutboundGateway.setRequestTimeout(timeOut);
tcpOutboundGateway.setSendTimeout(timeOut);
tcpOutboundGateway.setConnectionFactory(primeTcpCachedClientConnectionFactory);
return tcpOutboundGateway;
}
}

See this SO thread for more about Keep Alive: Does a TCP socket connection have a "keep alive"?.
According to current Java Net API we got this class:
/**
* Defines extended socket options, beyond those defined in
* {#link java.net.StandardSocketOptions}. These options may be platform
* specific.
*
* #since 1.8
*/
public final class ExtendedSocketOptions {
Which provides this constant:
/**
* Keep-Alive idle time.
*
* <p>
* The value of this socket option is an {#code Integer} that is the number
* of seconds of idle time before keep-alive initiates a probe. The socket
* option is specific to stream-oriented sockets using the TCP/IP protocol.
* The exact semantics of this socket option are system dependent.
*
* <p>
* When the {#link java.net.StandardSocketOptions#SO_KEEPALIVE
* SO_KEEPALIVE} option is enabled, TCP probes a connection that has been
* idle for some amount of time. The default value for this idle period is
* system dependent, but is typically 2 hours. The {#code TCP_KEEPIDLE}
* option can be used to affect this value for a given socket.
*
* #since 11
*/
public static final SocketOption<Integer> TCP_KEEPIDLE
= new ExtSocketOption<Integer>("TCP_KEEPIDLE", Integer.class);
So, what we need on the TcpNetClientConnectionFactory is this:
public void setTcpSocketSupport(TcpSocketSupport tcpSocketSupport) {
Implement that void postProcessSocket(Socket socket); to be able to do this:
try {
socket.setOption(ExtendedSocketOptions.TCP_KEEPIDLE, 349);
}
catch (IOException ex) {
throw new UncheckedIOException(ex);
}
According to that AWS doc you have shared with us.
See also some info in Spring Integration docs: https://docs.spring.io/spring-integration/docs/current/reference/html/ip.html#the-tcpsocketsupport-strategy-interface

Related

Caused by: java.net.SocketException: Connection reset by peer: socket write error

I'm trying to connect to rabbitMQ over SSL using Spring Boot 2.7.4 and java 11.0.14 I was following this example here:
I have added the following configurations:
properties file:
# RabbitMQ Server configuration file.
rabbit.username=admin
rabbit.password=admin
rabbit.host=localhost
rabbit.port=5671
rabbit.ssl=TLSv1.2
rabbit.keystore.name=client_key.p12
rabbit.keystore.password=rabbitstore
rabbit.truststore=server_store.jks
rabbit.truststore.password=rabbitstore
client_key.p12 and server_store.jks are in my classpath.
Configuration Class:
#Configuration
#PropertySource("classpath:rabbit.properties")
public class RabbitConfiguration {
/**
* Default sample channel name to respond for requests from clients.
*/
public static final String DEFAULT_QUEUE = "sample_queue";
/**
* Environment properties file from rabbitmq configuration.
*/
#Autowired
private Environment env;
/**
* Establish a connection to a rabbit mq server.
* #return Rabbit connection factory for rabbitmq access.
* #throws IOException If wrong parameters are used for connection.
*/
#Bean
public RabbitConnectionFactoryBean connectionFactoryBean() throws IOException {
RabbitConnectionFactoryBean connectionFactoryBean = new RabbitConnectionFactoryBean();
connectionFactoryBean.setHost(Objects.requireNonNull(env.getProperty("rabbit.host")));
connectionFactoryBean.setPort(Integer.parseInt(Objects.requireNonNull(env.getProperty("rabbit.port"))));
connectionFactoryBean.setUsername(Objects.requireNonNull(env.getProperty("rabbit.username")));
connectionFactoryBean.setPassword(Objects.requireNonNull(env.getProperty("rabbit.password")));
// SSL-Configuration if set
if(env.getProperty("rabbit.ssl") != null) {
connectionFactoryBean.setUseSSL(true);
connectionFactoryBean.setSslAlgorithm(Objects.requireNonNull(env.getProperty("rabbit.ssl")));
// This information should be stored safely !!!
connectionFactoryBean.setKeyStore(Objects.requireNonNull(env.getProperty("rabbit.keystore.name")));
connectionFactoryBean.setKeyStorePassphrase(Objects.requireNonNull(env.getProperty("rabbit.keystore.password")));
connectionFactoryBean.setTrustStore(Objects.requireNonNull(env.getProperty("rabbit.truststore")));
connectionFactoryBean.setTrustStorePassphrase(Objects.requireNonNull(env.getProperty("rabbit.truststore.password")));
}
return connectionFactoryBean;
}
/**
* Connection factory which established a rabbitmq connection used from a connection factory
* #param connectionFactoryBean Connection factory bean to create connection.
* #return A connection factory to create connections.
* #throws Exception If wrong parameters are used for connection.
*/
#Bean(name = "GEO_RABBIT_CONNECTION")
public ConnectionFactory connectionFactory(RabbitConnectionFactoryBean connectionFactoryBean) throws Exception {
return new CachingConnectionFactory(Objects.requireNonNull(connectionFactoryBean.getObject()));
}
/**
* Queue initialization from rabbitmq to listen a queue.
* #return An queue to listen for listen receiver.
*/
#Bean
public Queue queue() {
// Create an new queue to handle incoming responds
return new Queue(DEFAULT_QUEUE, false, false, false, null);
}
/**
* Generates a simple message listener container.
* #param connectionFactory Established connection to rabbitmq server.
* #param listenerAdapter Listener event adapter to listen for messages.
* #return A simple message container for listening for requests.
*/
#Bean
public SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(DEFAULT_QUEUE);
container.setMessageListener(listenerAdapter);
container.setAcknowledgeMode(AcknowledgeMode.AUTO);
return container;
}
/**
* Message listener adapter to generate a message listener.
* #param deviceMonitoringReceiver Device receive to for listening.
* #return A message listener adapter to receive messages.
*/
#Bean
public MessageListenerAdapter listenerAdapter(DeviceMonitoringReceiver deviceMonitoringReceiver) {
return new MessageListenerAdapter(deviceMonitoringReceiver, "receiveMessage");
}
}
Also I have updated rabbitMQ configurations:
[
{rabbit, [
{ssl_listeners, [5671]},
{ssl_options, [{cacertfile, "D:\\tls-gen\\basic\\result\\ca_certificate.pem"},
{certfile, "D:\\tls-gen\\basic\\result\\server_seliiwvdec53152_certificate.pem"},
{keyfile, "D:\\tls-gen\basic\\result\\server_seliiwvdec53152_key.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, true}]}
]}
].
But the application is not starting and throwing
Caused by: java.net.SocketException: Connection reset by peer: socket write error
I resolved the issue by adding this to the configurations:
ssl_options.password = xxx
It's mentioned in the official documentation it's optional I don't know why. But whatever the issue is now resolved.

how to set maximum number of connection retries with Spring AMQP

I have a scenario where my rabbit mq instance is not always available and would like to set the maximum number of times a connection retry happens, Is this possible with amqp?
Example,
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setUri("amqprl//");
factory ../ try uri connection for 4 times max then fail if still no connection
return factory;
}
Message producers will only try to create a connection when you send a message.
Message consumers (container factories) will retry indefinitely.
You can add a ConnectionListener to the connection factory and stop() the listener containers after some number of failures.
#FunctionalInterface
public interface ConnectionListener {
/**
* Called when a new connection is established.
* #param connection the connection.
*/
void onCreate(Connection connection);
/**
* Called when a connection is closed.
* #param connection the connection.
* #see #onShutDown(ShutdownSignalException)
*/
default void onClose(Connection connection) {
}
/**
* Called when a connection is force closed.
* #param signal the shut down signal.
* #since 2.0
*/
default void onShutDown(ShutdownSignalException signal) {
}
/**
* Called when a connection couldn't be established.
* #param exception the exception thrown.
* #since 2.2.17
*/
default void onFailed(Exception exception) {
}
}

Multithreaded Executor channel to speed up the consumer process

I have a message producer which produces around 15 messages/second
The consumer is a spring integration project which consumes from the Message Queue and does a lot of processing. Currently it is single threaded and not able to match with the rate at which the producer are sending the messages. hence the queue depth keeps on increasing
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(Jms.container(this.emsConnectionFactory, this.emsQueue).get()))
.wireTap(FLTAWARE_WIRE_TAP_CHNL)// push raw fa data
.filter(ingFilter, "filterMessageOnEvent").transform(eventHandler, "parseEvent")
.aggregate(a -> a.correlationStrategy(corStrgy, "getCorrelationKey").releaseStrategy(g -> {
boolean eonExists = g.getMessages().stream()
.anyMatch(eon -> ((FlightModel) eon.getPayload()).getEstGmtOnDtm() != null);
if (eonExists) {
boolean einExists = g.getMessages().stream()
.anyMatch(ein -> ((FlightModel) ein.getPayload()).getEstGmtInDtm() != null);
if (einExists) {
return true;
}
}
return false;
}).messageStore(this.messageStore)).channel("AggregatorEventChannel").get();
is it possible to use executor channel to process this in a multithreaded environment and speed up the consumer process
If yes, please suggest how can i achieve - To ensure ordering of the messages I need to assign the messages of same type (based on the id of the message) to the same thread of the executor channel.
[UPDATED CODE]
I have created the below executor channels
public static final MessageChannel SKW_DEFAULT_CHANNEL = MessageChannels
.executor(ASQ_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
public static final MessageChannel RPA_DEFAULT_CHANNEL = MessageChannels
.executor(ASH_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
Now from the main message flow I redirected to a custom router which forwards the message to Executor channel as shown below -
#Bean
public IntegrationFlow baseEventFlow1() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(Jms.container(this.emsConnectionFactory, this.emsQueue).get()))
.wireTap(FLTAWARE_WIRE_TAP_CHNL)// push raw fa data
.filter(ingFilter, "filterMessageOnEvent").route(route()).get();
}
public AbstractMessageRouter router() {
return new AbstractMessageRouter() {
#Override
protected Collection<MessageChannel> determineTargetChannels(Message<?> message) {
if (message.getPayload().toString().contains("\"id\":\"RPA")) {
return Collections.singletonList(RPA_DEFAULT_CHANNEL);
} else if (message.getPayload().toString().contains("\"id\":\"SKW")) {
return Collections.singletonList(SKW_DEFAULT_CHANNEL);
} else {
return Collections.singletonList(new NullChannel());
}
}
};
}
I will have individual consumer flow for the corresponding executor channel.
Please correct my understaning
[UPDATED]
#Bean
#BridgeTo("uaxDefaultChannel")
public MessageChannel ucaDefaultChannel() {
return MessageChannels.executor(UCA_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
}
#Bean
#BridgeTo("uaDefaultChannel")
public MessageChannel ualDefaultChannel() {
return MessageChannels.executor(UAL_DEFAULT_CHANNEL_NAME, Executors.newFixedThreadPool(1)).get();
}
#Bean
public IntegrationFlow uaEventFlow() {
return IntegrationFlows.from("uaDefaultChannel").wireTap(UA_WIRE_TAP_CHNL)
.transform(eventHandler, "parseEvent")
}
So BridgeTo on the executor channel will forward the messages
hence the queue depth keeps on increasing
Since it looks like your queue is somewhere on JMS broker that is really OK to have such a behavior. That's exactly for what messaging systems have been designed - to distinguish producer and consumer and deal with messages in a destination whenever it is possible.
if you want to increase a polling from JMS, you can consider to have a concurrency option on the JMS container:
/**
* The concurrency to use.
* #param concurrency the concurrency.
* #return current {#link JmsDefaultListenerContainerSpec}.
* #see DefaultMessageListenerContainer#setConcurrency(String)
*/
public JmsDefaultListenerContainerSpec concurrency(String concurrency) {
this.target.setConcurrency(concurrency);
return this;
}
/**
* The concurrent consumers number to use.
* #param concurrentConsumers the concurrent consumers count.
* #return current {#link JmsDefaultListenerContainerSpec}.
* #see DefaultMessageListenerContainer#setConcurrentConsumers(int)
*/
public JmsDefaultListenerContainerSpec concurrentConsumers(int concurrentConsumers) {
this.target.setConcurrentConsumers(concurrentConsumers);
return this;
}
/**
* The max for concurrent consumers number to use.
* #param maxConcurrentConsumers the max concurrent consumers count.
* #return current {#link JmsDefaultListenerContainerSpec}.
* #see DefaultMessageListenerContainer#setMaxConcurrentConsumers(int)
*/
public JmsDefaultListenerContainerSpec maxConcurrentConsumers(int maxConcurrentConsumers) {
this.target.setMaxConcurrentConsumers(maxConcurrentConsumers);
return this;
}
See more info the Docs: https://docs.spring.io/spring/docs/5.2.3.RELEASE/spring-framework-reference/integration.html#jms-receiving
But that won't allow you to "asign messages to the specific thread". There is just like no way to partition in JMS.
We can do that with Spring Integration using router according your "based on the id of the message" and particular ExecutorChannel instances configured with a singled-threaded Executor. Every ExecutorChannel is going to be its dedicated executor with only single thread. This way you will ensure an order for messages with the same partition key and you'll process them in parallel. All the ExecutorChannel can have the same subscriber or bridge to the same channel for processing.
However you need to keep in mind that when you are leaving JMS listener thread, you finish JMS transaction and you fail to process a message in that separate thread you may lose a message.

How to set Durable Subscriber in DefaultMessageListenerContainer in spring?

Producer of the message is not sending message as persistent and when i am trying to consume the message through MessageListener, and any exception(runtime) occurs, it retries for specific number of times (default is 6 from AMQ side) and message get lost.
Reason is that since producer is not setting the Delivery mode as Persistent, after certain number of retry attempt, DLQ is not being created and message does not move to DLQ. Due to this , i lost the message.
My Code is like this :-
#Configuration
#PropertySource("classpath:application.properties")
public class ActiveMqJmsConfig {
#Autowired
private AbcMessageListener abcMessageListener;
public DefaultMessageListenerContainer purchaseMsgListenerforAMQ(
#Qualifier("AMQConnectionFactory") ConnectionFactory amqConFactory) {
LOG.info("Message listener for purchases from AMQ : Starting");
DefaultMessageListenerContainer defaultMessageListenerContainer =
new DefaultMessageListenerContainer();
defaultMessageListenerContainer.setConnectionFactory(amqConFactory);
defaultMessageListenerContainer.setMaxConcurrentConsumers(4);
defaultMessageListenerContainer
.setDestinationName(purchaseReceivingQueueName);
defaultMessageListenerContainer
.setMessageListener(abcMessageListener);
defaultMessageListenerContainer.setSessionTransacted(true);
return defaultMessageListenerContainer;
}
#Bean
#Qualifier(value = "AMQConnectionFactory")
public ConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory amqConnectionFactory =
new ActiveMQConnectionFactory();
amqConnectionFactory
.setBrokerURL(System.getProperty(tcp://localhost:61616));
amqConnectionFactory
.setUserName(System.getProperty(admin));
amqConnectionFactory
.setPassword(System.getProperty(admin));
return amqConnectionFactory;
}
}
#Component
public class AbcMessageListener implements MessageListener {
#Override
public void onMessage(Message msg) {
//CODE implementation
}
}
Problem :- By setting the client-id at connection level (Connection.setclientid("String")), we can subscribe as durable subscriber even though message is not persistent. By doing this, if application throws runtime exception , after a certain number of retry attempt, DLQ will be created for the Queue and message be moved to DLQ.
But in DefaultMessageListenerContainer, connection is not exposed to client. it is maintained by Class itself as a pool, i guess.
How can i achieve the durable subscription in DefaultMessageListenerContainer?
You can set the client id on the container instead:
/**
* Specify the JMS client ID for a shared Connection created and used
* by this container.
* <p>Note that client IDs need to be unique among all active Connections
* of the underlying JMS provider. Furthermore, a client ID can only be
* assigned if the original ConnectionFactory hasn't already assigned one.
* #see javax.jms.Connection#setClientID
* #see #setConnectionFactory
*/
public void setClientId(#Nullable String clientId) {
this.clientId = clientId;
}
and
/**
* Set the name of a durable subscription to create. This method switches
* to pub-sub domain mode and activates subscription durability as well.
* <p>The durable subscription name needs to be unique within this client's
* JMS client id. Default is the class name of the specified message listener.
* <p>Note: Only 1 concurrent consumer (which is the default of this
* message listener container) is allowed for each durable subscription,
* except for a shared durable subscription (which requires JMS 2.0).
* #see #setPubSubDomain
* #see #setSubscriptionDurable
* #see #setSubscriptionShared
* #see #setClientId
* #see #setMessageListener
*/
public void setDurableSubscriptionName(#Nullable String durableSubscriptionName) {
this.subscriptionName = durableSubscriptionName;
this.subscriptionDurable = (durableSubscriptionName != null);
}

can we batch up groups of 10 message load in mosquitto using spring integration

this is how i have defined my mqtt connection using spring integration.i am not sure whether this is possible bt can we setup a mqtt subscriber works after getting a 10 load of messages. right now subscriber works after publishing a message as it should.
#Autowired
ConnectorConfig config;
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
factory.setServerURIs(config.getUrl());
factory.setUserName(config.getUser());
factory.setPassword(config.getPass());
return factory;
}
#Bean
public MessageProducer inbound() {
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter(config.getClientid(), mqttClientFactory(), "ALERT", "READING");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
adapter.setOutputChannel(mqttRouterChannel());
return adapter;
}
/**this is router**/
#MessageEndpoint
public class MessageRouter {
private final Logger logger = LoggerFactory.getLogger(MessageRouter.class);
static final String ALERT = "ALERT";
static final String READING = "READING";
#Router(inputChannel = "mqttRouterChannel")
public String route(#Header("mqtt_topic") String topic){
String route = null;
switch (topic){
case ALERT:
logger.info("alert message received");
route = "alertTransformerChannel";
break;
case READING:
logger.info("reading message received");
route = "readingTransformerChannel";
break;
}
return route;
}
}
i need to batch up groups of 10 messages at a time
That is not a MqttPahoMessageDrivenChannelAdapter responsibility.
We use there MqttCallback with this semantic:
* #param topic name of the topic on the message was published to
* #param message the actual message.
* #throws Exception if a terminal error has occurred, and the client should be
* shut down.
*/
public void messageArrived(String topic, MqttMessage message) throws Exception;
So, we can't batch them there on this Channel Adapter by nature of the Paho client.
What we can suggest you from the Spring Integration perspective is an Aggregator EIP implementation.
In your case you should add #ServiceActivator for the AggregatorFactoryBean #Bean before that mqttRouterChannel, before sending to the router.
That maybe as simple as:
#Bean
#ServiceActivator(inputChannel = "mqttAggregatorChannel")
AggregatorFactoryBean mqttAggregator() {
AggregatorFactoryBean aggregator = new AggregatorFactoryBean();
aggregator.setProcessorBean(new DefaultAggregatingMessageGroupProcessor());
aggregator.setCorrelationStrategy(m -> 1);
aggregator.setReleaseStrategy(new MessageCountReleaseStrategy(10));
aggregator.setExpireGroupsUponCompletion(true);
aggregator.setSendPartialResultOnExpiry(true);
aggregator.setGroupTimeoutExpression(new ValueExpression<>(1000));
aggregator.setOutputChannelName("mqttRouterChannel");
return aggregator;
}
See more information in the Reference Manual.

Resources