I'm trying to connect to rabbitMQ over SSL using Spring Boot 2.7.4 and java 11.0.14 I was following this example here:
I have added the following configurations:
properties file:
# RabbitMQ Server configuration file.
rabbit.username=admin
rabbit.password=admin
rabbit.host=localhost
rabbit.port=5671
rabbit.ssl=TLSv1.2
rabbit.keystore.name=client_key.p12
rabbit.keystore.password=rabbitstore
rabbit.truststore=server_store.jks
rabbit.truststore.password=rabbitstore
client_key.p12 and server_store.jks are in my classpath.
Configuration Class:
#Configuration
#PropertySource("classpath:rabbit.properties")
public class RabbitConfiguration {
/**
* Default sample channel name to respond for requests from clients.
*/
public static final String DEFAULT_QUEUE = "sample_queue";
/**
* Environment properties file from rabbitmq configuration.
*/
#Autowired
private Environment env;
/**
* Establish a connection to a rabbit mq server.
* #return Rabbit connection factory for rabbitmq access.
* #throws IOException If wrong parameters are used for connection.
*/
#Bean
public RabbitConnectionFactoryBean connectionFactoryBean() throws IOException {
RabbitConnectionFactoryBean connectionFactoryBean = new RabbitConnectionFactoryBean();
connectionFactoryBean.setHost(Objects.requireNonNull(env.getProperty("rabbit.host")));
connectionFactoryBean.setPort(Integer.parseInt(Objects.requireNonNull(env.getProperty("rabbit.port"))));
connectionFactoryBean.setUsername(Objects.requireNonNull(env.getProperty("rabbit.username")));
connectionFactoryBean.setPassword(Objects.requireNonNull(env.getProperty("rabbit.password")));
// SSL-Configuration if set
if(env.getProperty("rabbit.ssl") != null) {
connectionFactoryBean.setUseSSL(true);
connectionFactoryBean.setSslAlgorithm(Objects.requireNonNull(env.getProperty("rabbit.ssl")));
// This information should be stored safely !!!
connectionFactoryBean.setKeyStore(Objects.requireNonNull(env.getProperty("rabbit.keystore.name")));
connectionFactoryBean.setKeyStorePassphrase(Objects.requireNonNull(env.getProperty("rabbit.keystore.password")));
connectionFactoryBean.setTrustStore(Objects.requireNonNull(env.getProperty("rabbit.truststore")));
connectionFactoryBean.setTrustStorePassphrase(Objects.requireNonNull(env.getProperty("rabbit.truststore.password")));
}
return connectionFactoryBean;
}
/**
* Connection factory which established a rabbitmq connection used from a connection factory
* #param connectionFactoryBean Connection factory bean to create connection.
* #return A connection factory to create connections.
* #throws Exception If wrong parameters are used for connection.
*/
#Bean(name = "GEO_RABBIT_CONNECTION")
public ConnectionFactory connectionFactory(RabbitConnectionFactoryBean connectionFactoryBean) throws Exception {
return new CachingConnectionFactory(Objects.requireNonNull(connectionFactoryBean.getObject()));
}
/**
* Queue initialization from rabbitmq to listen a queue.
* #return An queue to listen for listen receiver.
*/
#Bean
public Queue queue() {
// Create an new queue to handle incoming responds
return new Queue(DEFAULT_QUEUE, false, false, false, null);
}
/**
* Generates a simple message listener container.
* #param connectionFactory Established connection to rabbitmq server.
* #param listenerAdapter Listener event adapter to listen for messages.
* #return A simple message container for listening for requests.
*/
#Bean
public SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(DEFAULT_QUEUE);
container.setMessageListener(listenerAdapter);
container.setAcknowledgeMode(AcknowledgeMode.AUTO);
return container;
}
/**
* Message listener adapter to generate a message listener.
* #param deviceMonitoringReceiver Device receive to for listening.
* #return A message listener adapter to receive messages.
*/
#Bean
public MessageListenerAdapter listenerAdapter(DeviceMonitoringReceiver deviceMonitoringReceiver) {
return new MessageListenerAdapter(deviceMonitoringReceiver, "receiveMessage");
}
}
Also I have updated rabbitMQ configurations:
[
{rabbit, [
{ssl_listeners, [5671]},
{ssl_options, [{cacertfile, "D:\\tls-gen\\basic\\result\\ca_certificate.pem"},
{certfile, "D:\\tls-gen\\basic\\result\\server_seliiwvdec53152_certificate.pem"},
{keyfile, "D:\\tls-gen\basic\\result\\server_seliiwvdec53152_key.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, true}]}
]}
].
But the application is not starting and throwing
Caused by: java.net.SocketException: Connection reset by peer: socket write error
I resolved the issue by adding this to the configurations:
ssl_options.password = xxx
It's mentioned in the official documentation it's optional I don't know why. But whatever the issue is now resolved.
Related
We have a java spring integration application running on aws (multiple pods within a Kubernetes cluster). We use TCP Outbound gateways to communicate with third party systems and cache these connections using a CachingClientConnectionFactory factory. On the factory we have set the sokeepalive as true however we still see that after 350 seconds the connection is dropped. Do we need anythign else in the configuration to keep pinging the server a little before 350 seconds of idle waiting time ? AWS talks about the 350s restriction here -
https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-troubleshooting.html#nat-gateway-troubleshooting-timeout
Configuration of our connection factory and gateway is as follows
#Bean
public AbstractClientConnectionFactory primeClientConnectionFactory() {
TcpNetClientConnectionFactory tcpNetClientConnectionFactory = new TcpNetClientConnectionFactory(host, port);
tcpNetClientConnectionFactory.setDeserializer(new PrimeCustomStxHeaderLengthSerializer());
tcpNetClientConnectionFactory.setSerializer(new PrimeCustomStxHeaderLengthSerializer());
tcpNetClientConnectionFactory.setSingleUse(false);
tcpNetClientConnectionFactory.setSoKeepAlive(true);
return tcpNetClientConnectionFactory;
}
#Bean
public AbstractClientConnectionFactory primeTcpCachedClientConnectionFactory() {
CachingClientConnectionFactory cachingConnFactory = new CachingClientConnectionFactory(primeClientConnectionFactory(), connectionPoolSize);
//cachingConnFactory.setSingleUse(false);
cachingConnFactory.setLeaveOpen(true);
cachingConnFactory.setSoKeepAlive(true);
return cachingConnFactory;
}
#Bean
public MessageChannel primeOutboundChannel() {
return new DirectChannel();
}
#Bean
public RequestHandlerRetryAdvice retryAdvice() {
RequestHandlerRetryAdvice retryAdvice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(500);
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(3);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(retryPolicy);
retryAdvice.setRetryTemplate(retryTemplate);
return retryAdvice;
}
#Bean
#ServiceActivator(inputChannel = "primeOutboundChannel")
public MessageHandler primeOutbound(AbstractClientConnectionFactory primeTcpCachedClientConnectionFactory) {
TcpOutboundGateway tcpOutboundGateway = new TcpOutboundGateway();
List<Advice> list = new ArrayList<>();
list.add(retryAdvice());
tcpOutboundGateway.setAdviceChain(list);
tcpOutboundGateway.setRemoteTimeout(timeOut);
tcpOutboundGateway.setRequestTimeout(timeOut);
tcpOutboundGateway.setSendTimeout(timeOut);
tcpOutboundGateway.setConnectionFactory(primeTcpCachedClientConnectionFactory);
return tcpOutboundGateway;
}
}
See this SO thread for more about Keep Alive: Does a TCP socket connection have a "keep alive"?.
According to current Java Net API we got this class:
/**
* Defines extended socket options, beyond those defined in
* {#link java.net.StandardSocketOptions}. These options may be platform
* specific.
*
* #since 1.8
*/
public final class ExtendedSocketOptions {
Which provides this constant:
/**
* Keep-Alive idle time.
*
* <p>
* The value of this socket option is an {#code Integer} that is the number
* of seconds of idle time before keep-alive initiates a probe. The socket
* option is specific to stream-oriented sockets using the TCP/IP protocol.
* The exact semantics of this socket option are system dependent.
*
* <p>
* When the {#link java.net.StandardSocketOptions#SO_KEEPALIVE
* SO_KEEPALIVE} option is enabled, TCP probes a connection that has been
* idle for some amount of time. The default value for this idle period is
* system dependent, but is typically 2 hours. The {#code TCP_KEEPIDLE}
* option can be used to affect this value for a given socket.
*
* #since 11
*/
public static final SocketOption<Integer> TCP_KEEPIDLE
= new ExtSocketOption<Integer>("TCP_KEEPIDLE", Integer.class);
So, what we need on the TcpNetClientConnectionFactory is this:
public void setTcpSocketSupport(TcpSocketSupport tcpSocketSupport) {
Implement that void postProcessSocket(Socket socket); to be able to do this:
try {
socket.setOption(ExtendedSocketOptions.TCP_KEEPIDLE, 349);
}
catch (IOException ex) {
throw new UncheckedIOException(ex);
}
According to that AWS doc you have shared with us.
See also some info in Spring Integration docs: https://docs.spring.io/spring-integration/docs/current/reference/html/ip.html#the-tcpsocketsupport-strategy-interface
I have a scenario where my rabbit mq instance is not always available and would like to set the maximum number of times a connection retry happens, Is this possible with amqp?
Example,
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setUri("amqprl//");
factory ../ try uri connection for 4 times max then fail if still no connection
return factory;
}
Message producers will only try to create a connection when you send a message.
Message consumers (container factories) will retry indefinitely.
You can add a ConnectionListener to the connection factory and stop() the listener containers after some number of failures.
#FunctionalInterface
public interface ConnectionListener {
/**
* Called when a new connection is established.
* #param connection the connection.
*/
void onCreate(Connection connection);
/**
* Called when a connection is closed.
* #param connection the connection.
* #see #onShutDown(ShutdownSignalException)
*/
default void onClose(Connection connection) {
}
/**
* Called when a connection is force closed.
* #param signal the shut down signal.
* #since 2.0
*/
default void onShutDown(ShutdownSignalException signal) {
}
/**
* Called when a connection couldn't be established.
* #param exception the exception thrown.
* #since 2.2.17
*/
default void onFailed(Exception exception) {
}
}
When using #KafkaListener with batches, the error handler logs the content of the full batch (all messages) in case of an exception.
How can I make this less verbose? I'd like to avoid spamming the log files with all the messages and only see the actual exception.
Here is a minimal example of how my consumer currently looks like:
#Component
class TestConsumer {
#Bean
fun kafkaBatchListenerContainerFactory(kafkaProperties: KafkaProperties): ConcurrentKafkaListenerContainerFactory<String, String> {
val configs = kafkaProperties.buildConsumerProperties()
configs[ConsumerConfig.MAX_POLL_RECORDS_CONFIG] = 10000
val factory = ConcurrentKafkaListenerContainerFactory<String, String>()
factory.consumerFactory = DefaultKafkaConsumerFactory(configs)
factory.isBatchListener = true
return factory
}
#KafkaListener(
topics = ["myTopic"],
containerFactory = "kafkaBatchListenerContainerFactory"
)
fun batchListen(values: List<ConsumerRecord<String, String>>) {
// Something that might throw an exception in rare cases.
}
}
What version are you using?
This container property was added in 2.2.14.
/**
* Set to false to log {#code record.toString()} in log messages instead
* of {#code topic-partition#offset}.
* #param onlyLogRecordMetadata false to log the entire record.
* #since 2.2.14
*/
public void setOnlyLogRecordMetadata(boolean onlyLogRecordMetadata) {
this.onlyLogRecordMetadata = onlyLogRecordMetadata;
}
It has been true by default since version 2.7 (which is why the javadocs now read that way).
This was the previous javadoc:
/**
* Set to true to only log {#code topic-partition#offset} in log messages instead
* of {#code record.toString()}.
* #param onlyLogRecordMetadata true to only log the topic/parrtition/offset.
* #since 2.2.14
*/
Also, starting with version 2.5, you can set the log level on the error handler:
/**
* Set the level at which the exception thrown by this handler is logged.
* #param logLevel the level (default ERROR).
*/
public void setLogLevel(KafkaException.Level logLevel) {
Assert.notNull(logLevel, "'logLevel' cannot be null");
this.logLevel = logLevel;
}
I have a kubernetes cluster with an activeMQ Artemis Queue and I am using hpa for autoscaling of micro services. The messages are send via QpidSender and received via JMSListener.
Messaging works, but I am not able to configure the Queue/Listener in a way, that autoscaling works as expacted.
This is my Qpid sender
public static void send(String avroMessage, String task) throws JMSException, NamingException {
Connection connection = createConnection();
connection.start();
Session session = createSession(connection);
MessageProducer messageProducer = createProducer(session);
TextMessage message = session.createTextMessage(avroMessage);
message.setStringProperty("task", task);
messageProducer.send(
message,
DeliveryMode.NON_PERSISTENT,
Message.DEFAULT_PRIORITY,
Message.DEFAULT_TIME_TO_LIVE);
connection.close();
}
private static MessageProducer createProducer(Session session) throws JMSException {
Destination producerDestination =
session.createQueue("queue?consumer.prefetchSize=1&heartbeat='10000'");
return session.createProducer(producerDestination);
}
private static Session createSession(Connection connection) throws JMSException {
return connection.createSession(Session.AUTO_ACKNOWLEDGE);
}
private static Connection createConnection() throws NamingException, JMSException {
Hashtable<Object, Object> env = new Hashtable<>();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.apache.qpid.jms.jndi.JmsInitialContextFactory");
env.put("connectionfactory.factoryLookup", amqUrl);
Context context = new javax.naming.InitialContext(env);
ConnectionFactory connectionFactory = (ConnectionFactory) context.lookup("factoryLookup");
PooledConnectionFactory pooledConnectionFactory = new PooledConnectionFactory();
pooledConnectionFactory.setConnectionFactory(connectionFactory);
pooledConnectionFactory.setMaxConnections(10);
return connectionFactory.createConnection(amqUsername, amqPassword);
}
This is my Listener config
#Bean
public JmsConnectionFactory jmsConnection() {
JmsConnectionFactory jmsConnection = new JmsConnectionFactory();
jmsConnection.setRemoteURI(this.amqUrl);
jmsConnection.setUsername(this.amqUsername);
jmsConnection.setPassword(this.amqPassword);
return jmsConnection;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(jmsConnection());
return factory;
}
And here is my Listener
#JmsListener(
destination = "queue?consumer.prefetchSize=1&heartbeat='10000'",
selector = "task = 'myTask'"
)
public void receiveMsg(Message message) throws IOException, JMSException {
message.acknowledge();
doStuff();
}
I send the message like this
QpidSender.send(avroMessage, "myTask");
This setting works. I can send different messages and as soon than there are more then 2, the second instance of my service starts and consumes the message. If later the message count is below 2, the service is terminated.
The problem is: I don't want the message to be acknowledged before the doStuff(). Because if something goes wrong or if the service is terminated before finishing doStuff(), the message is lost (right?).
But if I reorder it to
doStuff();
message.acknowledge();
the second instance can not receive a message from the broker, as long as the first service is still in doStuff() and hasn't acknowledged the message.
How do I configure this in a way, that more than one instance can consume a message from the queue, but the message isn't lost, if the service gets terminated or something else fails on doStuff()?
Use factory.setSessionTransacted(true).
See the javadocs for DefaultMessageListenerContainer:
* <p><b>It is strongly recommended to either set {#link #setSessionTransacted
* "sessionTransacted"} to "true" or specify an external {#link #setTransactionManager
* "transactionManager"}.</b> See the {#link AbstractMessageListenerContainer}
* javadoc for details on acknowledge modes and native transaction options, as
* well as the {#link AbstractPollingMessageListenerContainer} javadoc for details
* on configuring an external transaction manager. Note that for the default
* "AUTO_ACKNOWLEDGE" mode, this container applies automatic message acknowledgment
* before listener execution, with no redelivery in case of an exception.
Producer of the message is not sending message as persistent and when i am trying to consume the message through MessageListener, and any exception(runtime) occurs, it retries for specific number of times (default is 6 from AMQ side) and message get lost.
Reason is that since producer is not setting the Delivery mode as Persistent, after certain number of retry attempt, DLQ is not being created and message does not move to DLQ. Due to this , i lost the message.
My Code is like this :-
#Configuration
#PropertySource("classpath:application.properties")
public class ActiveMqJmsConfig {
#Autowired
private AbcMessageListener abcMessageListener;
public DefaultMessageListenerContainer purchaseMsgListenerforAMQ(
#Qualifier("AMQConnectionFactory") ConnectionFactory amqConFactory) {
LOG.info("Message listener for purchases from AMQ : Starting");
DefaultMessageListenerContainer defaultMessageListenerContainer =
new DefaultMessageListenerContainer();
defaultMessageListenerContainer.setConnectionFactory(amqConFactory);
defaultMessageListenerContainer.setMaxConcurrentConsumers(4);
defaultMessageListenerContainer
.setDestinationName(purchaseReceivingQueueName);
defaultMessageListenerContainer
.setMessageListener(abcMessageListener);
defaultMessageListenerContainer.setSessionTransacted(true);
return defaultMessageListenerContainer;
}
#Bean
#Qualifier(value = "AMQConnectionFactory")
public ConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory amqConnectionFactory =
new ActiveMQConnectionFactory();
amqConnectionFactory
.setBrokerURL(System.getProperty(tcp://localhost:61616));
amqConnectionFactory
.setUserName(System.getProperty(admin));
amqConnectionFactory
.setPassword(System.getProperty(admin));
return amqConnectionFactory;
}
}
#Component
public class AbcMessageListener implements MessageListener {
#Override
public void onMessage(Message msg) {
//CODE implementation
}
}
Problem :- By setting the client-id at connection level (Connection.setclientid("String")), we can subscribe as durable subscriber even though message is not persistent. By doing this, if application throws runtime exception , after a certain number of retry attempt, DLQ will be created for the Queue and message be moved to DLQ.
But in DefaultMessageListenerContainer, connection is not exposed to client. it is maintained by Class itself as a pool, i guess.
How can i achieve the durable subscription in DefaultMessageListenerContainer?
You can set the client id on the container instead:
/**
* Specify the JMS client ID for a shared Connection created and used
* by this container.
* <p>Note that client IDs need to be unique among all active Connections
* of the underlying JMS provider. Furthermore, a client ID can only be
* assigned if the original ConnectionFactory hasn't already assigned one.
* #see javax.jms.Connection#setClientID
* #see #setConnectionFactory
*/
public void setClientId(#Nullable String clientId) {
this.clientId = clientId;
}
and
/**
* Set the name of a durable subscription to create. This method switches
* to pub-sub domain mode and activates subscription durability as well.
* <p>The durable subscription name needs to be unique within this client's
* JMS client id. Default is the class name of the specified message listener.
* <p>Note: Only 1 concurrent consumer (which is the default of this
* message listener container) is allowed for each durable subscription,
* except for a shared durable subscription (which requires JMS 2.0).
* #see #setPubSubDomain
* #see #setSubscriptionDurable
* #see #setSubscriptionShared
* #see #setClientId
* #see #setMessageListener
*/
public void setDurableSubscriptionName(#Nullable String durableSubscriptionName) {
this.subscriptionName = durableSubscriptionName;
this.subscriptionDurable = (durableSubscriptionName != null);
}