Here are my Hornetq configuration in spring boot.
spring.hornetq.mode=embedded
spring.hornetq.embedded.enabled=true
spring.hornetq.embedded.persistent=true
spring.hornetq.port=5445
spring.hornetq.embedded.queues=jms.testqueue
Here is my Producer
public class Producer {#Inject
private JmsTemplate jmsTemplate;
public void resolveError( String message) {
try{
jmsTemplate.convertAndSend(DATA_QUEUE, message);
}catch(Exception e){
//log error
}
}}
Here is my Consumer
#JmsListener(destination = DATA_QUEUE)
public void consume(String message) throws InterruptedException {
log.info("Receiving event: {}", message);
try {
//do stuff with message
}catch (Exception e){
log.error(e.toString());
}
}
Here is my config file
#Configuration#EnableJms public class JmsConfig {
public static final String LOGGING_SCRAPPER_KEY ="DATA_SYNC_ERROR";
public static final String DATA_QUEUE = "jms.testqueue"; }
I want to slow down the consuming process of #JMSlistener, I don't want to the JMS listener hit the queue all the time any help is appreciated, thanks!!
The listeners that are created under the covers for each #JmsListener annotated method are held in a registry as explained in the documentation
If you want to pause your listener, it is very easy to look it up and stop it. Let's assume you have a way to invoke the following bean (JMX endpoint, secure rest mapping, whatever):
static class YourService {
private final JmsListenerEndpointRegistry registry;
#Autowired
public YourService(JmsListenerEndpointRegistry registry) {
this.registry = registry;
}
public void stopListener() {
this.registry.getListenerContainer("myListener").stop();
}
public void startListener() {
this.registry.getListenerContainer("myListener").start();
}
}
Then you need to associate the proper id to your listener (myListener) in the example above.
#JmsListener(id = "myListener", destination = DATA_QUEUE)
public void consume(String message) throws InterruptedException { ... }
I'm not able to set the consuming time of JmsListener but I found an alternative where I'm able to set delivery delay time limit on jmsTemplate instead, use jmsTemplate setDeliveryDelay which will delay sending it to the queue. Either way, it is delayed only if you go with delaying the consuming process of JMS listener you will have the message in the queue in my approach it won't be in the queue until the delay delivery time.
Related
I have 2 queues - PRIORITY_QUEUE and SIMPLE_QUEUE. The SIMPLE_QUEUE is large, the number of messages in it reaches several thousand, the PRIORITY_QUEUE can be empty and only sometimes receive a few messages. I need to ensure that messages from the SIMPLE_QUEUE are not read while there are messages in the PRIORITY_QUEUE, and if there are messages in the priority queue, reading the SIMPLE_QUEUE is blocked/paused. When the PRIORITY_QUEUE has received a message, we must pause reading from the SIMPLE_QUEUE.
prefetch property: we read messages by one
spring.rabbitmq.listener.simple.prefetch=1
listener example
#RabbitListener(queues = {priorityQueue , simpleQueue})
public void processMyQueue(String message) {
// if priorityQueue is not empty we shouldn't consume messages from simpleQueue
}
rabbit configuration
#Slf4j
#Configuration
#PropertySource({"...,..."})
#RequiredArgsConstructor
public class RabbitConfiguration {
#Value("${spring.rabbitmq.host}")
private String queueHost;
#Value("${spring.rabbitmq.username}")
private String username;
#Value("${spring.rabbitmq.password}")
private String password;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory cachingConnectionFactory = new
CachingConnectionFactory(queueHost);
cachingConnectionFactory.setUsername(username);
cachingConnectionFactory.setPassword(password);
return cachingConnectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(connectionFactory());
}
#Bean(value = "simpleQueue")
public Queue simpleQueue() {
return new Queue("simpleQueue");
}
#Bean(name = "priorityQueue")
public Queue priorityQueue() {
return new Queue("priorityQueue");
}
}
You can't do it with a single listener; use two listeners and stop the simple listener container when the priority listener receives any message; then start the simple queue listener when the priority listener goes idle.
You can use a ListenerContainerIdleEvent to detect an idle listener.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#consumer-events
Use the RabbitListenerEndpointRegistry to stop/start containers.
I have an application with WebSockets using spring-boot application as backend and Stomp/SockJS in the client side, the spring-boot application consume JMS queue messages and notify the changes to the right user. What is the problem? Sometimes works and sometimes doesn't work, same code and users could work or not.
The client side code is a bit more difficult to copy here because it's integrate over react/redux application but basically is a subscription to two different channels, both defined in the configuration of Spring. The sessions are created correctly according to debug information but just sometimes the message is processed to send it to connected sessions.
This is the configuration class for Spring.
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfiguration implements WebSocketMessageBrokerConfigurer {
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry
.addEndpoint("/stomp")
.setAllowedOrigins("*")
.withSockJS();
}
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry
.setApplicationDestinationPrefixes("/app")
.enableSimpleBroker("/xxxx/yyyy", "/ccccc");
}
#Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.interceptors(new ChannelInterceptor() {
#Override
public Message<?> preSend(Message<?> message, MessageChannel channel) {
StompHeaderAccessor accessor =
MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);
if (StompCommand.CONNECT.equals(accessor.getCommand())) {
Object raw = message
.getHeaders()
.get(SimpMessageHeaderAccessor.NATIVE_HEADERS);
if (raw instanceof Map) {
Object name = ((Map<?,?>) raw).get("email");
if (name instanceof LinkedList) {
String user = ((LinkedList<?>) name).get(0).toString();
accessor.setUser(new User(user));
}
}
}
return message;
}
});
}
}
This is the JMS listener to process queue message and send it to specific user.
#Component
public class UserEventListener {
private final Logger logger = LoggerFactory.getLogger(getClass());
private final SimpMessagingTemplate template;
#Autowired
public UserEventListener(SimpMessagingTemplate pTemplate) {
this.template = pTemplate;
}
#JmsListener(destination="user/events")
public void onStatusChange(Map<String, Object> props) {
if (props.containsKey("userEmail")) {
logger.debug("Event for user received: {}", props.get("userEmail"));
template.convertAndSendToUser((String)props.get("userEmail"), "/ccccc", props);
}
}
}
Edit 1:
After more debugging the times when doesn't work the "session" for WebSocket seems to be lost by Spring configuration. I don't see any log information about "Disconnected" messages or something similar, besides if I debug remotely the server when this happens the problem doesn't appears during debugging session. Some idea? The class from Spring where session disappear is DefaultSimpUserRegistry.
After more research I found a question with the same problem and the solution here. Basically the conclusion is this:
Channel interceptor is not the right place to authenticate user, we need to change it with a custom handshake handler.
My app successfully sends Kafka messages, but only after Kafka is initialized. Before that i get the error "Dispatcher has no subscribers". How do i wait for subscribers to finish being registered for channels?
Here's a trace of the order of events (timing in second.ms):
17.165 SenderClass created
17.816 initialization class, #PostConstruct starts PollingTask
24.781 PollingTask sends first Kafka message
24.816 First error: "Dispatcher has no subscribers"
25.778 Registering MessageChannel my-channel
still seeing Dispatcher errors
27.067 Channel my-channel' has 1 subscriber
No more errors after this, messages send fine
i'm not sure how to approach this. Wild guesses have included:
Place sending code in #PostConstruct
Add #AutoConfigureBefore(BindingServiceConfiguration.class) to Sender
Add #AutoConfigureAfter(BindingServiceConfiguration.class) to SenderClass
Add #AutoConfigureBefore(BindingServiceConfiguration.class) to Main
Place #DependsOn({"EnableBindingClass"}) on Task
Place #DependsOn({"ApplicationLifeCycle"}) on SchedulerClass, where ApplicationLifeCycle is a class that does nothing but
implements SmartLifecycle with getPhase returning MAX_INT
Making sure ComponentScan is on for whole package (a suggestion from other SO threads)
Various combinations of the above
Created a new app, made it as simple as i could:
public interface Source {
#Output(channelName)
MessageChannel outboundChannel();
}
#EnableBinding(Source.class)
#Component
public class Sender {
#Autowired
private Source source;
public boolean send(SomeObject object) {
return source.outboundChannel().send(MessageBuilder.withPayload(object).build());
}
#Service
public class Scheduler {
#Autowired
Sender sender;
#Autowired
ThreadPoolTaskScheduler taskScheduler;
#PostConstruct
public void initialize() {
taskScheduler.schedule(new PollingTask(), nextTime);
}
private class PollingTask implements Runnable {
#Override
public void run() {
List<SomeObject> objects = getDummyData();
for(SomeObject object : objects)
{
sender.send(interval);
}
Instant nextTime = Instant.now().plusMillis(1_000L);
try {
taskScheduler.schedule(new PollingTask(), nextTime);
} catch (Exception e) {
logger.error(e);
}
}
}
Edit to add Solution
It works now! In my scheduler that starts the things that send the messages i switched from starting things in #PostConstruct to SmartLifecycle::start().
#Service
public class Scheduler implements SmartLifecycle {
#Autowired
Sender sender;
#Autowired
ThreadPoolTaskScheduler taskScheduler;
#Override
public void start() {
taskScheduler.schedule(new PollingTask(), nextTime);
}
private class PollingTask implements Runnable {
#Override
public void run() {
List<SomeObject> objects = getDummyData();
for(SomeObject object : objects)
{
sender.send(interval);
}
Instant nextTime = Instant.now().plusMillis(1_000L);
try {
taskScheduler.schedule(new PollingTask(), nextTime);
} catch (Exception e) {
logger.error(e);
}
}
}
#PostConstruct is too early to send messages; the context is still being built.. Implememt SmartLifecycle, put the bean in a high phase (Integer.MAX_VALUE) and do the sends in start().
Or do the sends in an ApplicationRunner.
I faced a similar problem in Webflux + Spring Cloud Stream functional style. Spring Cloud Function in 2022 is the preferred way.
My hypothesis after a lot of debugging was that beans were not created in right order. The bean was probably not registered in spring-cloud-stream's dispatchers before kafka message processing started. similar to what #gary mentioned.
So I added #Order(1) before my consumer beans. Hoping that this bean would be created before it is dispatcher-registrations starts.
#Bean
#Order(1)
public Function<Flux<Message<Pojo>>, Mono<Void>> pojoConsumer() {
This seems to fix my issue for now.
I currently have a spring application with hibernate and a PlataformTransactionManager running on Jboss/wildfly.
Some of the methods that manipulate the database also call a bean which contains a LinkedBlockingQueue. This queue stores logging messages that are periodically dispatched to someplace else on another thread (using simple spring #Scheduler).
Would it be possible to make my queue (inside a bean) transactional? ie. if the transaction rollback would I be able to "undo" any operations made on my Collection? What's the best strategy to implement this ?
So, in short something like:
#Service
#Transactional
public PersonService {
#Autowired
EntityManager EM;
#Autowired
LoggingBuffer logger;
public void addPerson(String name) {
EM.persist(new Person(.....));
logger.add("New person!");
// A rollback here via some thrown exception would not affect the queue
}
}
#Component
public class LoggingBuffer {
private Queue<String> q= new LinkedBlockingQueue<String>();
public add(String msg){
q.add(msg);
}
}
Try something like this
#Transactional
public void addPerson(String name) {
EM.persist(new Person(.....));
//logger.add("New person!");
// A rollback here via some thrown exception would not affect the queue
}
public void wrapAddPerson(String name){
List<String> localBuffer = new ArrayList<>();
try{
addPerson(name);
localBuffer.add(".....");
}catch(Exception e)
{
localBuffer.clear();
}
finally{
localBuffer.forEach(logger::add);
}
}
So I got a advanced queue working with a Connectionfactory:
ConnectionFactory jmsQueueConnectionFactory() throws JMSException, SQLException {
final OracleDataSource dataSource = new OracleDataSource();
dataSource.setUser(username);
dataSource.setPassword(password);
dataSource.setURL(url);
dataSource.setImplicitCachingEnabled(true);
dataSource.setFastConnectionFailoverEnabled(true);
return AQjmsFactory.getConnectionFactory(dataSource);
}
This is running on a shared Database which might be restarted or sometimes the network just has a short hickup. Which results in no more messages from the Queue.
I use a spring MessageListener to retrieve messages and there is actually no indicator or what so ever that the queue is not running anymore. After restarting the application I then get a load of older messages that should have been processed already.
Is there a way or specific data source implementation which reconnects or something?
Update: Listener Impl
#Bean
OracleAqQueueFactoryBean etlQueueFactory() throws JMSException, SQLException {
final OracleAqQueueFactoryBean bean = new OracleAqQueueFactoryBean();
bean.setConnectionFactory(jmsQueueConnectionFactory());
bean.setOracleQueueUser("USER");
bean.setOracleQueueName("QUEUE");
return bean;
}
#Bean
DefaultMessageListenerContainer jmsContainer() throws JMSException, SQLException {
final DefaultMessageListenerContainer bean = new DefaultMessageListenerContainer();
bean.setConnectionFactory(jmsQueueConnectionFactory());
bean.setDestination(etlQueueFactory().getObject());
bean.setMessageListener(new MyListener());
bean.setSessionTransacted(false);
return bean;
}
public class MyListener implements MessageListener {
#Override
public void onMessage(Message message) {
...
}
}
I guess you have to do it on the JMS level, not DB level.
Not sure what type of listener you are using, but a DefaultMessageListenerContainer in Spring is implemented with a consumer.receive(timeout) loop. It's more robust than using a plain listener as it will attempt to reconnect on each poll cycle (if needed).