Im using Http Client Processor app in one of my stream which is a standard app of SCDF.The puspose of this app is to make http calls against provided URL with a message payload. I tried to enable the retry mechanism of this app by keeping the boolean httpclient.retry.enabled to 'true'.
But when I do that it try to repost the message against the http end point even if the first attempt is succecssfull. It looks like it is working with the concept of 'write at least once'. The problem with this approach is, it creates duplicates in the target system.
Is there a way we can configure it for 'write just once if the call is successful else, retry'. If not can we expect a fix from Spring ?
The Http Client Processor is no longer supported. I recommend upgrading to HttpRequestProcessor. This uses the common retry mechanism included in the messaging binder. The behavior is as you describe, the request will be retried only if the consumer fails to acknowledge the message. With at least once guarantees, you still have the potential for duplicates.
Related
I have very regularly experienced timeouts using ReplyingKafkaTemplate and request/reply pattern for synchronous calls. It has been suggested this is due to consumers and that appears to be correct because what resolves renaming the consumer group on the calling side of request/reply and redeploying the service. This is not an optimal solution however, and I would like to prevent this error from occurring. To recap I from Service A to Service B I make a synchronous request and after a while due to staleness or something eventually I see a timeout in this request. The solution I have found is to change name of consumer group in service A and redeploy but I need a better resolution. Any suggestions?
We are using rabbitmq 3.0.1 on CentOS 6, and as a client Spring spring-rabbit version 1.1.2.RELEASE. (I know these aren't the latest versions, see later).
We send messages to rabbitmq via this client. These messages are initiated via an external rest call. Someone else calls our web service updates the database and sends the amqp message. I would like to be informed if rabbitmq blocks the client - for instance if the disk_free_limit threshold is reached.
Importantly, I would like to be informed in the same thread as that processing the web request, so that I can rollback the transaction.
Our web service can also update a database (within a transaction obviously). Normally, this works fine. However, under certain circumstances, rabbitmq can block our web server - the most obvious being when the disk_free_limit is reached. This blocks the web server Thread, indefinitely. The external caller of the web service will obviously time out after a sensible period, but the thread in our web service doesn't - it stays around, and keeps the resources, and importantly the transaction open.
The web server is blocking the thread because it is transactional. It isn't the initial message which is blocking, it is the commit. I assume that rabbitmq is blocking because it can't persist it or something like that. The thread is blocking until rabbitmq sends the commit ok message back. The bit of code is deep within the rabbitmq implementation - com.rabbitmq.client.impl.ChannelN
public Tx.CommitOk More ...txCommit() throws IOException
{
return (Tx.CommitOk) exnWrappingRpc(new Tx.Commit()).getMethod();
}
and this eventually calls the following method from com.rabbitmq.client.impl.AMQChannel
public T More ...getReply() throws ShutdownSignalException
{
return _blocker.uninterruptibleGetValue();
}
The preferable solution for this would be some sort of timeout on the txCommit - then I could throw an exception and fail the web service with a 500 or whatever. I can't find any way of doing this.
What I have found is:
addBlockedListener - this adds a listener on a message sent by rabbitmq when this it is blocked. This is good, but the message will is treated by another thread - so I can't fail the web service. Using this I can at least log the fact that rabbitmq is blocked, through syslog or whatever. However, this isn't available on the version that we run - we would have to upgrade to the latest. We would prefer not to do this because of the testing it would imply.
setConnectionTimeout(int) - this sets the connection timeout for the initial connection to rabbitmq. This doesn't apply in my case, because rabbitmq is up and running and accepts the connection.
AmqpTemplate.setReplyTimeout() - as shown above, this reply timeout does not apply to the commit.
I fully understand that this situation (disk_free_limit threshold is breached) is a situation which should not occur in a production system. However, I would like to be able to cope nicely with this situation so that my application behaves nicely when one of its components (rabbitmq) has a problem.
So, what other options do I have? Is there any way, short of rewriting portions of the spring amqp client or removing the transactionality of doing what I want?
I have a project that is related to job postings. Consultants or employers register on my website and then start posting jobs. I want to make push notifications for all users. When a consultant or employer posts a job, all online users must get notified that an employer has posted this job without any page refreshes on jquery setInterval or timeout.
I am using Spring framework. I have searched for the solution but found nothing. I want to know whether Spring provided WebSockets in their latest version. Is this possible to do with WebSockets?
I want a proper resource so that I can implement it on my website.
There are two ways to satisfy your need;
First is polling in which you repeatedly send requests from client to the server. On server side you somehow need have a kind of message queue for each client to deliver the incidents on a request. There also is a different type of polling in which you send a request from client and never end the request on the server-side thus you have a kind of pipe between two ends. This is called long polling.
Disadvantage of polling is that you have to send requests to the server forever from the client and in many cases server sends empty messages as there is no events happened.
The real application of pushing messages is recently avaliable with websockets (thanks to html5). However this requires the application server to be capable of websocket functionality. afaik jetty and tomcat has this ability. Spring 4 has websocket here you can find the tutorial; http://syntx.io/using-websockets-in-java-using-spring-4/
You can find a related stackoverflow post here
We're use MassTransit with RabbitMQ. Is there a way to check that endpoints aren't available before we publish any messages? I want to setup our IoC to use another strategy if servicebus isn't available and I don't want to get to the point when I'll catch RabbitMQ.Client.Exceptions.BrockerUnreachableException on publishing messages.
If you're using a container, you could create a decorator that could monitor the outcome of the Publish method call, and if it starts throwing exceptions, you could switch the calls over to an alternative publisher.
Ideally such an implementation would include some type of progressive retry capability so that once the endpoint becomes available the calls resume back to the actual endpoint, as well as triggering some replay of the previously failed messages to the endpoint as well.
I figure you're already dealing with the need to have an alternative storage available, such as a local endpoint or some sort of local storage.
Not currently, you can submit an issue requesting that feature: https://github.com/MassTransit/MassTransit/issues. It's not trivial to implement, but maybe not impossible.
A couple of other options people have done include a remote cluster or having a local instance to forward/cluster across all machines included in the bus.
I'm pretty new to Spring Integration and still trying to get my head around it. Right now I'm just trying to understand if the example I've found here is actually safe across multiple threads:
https://github.com/spring-projects/spring-integration-samples/blob/master/basic/jms/src/test/java/org/springframework/integration/samples/jms/ChannelAdapterDemoTest.java
My use case is as follows:
Send request to queue with JMS Reply-to as a temporary queue
Wait for response to be received on the temporary queue
Need this to happen synchronously within a method -- I don't want to split it up and make it asynchronous across several methods
Will the above example work for this? If not, am I barking up the wrong tree?
Thanks in advance.
That sample is pretty simple; it just sends the message to stdout so, yes, it's perfectly thread safe.
For the request/reply scenario you are talking about, you need to use a <gateway/> - see the other example in that sample project. In that case, you can see that the message is handled by 'demoBean' which, again, is perfectly thread safe.
For a real application, the thread-safetyness depends on the code in the services invoked by the flow receiving the message.
If you wish, you can use Spring Integration on the client side too (with an outbound gateway).