RabbitMQ channel.addConfirmListener() , interface ackCallback Some callbacks are missing? - spring-boot

This is my code, channel.addConfirmListener() ackCallback Some callbacks will be lost, The message is indeed sent to the rabbitmq server and can be consumed normally , But I sleep for 2ms after sending the message, and all ack callbacks can be received,
I don't know if this is an error in my code or a rabbitmq bug
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import lombok.extern.log4j.Log4j2;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.TimeoutException;
#Log4j2
public class 异步确认发布{
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("");
connectionFactory.setPort(7005);
connectionFactory.setUsername("");
connectionFactory.setPassword("");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
// 开启确认发布
AMQP.Confirm.SelectOk selectOk = channel.confirmSelect();
channel.queueDeclare("hello", true, false, false, null);
// 异步确认发布消息 回调
channel.addConfirmListener(
(deliveryTag, multiple) -> {
log.info("消息deliveryTag=>{}, send successful", deliveryTag);
},
(deliveryTag, multiple) -> {
log.info("消息deliveryTag=>{}, fail in send", deliveryTag);
}
);
for (int i = 0; i < 5; i++) {
String message = "Hello World!!! " + i;
channel.basicPublish("", "hello", null, message.getBytes(StandardCharsets.UTF_8));
}
}
}
The console shows some callbacks missing
17:04:29.607 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>4, send successful
17:04:29.615 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>5, send successful
But I sleep for 2ms after sending the message, and all callbacks can be received
example code
for (int i = 0; i < 5; i++) {
String message = "Hello World!!! " + i;
channel.basicPublish("", "hello", null, message.getBytes(StandardCharsets.UTF_8));
Thread.sleep(2); // I sleep for 2ms after sending the message, and all ack callbacks can be received
}
console log
17:05:18.037 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>1, send successful
17:05:18.043 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>2, send successful
17:05:18.043 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>3, send successful
17:05:18.043 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>4, send successful
17:05:18.043 [AMQP Connection 27.11.210.232:7005] INFO me.demo.me.rabbitmq.consumer.发布确认.异步确认发布 - ackCallback, deliveryTag=>5, send successful
My RabbitMQ Server Version is 3.9.14 (No configuration has been modified. The default configuration is used), Erlang 24.3.2 ,
Maven Project dependency in
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit</artifactId>
<version>2.2.18.RELEASE</version>
</dependency>
I tried to prevent the main thread from shutting down, but it doesn't seem to be the reason for the main thread to shut down, because the main thread won't shut down automatically once the connection is created

I am not sure why you tagged this with spring-rabbit because you are not using the spring-rabbit APIs at all; you are using the amqp-client directly.
This is working as designed; for performance reasons, the confirm callback has the additional argument multiple when true; this means that all tags up to and including this one are confirmed with a single confirmation.
https://www.rabbitmq.com/tutorials/tutorial-seven-java.html
multiple: this is a boolean value. If false, only one message is confirmed/nack-ed, if true, all messages with a lower or equal sequence number are confirmed/nack-ed.

Related

Spring webflux websocket closed when receive message too fast

enter code herepublic Mono<Void> handle(#Nonnull WebSocketSession session) {
final WebSocketContext webSocketContext = new WebSocketContext(session);
Mono<Void> output = session.send(Flux.create(webSocketContext::setSink));
Mono<Void> input = session.receive()
.timeout(Duration.ofSeconds(adapterProperties.getSessionTimeout()))
.doOnSubscribe(subscription -> subscription.request(64))
.doOnNext(WebSocketMessage::retain)
.publishOn(Schedulers.boundedElastic())
.concatMap(msg -> {
// ....blocking operation
return Flux.empty();
}).then();
return Mono.zip(input, output).then();
When I use ws client to send information, because the message sending speed is too fast, when 2000 pieces of data are received, the connection is disconnected and there is no exception message,
After I slow down the sending speed of the message on the client side, there is no problem. How can I solve it?
Below is the Flux log information:
2022-07-14 17:19:40.295 adapter-iat [boundedElastic-5] INFO reactor.Flux.PublishOn.5 - | onNext(WebSocket TEXT message (13765 bytes))
2022-07-14 17:19:40.296 adapter-iat [boundedElastic-5] INFO reactor.Flux.PublishOn.5 - | onNext(WebSocket TEXT message (13765 bytes))
2022-07-14 17:19:40.300 adapter-iat [boundedElastic-5] INFO reactor.Flux.PublishOn.5 - | onComplete()????why

How to detect if RSocket connection is successfull?

I have the following program through which I can detect the connection failure i.e doBeforeRetry.
Can someone tell me how to detect the successful connection or reconnection. I want to integrate a Health Check program that monitors this connection, but I am unable to capture the event that informs the connections is successfull.
Thanks
requester = RSocketRequester.builder()
.rsocketConnector(connector -> {
connector.reconnect(Retry
.fixedDelay(Integer.MAX_VALUE,Duration.ofSeconds(1))
.doBeforeRetry(e-> System.out.println("doBeforeRetry===>"+e))
.doAfterRetry(e-> System.out.println("doAfterRetry===>"+e))
);
connector.payloadDecoder(PayloadDecoder.ZERO_COPY);
}
).dataMimeType(MediaType.APPLICATION_CBOR)
.rsocketStrategies(strategies)
.tcp("localhost", 7999);
I achieved the detection of successful connection or reconnection with the following approach.
Client Side (Connection initialization)
Mono<RSocketRequester> requester = Mono.just(RSocketRequester.builder()
.rsocketConnector(
// connector configuration goes here
)
.dataMimeType(MediaType.APPLICATION_CBOR)
.setupRoute("client-handshake")
.setupData("caller-name")
.tcp("localhost", 7999)));
One the server side
#ConnectMapping("client-handshake")
public void connect(RSocketRequester requester, #Payload String callerName) {
LOG.info("Client Connection Handshake: [{}]", callerName);
requester
.route("server-handshake")
.data("I am server")
.retrieveMono(Void.class)
.subscribe();
}
On the client side, when I receive the callback on the below method, I detect the connection is successfull.
#MessageMapping("server-handshake")
public Mono<ConsumerPreference> handshake(final String response){
LOG.info("Server Connection Handshake received : Server message [{}]", response.getCallerName());
connectionSuccess.set(true);
return Mono.empty();
}else{
throw new InitializationException("Invalid response message received from Server");
}
}
Additionally, created a application level heartbeat to ensure, the liveliness of the connection.
If you want to know if it's actually healthy, you should probably have a side task that is polling the health of the RSocket, by sending something like a custom ping protocol to your backend. You could time that and confirm that you have a healthy connection, record latencies and success/failures.

Jetty websocket client issue

Use Eclipse IDE in an attempt to create a java websocket client for an OpenHab project, thus using default Jetty websocket library. The OpenHab project intend to connect to a websocket endpoint requiring token authorization, send a request message and retrieve live/continuous measurements.
So, I have
public static final String SUBSCRIPTION_URL = "wss://api.tibber.com/v1-beta/gql/subscriptions";
Further, my WS open code:
public void open() throws Exception {
if (isConnected()) {
logger.warn("Open: connection is already open");
}
logger.warn("Connecting to: {}", SUBSCRIPTION_URL);
sslContextFactory = new SslContextFactory(true);
sslContextFactory.setTrustAll(true);
client = new WebSocketClient(sslContextFactory);
client.setMaxIdleTimeout(360 * 1000);
TibberWebSocketListener socket = new TibberWebSocketListener();
request = new ClientUpgradeRequest();
String token = new StringBuilder("Bearer ").append(configuration.getToken()).toString();
request.setHeader("Authorization", token);
request.setSubProtocols("graphql-subscriptions");
client.start();
client.connect(socket, new URI(SUBSCRIPTION_URL), request);
}
However, with this code it seems like I get connected, but end up with an IOException: Broken pipe after 1 minute. I get the same error both if I just connect without message / connect with send message.
13:59:15.987 [safeCall-1] WARN o.o.b.t.i.handler.TibberHandler:346 - Connecting to: wss://api.tibber.com/v1-beta/gql/subscriptions
13:59:16.390 [#1379116703-141] WARN o.o.b.t.i.handler.TibberHandler:385 - Connected to Server
14:00:16.430 [#1379116703-142] WARN o.o.b.t.i.handler.TibberHandler:392 - Closing a WebSocket due to Disconnected
14:00:16.434 [#1379116703-139] WARN o.o.b.t.i.handler.TibberHandler:399 - Error during websocket communication: Broken pipe
java.io.IOException: Broken pipe
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.flush(SslConnection.java:928)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
at org.eclipse.jetty.websocket.common.io.FrameFlusher.flush(FrameFlusher.java:264)
at org.eclipse.jetty.websocket.common.io.FrameFlusher.process(FrameFlusher.java:193)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.outgoingFrame(AbstractWebSocketConnection.java:516)
at org.eclipse.jetty.websocket.client.io.WebSocketClientConnection.outgoingFrame(WebSocketClientConnection.java:72)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.close(AbstractWebSocketConnection.java:184)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:458)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:428)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:426)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:320)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:158)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
at java.lang.Thread.run(Thread.java:748)
14:00:16.435 [#1379116703-139] WARN o.o.b.t.i.handler.TibberHandler:392 - Closing a WebSocket due to Broken pipe
java.io.IOException: Broken pipe
That means the OS or network below Java (and Jetty) detected that the connection was closed.
"Broken Pipe" is actually fairly common, especially with mobile devices or wireless networking.
Not much Java or Jetty can do about it either, it happened outside of its control.

okhttp3.OkHttpClient - A connection to any_host was leaked.Did you forget to close a response body?

In my Kotlin's project I use okhttp3
Here example to connect to remote host:
import okhttp3.*
import okhttp3.logging.HttpLoggingInterceptor
var оkHttpClient = OkHttpClient() оkHttpClient.newCall(requestNegotiate).enqueue(object : Callback {
override fun onResponse(call: Call, response: Response) {
...
}}
and is's work fine.
But after few minutes I get many times the next message:
2019-02-13 18:39:39,197 675399 [OkHttp ConnectionPool] WARN okhttp3.OkHttpClient - A connection to any_host_com was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

google-cloud pubsub leaves messages in queue after sending ack

I have this subscriber code:
try {
//subscriber
syncSubscriber.createSubscriber(SdkServiceConfig.s.SUBSCRIPTION_NAME_PARTNER_REQUEST);
final List<ReceivedMessage> messages = syncSubscriber.fetch(10, true);//get all current messages.
List<String> ackIds = new ArrayList<>();
for (ReceivedMessage message : messages) {
requestToCofmanSender.receiveMessage(message.getMessage());
ackIds.add(message.getAckId());
}
//preferred bulk ack, due to network performance
syncSubscriber.sendAck(ackIds);
requestToCofmanSender.getWazePublisher().shutdown();
}
and
public void sendAck(Collection<String> ackIdList) {
if (ackIdList != null && ackIdList.size() != 0) {
String subscriptionName = SubscriptionName.format(this.getProjectId(), this.subscriptionId);
AcknowledgeRequest acknowledgeRequest = AcknowledgeRequest.newBuilder().setSubscription(subscriptionName).addAllAckIds(ackIdList).build();
this.subscriber.acknowledgeCallable().call(acknowledgeRequest);
}
}
I poll the pubsub queue in loop
and even though the code sends ack i still get the same messages.
how should i ack otherwise?
My problem was that i had a break point between receiving the message and sending ack. My pubsub was configured to 10 seconds timeout.

Resources