I have a spring boot project that uses Spring Integration for TCP communications. I'm sending a request to server, server returns firstly an ACK and then returns actual response. Server wants me to send ACK when I get the response. How can I send this ACK on same socketto indicate that I got the response?
This my connection factory and outbound gateway:
<int-ip:tcp-connection-factory id="tcpClientFactory"
type="client"
host="${host}"
port="${port}"
so-keep-alive="true"
single-use="true"
using-nio="true"
serializer="messageSerializer"
deserializer="messageSerializer"/>
<ip:tcp-outbound-gateway id="outboundGateway"
connection-factory="tcpClientFactory"
request-channel="toTcpAdapterChannel"
reply-channel="fromTcpAdapterChannel"
auto-startup="true"
request-timeout="90000"
remote-timeout="90000"
reply-timeout="90000"/>
This is my serialized and deserializer:
#Override
public void serialize(byte[] bytes, OutputStream outputStream) throws IOException {
super.serialize(bytes, outputStream);
if (logger.isDebugEnabled()) {
ByteArrayOutputStream btO = new ByteArrayOutputStream();
super.serialize(bytes, btO);
logger.debug("Message(Serialized) bytes:" + HexUtils.toHexString(btO.toByteArray()));
}
}
#Override
public byte[] deserialize(InputStream inputStream) throws IOException {
boolean allMessageRead = false;
byte[] incomingArray = new byte[0];
while (!allMessageRead) {
int length = inputStream.available();
incomingArray = new byte[length];
this.read(inputStream, incomingArray, false);
if (length == 1) {
logger.info("Available length of inputStream is: {}", length);
logger.info("Length is 1. Incoming byte: {}", HexUtils.toHexString(incomingArray));
return null;
} else if (length > 1) {
System.out.println("Message DUMP-1! " + HexUtils.toHexString(incomingArray));
incomingArray = Arrays.copyOfRange(incomingArray, 1, length - 3);
allMessageRead = true;
}
}
return incomingArray;
}
You can't use the gateway to send arbitrary messages, only request/reply.
To do what you need, you must use collaborating channel adapters instead.
See the documentation:
https://docs.spring.io/spring-integration/docs/current/reference/html/ip.html#ip-collaborating-adapters
EDIT
Alternatively, probably a simpler solution would be to use connection interceptors. Your interceptor can send the ack when the result is received.
https://docs.spring.io/spring-integration/docs/current/reference/html/ip.html#ip-interceptors
Related
I have been looking around for sample code how to call a Restful service written in Spring boot (deployed in different server/ip) from an EJB client.
I couldn't find a simple example or reference to guide me on how to implement an EJB client that can call a restful service(deployed in different server/ip). Could you please point me to a document or example that shows or describe how the two can interface/talk to each other.
I need to call the endpoint by passing two header parameters for authentication, if authentication is success then only retrieve the details from Rest and send back the response to EJB client.
I use something like this, try
`public void calExternal() throws ProtocolException,
MalformedURLException,
IOException,
NoSuchAlgorithmException,
InvalidKeyException {
URL myurl = new URL("API END POINT URL");
ObjectMapper mapper = new ObjectMapper();
HttpURLConnection conn = (HttpURLConnection) myurl.openConnection();
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
String payLoad = mapper.writeValueAsString("your payload here");
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("AUTHORIZATION-TYPE", "HMAC");
try {
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(payLoad);
wr.flush();
InputStream in = null;
int responseCode = conn.getResponseCode();
if (responseCode == 200) {
in = conn.getInputStream();
} else {
in = conn.getErrorStream();
}
String encoding = conn.getContentEncoding() == null ? "UTF-8" : conn.getContentEncoding();
String response = IOUtils.toString(in, encoding);
} catch (Exception e) {
e.printStackTrace();
}
}
I am calling a third party rest API, some times it sends response with status code 500, 504.
I want to make a another hit to the API if it gives above status code.
My current logic of retry is:
public <T> ResponseEntity<T> sendGetRequest(String url,
Class<T> responseClazz,
HttpHeaders headers) {
ResponseEntity<T> response = null;
int count = 0;
int maxTries = 2;
while(true) {
try {
HttpEntity request = new HttpEntity(headers);
response = restTemplate.exchange(url, HttpMethod.GET, request, responseClazz);
if(response.getStatusCode() != HttpStatus.OK) {
log.error("null or Error response from server for ", url);
}
log.info("Response received {}", response.toString());
return response;
}catch (ResourceAccessException rae){
log.warn("retry count {} {}", count, rae);
if (++count == maxTries) throw new ServerErrorException("API timeout");
}
}
}
I have also used apache http where I use CloseableHttpClient to retry for status code 500 and 504.
I have also looks to the solution of spring-retry. Is there any other method to do this?
When calling HTTP request with RestTemplate, there are 2 main cases to retry:
Specific response HTTP statuses. For example:
503 Service Unavailable status can be retried.
404 Not Found can be proceeded without a retry attempt.
ResourceAccessException which can represent some IO exception received without getting the HTTP server response, like SocketException: Broken pipe.
For solution based on RestTemplate/HttpClient, while it exposes options to retry based on the HTTP response, combining it with IOException handling can be tricky.
Solution based on Spring RetryTemplate
#Bean(name = "restTemplateRetryTemplate")
public RetryTemplate restTemplateRetryTemplate() {
return createRestTemplateRetryTemplate();
}
private RetryTemplate createRestTemplateRetryTemplate(boolean retryOnServerErrors) {
Map<Class<? extends Throwable>, Boolean> retryableExceptions = new HashMap<>();
retryableExceptions.put(ResourceAccessException.class, true);
retryableExceptions.put(HttpServerErrorException.ServiceUnavailable.class, true);
retryableExceptions.put(HttpServerErrorException.BadGateway.class, true);
retryableExceptions.put(HttpServerErrorException.GatewayTimeout.class, true);
retryableExceptions.put(HttpClientErrorException.TooManyRequests.class, true);
return createRetryTemplate(retryableExceptions);
}
private RetryTemplate createRetryTemplate(Map<Class<? extends Throwable>, Boolean> retryableExceptions) {
RetryTemplate retryTemplate = new RetryTemplate();
ExponentialRandomBackOffPolicy exponentialRandomBackOffPolicy = new ExponentialRandomBackOffPolicy();
exponentialRandomBackOffPolicy.setInitialInterval(INITIAL_INTERVAL);
exponentialRandomBackOffPolicy.setMaxInterval(MAX_INTERVAL);
exponentialRandomBackOffPolicy.setMultiplier(MULTIPLIER);
retryTemplate.setBackOffPolicy(exponentialRandomBackOffPolicy);
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(MAX_ATTEMPTS, retryableExceptions));
// Optional, for additional logging on failures.
retryTemplate.registerListener(retryTemplateLogListener);
return retryTemplate;
}
Usage example
#Autowired
#Qualifier("restTemplateRetryTemplate")
private RetryTemplate retryTemplate;
...
String result = retryTemplate.execute(arg -> {
return longRestTemplate.getForObject(url, String.class);
});
i want to receive a HTTP Stream in SpringBoot but the InputStream of HttpServletRequest seems not to be an endless HTTP Stream and only contains the Content of first HTTP Body.
I want to process a chuncked HTTP Stream in SpringBoot on which is puhed some Value String from time to time.
Currently I tried something like this in a controller:
#Override
public void test(HttpServletRequest request,
HttpServletResponse response) {
System.out.println("StreamStart");
try {
byte[] buffer = new byte[1024];
while(true){
int len = request.getInputStream().read(buffer);
if(len!=-1) {
System.out.println("Len: " + len);
System.out.println(new String(buffer));
}
Thread.sleep(500);
}
}
catch(Exception x){
x.printStackTrace();
}
System.out.println("StreamEnd");
}
However the first Request Body after the header works, but the second does not appear in my Controller.
Does SpringBoot cancles the connection or the stream?
Can I have access to the complete HTTP Input stream to get my values from it?
Maybe Multipart request would be usefull for you?
That way you can recieve multiple parts of data
https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
Example:
#PostMapping("/upload")
public void uploadStream(#RequestParam MultipartFile[] multipartFiles){
for(MultipartFile multipartFile:multipartFiles){
try {
InputStream inputStream = multipartFile.getInputStream();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I'm sending a message to Kafka using the ReplyingKafkaTemplate and it's sending the message with a kafka_correlationId. However, when it hits my #KafkaListener method and forwards it to a reply topic, the headers are lost.
How do I preserve the kafka headers?
Here's my method signature:
#KafkaListener(topics = "input")
#SendTo("reply")
public List<CustomOutput> consume(List<CustomInput> inputs) {
... /* some processing */
return outputs;
}
I've created a ProducerInterceptor so I can see what headers are being sent from the ReplyingKafkaTemplate, as well as from the #SendTo annotation. From that, another strange thing is that the ReplyingKafkaTemplate is not adding the documented kafka_replyTopic header to the message.
Here's how the ReplyingKafkaTemplate is configured:
#Bean
public KafkaMessageListenerContainer<Object, Object> replyContainer(ConsumerFactory<Object, Object> cf) {
ContainerProperties containerProperties = new ContainerProperties(requestReplyTopic);
return new KafkaMessageListenerContainer<>(cf, containerProperties);
}
#Bean
public ReplyingKafkaTemplate<Object, Object, Object> replyingKafkaTemplate(ProducerFactory<Object, Object> pf, KafkaMessageListenerContainer<Object, Object> container) {
return new ReplyingKafkaTemplate<>(pf, container);
}
I'm not sure if this is relevant, but I've added Spring Cloud Sleuth as a dependency as well, and the span/trace headers are there when I'm sending messages, but new ones are generated when a message is forwarded.
Arbitrary headers from the request message are not copied to the reply message by default, only the kafka_correlationId.
Starting with version 2.2, you can configure a ReplyHeadersConfigurer which is called to determine which header(s) should be copied.
See the documentation.
Starting with version 2.2, you can add a ReplyHeadersConfigurer to the listener container factory. This is consulted to determine which headers you want to set in the reply message.
EDIT
BTW, in 2.2 the RKT sets up the replyTo automatically if there is no header.
With 2.1.x, it can be done, but it's a bit involved and you have to do some of the work yourself. The key is to receive and reply a Message<?>...
#KafkaListener(id = "so55622224", topics = "so55622224")
#SendTo("dummy.we.use.the.header.instead")
public Message<?> listen(Message<String> in) {
System.out.println(in);
Headers nativeHeaders = in.getHeaders().get(KafkaHeaders.NATIVE_HEADERS, Headers.class);
byte[] replyTo = nativeHeaders.lastHeader(KafkaHeaders.REPLY_TOPIC).value();
byte[] correlation = nativeHeaders.lastHeader(KafkaHeaders.CORRELATION_ID).value();
return MessageBuilder.withPayload(in.getPayload().toUpperCase())
.setHeader("myHeader", nativeHeaders.lastHeader("myHeader").value())
.setHeader(KafkaHeaders.CORRELATION_ID, correlation)
.setHeader(KafkaHeaders.TOPIC, replyTo)
.build();
}
// This is used to send the reply - needs a header mapper
#Bean
public KafkaTemplate<?, ?> kafkaTemplate(ProducerFactory<Object, Object> kafkaProducerFactory) {
KafkaTemplate<Object, Object> kafkaTemplate = new KafkaTemplate<>(kafkaProducerFactory);
MessagingMessageConverter messageConverter = new MessagingMessageConverter();
messageConverter.setHeaderMapper(new SimpleKafkaHeaderMapper("*")); // map all byte[] headers
kafkaTemplate.setMessageConverter(messageConverter);
return kafkaTemplate;
}
#Bean
public ApplicationRunner runner(ReplyingKafkaTemplate<String, String, String> template) {
return args -> {
Headers headers = new RecordHeaders();
headers.add(new RecordHeader("myHeader", "myHeaderValue".getBytes()));
headers.add(new RecordHeader(KafkaHeaders.REPLY_TOPIC, "so55622224.replies".getBytes())); // automatic in 2.2
ProducerRecord<String, String> record = new ProducerRecord<>("so55622224", null, null, "foo", headers);
RequestReplyFuture<String, String, String> future = template.sendAndReceive(record);
ConsumerRecord<String, String> reply = future.get();
System.out.println("Reply: " + reply.value() + " myHeader="
+ new String(reply.headers().lastHeader("myHeader").value()));
};
}
I want to create camel route in Spring Boot (2.1.1) project to get the data from some (rest) endpoint (http://localhost:8080/getAllUsers) and to send that data to activeMq.
I have tried with timer data to send it on activeMq and to consume it and it is working. But I have problem with collecting data from endpoint.
I have tried several things but no success. This is what I have tried.
In this example I am not sending the data to ActiveMq, I just want to see the response...
public void createNewRoute() {
CamelContext context = new DefaultCamelContext();
try {
ProducerTemplate template = context.createProducerTemplate();
context.start();
Exchange exchange = template.request("http://localhost:8080/getAllUsers",
new Processor() {
public void process(Exchange exchange) throws Exception {
}
});
if (null != exchange) {
Message out = exchange.getOut();
int responseCode = out.getHeader(Exchange.HTTP_RESPONSE_CODE, Integer.class);
System.out.println("Response: " + String.valueOf(responseCode));
}
Thread.sleep(1000 * 3);
context.stop();
} catch (Exception ex) {
System.out.println("Exception: " + ex);
}
System.out.println("DONE!!");
}
Another route:
from("servlet://localhost:8080/getAllUsers").to("activemq://all-users");
And another:
rest("//localhost:8080/getAllUsers")
.get().consumes("application/json")
.to("activemq://all-users");
I will go with your second example:
from("timer://test?repeatCount=1").routeId("newRoute")
.streamCaching()
.process(exchange -> exchange.getIn()
.setBody(exchange.getIn()
.getBody()))
.marshal()
.json(JsonLibrary.Jackson)
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
.to("http://localhost:8080/getAllUsers")
.log(LoggingLevel.INFO, "This is my body: ${body}")
.to("activemq:queue://new-queue");
This will trigger it once.
Try this without context.start() ....
CamelContext camelContext = new DefaultCamelContext();
ProducerTemplate template = camelContext.createProducerTemplate();
Exchange exchange = template.send("http://localhost:8080/getAllUsers", new Processor() {
public void process(Exchange exchange) throws Exception {}
});
Message out = exchange.getOut();
The http components are streaming based, so you can ask Camel to give you the response as string instead.
String s = exchange.getMessage().getBody(String.class);
See more in these links
http://camel.apache.org/stream-caching
http://camel.apache.org/why-is-my-message-body-empty.html