apache fluent client POST blocks forever when underlying call actually returned - my thread stuck - spring-boot

I'm calling the Post method of org.apache.http.client.fluent.Request class, and it's blocking - the next line of code is not executed. I'm getting logging that says that I got a successful response back too! My log messages (below) confirm that the message came back with an OK status (200).
>> ...with body JSON:
<< status: 200
So what happened? More importantly, what can I do to protect myself?
P.S. I've used this code thousands of times and this has, to my knowledge, never happened before.
Context
This is a Spring BOOT application (version 1.3.5). I'm using the RESTTemplate to communicate internally among microservices, but I'm using the apache fluent client to talk to an outside web service. I'm using org.apache.httpcomponents (version 4.5.2).
My code:
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.stream.Collectors;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.StatusLine;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpResponseException;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.fluent.Request;
import org.apache.http.entity.ContentType;
import org.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.eneura.devcom.microservices.exceptions.ExternalWebServiceCommunicationFailureException;
import com.eneura.devcom.microservices.exceptions.InvalidArgumentException;
import com.eneura.devcom.microservices.exceptions.PreconditionsFailureException;
final private String internalDoPostSingleCall(
String subUrlPath, String jsonString, List<String> parmList
)
throws InvalidArgumentException, ExternalWebServiceCommunicationFailureException, HttpResponseException
{
String bodyArg = (StringUtils.isNotBlank(jsonString)) ? jsonString : ""; //don't pass in null or blanks or ... just fold down to ""
String url = computeUrl(subUrlPath, parmList);
logger.info(">> sending POST " + url);
logger.info(">> ...with body JSON: " + bodyArg);
String res;
try {
res = Request
.Post( url )
.bodyString(
bodyArg, ContentType.APPLICATION_JSON
).execute()
.handleResponse(new ResponseHandler<String>() {
public String handleResponse(
final HttpResponse response) throws IOException {
StatusLine statusLine = response.getStatusLine();
HttpEntity entity = response.getEntity();
logger.info("<< status: " + statusLine.getStatusCode() );
if (statusLine.getStatusCode() >= 300) {
throw new HttpResponseException(statusLine
.getStatusCode(), statusLine
.getReasonPhrase());
}
String content =
(null == entity) ? (
"" // ok not to have any content back.
) : (
IOUtils.toString(entity.getContent(), "UTF-8")
);
return content;
}
});
} catch (HttpResponseException e) {
throw e;
} catch (IOException e) {
String msg =
"failed to perform POST to ExternalWebService - exception: " + e.getLocalizedMessage();
throw new ExternalWebServiceCommunicationFailureException(msg);
}
logger.info("<< body returned for POST (len="+ res.length() +"): " + res);
return res;
}
Log messages:
2020-03-14 18:46:14.119 INFO 15213 --- [http-nio-6444-exec-9] c.e.d.m.s.avcourier.AVRestDelegate : >> sending POST https://externalwebservice.com/api/v1/operations/a0a01212445566770011001130304040/cancel
2020-03-14 18:46:14.119 INFO 15213 --- [http-nio-6444-exec-9] c.e.d.m.s.avcourier.AVRestDelegate : >> ...with body JSON:
2020-03-14 18:46:14.393 INFO 15213 --- [http-nio-6444-exec-9] c.e.d.m.s.avcourier.AVRestDelegate : << status: 200
2020-03-14 18:50:07.589 INFO 15213 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration
2020-03-14 18:55:07.590 INFO 15213 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration
2020-03-14 19:00:07.591 INFO 15213 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration

Related

Using Spring EL to add optional postfix from properties to consumerGroup in #KafkaListener

I have simple spring boot application with Kafka Consumers that looks like
#KafkaListener(topics="topic", groupId="SOME_CONSTANT") {
....
}
What I am required to do Is to add optional spring boot property (from env variables but that is not important) lets say:
myapp.env: TEST
And when that variable is present I should automatically update consumer group to be
SOME_CONSTANT-TEST
I am playing with SPEL
#KafkaListener(topics="topic", groupId="#{ '${myApp.env}' == null ? 'SOME_CONSTANT' : 'SOME_CONSTANT' + '-' + '${myApp.env}}'") {
....
}
But that does not seem to work :/ Any Ideas?
You can use the T operator to read the constant's value, and use the colon ':' for the case when there's no env variable:
#KafkaListener(topics="topic", groupId="#{ '${my.app.env:}' == '' ? T(com.mypackage.MyListener).SOME_CONSTANT : T(com.mypackage.MyListener).SOME_CONSTANT + '-' + '${my.app.env:}'}")
Here's a sample application with this solution:
package org.spring.kafka.playground;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaOperations;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.stereotype.Component;
#SpringBootApplication
public class SO71291726 {
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(SO71291726.class, args);
try {
Thread.sleep(10000);
}
catch (InterruptedException e) {
Thread.interrupted();
throw new RuntimeException("Interrupted");
}
KafkaOperations kafkaTemplate = context.getBean("kafkaTemplate", KafkaOperations.class);
kafkaTemplate.send("topic", "My message");
}
Logger log = LoggerFactory.getLogger(this.getClass());
public static final String SOME_CONSTANT = "my-group-id-constant";
#Component
class MyListener {
#KafkaListener(topics="topic", groupId="#{ '${71291726.my.app.env:}' == '' ? T(org.spring.kafka.playground.SO71291726).SOME_CONSTANT : T(org.spring.kafka.playground.SO71291726).SOME_CONSTANT + '-' + '${71291726.my.app.env:}'}")
void listen(String message, #Header(KafkaHeaders.GROUP_ID) String groupId) {
log.info("Received message {} from group id {} ", message, groupId);
}
}
}
Output:
2022-02-28 14:26:14.733 INFO 18841 --- [ntainer#0-0-C-1] 1291726$$EnhancerBySpringCGLIB$$cf264156 : Received message My message from group id my-group-id-constant
If I add 71291726.my.app.env = TEST to the application.properties file:
2022-02-28 14:34:03.900 INFO 18870 --- [ntainer#0-0-C-1] 1291726$$EnhancerBySpringCGLIB$$e1a5933e : Received message My message from group id my-group-id-constant-TEST

Call BigQuery stored procedure(Routine) using spring boot

I'm trying to call a Google BigQuery stored procedure (Routine) using Spring boot. I tried all the methods of the routines to extract data. However, it didn't help.
Has anyone ever created and called a BigQuery stored procedure (Routine) through the Spring boot? If so, how?
public static Boolean executeInsertQuery(String query, TableId tableId, String jobName) {
log.info("Starting {} truncate query", jobName);
BigQuery bigquery = GCPConfig.getBigQuery(); // bqClient
// query configuration
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query)
.setUseLegacySql(false)
.setAllowLargeResults(true)
.setDestinationTable(tableId) .setWriteDisposition(JobInfo.WriteDisposition.WRITE_TRUNCATE).build();
try {
// build the query job.
QueryJob queryJob = new QueryJob.Builder(queryConfig).bigQuery(bigquery).jobName(jobName).build();
QueryJob.Result result = queryJob.execute();
} catch (JobException e) {
log.error("{} unsuccessful. job id: {}, job name: {}. exception: {}", jobName, e.getJobId(),
e.getJobName(), e.toString());
return false;
}
}
package ops.google.com;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryError;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.EncryptionConfiguration;
import com.google.cloud.bigquery.InsertAllRequest;
import com.google.cloud.bigquery.InsertAllResponse;
import com.google.cloud.bigquery.QueryJobConfiguration;
import com.google.cloud.bigquery.TableId;
import com.google.cloud.bigquery.TableResult;
import com.google.common.collect.ImmutableList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class SelectFromBigQueryFunction {
private static final Logger logger = LogManager.getLogger(SelectFromBigQueryFunction.class);
public boolean tableSelectFromJoin(String key_path) {
String projectID = "ProjectID";
String datasetName = "DataSetName";
String tableName1 = "sample_attribute_type";
String tableName2 = "sample_attribute_value";
boolean status = false;
try {
//Call BQ Function/Routines, functinon name->bq_function_name
//String query = String.format("SELECT DataSetName.bq_function_name(1, 1)");
//Call BQ Stored Procedure, procedure name-> bq_stored_procedure_name
String query = String.format("CALL DataSetName.bq_stored_procedure_name()");
File credentialsPath = new File(key_path);
FileInputStream serviceAccountStream = new FileInputStream(credentialsPath);
GoogleCredentials credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
// Initialize client that will be used to send requests. This client only needs to be created
BigQuery bigquery = BigQueryOptions.newBuilder()
.setProjectId(projectID)
.setCredentials(credentials)
.build().getService();
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();
TableResult results = bigquery.query(queryConfig);
results.iterateAll().forEach(row -> row.forEach(val -> System.out.printf("%s,", val.toString())));
logger.info("Query performed successfully with encryption key.");
status = true;
} catch (BigQueryException | InterruptedException e) {
logger.error("Query not performed \n" + e.toString());
}catch(Exception e){
logger.error("Some Exception \n" + e.toString());
}return status;
}
}

Spring Cloud Open Feign: linebreaks "\n" stripped from gzip response body

Linebreaks ("\n" or "\r") are been stripped out from gzip response body, when using response compression configuration (as below) in Spring Cloud Open Feign. There are no errors raised. Linebreaks are just been replace by an empty string "". The response has the correct "content-encoding: gzip" header, and a well formed gzipped body content.
Does someone has a clue? It seems a issue for me as I opened here spring-cloud-openfeign/issue400
feign.compression.response.enabled: true
feign.compression.response.useGzipDecoder: true
# Same behaviour using Apache Http as client
feign.httpclient.enabled: true
SpringCloudFeignClient.java
package springcloudfeigngzip;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
#FeignClient(name = "SpringCloudFeignClient", url = "http://localhost:8082")
public interface SpringCloudFeignClient {
#GetMapping(value = "/gzip")
String getGzippedString();
}
ApplicationTest.java:
package springcloudfeigngzip;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.contract.wiremock.AutoConfigureWireMock;
import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.junit.jupiter.api.Assertions.assertEquals;
#SpringBootTest
#AutoConfigureWireMock(port = 8082)
class ApplicationTest {
#Autowired
private SpringCloudFeignClient springCloudFeignClient;
#Test
void success_GzipOneLine() {
stubFor(get(urlEqualTo("/gzip")).withHeader("Accept-Encoding", containing("gzip"))
.willReturn(aResponse().withStatus(200).withBody("lineone")));
String response = springCloudFeignClient.getGzippedString();
assertEquals("lineone", response); //success
}
#Test
void fail_GzipLineBreak() {
stubFor(get(urlEqualTo("/gzip")).withHeader("Accept-Encoding", containing("gzip"))
.willReturn(aResponse().withStatus(200).withBody("lineone\nlinetwo")));
String response = springCloudFeignClient.getGzippedString();
assertEquals("lineone\nlinetwo", response); //fail!
}
}
bootstrap.yml:
feign:
compression:
response:
enabled: true
useGzipDecoder: true
Versions:
org.springframework.boot: 2.3.3.RELEASE
org.springframework.cloud: Hoxton.SR8
Full project here:
https://github.com/spring-cloud/spring-cloud-openfeign/files/5147346/spring-cloud-feign-gzip.zip

oracle.stellent.ridc.protocol.http.HttpProtocolException: Http status: HTTP/1.1 401 Authorization Required

I am facing some issues while pinging the server With RIDC code for oracle UCM 10 g and i am getting HTTP/1.1 401 Authorization Required exception.
import java.util.logging.Level;
import java.util.logging.Logger;
import oracle.stellent.ridc.IdcClient;
import oracle.stellent.ridc.IdcClientException;
import oracle.stellent.ridc.IdcClientManager;
import oracle.stellent.ridc.IdcContext;
import oracle.stellent.ridc.model.DataBinder;
import oracle.stellent.ridc.protocol.ServiceResponse;
public class PingGuest {
IdcClientManager manager = new IdcClientManager();
IdcClient idcClient;
public PingGuest() throws IdcClientException {
this.idcClient = manager.createClient("Server Address");
idcClient.getConfig ().setProperty ("http.library", "apache4");
IdcContext userPasswordContext = new IdcContext("user", "pass");
DataBinder dataBinder = this.idcClient.createBinder ();
dataBinder.putLocal ("IdcService", "PING_SERVER");
ServiceResponse response = idcClient.sendRequest (userPasswordContext, dataBinder);
}
public static void main(String args[]){
try {
new PingGuest();
} catch (IdcClientException ex) {
Logger.getLogger(PingGuest.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
This is the error.
oracle.stellent.ridc.protocol.http.HttpProtocolException: Http status: HTTP/1.1 401 Authorization Required
Any kind of help is appreciated.
You have omitted your server address. I understand why you omitted it, but getting that wrong might cause the issue you are seeing. If you've read newer documentation, keep in mind that it differs a bit in the older versions.

How to convert a string to an Apache HttpComponents HttpRequest

I have a String that contains an HTTP header. I want to turn this into an Apache HttpComponents HttpRequest object. Is there a way to do this without picking apart the string myself?
This tutorial: http://hc.apache.org/httpcomponents-core-dev/tutorial/html/fundamentals.html#d5e56 and the javadoc does not indicate as much.
A class to convert a string to apache request:
import org.apache.http.*;
import org.apache.http.impl.DefaultHttpRequestFactory;
import org.apache.http.impl.entity.EntityDeserializer;
import org.apache.http.impl.entity.LaxContentLengthStrategy;
import org.apache.http.impl.io.AbstractSessionInputBuffer;
import org.apache.http.impl.io.HttpRequestParser;
import org.apache.http.io.HttpMessageParser;
import org.apache.http.io.SessionInputBuffer;
import org.apache.http.message.BasicHttpEntityEnclosingRequest;
import org.apache.http.message.BasicLineParser;
import org.apache.http.params.BasicHttpParams;
import java.io.ByteArrayInputStream;
import java.io.IOException;
/**
*
*/
public class ApacheRequestFactory {
public static HttpRequest create(final String requestAsString) {
try {
SessionInputBuffer inputBuffer = new AbstractSessionInputBuffer() {
{
init(new ByteArrayInputStream(requestAsString.getBytes()), 10, new BasicHttpParams());
}
#Override
public boolean isDataAvailable(int timeout) throws IOException {
throw new RuntimeException("have to override but probably not even called");
}
};
HttpMessageParser parser = new HttpRequestParser(inputBuffer, new BasicLineParser(new ProtocolVersion("HTTP", 1, 1)), new DefaultHttpRequestFactory(), new BasicHttpParams());
HttpMessage message = parser.parse();
if (message instanceof BasicHttpEntityEnclosingRequest) {
BasicHttpEntityEnclosingRequest request = (BasicHttpEntityEnclosingRequest) message;
EntityDeserializer entityDeserializer = new EntityDeserializer(new LaxContentLengthStrategy());
HttpEntity entity = entityDeserializer.deserialize(inputBuffer, message);
request.setEntity(entity);
}
return (HttpRequest) message;
} catch (IOException e) {
throw new RuntimeException(e);
} catch (HttpException e) {
throw new RuntimeException(e);
}
}
}
and a test class showing how to use it:
import org.apache.http.HttpRequest;
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.http.message.BasicHttpEntityEnclosingRequest;
import org.junit.Test;
import java.io.IOException;
import java.net.URI;
import java.util.List;
import static org.junit.Assert.*;
/**
*
*/
public class ApacheRequestFactoryTest {
#Test
public void testGet() {
String requestString = "GET /?one=aone&two=atwo HTTP/1.1\n" +
"Host: localhost:7788\n" +
"Connection: Keep-Alive\n" +
"User-Agent: Apache-HttpClient/4.0.1 (java 1.5)";
HttpRequest request = ApacheRequestFactory.create(requestString);
assertEquals("GET", request.getRequestLine().getMethod());
List<NameValuePair> pairs = URLEncodedUtils.parse(URI.create(request.getRequestLine().getUri()), "ISO-8859-1");
checkPairs(pairs);
}
#Test
public void testPost() throws IOException {
String requestString = "POST / HTTP/1.1\n" +
"Content-Length: 17\n" +
"Content-Type: application/x-www-form-urlencoded; charset=ISO-8859-1\n" +
"Host: localhost:7788\n" +
"Connection: Keep-Alive\n" +
"User-Agent: Apache-HttpClient/4.0.1 (java 1.5)\n" +
"\n" +
"one=aone&two=atwo";
HttpRequest request = ApacheRequestFactory.create(requestString);
assertEquals("POST", request.getRequestLine().getMethod());
List<NameValuePair> pairs = URLEncodedUtils.parse(((BasicHttpEntityEnclosingRequest)request).getEntity());
checkPairs(pairs);
}
private void checkPairs(List<NameValuePair> pairs) {
for (NameValuePair pair : pairs) {
if (pair.getName().equals("one")) assertEquals("aone", pair.getValue());
else if (pair.getName().equals("two")) assertEquals("atwo", pair.getValue());
else assertTrue("got more parameters than expected:"+pair.getName(), false);
}
}
}
And a small rant:
WHAT ARE THE APACHE HTTP TEAM THINKING ? The api is incredibly awkward to use. Developers around the world are wasting time writing wrapper and conversion classes for what should be run of the mill every day usage (like this example the simple act of converting a string to an apache http request, and the bizarre way you need to extract the form parameters (also having to do it in two different ways depending on what type of request was made)). The global time wasted because of this is huge. When you write an API from the bottom up, starting with the specs, you MUST then start a layer from the top down (top being an interface where you can get typical work done without having to understand or look at the way the code is implemented), making every day usage of the library CONVENIENT and intuitive. Apache http libraries are anything but. It's almost a miracle that its the standard library for this type of task.

Resources