ElasticSearch Rest API in JAVA - elasticsearch

I am trying to write a simple JAVA REST Client through which I want to PUT/GET elasticsearch document information.
PUT is working fine, my json data got added into index.
But the problem is GET, Response Code is 200, but it is not returning any data.
Can anyone please help.
public static String httpGet(String resturl){
String output = null;
try {
URL url = new URL(resturl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Content-Type", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
//System.out.println(output);
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return output;
}
I am calling as
RestClient.httpGet("http://localhost:9200/gabsindex/employee/_search")

You println call is currently commented out. If you uncomment it, you'll get the response from the server on one single line, something like
{"took":50,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":7056,"max_score":1.0,"hits":[...]}}
If you are trying to follow this approach educationally, this can be ok. Remember, nonetheless, that there are a number of libraries that can be used exactly for this use. You should check out the clients page.

Related

Rest template exchange not accept ParameterizedTypeReference

Oops guys! Beauty? I'm trying to get a byte list[] in my Rest template's response, but my exchange isn't accepting the new ParameterizedTypeReference<List<byte[]>>() {} , could someone help me?
ResponseEntity<List<byte[]>> response = null;
try {
response = restTemplate.exchange(parametros.get("SERVICE_HUB2_BASE_URL") + "/fw/v1/pdf/kms/assinaturas",
HttpMethod.POST, entity, new ParameterizedTypeReference<List<byte[]>>() {});
} catch (HttpServerErrorException e) {
e.printStackTrace();
throw new ClientException(e.getStatusCode().value(), e.getStatusText());
} catch (HttpClientErrorException e) {
e.printStackTrace();
throw new ClientException(e.getStatusCode().value(), e.getStatusText());
} catch (Exception e) {
e.printStackTrace();
}
Hi As per the mouse hower it's pointing to different method.
Please check import statement for rest template.
both ways should be fine .
ResponseEntity<Collection<byte[]>> responseEntityOne = restTemplate.exchange(formattedUrl, HttpMethod.POST, entity,
new ParameterizedTypeReference<Collection<byte[]>>(){});
ResponseEntity<List<byte[]>> responseEntityOne1 = restTemplate.exchange(formattedUrl, HttpMethod.POST, entity,
new ParameterizedTypeReference<List<byte[]>>(){});

How to handle feign client connection timeout

I have below code to check this error but I am not getting timeout error its going to else condition
Response response = null;
try {
response = client.getResponse(URI.create(uri), headers, reuest);
} catch (Exception ex) {
if(ex instanceof SocketTimeoutException){
throw new ExternalClientException(Errors.TIMEOUT_ERROR);
} else {
throw new ExternalClientException(Errors.UNEXPECTED_ERROR);
}
}
You need to catch feign.RetryableException instead of SocketTimeoutException.
javadoc
Please send all code of class. You need to provide class of "client" variable so people can help.
One more thing, instead of checking instance of exception in "catch" clause, you should use multiple catching like this:
Response response = null;
try {
response = client.getResponse(URI.create(uri), headers, reuest);
} catch (SocketTimeoutException ex1) {
throw new ExternalClientException(Errors.TIMEOUT_ERROR, ex1);
} catch (Exception ex2) {
throw new ExternalClientException(Errors.UNEXPECTED_ERROR, ex2);
}

Nifi Processor gets triggered twice for single Input flow file

I am currently new on Apache Nifi and still exploring it.
I made a custom processor where I will fetch data from server with pagination.
I pass the input file which will contains the attribute "url".
Finally transfer the response in output flow file, as I fetch data with pagination, so I made a new output flow file for each page and transferred it to Successful relationship.
Below is the code part:
#Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
FlowFile incomingFlowFile = session.get();
String api = null;
if (incomingFlowFile == null) {
logger.info("empty input flow file");
session.commit();
return;
} else {
api=incomingFlowFile.getAttribute("url");
}
session.remove(incomingFlowFile);
if(api == null) {
logger.warn("API url is null");
session.commit();
return;
}
int page = Integer.parseInt(context.getProperty(PAGE).getValue());
while(page < 3) {
try {
String url = api + "&curpg=" + page;
logger.info("input url is: {}", url);
HttpResponse response = httpGetApiCall(url, 10000);
if(response == null || response.getEntity() == null) {
logger.warn("response null");
session.commit();
return;
}
String resp = EntityUtils.toString(response.getEntity());
InputStream is = new ByteArrayInputStream(StandardCharsets.UTF_16.encode(resp).array());
FlowFile outFlowFile = session.create();
outFlowFile = session.importFrom(is, outFlowFile);
session.transfer(outFlowFile, SUCCESSFUL);
} catch (IOException e) {
logger.warn("IOException :{}", e.getMessage());
return;
}
++page;
}
session.commit();
}
I am facing issue that for a single Input flow file, this processor get triggered twice and so it generates 4 flow files for a single input flow file.
I am not able to figure out this where I have done wrong.
Please help in this issue.
Thanks in advance.
======================================================================
processor group 1(Nifi_Parvin)
processor group 2 (News_Point_custom)

Android Asynctask return problems

I am facing a problem in value 'return' in Asynctask class in doInBackground method. I am getting an error, about 'missing return statement in below code.
`public class ForecastNetwork extends AsyncTask {
public final String TAG = ForecastNetwork.class.getSimpleName();
#Override
protected Void doInBackground(Void... params) {
HttpURLConnection urlConnection = null;
BufferedReader reader = null;
// Will contain the raw JSON response as a string.
String forecastJsonStr = null;
try {
// Construct the URL for the OpenWeatherMap query
// Possible parameters are avaiable at OWM's forecast API page, at
// http://openweathermap.org/API#forecast
URL url = new URL("http://api.openweathermap.org/data/2.5/forecast/daily?q=94043&mode=json&units=metric&cnt=7");
// Create the request to OpenWeatherMap, and open the connection
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.connect();
// Read the input stream into a String
InputStream inputStream = urlConnection.getInputStream();
StringBuffer buffer = new StringBuffer();
if (inputStream == null) {
// Nothing to do.
return null;
}
reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = reader.readLine()) != null) {
// Since it's JSON, adding a newline isn't necessary (it won't affect parsing)
// But it does make debugging a *lot* easier if you print out the completed
// buffer for debugging.
buffer.append(line + "\n");
}
if (buffer.length() == 0) {
// Stream was empty. No point in parsing.
return null;
}
forecastJsonStr = buffer.toString();
} catch (IOException e) {
Log.e(TAG, "Error ", e);
// If the code didn't successfully get the weather data, there's no point in attemping
// to parse it.
return null;
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
if (reader != null) {
try {
reader.close();
} catch (final IOException e) {
Log.e(TAG, "Error closing stream", e);
}
}
}
}`
What Should I return at the end?
I assume that you forgot to return the processing result
forecastJsonStr = buffer.toString();
return forecastJsonStr;

Any hints for https form get and post parse html project android?

I'm creating an Android app that should do the following;
Use a form on a https (SSL!) page to login and receive a cookie
Issue httpGET actions to get html
parse that html and show it in a view, list or something.
I've been fooling around with Jsoup, httpUnit and HTMLUnit for quite some time now, but I'm running in to several problems;
A. Login is fine, works.. (I get the website's welcome page) but then, when I issue a GET statement (and include the cookie), I am redirected to the login form. So the response html is not what I expected. (might have something to do with a keepalivestrategy?)
B. InputBuffers are too small to receive entire HTML pages and set them up for parsing.
NB : I do not have control over the webserver
I'm totally new at this, so a tutorial or code snippets would be helpful.
For instance, this is what I use to login to the website :
public int checkLogin() throws Exception {
ArrayList<NameValuePair> data = new ArrayList<NameValuePair>();
data.add(new BasicNameValuePair("userid", getUsername()));
data.add(new BasicNameValuePair("password", getPassword()));
data.add(new BasicNameValuePair("submit_login", "Logmein"));
Log.d(TAG, "Cookie name : " + getCookieName());
Log.d(TAG, "Cookie cont : " + getCookie());
HttpPost request = new HttpPost(BASE_URL);
request.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
request.getParams().setParameter("http.protocol.handle-redirects",false);
request.setEntity(new UrlEncodedFormEntity(data, "UTF-8"));
HttpResponse response;
httpsclient.getCookieStore().clear();
List<Cookie> cookies = httpsclient.getCookieStore().getCookies();
Log.d(TAG, "Number of Cookies pre-login : " + cookies.size());
response = httpsclient.execute(request);
cookies = httpsclient.getCookieStore().getCookies();
Log.d(TAG, "Number of Cookies post-login : " + cookies.size());
String html = "";
// Problem : buffer is too small!
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
str.append(line);
}
in.close();
html = str.toString();
Document doc = Jsoup.parse(html);
Log.v(TAG, "Ik heb nu dit : " + doc.toString());
if (cookies.size() > 0){
storeCookie(cookies.get(0).getName(), cookies.get(0).getValue());
return MensaMobileActivity.REQUEST_SUCCESS;
} else {
return MensaMobileActivity.REQUEST_ERROR;
}
}
You don't handle the SSL certificate at all, that's at least a part of the problem. I struggled starting to learn this recently as well. This block of code will grab the SSL cert from the webpage you're accessing.
try {
URL url = new URL(YOUR_WEBPAGE_HERE);
HttpsURLConnection connect = (HttpsURLConnection)url.openConnection();
connect.connect();
Certificate[] certs = connect.getServerCertificates();
if (certs.length > 0) {
cert = new File("YOUR_PATH_TO_THE_FILE");
//write the certificate obtained to the cert file.
OutputStream os = new FileOutputStream(cert);
os.write(certs[0].getEncoded());
return true;
}
}
catch (SSLPeerUnverifiedException e) {
e.printStackTrace();
}
catch (MalformedURLException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
catch (CertificateEncodingException e) {
e.printStackTrace();
}

Resources