Connect to Elastic Cloud from the Java Client - elasticsearch

I've been trying to connect to my ES instance on the Elastic cloud with no success using the Java client and following the Documentation
In the hostname I put https://myinstance-xx.europe-west1.gcp.cloud.es.io
private String hostName = "https://myinstance-xx.es.europe-west1.gcp.cloud.es.io";
private String username = "username";
private String password = "pass";
#Bean
public ElasticsearchClient client() {
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(username, password));
RestClientBuilder builder = RestClient.builder(new HttpHost(hostName, 9200))
.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
// Create the low-level client
RestClient restClient = builder.build();
// Create the transport with a Jackson mapper
ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
// And create the API client
return new ElasticsearchClient(transport);
}
However, I get an Connection Refused exception
java.net.ConnectException: Connection refused
What's the best way to connect to ES on Elastic Cloud through the Java Client?

Go to cloud.elastic.go/home after your are logged in, find the "Manage this deployment" link. It should be something like https://cloud.elastic.co/deployments/xxxxxxxx
Under "Applications" you should have an "Elasticsearch" enpoint (click on "Copy endpoint")
On the API console create an API Key:
POST /_security/api_key
{
"name": "my_key_name",
"expiration": "15d"
}
That's gonna give you something like:
{
"id": "API_KEY_ID",
"name": "my_key_name",
"expiration": 1670471984009,
"api_key": "API_KEY",
"encoded": "XxxxxxxxXXXXXxxxxxxXXXXxxxxxXXXX=="
}
Now in your Java code:
// endpoint you copied
String hostname = "youdepname.es.us-west1.gcp.cloud.es.io";
String apiKeyId = "API_KEY_ID";
String apiKeySecret = "API_KEY";
String apiKeyAuth =
Base64.getEncoder().encodeToString(
(apiKeyId + ":" + apiKeySecret)
.getBytes(StandardCharsets.UTF_8));
RestClientBuilder builder = RestClient.builder(
new HttpHost(hostname, 9243, "https"))
.setRequestConfigCallback(
new RestClientBuilder.RequestConfigCallback() {
#Override
public RequestConfig.Builder customizeRequestConfig(
RequestConfig.Builder requestConfigBuilder) {
return requestConfigBuilder
.setConnectTimeout(5000)
.setSocketTimeout(60000);
}
});
Header[] defaultHeaders =
new Header[]{new BasicHeader("Authorization",
"ApiKey " + apiKeyAuth)};
builder.setDefaultHeaders(defaultHeaders);
RestClient restClient = builder.build();
// Create the transport with a Jackson mapper
ElasticsearchTransport transport = new RestClientTransport(
restClient, new JacksonJsonpMapper());
// And create the API client
ElasticsearchClient client = new ElasticsearchClient(transport);
That should do it. There are other ways to connect, but this is my preferred way.

Related

How to connect elastic server(with protected) from java springboot?

I can able to connect with elastic server without security by using below code.But i can connect elastic server with user credentials.
#Bean
public RestClient getRestClient() {
RestClient restClient = RestClient.builder(
new HttpHost("localhost", 9200)).build();
return restClient;
}
#Bean
public ElasticsearchTransport getElasticsearchTransport() {
return new RestClientTransport(
getRestClient(), new JacksonJsonpMapper());
}
#Bean
public ElasticsearchClient getElasticsearchClient(){
ElasticsearchClient client = new ElasticsearchClient(getElasticsearchTransport());
return client;
}`your text`
But my Elasticyour textSearch is protected with username and password. Httphost doesn't have username and password parameters,Could you please help me with that....
Thanks for the Advance.
I need to connect elastic server from springboot(java) while my elastic server has username and password .
I need to connect elastic server from springboot ,and i need to create index from springboot.

Spring Boot Proxy via HttpClient doesn't work

I'm trying to setup a WebClient connection in Spring Boot using a proxy. My implementation looks like the following:
final WebClient.Builder webclientBuilder = WebClient.builder();
final HttpClient httpClient = HttpClient.create();
httpClient.proxy(proxy -> proxy
.type(Proxy.HTTP)
.host(proxyName)
.port(Integer.parseInt(proxyPort)));
final ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
webclientBuilder.clientConnector(connector);
final WebClient webClient = webclientBuilder
.baseUrl(baseUrl)
.build();
After running it and sending an API call, I receive a "Connection timed out: no further information". I should get back a Bad Request (in case my call is wrong), but I don't.
Is the implementation wrong?
the proxyName is written like this: "proxy.blabla.de"
After some trial and error and comparing I found a solution working for me:
String baseUrl = "https://mybaseurl";
String proxyName = "proxy.blabla.de";
int proxyPort = 1234;
public InitResponse addAccount() {
// for logging purposes, nothing to do with the proxy
LOGGER.info("LOGGER.info: addAccount()");
final InitRequest request = buildRequest();
HttpClient httpClient = HttpClient.create()
.proxy(proxy -> proxy.type(Proxy.HTTP)
.host(proxyName)
.port(proxyPort));
ReactorClientHttpConnector conn = new ReactorClientHttpConnector(httpClient);
WebClient webClient = WebClient.builder().clientConnector(conn).baseUrl(baseUrl).build();

HttpClient have both SSL and Proxy authentication configured?

I have two pieces of code using HttpClient,
First part in case that the end point requires SSL
Second is proxy connection with basic authentication
My question Is how can I make this code conditional so in cases i have SSL + Proxy or SSL only
I have hard time figuring out how to set the default credentials for example after I created the client using the client in the SSL part
.setDefaultCredentialsProvider(credsProvider)
This part is how I create the Client when I need SSL
CloseableHttpClient client = null;
if(conf.isUseSslConfig()) {
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new File(conf.getTrustStoreLocation()), conf.getTrustStorePassword().toCharArray(), new TrustSelfSignedStrategy()).build();
// Allow protocols
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslcontext,conf.getTlsVersions(), null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
client = HttpClients.custom().setSSLSocketFactory(sslsf).build();
}else {
client= HttpClients.createDefault();
}
And this part is how I create the Client when I need Proxy authentication:
if(conf.isUseProxyConfig()){
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope("fakeProxy.xerox.com", 80),
new UsernamePasswordCredentials("xeroxUser","fakePassword123"));
HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider).build();
}
So the bottom line is how to make the two sections work together so in case
Call with SSL + Proxy and authentication
Call with only SSL
Call with only Proxy and authentication
You can write code this way to get multiple conditions resolved :
CloseableHttpClient client = null;
if(conf.isUseSslConfig() && conf.isUseProxyConfig()) {
setSSLSetting(client);
setProxy()
}else if(conf.isUseSslConfig()) {
setSSLSetting(client);
}else {
client= HttpClients.createDefault();
}
private void setProxy(){
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(new AuthScope("fakeProxy.xerox.com", 80),new UsernamePasswordCredentials("xeroxUser","fakePassword123"));
}
private void setSSLSetting(CloseableHttpClient client){
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new File(conf.getTrustStoreLocation()), conf.getTrustStorePassword().toCharArray(), new TrustSelfSignedStrategy()).build();
// Allow protocols
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslcontext,conf.getTlsVersions(), null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
client = HttpClients.custom().setSSLSocketFactory(sslsf).build();
}
or you can create methods that return client with different settings and configs like this :
final Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create().register("http", new PlainConnectionSocketFactory()).register("https", sslsf).build();
final PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(registry);
private CloseableHttpClient createHttpClient(String headerName, String value) throws NoSuchAlgorithmException, KeyManagementException,KeyStoreException {
SSLContextBuilder builder = new SSLContextBuilder();
builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
Header header = new BasicHeader(headerName,value);
List<Header> headers = new ArrayList<>();
headers.add(header);
RequestConfig reqConfig = RequestConfig.custom().setConnectionRequestTimeout(long milli seconds).build();
CloseableHttpClient httpclient = HttpClients.custom().
setDefaultHeaders(headers).
setDefaultRequestConfig(reqConfig).
setConnectionManager(cm).
build();
return httpclient;
}

Special character with Apache and Spring

I use the below code where I set credentials for basic https authentication to my server that uses Spring Security. Unfortunately I have problem with special characters like é,ò etc... I receive on server the question mark ? instead of correct character (both username and password). Someone know how to resolve it?
private RestClient(String username, String password) {
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(null, -1),
new UsernamePasswordCredentials(username, password));
HttpClient httpClient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
setRequestFactory(new HttpComponentsClientHttpRequestFactory(httpClient));
}
public static synchronized RestClient getInstance(String username, String password){
if (instance == null){
instance = new RestClient(username, password);
instance.getMessageConverters().add(0, new StringHttpMessageConverter(Charset.forName("UTF-8")));
}
return instance;
}
and then I use for example
RestClient restClient = RestClient.getInstance(username, password);
if (queryParams!=null && queryParams.length!=0){
response = restClient.getForObject(addQueryParam(url, queryParams), Response.class);
}
I've tried even with URIEncoding="UTF-8" into server.xml of tomcat but the issue is still present.
UPDATE maybe I have fixed the issue setting the header manually:
private RestClient(String username, String password) {
String credential = Base64.getEncoder().encodeToString((username+":"+password).getBytes());
// create custom http headers for httpclient
List<BasicHeader> defaultHeaders = Arrays.asList(new BasicHeader(HttpHeaders.AUTHORIZATION, "Basic "+credential));
HttpClient httpClient = HttpClients.custom().setDefaultHeaders(defaultHeaders).build();
setRequestFactory(new HttpComponentsClientHttpRequestFactory(httpClient));
}
public static synchronized RestClient getInstance(String username, String password){
if (instance == null){
instance = new RestClient(username, password);
}
return instance;
}
I'm testing, let you know

Proxy configuration in OAuth2RestTemplate

I need to consume an API which is secured by OAuth2. For that I am using OAuth2RestTemplate.
But am getting below error:
java.net.ConnectException: Connection timed out: connect
This is happening due to proxy issue. I Know how to set proxy in RestTemplate :
SimpleClientHttpRequestFactory clientHttpRequestFactory = new SimpleClientHttpRequestFactory();
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("Proxy host", 8080));
clientHttpRequestFactory.setProxy(proxy);
RestTemplate restTemplate = new RestTemplate(clientHttpRequestFactory);
The same way I tried to set for OAuth2RestTemplate :
#Bean
public OAuth2RestOperations restTemplate(OAuth2ClientContext oauth2ClientContext) {
OAuth2RestTemplate client = new OAuth2RestTemplate(resource(), oauth2ClientContext);
SimpleClientHttpRequestFactory clientHttpRequestFactory = new SimpleClientHttpRequestFactory();
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(PROXY_HOST, PROXY_PORT));
clientHttpRequestFactory.setProxy(proxy);
client.setRequestFactory(clientHttpRequestFactory);
return client;
}
But it is not working and giving "Connection timed out" exception. This is happening because of this first line OAuth2RestTemplate client = new OAuth2RestTemplate(resource(), oauth2ClientContext); which tries to get Access token that means there also it needs proxy setting. if I add below lines then it works:
System.setProperty("https.proxyHost", "urproxy.com");
System.setProperty("https.proxyPort", "8080");
But I can not use System.setProperties("","") option as we do not have permission to set on tomcat server.
I researched but could not find any way to set proxy in OAuth2RestTemplate while creating this object.
Any help would be appreciated. Thanks
OAuth2RestTemplate just creates a set of AccessTokenProvider to retrieve the token from authorization server according to different kinds of grant types. For example AuthorizationCodeAccessTokenProvider is used to retrieve access token with grant type authorization_code. The token providers themselves initiate some RestTemplate to send the request but do not use OAuth2RestTemplate just created. One way might resolve the issue. That is to create you own AccessTokenProvider and set the request factory.
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
Proxy proxy= new Proxy(Type.HTTP, new InetSocketAddress(PROXY_HOST, PROXY_PORT));
requestFactory.setProxy(proxy);
AuthorizationCodeAccessTokenProvider authorizationCodeAccessTokenProvider = new AuthorizationCodeAccessTokenProvider();
authorizationCodeAccessTokenProvider.setRequestFactory(requestFactory);
ImplicitAccessTokenProvider implicitAccessTokenProvider = new ImplicitAccessTokenProvider();
implicitAccessTokenProvider.setRequestFactory(requestFactory);
AccessTokenProvider accessTokenProvider = new AccessTokenProviderChain(
Arrays.<AccessTokenProvider> asList(authorizationCodeAccessTokenProvider, implicitAccessTokenProvider));
OAuth2RestTemplate client = new OAuth2RestTemplate(github(), oauth2ClientContext);
client.setAccessTokenProvider(accessTokenProvider);
You could also add ResourceOwnerPasswordAccessTokenProvider and ClientCredentialsAccessTokenProvider to the OAuth2RestTemplate.
This RestTemplate provides a workaround:
/**
* An OAuth2RestTemplate with proxy support.
*
* #author E.K. de Lang
*/
public class ProxySupportingOAuth2RestTemplate
extends OAuth2RestTemplate
{
private static final Logger LOG = LogFactory.getLogger(ProxySupportingOAuth2RestTemplate.class);
private final SimpleClientHttpRequestFactory factory;
public ProxySupportingOAuth2RestTemplate(OAuth2ProtectedResourceDetails resource, OAuth2ClientContext context,
AccessTokenProvider accessTokenProvider)
{
super(resource, context);
factory = new SimpleClientHttpRequestFactory();
super.setRequestFactory(factory);
super.setAccessTokenProvider(accessTokenProvider);
// To fix issue: https://github.com/spring-projects/spring-security-oauth/issues/459 also set the factory of the token-provider.
if (accessTokenProvider instanceof OAuth2AccessTokenSupport) {
((OAuth2AccessTokenSupport) accessTokenProvider).setRequestFactory(factory);
}
else {
throw new UnsupportedOperationException("accessTokenProvider must extend OAuth2AccessTokenSupport");
}
}
public void setProxy(Proxy proxy)
{
if (LOG.isDebugEnabled()) {
LOG.debug("setProxy:" + proxy);
}
if (super.getRequestFactory() == factory) {
factory.setProxy(proxy);
}
else {
throw new UnsupportedOperationException("RequestFactory has changed.");
}
}
}

Resources