I am getting the below Warning messages for every query that is sent from the Spring Boot API and would like to remove it from the Logs.
2022-08-17 12:41:31.123 WARN 61390 --- [nio-9002-exec-2] org.elasticsearch.client.RestClient : request [POST http://localhost:9200/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true] returned 1 warnings: [299 Elasticsearch-7.14.1-66b55ebfa59c92c15 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.14/security-minimal-setup.html to enable security."]
I am using the RestHighLevelClient. According to Elasticsearch, this can be solved by migrating to The Elasticsearch Java API Client that they recently introduced. However, this would take a long time for us make that kind of change.
The Elasticsearch version is 7.14.1 and unfortunately, we can not upgrade this.
I was wondering if there is a simpler solution to this problem.
This warning are coming because you have not enable security for elasticsearch cluster and anyone can access your elasticsearch cluster using URL. You can enable basic authentication to your elasticsearch cluster to remove this warning.
Please check this documentation for how to enable security for your elasticsearch cluster.
Also, once you enable security, you can use below code to pass authentication details to Java High Level client for connecting to secure elasticsearch cluster.
RestClientBuilder builder = RestClient.builder(new HttpHost(hostname, port));
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(elasticUserName, password));
builder.setHttpClientConfigCallback(httpClientBuilder -> {
httpClientBuilder.disableAuthCaching();
return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
});
RestHighLevelClient restHighLevelClient = new RestHighLevelClient(builder);
I solved the issue simply by adding the options to the .properties for the Logback as below;
logging.level.root=ERROR
logging.level.org.springframework.web=DEBUG
logging.level.org.hibernate=ERROR
Thanks for the top answer on this post.
Related
io.micrometer contains disk space metrics (io.micrometer.core.instrument.binder.jvm.DiskSpaceMetrics) but it doesn't seems to be enabled by default. There are no metric data. How do i enable this metric that it can be used by prometheus?
Metrics about disk space are exposed as part of the health endpoint, which is provided by Spring Boot Actuator (dependency: org.springframework.boot:spring-boot-starter-actuator).
The health endpoint can be enabled as follows in the application.properties file (by default, it should be enabled):
management.endpoints.web.exposure.include=health
Then, you can enable detailed disk space information as follows:
management.endpoint.health.show-components=always
management.endpoint.health.show-details=always
management.health.diskspace.enabled=true
In production, you might want to use when_authorized instead of always, so that the information is not publicly available.
Finally, you can see the disk info through the HTTP endpoint /actuator/health.
More info in the official docs.
The same metrics for Prometheus will be added in a future Spring Boot version. There's an open PR to add auto configuration for that. In the meantime, you can configure a bean yourself taking inspiration from the PR.
#Bean
public DiskSpaceMetrics diskSpaceMetrics() {
return new DiskSpaceMetrics(new File("."));
}
My spring-boot (version 2.4.1) application was successfully connected to an ElasticSearch(v7.9.3) instance using the autowired org.elasticsearch.client.RestHighLevelClient (I just had to specify the application properties and it worked).
In a new phase of the project a dependency with spring-boot-starter-webflux was added to use some reactive logic to call an external webservice. (which has nothing to do with my elasticsearch connection)
But now suddenly the elasticsearch client also tries to connect using reactor and I got errors like:
reactor.core.Exceptions$ErrorCallbackNotImplemented: org.springframework.data.elasticsearch.client.NoReachableHostException:
Host 'https://elastic-dev.company.intra:9200:9200' not reachable. Cluster state is offline.
Caused by: org.springframework.data.elasticsearch.client.NoReachableHostException:
Host 'https://elastic-dev.company.intra:9200:9200' not reachable. Cluster state is offline.
at org.springframework.data.elasticsearch.client.reactive.SingleNodeHostProvider.lambda$lookupActiveHost$4(SingleNodeHostProvider.java:108) ~[spring-data-elasticsearch-4.1.2.jar!/:4.1.2]
I know there is a configuration issue with :9200:9200 but I would like to just disable the use of reactor for my Elasticsearch client so it just uses the old way (I still need my Elasticseach client). Is this possible ?
Thanks.
After searching further I found a solution which was also suggested by P.J.Meish: disable the AutoConfiguration classes regarding the reactive elasticsearch:
I preferred the config in application.properties:
spring.autoconfigure.exclude=\
org.springframework.boot.actuate.autoconfigure.elasticsearch.ElasticSearchReactiveHealthContributorAutoConfiguration,\
org.springframework.boot.autoconfigure.data.elasticsearch.ReactiveElasticsearchRepositoriesAutoConfiguration,\
org.springframework.boot.autoconfigure.data.elasticsearch.ReactiveElasticsearchRestClientAutoConfiguration
I am in a dilemma over to use spring's rest template or elasticsearch's own high/low rest client while searching in es . Does es client provide any advantage like HTTP connection pooling , performance while compared to spring rest template . Which of the two take less time in getting response from the server . Can some one please explain this ?
The biggest advantage of using Spring Data Elasticsearch is that you don't have to bother about the things like converting your requests/request bodies/responses from your POJO domain classes to and from the JSON needed by Elasticsearch. You just use the methods defined in the ElasticsearchOperations class which is implemented by the *Template classes.
Or going one abstraction layer up, use the Repository interfaces the all the Spring Data modules provide to store and search/retrieve your data.
Firstly, This is a very broad question. Not sure if it suits the SO guidelines.
But my two cents:
High Level Client uses Low Level client which does provide connection pooling
High Level client manages the marshalling and unmarshalling of the Elastisearch query body and response, so it might be easier to work using the APIs.
On the other hand, if you are familiar with the Elasticsearch querying by providing the JSON body then you might find it a bit difficult to translate between the JSON body and the Java classes used for creating the query (i.e when you are using Kibana console or other REST API tools)
I generally overcome this by logging the query generated by the Java API so that I can use it with Kibana console or other REST API tools.
Regarding which one is efficient- the library will not matter that much to affect the response times.
If you want to use Spring Reactive features and make use of WebClient, ES Libraries do provide support for Async search.
Update:
Please check the answer by Wim Van den Brande below. He has mentioned a very valid point of using Transport Client which has been deprecated over REST API.
So it would be interesting to see how RestTemplate or Spring Data ElasticSearch will update their API to replace TransportClient.
One important remark and caveat with regards to the usage of Spring Data Elasticsearch. Currently, Spring Data Elasticsearch doesn't support the communication by the High Level REST Client API. They are using the transport client. Please note, the TransportClient is deprecated as of Elasticsearch 7.0.0 and is expected to be removed in Elasticsearch 8.0!!!
FYI, this statement has been confirmed already by another post: Elasticsearch Rest Client with Spring Data Elasticsearch
Goodmorning everyone. I am trying to Flink connector Elasticsearch 5.2.1 and I have problems with the authentication XPACK
Using a different transport clients is currently (March 2017, Flink 1.2) not supported in Flink.
However, I've filed a JIRA to add the feature: FLINK-6065
Make TransportClient for ES5 pluggable
Until this has been implemented into Flink, I recommend overriding the ElasticsearchSink and using a different call bridge calling the PreBuiltXPackTransportClient.
I am using the camel elasticsearch component : http://camel.apache.org/elasticsearch.html
My assumption, based on the docs, is that the elasticsearch server must be on the same network as the running camel route in order to work. Is this correct?
To clarify, the only connection property available is 'clustername'. I assume this is discovered by searching the network via multicast for the cluster.
My code needs to connect to a remote service. Is this just not possible?
I am fairly new to elasticsearch in general.
I had a similar problem with the autodiscovery of elasticsearch. I had a camel route that tried to index some exchanges, but the cluster was located in another subnet and thus not discoverd.
With the java api of ES it is possible to connect to a remote cluster with a TransportClient specifying an IP adress. I don't have acces to the code at the moment but the Java API in the ES documentation provides clean example code. You could make such a connection from within a bean in the route for example.
I also submitted a patch to Camel to add an ip parameter to the route, which should then connect to the remote cluster with such a TransportClient. The documentation states that should be available with Camel 2.12
Hope this helps.