I have following code for ES search:
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", 9200, "http")));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
qb = QueryBuilders.termQuery("title", "Java");
searchSourceBuilder.query(qb);
searchSourceBuilder.from(0);
searchSourceBuilder.size(5);
SearchRequest searchRequest = new SearchRequest("myindex");
searchRequest.types("books");
searchRequest.source(searchSourceBuilder);
SearchResponse sr = client.search(searchRequest);
System.out.println(sr.getHits().totalHits);
It gives me no results, even though there are documents meeting this criteria.
When I run similar term query from Kibana, I get results:
GET /myindex/books/_search
{
"query":
{
"term" : {
"title" : {
"value" : "java"
}
}
}
}
The difference is Java (with uppercase) in your Java code and java (in lowercase) in your Kibana query.
Since you're doing a term query, the case matters. If you used a match query, the case doesn't matter and both would work as you expect.
Related
Hi Team I am using elasticsearch after a long time and facing some difficulties with multi_match queries.
Essentially my query need to have an exact match on 2 fields and should do a text search on 4 fields.
Here is the exact query that I have prepared which is giving expected results.
GET /maintenance_logs/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"vinNumber.keyword": "DH34ASD7SDFF84742"
}
},
{
"term": {
"organizationId": 1
}
}
],
"minimum_should_match": 1,
"should": [
{
"multi_match": {
"query": "Cabin pressure",
"fields": [
"dtcCode",
"subSystem",
"maintenanceActivity",
"description"
]
}
}
]
}
}
}
However when I am trying to convert this into Elasticsearch java, I am unable to get data back.
Here is the error:
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://localhost:9200], URI [/maintenance_logs/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: For input string: \"DH34ASD7SDFF84742\"","index_uuid":"VnZg-NkFQmS-nSHbYemZkQ","index":"maintenance_logs"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"maintenance_logs","node":"_XMtzvY5TW-02IijVrR2Ww","reason":{"type":"query_shard_exception","reason":"failed to create query: For input string: \"DH34ASD7SDFF84742\"","index_uuid":"VnZg-NkFQmS-nSHbYemZkQ","index":"maintenance_logs","caused_by":{"type":"number_format_exception","reason":"For input string: \"DH34ASD7SDFF84742\""}}}]},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:326) ~[elasticsearch-rest-client-7.12.1.jar:7.12.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296) ~[elasticsearch-rest-client-7.12.1.jar:7.12.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270) ~[elasticsearch-rest-client-7.12.1.jar:7.12.1]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1654) ~[elasticsearch-rest-high-level-client-7.12.1.jar:7.12.1]
Here is the Request:
public static SearchRequest buildSearchRequest(final String indexName,
final ElasticSearchDTO dto,
final MaintenanceLogsSearchDto maintenanceLogsSearchDto,
Pageable pageable) {
try {
final int page = pageable.getPageNumber();
final int size = pageable.getPageSize();
final int from = page <= 0 ? 0 : pageable.getPageSize();
SearchRequest searchRequest = new SearchRequest(indexName);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
final QueryBuilder vinQuery = QueryBuilders.termsQuery("vinNumber.keyword", maintenanceLogsSearchDto.getVinNumber());
final QueryBuilder orgIdQuery = QueryBuilders.termsQuery("organizationId", maintenanceLogsSearchDto.getVinNumber());
boolQueryBuilder.must(vinQuery);
boolQueryBuilder.must(orgIdQuery);
boolQueryBuilder.minimumShouldMatch(1);
MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder(dto.getSearchTerm(), "dtcCode", "subSystem", "maintenanceActivity", "description");
multiMatchQueryBuilder.operator(Operator.AND);
boolQueryBuilder.should(multiMatchQueryBuilder);
searchSourceBuilder.query(boolQueryBuilder);
searchSourceBuilder
.from(from)
.size(size)
.sort(SortBuilders.fieldSort("statsDate")
.order(SortOrder.DESC));
searchRequest.source(searchSourceBuilder);
return searchRequest;
} catch (final Exception e) {
e.printStackTrace();
return null;
}
}
I would also like to improve my search functionality with another api where I would like to run something like this :
"select * from maintenance_logs where subSystem in ["cabin pressure", "Engine, ABS"] or dtcCoe in ["p100", "p200", "p300"]"
Looks like issue is with your vinQuery as it uses this DH34ASD7SDFF84742 and it seems you are trying to assign the number to your vinNumber.keyword using your Java DTO maintenanceLogsSearchDto.getVinNumber(), and once you query it in Elasticsearch, Elasticsearch is not able to convert it to number as it can't be converted to number.
Below log line in your logs give hint about the issue.
reason":"failed to create query: For input string:
"DH34ASD7SDFF84742"","index_uuid":"VnZg-NkFQmS-nSHbYemZkQ","index":"maintenance_logs","caused_by":{"type":"number_format_exception","reason":"For
input string: "DH34ASD7SDFF84742""}}}]},"status":400}
I am using 7.10. version of elasticsearch. I created an index and did settings-mappings. Then I sent query to index by using http requests. I got the results I need, but I want to do same thing with Java API. However, I couldn't.
Can you help me to send request and get the result as list in java from scratch ?
And here it is my query that I used for obtain suggestions:
{
"suggest": {
"text": "some title I want to search",
"phrase_suggester": {
"phrase": {
"field": "title.shingle",
"max_errors": 2,
"size": 5,
"confidence": 0.0,
"direct_generator": [{
"field": "title.shingle",
"max_edits": 2
}
]
}
}
}
}
How can I write this query with Elasticsearch Java API. Can you help me figure this out ?
This would be the way to build the request:
client.search(searchRequestBuilder -> searchRequestBuilder
.suggest(suggestBuilder -> suggestBuilder
.text("some title I want to search")
.suggesters("phrase_suggester", fieldSuggesterBuilder -> fieldSuggesterBuilder
.phrase(phraseBuilder -> phraseBuilder.field("title.shingle")
.maxErrors(2d)
.size(5)
.confidence(0.0)
.directGenerator(directGeneratorBuilder -> directGeneratorBuilder
.field("title.shingle")
.maxEdits(2))))),
YourEntity.class);
Btw, the new client was in 7.16, you wrote 7.14?
Finaly I've found my own answers. It was so hard to find the solution due to lack of documents about these kind of specific topics. I'm sharing my solution for those who wondered:
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
PhraseSuggestionBuilder builder = SuggestBuilders.phraseSuggestion("title.shingle")
.addCandidateGenerator(new DirectCandidateGeneratorBuilder("title.shingle")
.suggestMode("always"))
.text(query)
.maxErrors(2f)
.confidence(0f);
SuggestBuilder suggestBuilder = new SuggestBuilder().addSuggestion("suggestion", builder);
searchSourceBuilder.suggest(suggestBuilder);
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("index_name");
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
I am using Elasticsearch 6.3 with Jest client 6.3 (Java API)
Search search = new Search.Builder(jsonQueryString)
.addIndex("SOME_INDEX")
.build();
SearchResult result = jestClient.execute(search);
And this is my sample JSON query
{
"query": {
"bool" : {
"filter": {
"match" :{
"someField" : "some value"
}
}
}
}
}
The JSON query string is accepted as a POST request body and then passed to the Jest client. Before I can execute the json query on the Jest client, I need to add conditions to the query for e.g.
{
"query": {
"bool" : {
"filter": {
"match" :{
"someField" : "some value"
}
}
},
"must": {
"match" :{
"systemField" : "pre-defined value"
}
}
}
}
}
Is there an API that allows to parse the JSON query and add conditions to it before it can be executed on Jest client? The JSON query can be any query supported by Query DSL and not necessarily contain bool condition. I need to add a pre-defined condition to the query. I appreciate any help on this. Thanks very much.
There is no out of the box Elasticsearch or Jest API to achieve the above, the workaround I implemented is using Jackson ObjectMapper
// convert the search request body into object node
ObjectNode searchRequestNode = objectMapper.readValue(queryString, ObjectNode.class);
// extract the query
String query = searchRequestNode.get("query").toString();
// wrap the original query and add conditions
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(QueryBuilders.wrapperQuery(query));
boolQueryBuilder.filter(QueryBuilders.termsQuery("fieldA", listOfValues));
boolQueryBuilder.filter(QueryBuilders.termQuery("fieldB", value));
// convert querybuilder to json query string
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(queryBuilder);
String queryWithFilters = searchSourceBuilder.toString();
// convert json string to object node
ObjectNode queryNode = objectMapper.readValue(queryWithFilters, ObjectNode.class);
// replace original query with the new query containing added conditions
searchRequestNode.set("query", queryNode.get("query"));
String finalSearchRequestWithOwnFilters = searchRequestNode.toString();
For debugging purposes, I need to know what query spring-data-elasticsearch is sending to the ElasticSearch cluster. I have tried to call the toString method on the SearchQuery object, and doesn't return what I need.
What I am doing in Java (using spring-data-elasticsearch) is:
private FilterBuilder getFilterBuilder(String id) {
return orFilter(
termFilter("yaddayaddayadda.id", id),
termFilter("blahblahblah.id", id)
);
}
SearchQuery sq = NativeSearchQueryBuilder()
.withQuery(new MatchAllQuery())
.withFilter(fb)
.build();
And I expect to return something like this plain query executed in ES cluster REST API is returning:
{
"query": {
"filtered": {
"filter": {
"or": [
{
"term": {
"yaddayaddayadda.id": "9"
}
},
{
"term": {
"blahblahblah.id": "9"
}
}
]
}
}
}
}
Thanks in advance!
One way to achieve this is to log the queries on the ES/server-side into the slowlog file. Open your elasticsearch.yml config file and towards the bottom uncomment/edit the two lines below:
...
index.search.slowlog.threshold.query.info: 1ms
...
index.search.slowlog.threshold.fetch.info: 1ms
...
The advantage of this solution is that whatever client technology you're using to query your ES server (Spring Data, Ruby, Browser, Javascript, etc), you'll be able to dump and debug your queries in a single location.
SearchQuery Interface has a method getQuery() and getFilter() to get the information you need.
System.out.println(searchQuery.getQuery());
System.out.println(searchQuery.getFilter());
Hope this helps.
When using SearchRequest or SearchSourceBuilder, calling .toString() method on their instance will get you actual JSON query:
SearchRequest searchRequest = new SearchRequest("index");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
// building the query
// ...
searchSourceBuilder.query(query);
searchRequest.source(searchSourceBuilder);
System.out.println(searchSourceBuilder.toString()); // prints json query
System.out.println(searchRequest.toString()); // prints json query + other information
My data are stored in elastic search as shown below
{
"identifier":{
"source":"source 1",
"id":"22081070"
},
"title":"Book 1",
"published":2011,
"types":[
"type1",
"type2,
"type3"
]
}
Is there a way to retrieve a document with specific "identifier.id" and "identifier.source" parameters? For example I am retrieving the above document with its id as an input with the following:
QueryBuilder queryBuilder = QueryBuilders.matchQuery("identifier.id", "22081070");
SearchResponse searchResponse = client.prepareSearch("test-index")
.setTypes("type").setQuery(queryBuilder).execute().actionGet();
but I know know how to add the "identifier.source" as a match parameter.
Try this:
BoolQueryBuilder boolQuery = new BoolQueryBuilder();
QueryBuilder queryBuilder1 = QueryBuilders.matchQuery("identifier.id", "22081070");
QueryBuilder queryBuilder2 = QueryBuilders.matchQuery("identifier.source", "source 1");
boolQuery.must(queryBuilder1).must(queryBuilder2);
SearchResponse searchResponse = client.prepareSearch("test-index")
.setTypes("type").setQuery(boolQuery).execute().actionGet();