final QueryBuilder contentTagQuery = QueryBuilders.boolQuery()
.filter( QueryBuilders.termQuery("tenantId" , "en"));
SearchHits<Content> searchHits =
elasticsearchOperations.search(
new NativeSearchQuery(contentTagQuery), Content.class,
IndexCoordinates.of("index"));
How can I add size in this query, as I only need the first result.
You can set this on the Query instance:
NativeSearchQuery query = new NativeSearchQuery(contentTagQuery);
query.setMaxResults(1);
Related
I would like to retrieve all the documents from Elasticsearch, so I referred to the Search Scroll API.
But my question is, it is not returning all the documents, I have 36 documents in one index, for that it was returning 26 only.
Even when I checked with another index, where I have more than 10k documents, there it is also not returning the last 10 documents.
I really don't know why it was returning it like that! Any help will be appreciated! Thanks in advance!
Below the code I've tried:
final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
SearchRequest searchRequest = new SearchRequest("myindex");
searchRequest.scroll(scroll);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query("")//here some query;
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
String scrollId = searchResponse.getScrollId();
SearchHit[] searchHits = searchResponse.getHits().getHits();
while (searchHits != null && searchHits.length > 0) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
scrollRequest.scroll(scroll);
searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
scrollId = searchResponse.getScrollId();
searchHits = searchResponse.getHits().getHits();
for (SearchHits hit: searchHits){
String source=hit.getSourceAsString();
}
}
ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
clearScrollRequest.addScrollId(scrollId);
ClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
boolean succeeded = clearScrollResponse.isSucceeded();
Today I faced with the same problem while working with an example from:
Elastic Scroll API
First of all, about documents you missed - 10 is default value for the size of requests and based on this we can suppose that one of your requests wasn't handled properly.
In your code first batch of 10 documents isn't handled:
SearchHit[] searchHits = searchResponse.getHits().getHits();
Before while loop you should iterate over your searchHits .
From the first time it was not clear to me in the official documents.
You should change your while loop logic to execute the hit iteration first and the scroll after.
while (searchHits != null && searchHits.length > 0) {
// execute this block first otherwise the scroll will overwrite the initial hits.
for (SearchHits hit: searchHits){
String source=hit.getSourceAsString();
}
SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
scrollRequest.scroll(scroll);
searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
scrollId = searchResponse.getScrollId();
searchHits = searchResponse.getHits().getHits();
}
Another thing to consider is that you can increase the response hit size. from the docs:
The index.max_result_window which defaults to 10,000 is a safeguard, search requests take heap memory and time proportional to from + size.
So the defaulted value for max_result_window is 10k hits, you can also set this value to be something else. this means you can fetch up to 10k hits in 1 search call instead of executing redundant paginations.
You can do this by specifying the size property for searchSourceBuilder before executing the search call like so:
searchSourceBuilder.size(10000);
I want to use a pageable primfaces datatable to display all entries in an elastic seach index.
Unfortunately the following code only returns 10 entries.
public ArrayList<Video> getAllVideos(String indexName) throws IOException{
makeConnection();
SearchRequest searchRequest = new SearchRequest(indexName);
Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
searchRequest.scroll(scroll);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.matchAllQuery());
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
return buildResponse(searchResponse, restHighLevelClient);
}
Does anybody have a working example for getting all records? Thanks
I'm having issue migrating from transport client to Rest high-level client.
The following code will not work with RestHighLevelClient which I want to use to get a response of aggregated pages of type Class.
elasticsearchTemplate.queryForPage(searchQuery, Class.class)
Any suggestions to achieve the same with other method is also welcomed.
My workaround using restHighLevelClient without Spring data elasticsearch consist in this code (this is not the solution but perhaps could be help for your solution):
BoolQueryBuilder criteriaQuerySpecification = getCriteriaQuerySpecification(transactionFilter);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.sort(new FieldSortBuilder("operation_created_at").order(SortOrder.DESC));
sourceBuilder.query(criteriaQuerySpecification);
SearchRequest searchRequest = generateSearchRequest(totalElementsInt, pageNumberInt, sourceBuilder);
SearchResponse search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
List<OperationDto > operations = Arrays.asList(hits).stream().map(hit -> {
hit.getSourceAsString();
// get operation is a method mapping from hit to your dto using Map<String, Object> sourceAsMap = hit.getSourceAsMap();
OperationDto operation = getOperationDto(hit);
//convert hit to OperationDto
return operation;
}).collect(Collectors.toList());
private SearchRequest generateSearchRequest(Integer totalElementsInt, Integer pageNumberInt, SearchSourceBuilder sourceBuilder) {
SearchRequest searchRequest = new SearchRequest("operation-index").types("operation");
int offset = pageNumberInt *totalElementsInt;
sourceBuilder.from(offset);
sourceBuilder.size(totalElementsInt);
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
searchRequest.source(sourceBuilder);
return searchRequest;
}
This worked for me
public Page<T> search(){
Query query;
SearchHits<T> t;
Criteria nameCriteria = new Criteria("name").is(text).and(new
Criteria("jsonNode").in(String ...)); //This can be any Aggregator
query = new CriteriaQuery(nameCriteria).setPageable(paging);
searchHits = elasticsearchOperations.search(query, T.class);
return (Page) SearchHitSupport.searchPageFor(searchHits, query.getPageable());;
}
matchAllQuery() in Elasticsearch gets me only 10 results how do I increase its output so that I can get as many results as per my requirement.
Code
QueryBuilder query = QueryBuilders.matchAllQuery();
By default 10 results are returned, you need to increase the size parameter:
SearchRequestBuilder request = client.prepareSearch(index)
.setQuery(QueryBuilders.matchAllQuery())
.setSize(100);
Yes u can do , here you can pass aPageRequestcount whatever you want and If you want no of records exist in Elastic search than repository.count() will work for that :-
int aPageRequestcount = (int) repository.count();
NativeSearchQueryBuilder aNativeSearchQueryBuilder = new NativeSearchQueryBuilder();
aNativeSearchQueryBuilder.withIndices(indexName).withTypes(type).withPageable(new PageRequest(0, aPageRequestcount));
final BoolQueryBuilder aQuery = new BoolQueryBuilder();
NativeSearchQuery nativeSearchQuery = aNativeSearchQueryBuilder.withQuery(aQuery).build();
= elasticsearchTemplate.queryForList(nativeSearchQuery, A.class);
Say we can match ("name"="A" AND "name!="B") as,
BoolQueryBuilder queryBuilder1 = QueryBuilders.boolQuery()
.must(QueryBuilders.matchQuery("name","A"))
.mustNot(QueryBuilders.matchQuery("name","B"));
How can I imply a "shouldNot" operation for ("name"="A" OR "name"!="B")
like,
BoolQueryBuilder queryBuilder1 = QueryBuilders.boolQuery()
.should(QueryBuilders.matchQuery("name","A"))
.sholdNot(QueryBuilders.matchQuery("name","B")); // This API not exists..!
You can use this:
BoolQueryBuilder queryBuilder1 = QueryBuilders.boolQuery()
.should(QueryBuilders.matchQuery("name","A"))
.should(QueryBuilders.boolQuery()
.mustNot(QueryBuilders.matchQuery("name","B")))