Paginate in elasticsearch - elasticsearch

I am new to elasticsearch and at least for the flask extension (flask-elasticsearch) the documentation is very bad. Any help is much appreachiated.
I have a elasticsearch query which looks like this
res = es.search(index="user-index", from_=(page-1)*10, size=10, body={"query": {"match_all": {}}})
I query the user table and want only 10 results. Now I still have to get the actual table rows. So I query them like this
list_of_ids = []
for hit in res['hits']['hits']:
list_of_ids.append(hit["_id"])
search_result = models.User.query.filter(models.User.id.in_(list_of_ids)).paginate(page, 10, False)
as you can see I am using the paginate() function. However, since I passed a list of 10 ids into the query, the paginate function does not know about the total number of results in the search query. So the paginate function can't work this way...
I could just do all the paginate functionality by myself, but I was wondering whether there is a nice way to keep the paginate functionality and somehow link it with the elastic search?
thanks
carl

I found a solution... its possible to just assign the relevant pagination values like
search_result.total = res['hits']['total']
search_result.page = page
cheers

Related

How to get list of ids from pagination query of the current page?

I have 1 pagination query for every:
$pagination_query = Table1::...->paginate(100);
then I have a second query that needs to use the list of ids from the pagination query:
$second_query = Table2::whereIn('id', $list_of_ids_from_pagination_query);
How can I get the list of ids only from the paginated page so that I get 100 ids and not thousands of ids which will end up in too many bound variables error?
Ty!
You can use something like this .
$ids = Table1::...->paginate(100)->pluck('id');
$second_query = Table2::wherein('id', $ids);

How do I negate a query in Parse's API (Back4App)? Specifically, how do I get everything not in a relation?

Does anyone know if there's an easy way to negate a parse query? Something like this:
Parse.Query.not(query)
More specifically I want to do a relational query that gets everything except for the objects within the query. For example:
const relation = myParseObject.relation("myRelation");
const query = relation.query();
const negatedQuery = Parse.Query.not(query);
return await negatedQuery.find();
I know one solution would be to fetch the objects in the relation and then create a new query by looping through the objectIds using query.notEqualTo("objectId", fetchedObjectIds[i]), but this seems really circuitous...
Any help would be much appreciated!
doesNotMatchKeyInQuery is the solution as Davi Macedo pointed out in the comments.
For example, if I wanted to get all of the Comments that are not in an Article's relation, I would do the following:
const relationQuery = article.relation("comments").query();
const notInRelationQuery = new Parse.Query("Comment");
notInRelationQuery.doesNotMatchKeyInQuery("objectId", "objectId", relationQuery);
const notRelatedComments = await notInRelationQuery.find();
How I understand it is that the first argument is specifying the key in the objects that we are fetching. The second argument is specifying the key in the objects that are in the query that we're about to argue. And lastly we argue a query for the objects we don't want. So, it essentially finds the objects you don't want and then compares the values of the objects you do want to the values of the objects you don't want for the argued keys. It then returns all the objects you do want. I could probably write that more succinctly, but w/e.

How can we fetch column values which are between two limits in MongoTemplate?

for example i want to find age between 16 and 25 from a collection in mongoDB.
my query is..
Query query = new Query(Criteria.where("visibility").is(1)
.and("type").is("guide").and("age").gte(16).and("age").lte(25));
but it is giving exception. reason is mongo template do not support lte() and gte() with same column. so how can i handle it ? is their any solution ?
Try not to include an extra and("age") part in your criteria. What you need is this:
Query query = new Query(Criteria.where("visibility").is(1)
.and("type").is("guide").and("age").gte(16).lte(25));

Spring data elastic search - Query - Full text search

I am trying to use elastic search for full text search and Spring data for integrating elastic search with my application.
For example,
There are 6 fields to be indexed.
1)firstName
2)lastName
3)title
4)location
5)industry
6)email
http://localhost:9200/test/_mapping/
I can see these fields in the mapping.
Now, I would like to make a search against these fields with a search input.
For example, When I search "mike 123", it has to search against all these 6 fields.
In Spring data repository,
The below method works to search only in firstName.
Collection<Object> findByFirstNameLike(String searchInput)
But, I would like to search against all the fields.
I tried,
Collection<Object> findByFirstNameLikeOrLastNameLikeOrTitleLikeOrLocationLikeOrIndustryLikeOrEmailLike(String searchInput,String searchInput1,String searchInput2,String searchInput3,)
Here, even the input string is same, i need to pass the same input as 6 params. Also the method name looks bigger with multiple fields.
Is there anyway to make it simple with #Query or ....
Like,
Collection<Object> findByInput(String inputString)
Also, boosting should be given for one of the field.
For example,
When i search for "mike mat", if there is any match in the firstName, that should be the first one in the result even there are exact match in the other fields.
Thanks
Lets suppose your search term is in the variable query, you can use the method search in ElasticsearchRepository.
repo.search(queryStringQuery(query))
to use queryStringQuery use the following import
import static org.elasticsearch.index.query.QueryBuilders.queryStringQuery;
I found the way to achieve this and posting here. Hope, this would help.
QueryBuilder queryBuilder = boolQuery().should(
queryString("Mike Mat").analyzeWildcard(true)
.field("firstName", 2.0f).field("lastName").field("title")
.field("location").field("industry").field("email"));
Thanks
Not a spring-data elasticsearch expert. But I see two directions you can go. The first would be to use the #Query option. That way you can create your own query. The second would be to use the example in the Filter builder section:
http://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.misc.filter
Within elasticearch you would want to use the multi_match query:
http://www.elastic.co/guide/en/elasticsearch/reference/1.5/query-dsl-multi-match-query.html
In java such a query would look like this:
QueryBuilder qb = multiMatchQuery(
"kimchy elasticsearch",
"user", "message"
);
Example coming from: http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/query-dsl-queries.html#multimatch
We can write our own custom query as below.
we can specific index, routing value (this is used if alias is used)
SearchQuery searchQuery = new NativeSearchQueryBuilder().withIndices(INDEX)
.withRoute(yourQueryBuilderHelper.getRouteValue())
.withQuery(yourQueryBuilderHelper.buildQuery(yourSearchFilterRequestObject))
.withFilter(yourQueryBuilderHelper.buildFilter(yourSearchFilterRequestObject)).withTypes(TYPE)
.withSort(yourQueryBuilderHelper.buildSortCriteria(yourSearchFilterRequestObject))
.withPageable(yourQueryBuilderHelper.buildPaginationCriteria(yourSearchFilterRequestObject)).build();
FacetedPage<Ticket> searchResults = elasticsearchTemplate.queryForPage(searchQuery, YourDocumentEntity.class);
Its good to use your own queryBuilder helper which can seperate your elasticSearchService from queryBuilder responsibility.
Hope this helps
Thanks
QueryBuilder class is helpful to query from spring Dao to elastic search:
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.QueryBuilder;
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.termQuery("state", "KA"));
.must(QueryBuilders.termQuery("content", "test4"))
.mustNot(QueryBuilders.termQuery("content", "test2"))
.should(termQuery("content", "test3"));
.should(termQuery("content", "test3"));
Try like this, you can even set importance of the field
QueryBuilder queryBuilder = QueryBuilders.multiMatchQuery(query)
.field("name", 2.0f)
.field("email")
.field("title")
.field("jobDescription", 3.0f)
.type(MultiMatchQueryBuilder.Type.PHRASE_PREFIX);
Another way is using Query String query
Query searchQuery = new StringQuery(
"{\"query\":{\"query_string\":{\"query\":\""+ your-query-here + "\"}}}\"");
SearchHits<Product> products = elasticsearchOperations.search(
searchQuery,
Product.class,
IndexCoordinates.of(PRODUCT_INDEX_NAME));
This will search all the field from your document of specified index

RethinkDB filter and retrieve value from nested array

Using the following query:
r.db('somedb').table('sometable')('users')
I get the following data from the result:
[
   [
      {
         "fn": "dpw",
         "u": "usertwo"
      },
      {
         "fn": "dwd",
         "u": "userone"
      }
   ]
]
I would like to take the field "u" and specify lets say "usertwo" and get the value of "fn" for that "u". I want to have the result filtered using ReQL so that I am not just parsing the json result in nodejs as the result will be enormous eventually. What would be the best and most efficient approach. I am new to RethinkDB and would appreciate if you could explain the answer as best you can.
I'm not sure of what you exactly want, but from my understanding, this is what you are looking for:
r.db('somedb').table('sometable')('users').filter(function(user) {
return user("u").eq("usertwo")
})("fn")
You seem to have an array of array of users. if that was not a typo, the query should probably be
r.db('somedb').table('sometable')('users').nth(0).filter(function(user) {
return user("u").eq("usertwo")
})("fn")

Resources