I have a scenario where i need to extend search behavior and add some additional filter logic based on the document id . If the document can be visible to the user searching and then show in search result but have not found any.
Currently as part of search , search query is executed after applying all the filters. After the search results are fetched, we need to know if the resource can be shown to this user. Pretty much like ACL.
Now, if i apply these authorization/ audience type filters myself after getting results from elastic search , it creates a lot of problems like the aggregations count changes post filter. Also, the pagination of results gets impacted.
Is a there a way to implement such rules as hook provided by elastic search. That is if i can implement some logic implementing some interface and then call some web service returning a boolean and then adding the search result to the final collection based on that.
Some insight would be very useful.
Related
I am new to Elastic Search. I would like to know if the following steps are how typically people use ES to build a search engine.
Use Elastic Search to get a list of qualified documents/results based on a user's input.
Build and use a search ranking model to sort this list.
Use this sorted list as the output of the search engine to the user.
I would probably add a few steps
Think about your information model.
What kinds of documents are you indexing?
What are the important fields and what field types are they?
What fields should be shown in the search result?
All this becomes part of your mapping
Index documents
Are the underlying data changing or can you index it just once?
How are you detecting new docuemtns/deletes/updates?
This will be included in your connetors, that can be set up in multiple ways, for example using the Documents API
A bit of trial and error to sort out your ranking model
Depending on your use case, the default ranking may be enough.
have a look at the Search API to try out different ranking.
Use the search result list to present the results to the end user
I am trying to create one query in the Kibana search bar to retrieve some specific documents.
The goal is to get the documents that either have the field "myDate" before 2019-10-08 or "myDate" does not exist.
I have documents that meet one or the other condition.
I started by creating this query :
myDate:<=2019-10-08 OR NOT _exists_:myDate
But no documents were returned.
Since it did not work, I tried some other ways i found online :
myDate:<=2019-10-08 OR NOT (_exists_:myDate)
myDate:<=2019-10-08 OR !(_exists_:myDate)
myDate:<=2019-10-08 OR NOT (myDate:*)
But still, no results.
When I use either "part" of the "OR" condition, it works perfectly : I get either the documents who have myDate<=2019-10-08 or the ones that do not have a "myDate" field filled.
But when I try with both conditions, I get no document.
I have to use only the search bar to find these documents, neither an elasticsearch rest query nor by using kibana filters.
Thank you for your help :)
Below query works. Use Inspect button in kibana to see what query is actually being fired and make sure you are using correct index pattern as well.
(myDate:<=2019-12-31) OR (NOT _exists_:myDate)
Take a look at Query DSL documentation for Boolean operators for more better understanding with different use cases
I am try to build a full text search engine using elasticsearch. We have a application which has conferences running across the globe. We have the future and past conferences data. For a POC we have already loaded the conferences details into elasticsearch and it contains fields like title,date,venue,geo_location of the venue as document.
I am able to do simple search using match all query. And also by using function_score I can get the current on going conferences and also using user geo location i can get nearby conferences to users location.
But there are some uses cases where i got stuck and could not proceed. Use cases are.
1) If user try to search with "title + location" then I should not use the user current geo location rather whatever user has provided the city_name use that place geo location and retrieve those doc. Here I know some programming is also required.
2) User search with "title + year", for ex. cardio 2014. User interested to see all the caridology conf of 2014 and it should retrieve that year documents only. But using function score it is retrieving the current years documents.
First of all let me know that above two use cases can be handled in single query. I am thinking to handle it one request, but got stuck.
A proper solution would require you to write your own query parser in your application (outside of elasticsearch) that will parse the query and extract dates, locations, etc. Once all features are extracted, the parser should generate a bool query where each feature would become an appropriate must clause. So, the date would became a range query, the location - geo_location query and everything else would go into a match query for full text matching. Then this query can be sent to elasticsearch.
We do understand the behavior of user by analyzing the tags he usually search for.
Now we need to give higher precedence for such tags for these users. I would like to know how we can achieve this using Elasticsearch in an elegant manner.
Well the best approach for this would be to
Analyse the behavior of the user
See which all keywords are of his interests
Maintain one document per user in another index which have all these keywords.
On the searches for that user , boost the occurrence of these keywords using function_score query
You can use terms filter inside boost function to achieve this.Add the boost function under functions in the function score query
In terms filter , you can point to this users document and get the values dynamically
Use custom filter key so that the cache key constructed wont eat too much memory
In this approach , you can avoid lots of code paths in client code.
I'm integrating elasticsearch into an asset tracking application. When I setup the mapping initially, I envisioned the 'brand' field being a single-term field like 'Hitachi', or 'Ford'. Instead, I'm finding that the brand field in the actual data contains multiple terms like: "MB 7 A/B", "B-7" or even "Brush Bull BB72X".
I have an autocomplete component setup now that I configured to do autocomplete against an edgeNGram field, and perform the actual search against an nGram field. It's completely useless the way I set it up because users expect the search results to be restricted to what the autocomplete matches.
Any suggestions on the best way to setup my mapping to support autocomplete and subsequent searches against a multiple term field like this? I'm considering a terms query against a keyword field, or possibly a match query with 'and' as the operator? I also have to deal with hyphens like "B-7".
you can use phrase suggest, the guide is here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters.html
the phrase suggest guide is here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-phrase.html