I am try to build a full text search engine using elasticsearch. We have a application which has conferences running across the globe. We have the future and past conferences data. For a POC we have already loaded the conferences details into elasticsearch and it contains fields like title,date,venue,geo_location of the venue as document.
I am able to do simple search using match all query. And also by using function_score I can get the current on going conferences and also using user geo location i can get nearby conferences to users location.
But there are some uses cases where i got stuck and could not proceed. Use cases are.
1) If user try to search with "title + location" then I should not use the user current geo location rather whatever user has provided the city_name use that place geo location and retrieve those doc. Here I know some programming is also required.
2) User search with "title + year", for ex. cardio 2014. User interested to see all the caridology conf of 2014 and it should retrieve that year documents only. But using function score it is retrieving the current years documents.
First of all let me know that above two use cases can be handled in single query. I am thinking to handle it one request, but got stuck.
A proper solution would require you to write your own query parser in your application (outside of elasticsearch) that will parse the query and extract dates, locations, etc. Once all features are extracted, the parser should generate a bool query where each feature would become an appropriate must clause. So, the date would became a range query, the location - geo_location query and everything else would go into a match query for full text matching. Then this query can be sent to elasticsearch.
Related
I want to make dependent search like when user type country and select country then on next dropdown/text search result would be from that particular Countries state, after selecting state on next text search would only based on that selected state. can anyone help to achieve this kind thing via elastic search.
i am new to elastic Search and i had basic idea of it, but didn't get idea how to do this kind of stuff where i need to search from child and feel data like map
Fist, it is important to understand how Elasticsearch store its data. You can find this kind of info here: https://www.elastic.co/guide/en/elasticsearch/reference/master/documents-indices.html
So, basically what you need is build a query with two must terms.
One for the object type (Country, State, etc).
Other for the name ("Los Angeles", "Massachussets", etc). If you want a autocomplete feature you could add a wildcard query in your list. https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html
Obs: When you store your State object do not forget to store the Country name together. Since Elasticsearch is non relational you have to have Country name indexed in the State document.
Hope that it helps
I am new to Elastic Search. I would like to know if the following steps are how typically people use ES to build a search engine.
Use Elastic Search to get a list of qualified documents/results based on a user's input.
Build and use a search ranking model to sort this list.
Use this sorted list as the output of the search engine to the user.
I would probably add a few steps
Think about your information model.
What kinds of documents are you indexing?
What are the important fields and what field types are they?
What fields should be shown in the search result?
All this becomes part of your mapping
Index documents
Are the underlying data changing or can you index it just once?
How are you detecting new docuemtns/deletes/updates?
This will be included in your connetors, that can be set up in multiple ways, for example using the Documents API
A bit of trial and error to sort out your ranking model
Depending on your use case, the default ranking may be enough.
have a look at the Search API to try out different ranking.
Use the search result list to present the results to the end user
I'm trying to refine the search results received by my application by including the sort parameter in my HTTP requests. I've combed through the documentation here, but I can't find exactly what I'm looking for.
I'm searching for DOC filetypes, and I am able to sort by date or sort by metadata, as in alphabetizing by title, author, etc. I can also filter by whether or not the title contains certain keywords. What I want to do is to sort by whether or not the title contains certain keywords (these documents appearing first in the results), but to still keep the other results.
For example, with keywords [winter, Christmas, holiday] I could do a descending sort by the sum of inmeta:title~winter, inmeta:title~Christmas, inmeta:title~holiday and the top result might be
Winter holidays other than Christmas
followed by documents with one or two of the keywords, followed by documents that meet the other search parameters but contain no keywords.
Is this possible in GSA?
I finally achieved what I was trying to do, so figured I'd post in case it helps anyone else.
As far as I know, it is impossible to create a query with this capability, but with Google's Custom Search API, you can create a search engine with the desired keywords in the context file (by editing the XML file directly or by adding keywords through the CSE console). Then you can formulate the query as usual, but perform the search on your personalized engine.
https://developers.google.com/custom-search/docs/ranking
For a matchmaking portal, we have one requirement where in, if a customer viewed complete profile details of a bride or groom then we have to exclude that profile from further search results. Currently, along with other detail we are storing the viewed profile ids in a field (Comma Separated) against that bride or groom's details.
Eg., if A viewed B, then in B's record under the field saw_me we will add A (comma separated).
while searching let say the currently searching members id is 123456 then we will fire a query like
Select * from profiledetails where (OTHER CON) AND 123456 not in saw_me;
The problem here is the saw_me field value is growing like anything, is there any better way to handle this requirement? Please guide.
If this is using Solr:
first, DON'T add the 'AND NOT ...' clauses along with the main query in q param, add them to fq. This have many benefits (the fq will be cached)
Until you get to a list of values that is maybe 1000s this approach is simple and should work fine
After you reach a point where the list is huge, maybe it time to move to a post filter with a high cost ( so it is looked up last). This would look up docs to remove in an external source (redis, db...).
In my opinion no matter how much the saw_me field grows, it will not make much difference in search time.Because tokens are indexed inversely and doc_values are created at index time in column major fashion for efficient read and has support for caching from OS. ES handles these things for you efficiently.
Is there any formula or functions which I can use to remove URL and #tags etc which come when we get the tweets using twitter in the node. I also need to make new columns which would contain the tweets which have specific keywords. But when I use the inbuilt formula "contains" in IBM BigSheets it is not of much help.
Finally, I need my result to extract the tweets into different columns based on the sentiment score they have got. I have attached my "twitter in" node with sentiment node to get the sentiment score. Therefore, it should make separate columns based on the score it has got and also based on the keywords it contains. Can we create a function to do so?