Extracting matching conditions from querystring - elasticsearch

ElasticSearch Query is formed using query string with multiple AND / OR operators. i.e. ((Condition 1 OR Condition 2) AND (Condition 3 OR Condition 4 OR Condition 5)), based on the condition it provides me multiple documents. For getting exact condition I again loop through all the resultant documents again and mark particular conditions. Is there any simple way to get resultant conditions specific to documents ?
Can anyone provide the better example using NEST API?

I think that what you need is to Highlight the data that made the hit on your query. Highlight functionality of elasticsearch actually marks the text from each search result so the user can see why the document matched the query. The marked text is returned in the response.
Please refer in the elasticsearch documentation in order to understand how this api actually works. Refer in the Nest Documentation in order to see how you can implement it with the Nest library.
For example, using the elasticsearch api imagine the below example:
GET /someIndex/someType/_search
{
"query" : {
"match_phrase" : {
"about" : "rock climbing"
}
},
"highlight": {
"fields" : {
"about" : {}
}
}
}
The same with Nest:
var result = _client.Search<someIndex>(s => s
.Query(q => q
.MatchPhrase(qs => qs
.OnField(e => e.about)
.Query("rock climbing")
)
)
.Highlight(h => h
.OnFields(f => f
.OnField(e => e.about)
)
)
);
The response will be of the below form for each search result (notice the highlight part)
"_score": 0.23013961,
"_source": {
"first_name": "John",
"last_name": "Smith",
"age": 25,
"about": "I love to go rock climbing",
"interests": [ "sports", "music" ]
},
"highlight": {
"about": [
"I love to go <em>rock</em> <em>climbing</em>"
]
}

Related

Elastic Search | How to get original search query with corresponding match value

I'm using ElasticSearch as search engine for a human resource database.
The user submits a competence (f.ex 'disruption'), and ElasticSearch returns all users ordered by best match.
I have configured the field 'competences' to use synonyms, so 'innovation' would match 'disruption'.
I want to show the user (who is performing the search) how a particular search result matched the search query. For this I use the explain api (reference)
The query works as expected and returns an _explanation to each hit.
Details (simplified a bit) for a particular hit could look like the following:
{
description: "weight(Synonym(skills:innovation skills:disruption)),
value: 3.0988
}
Problem: I cannot see what the original search term was in the _explanation. (As illustrated in example above: I can see that some search query matched with 'innovation' or 'disruption', I need to know what the skill the users searched for)
Question: Is there any way to solve this issue (example: parse a custom 'description' with info about the search query tag to the _explanation)?
Expected Result:
{
description: "weight(Synonym(skills:innovation skills:disruption)),
value: 3.0988
customDescription: 'innovation'
}
Maybe you can put the original query in the _name field?
Like explained in https://qbox.io/blog/elasticsearch-named-queries:
GET /_search
{
"query": {
"query_string" : {
"default_field" : "skills",
"query" : "disruption",
"_name": "disruption"
}
}
}
You can then find the proginal query in the matched queries section in the return object:
{
"_index": "testindex",
"_type": "employee",
"_id": "2",
"_score": 0.19178301,
"_source": {
"skills": "disruption"
},
"matched_queries": [
"disruption"
]
}
Add the explain to the solution and i think it would work fine...?

How can i get unique suggestions without duplicates when i use completion suggester?

I am using elastic 5.1.1 in my environment. I have chosen completion suggester on a field name post_hashtags with an array of strings to have suggestion on it. I am getting response as below for prefix "inv"
Req:
POST hashtag/_search?pretty&&filter_path=suggest.hash-suggest.options.text,suggest.hash-suggest.options._source
{"_source":["post_hashtags" ],
"suggest": {
"hash-suggest" : {
"prefix" : "inv",
"completion" : {
"field" : "post_hashtags"
}
}
}
Response :
{
"suggest": {
"hash-suggest": [
{
"options": [
{
"text": "invalid",
"_source": {
"post_hashtags": [
"invalid"
]
}
},
{
"text": "invalid",
"_source": {
"post_hashtags": [
"invalid",
"coment_me",
"daya"
]
}
}
]
}
]
}
Here "invalid" is returned twice because it is also a input string for same field "post_hashtags" in other document.
Problems is if same "invalid" input string present in 1000 documents in same index then i would get 1000 duplicated suggestions which is huge and not needed.
Can I apply an aggregation on a field of type completion ?
Is there any way I can get unique suggestion instead of duplicated text field, even though if i have same input string given to a particular field in multiple documents of same index ?
ElasticSearch 6.1 has introduced the skip_duplicates operator. Example usage:
{
"suggest": {
"autocomplete": {
"prefix": "MySearchTerm",
"completion": {
"field": "name",
"skip_duplicates": true
}
}
}
}
Edit: This answer only applies to Elasticsearch 5
No, you cannot de-duplicate suggestion results. The autocomplete suggester is document-oriented in Elasticsearch 5 and will thus return suggestions for all documents that match.
In Elasticsearch 1 and 2, the autocomplete suggester automatically de-duplicated suggestions. There is an open Github ticket to bring back this functionality, and it looks like it is possible to do so in a future version.
For now, you have two options:
Use Elasticsearch version 1 or 2.
Use a different suggestion implementation not based on the autocomplete suggester. The only semi-official suggestion I have seen so far involve putting your suggestion strings in a separate index.

Elasticsearch: Field level custom scores in text searches

I just started exploring elasticsearch. I need to find an approach for specifying custom scores at field level. For example:
I have a collection named blog whose documents have following format:
{
"_id": "1736hst26672829",
"name": "Learning regular expressions basics",
"author": "John Lee",
"summery": "Here is summery.",
"body": "Content of the blog."
}
If I search a text 'xyz' in the collection then the result should reflect following score criteria
match in the field 'name' has priority 1.
match in the author field has the 2nd priority.
match in the summery has 3rd priority.
match in the body has least priority.
I need top 10 results on the basis of the above criteria.
:
Scoring in ElasticSearch is extremely customizable, the following applies to query time based custom scoring. There are various other scoring options, by index, in your mapping (and thus applied to every query), on Filters or Facets, using boosts or custom scoring.
While Custom Score Query is generally the most powerful solution, here are the docs for various custom scoring methods to read up on.
Boosting Query
Custom Boost Factor Query
Custom Score Query
The following is probably the simplest methods to apply custom scoring in query time, although I suggest you read up on Custom Score Query.
"query": {
"filtered": {
"query":
"bool": {
"must": [
{"multi_match": {
"fields": [
"name^4",
"author^3",
"summery^2",
"body^1"
],
"query": "xyz",
"operator": "AND",
"type": "cross_fields",
"analyzer": "standard"
}}
]
}
}
}
}
For people who search this answer but wish to use NEST, bellow is the same query using NEST. Use the ^ character to boost specific fields or use OnFieldsWithBoost to give fields custom scoring, and the query is sorted by score.
var query = "xyz";
//Add your field names to a string in lower camelCase as is ES default.
List<string> searchIn = new List<string(new string[] {"_id","name","author","summery","body"});
.Type("blogType")
.SortDescending("_score")
.Query(
q => q.MultiMatch(
t => t.OnFields(
searchIn
.Select(qs => qs == "name" ? "name^4" : qs)
.Select(qs => qs == "author" ? "author^3" : qs)
.Select(qs => qs == "summery" ? "summery^2" : qs)
.Select(qs => qs == "body" ? "body" : qs)
)
.Query(query)
)
)
If you have the correct (default) mapping in ES (C# Object to ES indexed JSON Object), you can also use the following within the OnFields:
t => t.OnFieldsWithBoost(qs => qs.Add(entry => entry.Name, 4.0)
.Add(entry => entry.Author, 3.0)
.Add(entry => entry.Summary, 2.0)
.Add(entry => entry.Body, 1.0))

How to define document ordering based on filter parameter

Hi Elasticsearch experts.
I have a problem which might be realted to the fact I am indexing DB relational data.
My scenario is the following:
I have two entities:
documents and meetings.
Documents and meetings are independent entities. Although it is possible to assign documents to meetings in a given order.
We are using a join table for this in the DB.
meetings(id,name,date)
document(id,title,author)
meeting_document(doc_id,meeting_id,order)
In elasticsearch I am indexing the documents_id as NESTED property of the meeting
meeting example:
{
id: 25
name:"test",
documents: [22,12,24,55]
}
I will fetch the meeting, after this I would like to send a request to the documents filtering on document.id and asking elasticsearch to return the list in the same order I passed in the list of ids to the filter.
What is the best way to implement this ?
Thanks
Nice Question,
I've spent some time figuring a solution for you and come up with a solution, It might be tricky one but works.
Lets have a look to my query,
I've used script score, for sorting by user defined list.
POST index/type/_search
{
"query": {
"function_score": {
"functions": [
{
"script_score": {
"script": "ar.size()-ar.indexOf(doc['docid'].value)",
"params": {
"ar": [
"1",
"2",
"4",
"3"
]
}
}
}
]
}
},
"filter": {
"terms": {
"docid": [
"1",
"2",
"4",
"3"
]
}
}
}
The thing you have to take care is,
send, same value for filter and in params. Like in the above query.
This returns me hits with doc ids, 1, 2, 4, 3 .
You have to change field name inside script and in filter, and you can use termQuery inside query object.
I've tested the code, Hope this helps!!
Thanks

Find matching array items in MongoDB document

I am developing a web app using Codeigniter and MongoDB.
In the database I got a document that look like this:
{
"_id": {
"$id": "4f609932615a935c18r000000"
},
"basic": {
"name": "The project"
},
"members": [
{
"user_name": "john",
"role": "user",
"created_at": {
"sec": 1331730738,
"usec": 810000
}
},
{
"user_name": "markus",
"role": "user",
"created_at": {
"sec": 1331730738,
"usec": 810000
}
}
]
}
I need to search this document using both user_name and role. Right now when I am using the below code I get both. I only want to get array items matching both user_name and role.
$where = array (
'_id' => new MongoId ($account_id),
'members.user_id' => new MongoId ($user_id),
'members.role' => $role
);
$this -> cimongo -> where ($where) -> count_all_results ('accounts');
This is an old question, but as of MongoDB 2.2 or so you can use the $ positional operator in a projection so that only the matched array element is included in the result.
So you can do something like this:
$this->cimongo->where($where)->select(array('members.$'))->get('accounts');
This is a repeat of this question:
Get particular element from mongoDB array
Also you might want to use $elemMatch
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-ValueinanArray
Here is the rub -- you aren't going to be able to get the array items that match because mongo is going to return the entire document if those elements match. You will have to parse out the code client side. Mongo doesn't have a way to answer, "return only the array that matches."

Resources