In my ElasticSearch instance, I have two types in a single index. Think of them as "Profile" and "ProfileMetadata". There may be many ProfileMetadata items pointing to a single Profile.
Profile contains basic user info. Say firstname. ProfileMetadata contains metadata for the user, say "Tags".
What I want to be able to do, is run a single query that may look like the following. "Firstname NOT tag". The user would type this into the search bar. It would be a single search bar to search across both types at once.
The two queries are below :
Profile Query
GET _search
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [
"PersonalDetail.FirstName",
"PersonalDetail.LastName",
"PersonalDetail.Email"
],
"query": "John Smith NOT tag"
}
}
}
}
}
ProfileMetadata Query
GET _search
{
"query": {
"filtered": {
"query": {
"has_child":
{
"type": "ProfileMetadata",
"query":
{
"query_string": {
"fields": [
"Tags"
],
"query": "John Smith NOT Tag"
}
}
}
}
}
}
}
Is there any way to combine these queries, so that we get all John Smiths without that particular tag. I am using NEST in C#, and at the moment I am taking both of these queries (In NEST form), and using an OR between them, which isn't working as I need it to. So I'm trying to break it down into pure ES form first.
Maybe you could use only the second query, it will return all the matching parent document and then pass a filter on it representing your first query.
In this way you would not have to do a OR between two queries and might gain in performance with only one query+ filter.
Related
I'm using the following query for searching across multiple fields:
{
"query": {
"multi_match": {
"query": "italian sports car",
"fields": ["car_name", "car_brand", "car_description", "car_country"],
"type": "most_fields"
}
}
}
In this example, I'm looking for sports cars made in Italy (hence the car_country field). However, this will return all the cars made in Italy even if they are not sports cars. I want car_country to be just an auxiliary search field, so I don't want hits when the only matched field is car_country. Is this possible? I know I can set a lower score for that field, but I want hits with only this matching field to be completely ignored.
There can be different ways you handle this problem depending on the scoring etc. you require from you results. For instance -
Use a bool query with 2 parts
Must query - include queries that must match for the document to be in the resultset
Should query - include queries that should match(and impact scoring) but do not decide if a document should or should not be in the result set.
Add the multi-match query without the car_country field in must query and a match query for car_country field in should query.
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "italian sports car",
"fields": [
"car_name",
"car_brand",
"car_description"
],
"type": "most_fields"
}
}
],
"should": [
{
"match": {
"car_country": {
"query": "italian sports car"
}
}
}
]
}
}
}
I have a field with tokenizer that splits by dots.
on search, the following value aaa.bbb will be splitted to two terms aaa and bbb.
My question is how to write bool query that will contain multiple conditions on the same term?
For example, i want to get all docs where its field contains a term that matches a fuzzy search for gmail but also the same term must not contain gamil.
Here are some examples of what i want to achieve:
bmail // MATCH: since its matches fuzzy search and is not gamil
gamil.bmail // MATCH: since the term bmail matches fuzzy search and is not gamil
gamil // NO MATCH: since its matches fuzzy search and but equals gamil
NOTE: the following query does NOT appear to be working since it looks as if one term matches one condition and the second term matches the other, it will be considered a hit.
{
...
"body": {
"query": {
"bool": {
"must": [
{
"fuzzy": {
"my_field": {
"value": "gmail",
"fuzziness": 1,
"max_expansions": 2100000000
}
}
},
{
"bool": {
"must_not": [
{
"query_string": {
"default_field": "my_field",
"query": "*gamil*",
"analyzer": "keyword"
}
}
]
}
}
]
}
}
},
}
I ended up using Highlight by executing fuzzy (or any other) query, and then programatically filter the results by the returned highlight object.
span queries might also be a good option if you don't need regular expression or you can make sure you don't exceed the boolean query limit.
(see more details in the provided link)
I've setup an index that has many types representing user data such as a ShoppingList, Playlist, etc. Each type has an "identity_id" field for the user's unique identifier. I use the following query to search across all types and fields for a user (for a search function in a website):
GET _search
{
"query": {
"filtered": {
"query": {
"match_phrase_prefix": {
"_all": "awesome"
}
},
"filter": {
"match": {
"identity_id": 1
}
}
}
}
}
My questions are:
Is there a way to give a higher score to matches on fields that have "name" in the field name? For example, the ShoppingList type will have a shopping_list_name field, and I want a match on that to be higher than its other fields.
Is the above way of doing a full text search for a particular user (query then filter) the most efficient way? What about creating an index per user?
How about this query that boosts certain fields:
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "awesome",
"fields": [
"*_name",
"field*"
]
}
},
"functions": [
{
"weight": 2,
"filter": {
"multi_match": {
"query": "awesome",
"fields": [
"*_name"
]
}
}
},
{
"weight": 1,
"filter": {
"multi_match": {
"query": "awesome",
"fields": [
"field*"
]
}
}
}
]
}
}
}
What the query above does is to boost (weigth: 2) the *_name fields query and not do apply any boosting to fields called field*.
Is the above way of doing a full text search for a particular user (query then filter) the most efficient way? What about creating an index per user?
Regarding this ^ question, that's more complicated and you also need to consider how many users you have, the hardware resources the cluster has, structure of data, queries used etc.
I have a database with entries like
title: This is my awesome title
abstract: A more detailed descriptions of what [...]
I would like to build an Elasticsearch query that matches the above document with, e.g.,
awe detai
In words: A multi_match phrase_prefix query with multiple search terms. (This is intended to be used as a search-as-you-type feature.)
I see how you can combine multi_match and phrase_prefix, but it's unclear to me how to do this for multiple search terms.
Any hints?
Well there is few ways to do that
POST stack/autocomplete/1
{
"title": "This is my awesome title",
"abstract": "A more detailed descriptions of what"
}
Then you can search using query string with star but problem here is that you need to append asterix to query
POST stack/autocomplete/_search
{
"query": {
"query_string": {
"fields": [
"title",
"abstract"
],
"query": "awe* detai*"
}
}
}
If you want to match on user query then you can use like that
POST stack/autocomplete/_search
{
"query": {
"multi_match": {
"fields": [
"title",
"abstract"
],
"query": "awesome tit",
"type": "phrase_prefix"
}
}
}
One more option to consider would be to use nGram with query string so you will not need to modify user query "awe* detai*"
What is the difference between such query:
"query": {
"bool": {
...
"should": [
{
"match": {
"description": {
"query": "test"
}
}
},
{
"match": {
"address": {
"query": "test",
}
}
},
{
"match": {
"country": {
"query": "test"
}
}
},
{
"match": {
"city": {
"query": "test"
}
}
}
]
}}
and that one:
"query": {
"bool": {
...
"should": [
{
"query_string": {
"query": "test",
"fields": [
"description",
"address",
"country",
"city"
]
}
}
]
}}
Performance, relevance?
Thanks in advance!
The query is analyzed depending on the field analyzer (unless you specify the analyzer in the query itself), thus querying multiple fields with a single query doesn't necessarily mean analyzing the query only once.
Keep in mind that the query_string supports the lucene query syntax: AND and OR operators, querying on specific fields, wildcard, phrase queries etc. therefore it needs to be parsed, which I don't think makes a lot of difference here in terms of performance, but it is error prone and might lead to errors. If you don't need all that power, stick to the match query, and if you want to perform the same query on multiple fields, have a look at the multi_match query, which does what you did with your query_string but translates internally to multiple match queries.
Also, the scores returned if you compare the output of multiple match queries and your query_string might be quite different. Using a bool query you effectively build a lucene boolean query, while the query_string uses by default "use_dis_max":"true", which means it uses internally a dis_max query by default. Same happens using the multi_match query. If you set use_dis_max to false a bool query is going to be used internally instead.
I terms of performance, I would say that the second query will have performance benefits because, the first query requires the query string to be analyzed for all the four match sections, while in the second there is only one query string that needs to be analyzed.
Apart from that, there are some comparisons done over here that you can look at.
I am not quite sure about the relevancy differences, but that you can always fire these two queries and see if there is any difference in relevance from the results fetched.