I am trying to copy two fields onto a third field, which should have the type 'keyword' (because I want to be able to aggregate by it, and do not need to perform a full-text search)
PUT /test/_mapping/_doc
{
"properties": {
"first": {
"copy_to": "full_name",
"type": "keyword"
},
"last": {
"copy_to": "full_name",
"type": "keyword"
},
"full_name": {
"type": "keyword"
}
}
}
I then post a new document:
POST /test/_doc
{
"first": "Bar",
"last": "Foo"
}
And query it using the composite field full_name:
GET /test2/_search
{
"query": {
"match": {
"full_name": "Bar Foo"
}
}
}
And no hits are returned.
If the type of the composite field full_name were text then it works as expected and described in the docs:
https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html
Is it not possible to copy onto a keyword-type field?
The problem is that you use match query - When you index your docs you use keyword type which according to the ES documentation are "...only searchable by their exact value."
However when you query that field you use match query which is using the standard analyzer which, among other stuff, also does lower-casing which causes your terms to not match nothing.
You have few options I can think of in this case:
Change the field type to text which will perform the same analysis as the match query.
Create a custom field type with custom analyzer which will perform lower casing
Don't query more than a single term at a time and use term query instead of match
It seems that the type of destination field of copy_to must be text type.
Related
I am pretty new to Elastic Search. I have a dataset with multiple fields like name, product_info, description etc., So while searching a document, the search term can come from any of these fields (let us call them as "search core fields").
If I start storing the data in elastic search, should I derive a field which is a concatenated term of all the "search core fields" ? and then index this field alone ?
I came across _all mapping concept and little confused. Does it do the same ?
no, you don't need to create any new field with concatenated terms.
You can just use _all with match query to search a text from any field.
About _all, yes, it searches the text from any field
The _all field has been removed in ES 7, so it would only work in ES 6 and previous versions. The main reason for this is that it used too much storage space.
However, you can define your own all field using the copy_to feature. You basically specify in your mapping which fields should be copied to your custom all field and then you can search on that field.
You can define your mapping like this:
PUT my-index
{
"mappings": {
"properties": {
"name": {
"type": "text",
"copy_to": "custom_all"
},
"product_info": {
"type": "text",
"copy_to": "custom_all"
},
"description": {
"type": "text",
"copy_to": "custom_all"
},
"custom_all": {
"type": "text"
}
}
}
}
PUT my-index/_doc/1
{
"name": "XYZ",
"product_info": "ABC product",
"description": "this product does blablabla"
}
And then you can search on your "all" field like this:
POST my-index/_search
{
"query": {
"match": {
"custom_all": {
"query": "ABC",
"operator": "and"
}
}
}
}
What is the best way to query exact value of a field in elasticsearch? Say for example I have:
profile: {
email: "test#email.com"
}
How do I check if there's exactly the same test#email.com email in a profile?
Whenever you require exact search you can define data type of that field as keyword. If you require both partial search (analyzed) and exact search on the same field you can define a sub field for the same and refer to that subfield when exact search is required.
So the field definition looks as below:
"email": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
You can then use term query to perform exact search.
{
"query": {
"term": {
"email.keyword": "test#email.com"
}
}
}
NOTE: Defining the type as keyword results in case-sensitive exact search.
I have a field that is mapped as text and which includes another field mapped as keyword (fields keyword). I insert data and ensure it can retrieved using any query. However, when I query the additional field (mapped as keyword), I cannot find any data at all.
Here is the example (simplified):
POST people/_mapping/_doc
{
"properties": {
"name": {
"type": "text"
},
"bio": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
And here is a query:
POST people/_search
{
"query": {
"match": {
"bio.keyword": "Portugal"
}
}
}
Same happens regardless of the casing (Portugal vs portugal). What is the reason for this behavior?
In elasticseach, if you have a text type field say description.
One value of description is: He likes dog but hate cat.
The revert index for this field is: He/like/dog/but/hate/cat
And there is another 'keyword' field which is description.keyword, which is exactly He likes dog but hate cat.
The keyword field requires 100% match.
Got it:
Keyword fields are only searchable by their exact value
References: https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html
Good morning, In my code I can't search data which contain separate words. If I search on one word all good. I think problem in mapping. I use postman. When I put in URL http://192.168.1.153:9200/sport_scouts/video/_mapping and use method GET I get:
{
"sport_scouts": {
"mappings": {
"video": {
"properties": {
"hashtag": {
"type": "string"
},
"id": {
"type": "long"
},
"sharing_link": {
"type": "string"
},
"source": {
"type": "string"
},
"title": {
"type": "string"
},
"type": {
"type": "string"
},
"user_id": {
"type": "long"
},
"video_preview": {
"type": "string"
}
}
}
}
}
}
All good title have type string but if I search on two or more words I get empty massive. My code in Trait:
public function search($data) {
$this->client();
$params['body']['query']['filtered']['filter']['or'][]['term']['title'] = $data;
$search = $this->client->search($params)['hits']['hits'];
dump($search);
}
Then I call it in my Controller. Can you help me with this problem?
The reason that your indexed data can't be found is caused by a mismatch of the analyzing during indexing and a strict term filter when querying the data.
With your mapping configuration, you are using the default analyzing which (besides many other operations) does a tokenizing. So every multi-word data you insert is split at punctuation or whitespaces. If you insert for example "some great sentence", elasticsearch maps the following terms to your document: "some", "great", "sentence", but not the term "great sentence". So if you do a term filter on "great sentence" or any other part of the original value containing a whitespace, you will not get any results.
Please see the elasticsearch docs on how to configure your mapping for indexing without analyzing (https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping-intro.html#_index_2) or consider doing a match query instead of a term filter on the existing mapping (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html).
Please be aware that if you switch to not_analyzed you will be disabling many of the great fuzzy fulltext query functionality. Of course you can set up a mapping that does both, analyzed and not_analyzed in different fields. Then it's up on you to decide on which field you want to query on.
Does elastic search provide a functionality to map different fields to a single field and use that single field for search.
For eg _all refers to all the fields in the docs.
Similarly do we have any mapping configuration to define a field which would be referring to multiple fields.
Eg : I have a field called Brand,Name,Category.
I need to map Brand and Name to a single field custome_field.
I want it during mapping time and not during query time. I know cross fields does that during query time.
Take a look at copy_to functionality. It acts just like a custom _all. See here more about this:
In Metadata: _all field we explained that the special _all field
indexes the values from all other fields as one big string. Having all
fields indexed into one field is not terribly flexible though. It
would be nice to have one custom _all field for the person’s name, and
another custom _all field for their address.
Elasticsearch provides us with this functionality via the copy_to
parameter in a field mapping:
PUT /my_index {
"mappings": {
"person": {
"properties": {
"first_name": {
"type": "string",
"copy_to": "full_name"
},
"last_name": {
"type": "string",
"copy_to": "full_name"
},
"full_name": {
"type": "string"
}
}
}
} }