My intent is to fetch records matching a specified date range and uuid (Unique User ID). The query seems to fail:
If I try the same query with username then it works.
I suspected the : character in the search string and tried escaping it with a \ but it still didn't work.
What could be the problem ?
The mappings are:
{
"top_flows": {
"mappings": {
"top flows": {
"properties": {
"name": {
"type": "string"
},
"postDate": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
},
"uuid": {
"type": "string"
}
}
}
}
}
}
The JSON document being added into elasticsearch is:
doc = {
'postDate': timestamp_str,
'uuid': uuid,
'name': 'sam'
}
timestamp = time.time()
timestamp_str = datetime.fromtimestamp(timestamp).strftime('%Y-%m-%dT%H:%M:%S')
res = self.__eshandle.index(index="top_flows", doc_type='top flows', id=int(timestamp), body=doc)
Possible problem is that you did not turned of analysis on term field in mapping.
Put mapping
PUT /top_flows
{
"mappings": {
"users": {
"properties": {
"uuid": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
Save document
POST /top_flows/blogpost
{
"uuid": "FF:FF:FF:FF"
}
Search
POST /top_flows/_search
{
"query": {
"match_all": {}
} ,
"post_filter": {
"term": {
"uuid": "FF:FF:FF:FF"
}
}
}
Below worked out finally !
GET /top_flows/_search
{
"query" : {
"range" : {
"postDate" : {
"from" : "now-5m",
"to" : "now"
}
}
},
"filter": {
"query_string": {
"query": "FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:FF\\:11\\:8F\\:FF\\:FF\\:FF",
"fields": ["uuid"]
}
}
}
Not sure still why the earlier approaches failed :(
Related
text or keyword data type allows numeric values to be inserted.
index template is as shown below:
{
"index_patterns" : [ "temp-index" ],
"mappings": {
"docs" : {
"properties": {
"username" : {
"type": "keyword"
}
}
}
}
}
document inserted :
{
"username" : 10
}
as the above document is inserted successfully, is it possible to reject such numeric values for text/keyword datatype.
You can use the coerce setting but I think it only works vice versa -- i.e. not allowing strings to be indexed as numbers:
PUT roddo
{
"mappings": {
"properties": {
"username": {
"type": "integer",
"coerce": false
}
}
}
}
This way, attempting
POST roddo/_doc
{
"username": "10"
}
will throw an Integer value passed as String error.
Alternatively, you could extend the username functionality:
PUT roddo2
{
"mappings": {
"properties": {
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"integer": {
"type": "integer",
"coerce": true,
"ignore_malformed": true
}
}
}
}
}
}
which is going to attempt to coerce but will not throw an exception when failed.
After syncing your documents you'll be able to find those that did not have integer usernames:
GET roddo2/_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "username.integer"
}
}
}
}
}
want to search in elastic search, like we do in SQL query = (age = 25 and name = xyz).
This for a single field and single data.
Yes, its very much possible, just use below ES Mapping and query:
Mapping
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"age" :{
"type" : "integer"
}
}
},
"settings": {
"index": {
"number_of_shards": "1",
"number_of_replicas": "1"
}
}
}
Index a doc
{
"name": "xyz",
"age" : 25
}
Query
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "xyz"
}
},
{
"match": {
"age": 25
}
}
]
}
}
}
In addition to the accepted answer
POST _sql?format=txt {
"query":"SELECT age, name FROM collection WHERE age=25 AND name ='xyz'"
}
See also
https://www.elastic.co/what-is/elasticsearch-sql
I have tried in elasticsearch 6.x. But no result.
PUT suciptox
{
"mappings": {
"data": {"properties": {
"time_format":{ "type": "date" },
"fieldx": { "type": "text", "fielddata":true, "index": "not_analyzed"
}
}
}
}
}
Get query
"query": {
"terms": {
"fieldx": [ "L333","xxxx"]
}
}
According to the mapping which you provided, there is a field data which is an object type and has another field called fieldx.
so your query should be like
{
"query": {
"terms": {
"data.fieldx": ["L333", "xxxx"]
}
}
}
I am trying to retrieve data from elasticsearch based on 2 conditions, It should match the jarFileName and dependentClassName. The query runs fine with jarFileName but it matches dependendentClassName partially.
This is the query I used.
{
"query": {
"bool": {
"must": [
{
"match": {
"dependencies.dependedntClass": "java/lang/String"
}
},
{
"match": {
"JarFileName": {
"query": "Client.jar"
}
}
}
]
}
}
}
Query fully matches the jarFileName but for the dependentClassName it even matched and returned any part of the value mentioned. For an example if I used java/lang/String, it returns any type that has java or lang or String in their dependentClassName. I think its because of the "/". How can I correct this one?
EDIT
I used this query for mapping,
{
"classdata": {
"properties": {
"dependencies": {
"type": "object",
"properties": {
"dependedntClass": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
You can set the index of dependencies.dependedntClass to not_analyzed so that your given string will not be analyzed with standard analyzer. If you are using ES 2.x then the below mapping should work fine.
PUT /your_index
{
"mappings": {
"your_type":{
"properties": {
"dependencies":{
"type": "string",
"fields": {
"dependedntClass":{
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
Then, your query should also work fine.
EDIT (if dependencies field is of nested type)
If your dependencies field is of nested or array type, then change the mapping as like :
POST /your_index
{
"mappings": {
"your_type":{
"properties": {
"dependencies":{
"type": "nested",
"properties": {
"dependedntClass":{
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
And the query should be changed as like below:
GET /your_index/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "dependencies",
"query": {
"match": {
"dependencies.dependedntClass": "java/lang/String"
}
}
}
},
{
"match": {
"JarFileName": {
"query": "Client.jar"
}
}
}
]
}
}
}
I have a mapping with an inner object as follows:
{
"mappings": {
"_all": {
"enabled": false
},
"properties": {
"foo": {
"name": {
"type": "string",
"index": "not_analyzed"
},
"address": {
"type": "object",
"properties": {
"address": {
"type": "string"
},
"city": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
When I try the following aggregation it does not return any data:
post data:*/foo/_search?search_type=count
{
"query": {
"match_all": {}
},
"aggs": {
"unique": {
"cardinality": {
"field": "address.city"
}
}
}
}
When I try to put field city or address.city, aggregation returns zero but if i put foo.address.city it is then when i get the correct respond by elasticsearch. This also affects kibana behavior
Any ideas why this is happening? I saw there is a mapping refactoring that might affects this. I use elasticsearch version 1.7.1
To add on this if, I use the relative path in a search query as follows it works normally:
"query": {
"filtered": {
"filter": {
"term": {
"address.city": "london"
}
}
}
}
Seems its this same issue.
This is seen when the type name and field name is same.