I have indexed some entries containing a GEOJson point object :
exemple :
{
"_id": "48LEDd5imvEpFnCQx",
"loc": {
"type": "Point",
"coordinates": [-2.7577078342437744, 47.65381454210301]
},
"geoip": false,
"trackerId": "RG-DEMO-1",
"date": "2015-07-25T21:12:07.286Z"
}
The mapping:
{
'trace': {
'properties': {
'loc': {
'type': 'nested',
'properties': {
'type': {
'type': 'string'
},
'coordinates':{
'type': 'geo_point',
'geohash':true,
'geohash_prefix':true,
'lat_lon':true,
'fielddata' : {
'format' : 'compressed',
'precision' : '1cm'
}
}
}
...
The geohash is generated but Kibana says that loc.coordinates is not indexed and I can't use the visualisation map Unindexed fields can not be searched
What is the trick to force index on this type of field?
As stated in the doc nestedtype sub object are not indexed:
So there are two workarounds:
For keeping loc as a nested object, add a include_in_parent wildcard to true
Or, turn the loc type to Object.
Related
I have the following 2 documents indexed.
{
region: 'US',
manager: {
age: 30,
name: {
first: 'John',
last: 'Smith',
},
},
},
{
region: 'US',
manager: {
age: 30,
name: {
first: 'John',
last: 'Cena',
},
},
}
I am trying to search and sort them by their last name. I have tried the following query.
{
sort: [
{
'manager.name.first': {
order: 'desc',
nested: {
path: 'manager.name.first',
},
},
},
],
query: {
match: {
'manager.name.first': 'John',
},
},
},
I am getting the following error in response. What am I doing wrong here (I am very new to this elasticsearch, so apologize if this is a very basic thing I am not aware of)
ResponseError: search_phase_execution_exception: [query_shard_exception] Reason: [nested] failed to find nested object under path [manager.name.first]
I also tried path: 'manager.name', but that also didn't work.
You need to use only manager as nested path as that is only field define as nested type.
{
"sort": [
{
"manager.name.first.keyword": {
"order": "desc",
"nested": {
"path": "manager"
}
}
}
]
}
Use manager.name.first as field name if it is defined as keyword type otherwise use manager.name.first.keyword if it is define as multi type field with text and keyword both.
I'm creating a winston-elasticsearch logger usin Kibana, where I need to save all what is happen in the app (request, apis, ...).I used this( https://github.com/vanthome/winston-elasticsearch) I defined a json mapping template. Also a transformer.js file where I define a Transformer function to transform log data as provided by winston into a message structure which is more appropriate for indexing in ES.
then I use the winston-elasticsearch to create new Es instance:
exports.mappingTemplate = require('../../../index-template-mapping.json');
const TransportersElastic = new Elasticsearch({
index: 'soapserver',
level: 'info',
indexPrefix: 'logs',
transformer: transformer,
ensureMappingTemplate: true,
mappingTemplate: mappingTemplate,
flushInterval: 2000,
waitForActiveShards: 1,
handleExceptions: false,...
But I keep getting Elasticsearch index error of type: 'mapper_parsing_exception' reasons like : failed to parse field [fields.result.status] of type [text]'
illegal_argument_exception' reason : 'failed to parse field [fields.rows.tmcode] of type [long]'
Here transformer.js:
exports.mappingTemplate = require('../../../index-template-mapping.json');
exports.transformer = (logData) => {
const transformed = {};
transformed['#timestamp'] = new Date().toISOString();
transformed.source_host = os.hostname();
transformed.message = logData.message;
if (typeof transformed.message === 'object') {
transformed.message = JSON.stringify(transformed.message);
I need help, suggestion how to resolve those error and succeed the Mapping.
Here the index-mapping-template.json
{
"mapping": {
"_doc": {
"properties": {
"#timestamp": {
"type": "date"
},
"fields": {
"ignore_malformed": true,
"dynamic": "true",
"properties": {}
},
"message": {
"type": "text"
},
"severity": {
"type": "keyword"
}
}
}
}
}
Our system stores account in the following format: acct:username#domain
But for many searches we only need to username, so for the user created memos I've decided to make the user field a multi_field like this:
{
'text': {
'type': 'string'
}
'user': {
'type': 'multi_field',
'path': 'just_name',
'fields': {
'user': {
'type': 'string',
'index': 'analyzed',
'analyzer': 'lower_keyword'
},
'username': {
'type': 'string',
'index': 'analyzed',
'analyzer': 'username'
}
}
}
}
and other settings:
__settings__ = {
'analysis': {
'tokenizer': {
'username': {
'type': 'pattern',
'group': 1,
'pattern': '^acct:(.+)#.*$'
}
},
'analyzer': {
'lower_keyword': {
'type': 'custom',
'tokenizer': 'keyword',
'filter': 'lowercase'
},
'username': {
'tokenizer': 'username',
'filter': 'lowercase'
}
}
}
}
Now, if I make a query for the username it works. I.e if I have the following user: acct:testuser#testdomain
and I make a query like this:
{
"query": {
"bool": {
"must": [
{
"terms": {
"username": [
"testuser"
]
}
}
],
"minimum_number_should_match": 1
}
},
"size": 50
}
It works (I know it can be done much easier but this is a system generated query).
But, I need to make searches which looks for a string in both the text and the username fields.
I've decided to use a multi-match query for this.
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"operator": "and",
"query": "testuser",
"type": "cross_fields",
"fields": [
"text",
"username"
]
}
}
],
"minimum_number_should_match": 1
}
},
"size": 50
}
Now the problem is, that this query does not work for the username field. It does for the text field, and for other fields if I include them, but does not bring back any result for the username field.
Can you help me what am I doing wrong?
I've forgotten that the username analyzer would also tokenize my searches for match/multi match queries. That way the string 'testuser' was analyzed and it generated zero token.
So the solution is to change the username's field mapping to:
'username': {
'type': 'string',
'index': 'analyzed',
'index_analyzer': 'username',
'search_analyzer': 'lower_keyword'
}
and now both queries are working.
I have a Kendo datasource defined to perform server filtering. The problem is the filter it is generating looks like this:
$filter=StatusId eq '1'
which is generating the following error:
{
"error": {
"code": "",
"message": {
"lang": "en-US", "value": "Operator 'eq' incompatible with operand types 'System.Nullable`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]' and 'System.String' at position 9."
}
}
}
The message generated is in fact incorrect. The problem is with the '1' in the filter treating the number as a string. If I manually send the request as:
$filter=StatusId eq 1 I receive the expected results back. If I manualle send the first filter, I receive the error as well.
The data source for my filter UI is defined as:
type: "odata",
transport: {
read:{
url: plm.baseUrl + "_vti_bin/ListData.svc/NpiProcessSteps?$select=Title,Id",
dataType: "json"
}
},
schema:{
model: {
id: "Id",
fields:{
Id: {
type: "number",
from: "Id"
},
Title: {
type: "string",
from: "Title"
},
__metadata: {
type: "object",
from: "__metadata"
}
}
}
}
The column in question is defined as:
{
"field": "StatusId",
"title": "Status",
"template": kendo.template("${Status.Title}"),
"width": "300px",
"filterable": {
"ui": plmApp.NpiStatusFilter,
extra: false,
operators: {
number: {
eq: "Is equal to",
neq: "Is not equal to"
}
}
}
}
Have I done anything wrong to have the filter sent in this way, or is these any way to ensure that the filter is sent as a number (1) rather than as a string ('1')?
You need to specify the type of the StatusId field as well:
fields:{
Id: {
type: "number",
from: "Id"
},
StatusId: {
type: "number"
},
Title: {
type: "string",
from: "Title"
},
__metadata: {
type: "object",
from: "__metadata"
}
}
In addition you don't need to specify from if it is the same field.
This is different than your problem and more for people who are still googling the phrase "Operator 'eq' incompatible with operand types." In my case, it wasn't a problem with the schema but with the object's model which used an int instead of a string.
Always remember to check all the way back, kids.
We store documents that look something like this:
{
"id": "dQesbpxeQniUWXpsnjPQ",
"title": "Golf in Hamburg, Altona",
"user": "CtGjEaDxSrhPbf7W7NcH",
"location": {
"id": "Q6sZhRHdiS3mP2innbJ9",
"name": "Hamburg, Altona",
"lat": 53.55,
"lon": 9.93333,
"slug": "hamburg-altona"
},
"_type": "announcement"
}
We need the announcement.location.slug to be not_analyzed (it's a slug, after all)
However the mapping won't take, we have these settings:
Tire.index(##index_name) do
delete
create(mappings: {
announcement: {
properties: {
"id" => { type: 'string', index: 'not_analyzed' },
"user" => { type: 'string', index: 'not_analyzed' },
"location" => {
type: 'geo_point',
properties: {
"slug" => { type: 'string', index: 'not_analyzed' }
}
},
"times" => { type: 'string', analyzer: 'keyword' },
"sport" => {
"properties" => {
"slug" => { type: 'string', index: 'not_analyzed' }
}
}
}
}
},
settings: {
index: {
number_of_shards: 1,
number_of_replicas: 0
}
})
refresh
end
Note: The same mapping in curl syntax also doesn't work, but is less readable for SO, so I'm posting the Ruby code.
It seems like geo_point is overriding all other mappings on that part of the document. The documentation seems to agree.
I'm sure there's a way to use the lat_lon option, but I can't find any documentation on how that might work. (I assume one maps the individual lat and lon fields with lat_lon settings)
It might also be possible, I had hoped to use the multi field type but that doesn't seem to apply to whole sub-trees of the main document attributes.
How can I proceed without having to change my whole data model?
I'm afraid that you have to change your model as geo_point is a full data type and you can not add properties (meta) in it.