We store documents that look something like this:
{
"id": "dQesbpxeQniUWXpsnjPQ",
"title": "Golf in Hamburg, Altona",
"user": "CtGjEaDxSrhPbf7W7NcH",
"location": {
"id": "Q6sZhRHdiS3mP2innbJ9",
"name": "Hamburg, Altona",
"lat": 53.55,
"lon": 9.93333,
"slug": "hamburg-altona"
},
"_type": "announcement"
}
We need the announcement.location.slug to be not_analyzed (it's a slug, after all)
However the mapping won't take, we have these settings:
Tire.index(##index_name) do
delete
create(mappings: {
announcement: {
properties: {
"id" => { type: 'string', index: 'not_analyzed' },
"user" => { type: 'string', index: 'not_analyzed' },
"location" => {
type: 'geo_point',
properties: {
"slug" => { type: 'string', index: 'not_analyzed' }
}
},
"times" => { type: 'string', analyzer: 'keyword' },
"sport" => {
"properties" => {
"slug" => { type: 'string', index: 'not_analyzed' }
}
}
}
}
},
settings: {
index: {
number_of_shards: 1,
number_of_replicas: 0
}
})
refresh
end
Note: The same mapping in curl syntax also doesn't work, but is less readable for SO, so I'm posting the Ruby code.
It seems like geo_point is overriding all other mappings on that part of the document. The documentation seems to agree.
I'm sure there's a way to use the lat_lon option, but I can't find any documentation on how that might work. (I assume one maps the individual lat and lon fields with lat_lon settings)
It might also be possible, I had hoped to use the multi field type but that doesn't seem to apply to whole sub-trees of the main document attributes.
How can I proceed without having to change my whole data model?
I'm afraid that you have to change your model as geo_point is a full data type and you can not add properties (meta) in it.
Related
I have the following 2 documents indexed.
{
region: 'US',
manager: {
age: 30,
name: {
first: 'John',
last: 'Smith',
},
},
},
{
region: 'US',
manager: {
age: 30,
name: {
first: 'John',
last: 'Cena',
},
},
}
I am trying to search and sort them by their last name. I have tried the following query.
{
sort: [
{
'manager.name.first': {
order: 'desc',
nested: {
path: 'manager.name.first',
},
},
},
],
query: {
match: {
'manager.name.first': 'John',
},
},
},
I am getting the following error in response. What am I doing wrong here (I am very new to this elasticsearch, so apologize if this is a very basic thing I am not aware of)
ResponseError: search_phase_execution_exception: [query_shard_exception] Reason: [nested] failed to find nested object under path [manager.name.first]
I also tried path: 'manager.name', but that also didn't work.
You need to use only manager as nested path as that is only field define as nested type.
{
"sort": [
{
"manager.name.first.keyword": {
"order": "desc",
"nested": {
"path": "manager"
}
}
}
]
}
Use manager.name.first as field name if it is defined as keyword type otherwise use manager.name.first.keyword if it is define as multi type field with text and keyword both.
I want to set an images field in my document to an array of type [SanityImage].
I have tried setting it to type: 'image' but when I query the field I get that the type is [SanityCollectionItemsImages]
The images field is found inside an object (productVariant) which is then included in the main document (collection) where I have an array of productVariants in my items field.
Inside the fields array of my collection document:
...
{
title: "Items",
name: "items",
type: "array",
of: [
{
title: "Item",
type: "productVariant",
},
],
},
...
The productVariant type object:
export default {
title: "Product variant",
name: "productVariant",
type: "object",
fields: [
{
name: "images",
title: "Images",
type: "array",
of: [
{
type: "image",
options: {
hotspot: true,
},
},
],
},
...
My query using GraphQL:
allSanityCollection {
edges {
node {
_id
title
description
items {
_key
title
images {
asset {
_ref
_id
fluid(maxHeight: 600) {
base64
aspectRatio
src
srcSet
srcWebp
srcSetWebp
sizes
}
}
}
}
}
}
}
When I click on the the images field inside the GraphiQL web IDE:
Using Sanity.io HTTP API to get the whole document:
"images": [
{
"_key": "5605f5383975",
"_type": "image",
"asset": {
"_ref": "image-14f9b7688912499f187b7c20e57816b3cdf42c1e-4016x4688-jpg",
"_type": "reference"
}
},
...
My question is how to set the type of a field in my schema to be explicitly [SanityImage] instead of [SanityCollectionItemsImages] and why do I get that weird type?
The Sanity schema looks totally sane, and the API delivers the proper data structure as you say, so the problem seems to be related to the Gatsby and GraphQL layer of things.
I'll recommend making sure you are using the latest versions of the CLI, Studio and Sanity Gatsby plugin and see if that makes things better as development on the Gatsby source plugin is moving fast.
I fetch a document from ES and then try to update it using js client:
Structure of document is as(partial):
{
_id = "2c3cb61e-fd2a-11e8-8ac1-0242ac110008",
_index = "my_index",
_source: {taskName: "Task_name",....}
}
Code:
hit._source.taskName = 'New_task_name';
esClient.bulk({
body: [
{ update: { _index: 'my_index', _type: 'default', _id: hit._id }},
{ doc: hit}
]
}, function (err, resp) {
// ...
console.log(err, resp);
});
But it gives error as:
{ "type": "mapper_parsing_exception", "reason": "Field [_index] is
a metadata field and cannot be added inside a document. Use the index
API request parameters." }
Does it mean I can't have _index as field name in my document?
Great job so far, you're almost there!! You only need to send what's in the _source, do it like this instead:
hit._source.taskName = 'New_task_name';
esClient.bulk({
body: [
{ update: { _index: 'my_index', _type: 'default', _id: hit._id }},
{ doc: hit._source} <-- change here
]
}, function (err, resp) {
// ...
console.log(err, resp);
});
I do have have products and some of them are reduced in price for a specific date range.
(simplified) example products:
{
"id": 1,
"price": 2.0,
"specialPrice": {
"fromDate": null,
"tillDate": null,
"value": 0,
},
},
{
"id": 2,
"price": 4.0,
"specialPrice": {
"fromDate": 1540332000,
"tillDate": 1571781600,
"value": 2.5,
},
},
{
"id": 3,
"price": 3.0,
"specialPrice": {
"fromDate": null,
"tillDate": null,
"value": 0,
},
}
Filtering by price was no problem. That I could do with a simple bool query.
But I could not yet find a good example for ElasticSearch scripts that could point me in the right direction, even though it should be quite simple, given you know the syntax.
My pseudocode: price = ('now' between specialPrice.fromDate and specialPrice.tillDate) ? specialPrice.value : price
Is there a way to translate this into something that would work in an ElasticSearch sorting?
To clarify further: By default, all products are already sorted by several conditions. The user can also search for any terms and filter the results while also being able to select multiple sorting parameters. Items can for example be sorted by tags and then by price, it's all very dynamic and it does still sort those results by some other properties (including the _score) afterwards.
So just changing the _score would be bad, since that is already calculated in a complex matter to show the best results for the given search terms.
Here is my current script, which does fail at the first params.currentDate:
"sort": {
"_script": {
"type": "number",
"script": {
"source": "if(doc['specialPrice.tillDate'] > params.currentDate) {params.currentPrice = doc['specialPrice.value']} return params.currentPrice",
"params": {
"currentDate": "now",
"currentPrice": "doc['price']"
}
}
}
How it does work now:
One problem was the nesting of some of the properties.
So one of my steps was to duplicate their content to new fields for the product (which I'm not that happy about, but whatever).
So in my mapping, I created new properties for products (specialFrom, specialTill, specialValue) and gave the corresponding fields in my specialPrice "copy_to" properties with the new property names.
The part is in php array syntax, since I'm using ruflin/elastica:
'specialPrice' => [
'type' => 'nested',
'properties' => [
'fromDate' => [
'type' => 'date',
'format' => 'epoch_second',
'copy_to' => 'specialFrom',
],
'tillDate' => [
'type' => 'date',
'format' => 'epoch_second',
'copy_to' => 'specialTill',
],
'value' => [
'type' => 'float',
'copy_to' => 'specialValue',
],
],
],
'specialFrom' => [
'type' => 'date',
'format' => 'epoch_second',
],
'specialTill' => [
'type' => 'date',
'format' => 'epoch_second',
],
'specialValue' => [
'type' => 'float',
],
Now my sorting sorting script does look like this (in my testing client, still working on implementing it within elastica):
"sort": {
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": "params.param = ((doc['specialTill'].value - new Date().getTime()) > 0 && (new Date().getTime() - doc['specialFrom'].value) > 0) ? doc['specialValue'].value : doc['price'].value; return params.param;",
"params": {
"param": 0.0
}
}
}
}
I'm not 100% happy with this because I have redundant data and scripts (calling new Date().getTime() twice in the script), but it does work and that is the most important thing for now :)
I've updated the below query post your clarifications. Let me know if that works!
POST dateindex/_search
{
"query":{
"match_all":{ // you can ignore this, I used this to test at my end
}
},
"sort":{
"_script":{
"type":"number",
"script":{
"lang":"painless",
"inline":" params.param = ((doc['specialPrice.tillDate'].value - new Date().getTime()) > 0) ? doc['specialPrice.value'].value : doc['price'].value; return params.param;",
"params":{
"param":0.0
}
},
"order":"asc"
}
}
}
You can try using source instead of inline in the above query as I've been testing on ES5.X version on my machine.
Hope it helps!
I have indexed some entries containing a GEOJson point object :
exemple :
{
"_id": "48LEDd5imvEpFnCQx",
"loc": {
"type": "Point",
"coordinates": [-2.7577078342437744, 47.65381454210301]
},
"geoip": false,
"trackerId": "RG-DEMO-1",
"date": "2015-07-25T21:12:07.286Z"
}
The mapping:
{
'trace': {
'properties': {
'loc': {
'type': 'nested',
'properties': {
'type': {
'type': 'string'
},
'coordinates':{
'type': 'geo_point',
'geohash':true,
'geohash_prefix':true,
'lat_lon':true,
'fielddata' : {
'format' : 'compressed',
'precision' : '1cm'
}
}
}
...
The geohash is generated but Kibana says that loc.coordinates is not indexed and I can't use the visualisation map Unindexed fields can not be searched
What is the trick to force index on this type of field?
As stated in the doc nestedtype sub object are not indexed:
So there are two workarounds:
For keeping loc as a nested object, add a include_in_parent wildcard to true
Or, turn the loc type to Object.