Using Kibana/ Elasticsearch version 6.6.
Trying to run the below simple painless script:
String val = "Vanished";
if(doc.containsKey('type')) {
return doc['type'].value;
}
return val;
In the Preview Results section, when I try to run the code, the First 10 results section is always:
[]
However, if I alter the code like below:
String val = "Vanished";
if(doc.containsKey('type')) {
return "Present";
}
return val;
I am getting the below result in the same Preview Results section:
[
{
"_id": "Kha1NmkBcY4KotEKXsZz",
"test112": [
"Present"
]
},
{
"_id": "1oS1NmkBjBc6pl9UX0IW",
"test112": [
"Present"
]
},
{
"_id": "14S1NmkBjBc6pl9UX0IW",
"test112": [
"Present"
]
},
{
"_id": "whC1NmkBCa8dRNQVXzEW",
"test112": [
"Present"
]
},
{
"_id": "X221NmkBZQRXPOstYIHB",
"test112": [
"Present"
]
},
{
"_id": "Rca1NmkBZrtXVVVdY50r",
"test112": [
"Present"
]
},
{
"_id": "CMS1NmkBwiujVR8BZAt2",
"test112": [
"Present"
]
},
{
"_id": "xhC1NmkBCa8dRNQVZTFf",
"test112": [
"Present"
]
},
{
"_id": "yBC1NmkBCa8dRNQVZTFf",
"test112": [
"Present"
]
},
{
"_id": "yRC1NmkBCa8dRNQVZTFf",
"test112": [
"Present"
]
}
]
Can someone please help figure out why the doc['type'].value is failing?
This issue is solved by changing the script to the below:
String val = "Vanished";
if(doc.containsKey('type.keyword')) {
return doc['type.keyword'].value;
}
return val;
Note the use of type.keyword instead of type as the field name.
Related
I am having the following data in the elasticsearch
{
"_index": "media",
"_type": "information",
"_id": "6838",
"_source": {
"demographics_countries": {
"AE": 0.17543859649122806,
"CA": 0.013157894736842105,
"FR": 0.017543859649122806,
"GB": 0.043859649122807015,
"IT": 0.02631578947368421,
"LB": 0.013157894736842105,
"SA": 0.49122807017543857,
"TR": 0.017543859649122806,
"US": 0.09210526315789472
}
}
},
{
"_index": "media",
"_type": "information",
"_id": "57696",
"_source": {
"demographics_countries": {
"TN": 0.8125,
"MA": 0.034375,
"DZ": 0.032812,
"FR": 0.0125,
"EG": 0.0125,
"IN": 0.009375,
"SA": 0.009375
}
}
]
Expected result:
Find out an document having specific country SA (saudi arabia) is among top 3 in demographics_countries
For example:
"_id": "6838" (first document) is matched because SA (saudi arabia) is among top 3 in the demographics_countries in the above mentioned example document.
Tried ? : I have tried to filter using top_hits, But it's not working as expected.
Any suggestion will be grateful
With the current data model it's quite difficult to do that. What I'd suggest might be not the easiest way to do it, but it will definitely be the fastest to query eventually.
I'd suggest remodelling your documents to already include top countries:
[
{
"_index": "media",
"_type": "information",
"_id": "6838",
"_source": {
"top_demographics_countries": ["TN", "MA", "DZ"],
"demographics_countries": {
"AE": 0.17543859649122806,
"CA": 0.013157894736842105,
"FR": 0.017543859649122806,
"GB": 0.043859649122807015,
"IT": 0.02631578947368421,
"LB": 0.013157894736842105,
"SA": 0.49122807017543857,
"TR": 0.017543859649122806,
"US": 0.09210526315789472
}
}
},
{
"_index": "media",
"_type": "information",
"_id": "57696",
"_source": {
"top_demographics_countries": ["TN", "MA", "DZ"],
"demographics_countries": {
"TN": 0.8125,
"MA": 0.034375,
"DZ": 0.032812,
"FR": 0.0125,
"EG": 0.0125,
"IN": 0.009375,
"SA": 0.009375
}
}
}
]
Ignore values I've picked for top_demographics_countries. With this kind of approach, you can always precalculate top and then you could use a simple terms query to check if document contains that value or not:
{
"query": {
"bool": {
"filter": {
"term": {
"top_demographics_countries": "SA"
}
}
}
}
}
It's going to be cheaper to compute them once during saving compared to always building that clause dynamically.
#Evaldas is right -- it's better to extract the top 3 beforehand.
But if you can't help yourself and feel compelled to use java/painless, here's one approach:
{
"query": {
"bool": {
"must": [
{
"exists": {
"field": "demographics_countries.SA"
}
},
{
"script": {
"script": {
"source": """
def tuple_list = new ArrayList();
for (def c : params.all_countries) {
def key = 'demographics_countries.'+c;
if (!doc.containsKey(key) || doc[key].size() == 0) {
continue;
}
def val = doc[key].value;
tuple_list.add([c, val]);
}
// sort tuple list by the country values
Collections.sort(tuple_list, (arr1, arr2) -> arr1[1] < arr2[1] ? 1 : -1);
// slice & take only the top 3
def top_3_countries = tuple_list.subList(0, 3).stream().map(arr -> arr[0]).collect(Collectors.toList());
return top_3_countries.size() >=3 && top_3_countries.contains(params.country_of_interest);
""",
"params": {
"country_of_interest": "SA",
"all_countries": [
"AE",
"CA",
"FR",
"GB",
"IT",
"LB",
"SA",
"TR",
"US",
"TN",
"MA",
"DZ",
"EG",
"IN"
]
}
}
}
}
]
}
}
}
I have the following data:
{
"_id": ObjectID("5e2fa881c3a1a70006c5743c"),
"name": "Some name",
"policies": [
{
"cId": "dasefa-2738-4cf0-90e0d568",
"weight": 12
},
{
"cId": "c640ad67dasd0-92f981583568",
"weight": 50
}
]
}
I'm able to query this with Spring Mongo fine, however I want to be able to order the policies by weight
At the moment I get my results fine with:
return mongoTemplate.find(query, CArea::class.java)
However say I make the following aggregations:
val unwind = Aggregation.unwind("policies")
val sort = Aggregation.sort(Sort.Direction.DESC,"policies.weight")
How can I go and actually apply those to the returned results above? I was hoping that the dot annotation would do the job in my query however didnt do anything e.g. Query().with(Sort.by(options.sortDirection, "policies.weight"))
Any help appreciated.
Thanks.
I am not familier with Spring Mongo, but I guess you can convert the following aggregation to spring code.
db.collection.aggregate([
{
$unwind: "$policies"
},
{
$sort: {
"policies.weight": -1
}
},
{
$group: {
_id: "$_id",
"policies": {
"$push": "$policies"
},
parentFields: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
"$parentFields",
{
policies: "$policies"
}
]
}
}
}
])
This will result:
[
{
"_id": "5e2fa881c3a1a70006c5743c",
"name": "Some name",
"policies": [
{
"cId": "c640ad67dasd0-92f981583568",
"weight": 50
},
{
"cId": "dasefa-2738-4cf0-90e0d568",
"weight": 12
}
]
}
]
Playground
I use following terraform code to get a list of available db resources:
data "alicloud_db_instance_classes" "resources" {
instance_charge_type = "PostPaid"
engine = "PostgreSQL"
engine_version = "10.0"
category = "HighAvailability"
zone_id = "${data.alicloud_zones.rds_zones.ids.0}"
multi_zone = true
output_file = "./classes.txt"
}
And the output file looks like this:
[
{
"instance_class": "pg.x4.large.2",
"storage_range": {
"max": "500",
"min": "250",
"step": "250"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "pg.x8.medium.2",
"storage_range": {
"max": "250",
"min": "250",
"step": "0"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "rds.pg.c1.xlarge",
"storage_range": {
"max": "2000",
"min": "5",
"step": "5"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "rds.pg.s1.small",
"storage_range": {
"max": "2000",
"min": "5",
"step": "5"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
}
]
And I want to get the one that's cheapest.
One way to do so is by sorting with storage-range.min， but how do I sort this list based on 'storage_range.min'?
Or I can filter by 'instance_class', but "alicloud_db_instance_classes" doesn't seem to like filter as it says: Error: data.alicloud_db_instance_classes.resources: : invalid or unknown key: filter
Any ideas?
The sort() function orders lexicographical and you have no simple key here.
You can use filtering with some code like this (v0.12)
locals {
best_db_instance_class_key = "rds.pg.s1.small"
best_db_instance_class = element( alicloud_db_instance_classes.resources, index(alicloud_db_instance_classes.resources.*.instance_class, best_db_instance_class_key) )
}
(Untested code)
db.getCollection('questionbank').aggregate([
{ "$group": {
"_id": {
"technology": "$technology",
"level":"$level",
"type":"$type"
},
"Count": { "$sum": 1 }
}},
{ "$group": {
"_id": "$_id.technology",
"QuestionCount": {
"$push": {
"level":"$_id.level",
"type":"$_id.type",
"count": "$Count"
},
}
}}
])
I am trying to get the same output structure.
Can anyone please help me to write above query in spring.
I have tried a lot but failed.
You can use the following .
group("technology", "level", "type").count().as("count"), group("_id.technology") .push( new BasicDBObject("level", "$_id.level") .append("type", "$_id.type") .append("count", "$count")) .as("questionCount")
I have this document which I only want part of it. But I'm not sure how to do this in Mongoid query.
{
"_id": {
"$oid": "5297d6773865640002000000"
},
"saved_tweets": [
{
"_id": {
"$oid": "52b0856b6535380002000000"
},
"saved_id": "123456",
"tweet_ids": [
"1",
"2"
]
},
{
"_id": {
"$oid": "52b0856b6535380002000001"
},
"saved_id": "78901",
"tweet_ids": [
"3",
"4"
]
}
]}
What I want is all the tweet_ids according to the saved_id. This is what I'm doing right now which I think it's very ineffective.
existing_user = User.find_by(:social_id => social_id)
existing_user.saved_tweets.each do |saved_tweet|
if saved_id == saved_tweet.saved_id
#saved_tweet_ids = saved_tweet.tweet_ids
end
end
did you try something like that?
user.saved_tweets.where(saved_id: user.saved_id).map(&:tweet_ids)
?