I have to filter payloads like this on an ElasticSearch query:
{
"bestPrices": {
"cia1": {},
"cia2": {}
}
}
I must get only results like:
{
"bestPrices": {
"cia1": {
"gol": {
"price1": 799,
"price2": null,
"miles": 25000
}
},
"cia2": {
"gol": {
"price1": null,
"price2": null,
"miles": null
}
}
}
}
I'm trying exists query, but seems that it do not apply to this particular situation:
{
"exists": {
"field": "searchIntention.bestSalePrices.cia1"
}
}
I'm using ElasticSearch 6.1
The Elasticsearch Documentation for the Exists Query specifies that null, [], and [null] qualify as non-existent values. Therefore, I believe all other values, including an empty object ({}) would be considered non-null. If the go1 member of the cia object is always populated, you could try using exists on that field instead.
{
"exists": {
"field": "searchIntention.bestSalePrices.cia1.go1"
}
}
Related
I would like to query the Elastic Search with map of values and retrieve the documents.
Example:
I have indexed the below two documents
1. {
"timestamp": 1601498048,
"props": {
"cp1": "cv1",
"cp2": "cv2"
}
}
2. {
"timestamp": 1601498098,
"props": {
"key1": "v1",
"key2": "v2"
}
}
So, I wanted to query with the entire map values props with
"props"
{
"cp1": "cv1",
"cp2": "cv2"
}
and return documents only for the entired matched map values. So in this case the result would be only first document, since it matched the given props.
I can able to query with only single map value like below , but need to search for entire map.
curl -X GET "localhost:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool" : {
"must" : [
{
"terms" : {
"customProperties.cp1.keyword" : [ "cv1" ]
}
}
]
}
}
}
'
So how we query for entire map props and return documents only if all map key-values matched.
Update
Mainly I need a QueryBuilder to search with map of values. I could do for set of values like below
val sampleSet = setOf("foo", "bar")
val query = NativeSearchQueryBuilder()
.withQuery(
QueryBuilders.termsQuery(
"identifiers.endpointId.keyword", sampleSet)
)
.build()
I need QueryBuilder to search with map of values in the ES index and return document only if entire map values matches.
Suggestions please.
you must apply double match clausule.
{
"query": {
"bool": {
"must": [
{
"match": {
"props.cp1": "cv1"
}
},
{
"match": {
"props.cp2": "cv2"
}
}
]
}
}
}
Or Term.
{
"query": {
"bool": {
"must": [
{
"term": {
"props.cp1.keyword": "cv1"
}
},
{
"term": {
"props.cp2.keyword": "cv2"
}
}
]
}
}
}
This worked. I just looped through the queryBuilder with map values props.
val builder = QueryBuilders.boolQuery()
for (prop in props) {
builder.must(QueryBuilders.matchQuery("customProperties.${prop.key}", prop.value))
}
val query = NativeSearchQueryBuilder().withQuery(builder)
println("results + $queryForList(query)")
passed query to this function
internal fun queryForList(query: NativeSearchQuery): List<DocumentType> {
val resp = searchOperations.search(query, type, IndexCoordinates.of(indexName))
return resp.searchHits.map { it.content }
}
I'm trying to set an attribute of a document inside an array to uppercase.
This is a document example
{
"_id": ObjectId("5e786a078bc3b3333627341e"),
"test": [
{
"itemName": "alpha305102992",
"itemNumber": ""
},
{
"itemName": "beta305102630",
"itemNumber": "P5000"
},
{
"itemName": "gamma305102633 ",
"itemNumber": ""
}]
}
I already tried a lot of thing.
private void NameElementsToUpper() {
AggregationUpdate update = AggregationUpdate.update();
//This one does not work
update.set("test.itemName").toValue(StringOperators.valueOf(test.itemName).toUpper());
//This one also
update.set(SetOperation.set("test.$[].itemName").withValueOfExpression("test.#this.itemName"));
//And every variant in between these two.
// ...
Query query = new Query();
UpdateResult result = mongoTemplate.updateMulti(query, update, aClass.class);
log.info("updated {} records", result.getModifiedCount());
}
I see that Fields class in spring data is hooking into the "$" char and behaving special if you mention it. Do not seem to find the correct documentation.
EDIT: Following update seems to work but I do not seem to get it translated into spring-batch-mongo code
db.collection.update({},
[
{
$set: {
"test": {
$map: {
input: "$test",
in: {
$mergeObjects: [
"$$this",
{
itemName: {
$toUpper: "$$this.itemName"
}
}
]
}
}
}
}
}
])
Any solutions?
Thanks!
For now I'm using which does what i need. But a spring data way would be cleaner.
mongoTemplate.getDb().getCollection(mongoTemplate.getCollectionName(Application.class)).updateMany(
new BasicDBObject(),
Collections.singletonList(BasicDBObject.parse("""
{
$set: {
"test": {
$map: {
input: "$test",
in: {
$mergeObjects: [
"$$this",
{
itemName: { $toUpper: "$$this.itemName" }
}
]
}
}
}
}
}
"""))
);
I'm trying to query data within a Strapi Dynamic Zone in Gatsby. In the Graphql Playground I can get this to work, but using the same query in Gatsby I receive the following error in the terminal:
error Unknown type "ComponentTextArticleCopy" graphql/template-strings
And my query in article.js
export const query = graphql`
query ArticleTemplate($id: String!) {
strapiArticle(id: { eq: $id }) {
articleHeader {
articleTitle
articleSnippet
}
articleContent {
__typename
... on ComponentTextArticleCopy {
contentCopy
}
... on ComponentImageContentImg {
imgCaption
}
... on ComponentTextArticleQuote {
contentQuote
}
}
}
}
`
According to the Graphql docs, Inline Fragment would seem to be the right approach but clearly I've got something wrong somewhere.
The following query 'works' on Gatsby but tries to resolve for all components:
query MyQuery {
allStrapiArticle {
edges {
node {
__typename
articleContent {
contentCopy
contentQuote
}
}
}
}
}
{
"data": {
"allStrapiArticle": {
"edges": [
{
"node": {
"__typename": "StrapiArticle",
"articleContent": [
{
"contentCopy": null,
"contentQuote": null
},
{
"contentCopy": "What a great city Gothenburg is. We even took a trip out to the archipelago. ",
"contentQuote": null
},
{
"contentCopy": null,
"contentQuote": null
},
{
"contentCopy": null,
"contentQuote": "You must visit at have fika"
}
]
}
}
]
}
},
Deleting Cache folder and running again worked for me.
I need some help in creating an AggregationDictionary from the following elasticsearch query
GET organisations/_search
{
"size": 0,
"aggs": {
"by_country": {
"nested": {
"path": "country"
},
"aggs": {
"by_country2": {
"filter": {
"bool": {
"must": [
{
"term": {
"country.isDisplayed": "true"
}
}
]
}
},
"aggs": {
"by_country3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
}
}
}
}
}
}
I managed to write this horrible piece of code which I am pretty sure it is wrong, I am totally new to this.
AggregationDictionary aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[] {
new NestedQuery
{
Query = new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Can someone tell me if I am in the correct direction? I am using Nest 6.6.0. Is there any tool that helps with these translations?
What you have so far is pretty solid, but when you try to execute this aggregation with the following call
var searchAsync = await client.SearchAsync<Document>(s => s.Size(0).Aggregations(aggs));
you will get this error
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
}
],
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
},
"status" : 400
}
Checking request which was sent to elasticsearch give us the answer why it happened
{
"aggs": {
"countries_step1": {
"aggs": {
"countries_step2": {
"aggs": {
"countries_step3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
},
"filter": {}
}
},
"nested": {
"path": "country"
}
}
},
"size": 0
}
filter clause is empty, this is because you tried to used nested query but you didn't pass path parameter. We don't need nested query here (as shown in your example query), we can simplify the whole query to
var aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Now we have a valid request sent to elasticsearch.
There are a couple of things we can improve here:
1. Remove unnecessary bool query
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
to
Filter =
new TermQuery
{
Field = "country.isDisplayed",
Value = true
},
2. Replace string field names
Usually, when doing calls from .Net there is some kind of POCO type which is helping us with writing strongly-typed requests to elasticsearch which helps us managing clean code and refactoring. With this, we can change field definition from
"country.displayName.keyword"
to
Infer.Field<Document>(f => f.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
my types definition
public class Document
{
public int Id { get; set; }
[Nested]
public List<Country> Country { get; set; }
}
public class Country
{
public bool IsDisplayed { get; set; }
public string DisplayName { get; set; }
}
3. Consider using a fluent syntax
With NEST you can write queries in two ways: using object initializer syntax (which you did) or with help of fluent syntax. Have a look. Trying to write above query with the fluent syntax you will get something like
var searchResponse = await client.SearchAsync<Document>(s => s
.Size(0)
.Aggregations(a => a.Nested("by_country", n => n
.Path(p => p.Country)
.Aggregations(aa => aa
.Filter("by_country2", f => f
.Filter(q => q
.Term(t => t
.Field(field => field.Country.FirstOrDefault().IsDisplayed)
.Value(true)))
.Aggregations(aaa => aaa
.Terms("by_country3", t => t
.Field(field => field.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
.Size(9999)
)))))));
which I find a little bit easier to follow and write, maybe it will be better for you as well.
As a final note, have a look into docs and check how you can debug your queries.
Hope that helps.
Part of my mapping looks like this:
{
...
INFO_NODO: {
properties: {
CODIGO: {
type: string
}
ESTADO: {
type: string
}
IN_HOME: {
type: string
}
TEXTO: {
type: string
}
ID_NODO: {
type: integer
}
...
}
}
}
I need to make a facet that will return the fields: ID_NODO, TEXTO, IN_HOME, ESTADO, CODIGO, and COUNT to parse it and feed it to my application. The key is that all these fields except COUNT are dependant on the ID_NODO, that is, if the field INFO_NODO is the same the rest of the information is the same... with that being said ideally I would like to make my facet dependent on the whole INFO_NODO field and not its sub-fields.
I found several solutions but I keep either failing to implement them properly or they are just not working. Any thoughts on my weird situation?
EDIT: What I'd need to do is:
{
"facets": {
"FACET_X_NODO": {
"terms": {
"field": "INFO_NODO"
}
}
}
}
I just can't get the syntax in no documentation since INFO_NODO is a subdocument and not a field.
If I understood you correctly, you should be able to do something like this:
{
"query" : {
"match_all" : { }
},
"facets" : {
"info_node_facet" : {
"terms" : {
"script_field" : "_source.INFO_NODO.CODIGO + _source.INFO_NODO.ESTADO",
"size" : 10
}
}
}
}