How can I properly integrate/mask a GraphQL query in PowerQuery?
I have here a GraphQL query that runs perfectly in my GraphQL test environment.
query {
getProductListing(
filter: "{\"$and\":[{\"articleNumber\":{\"$not\":null}},{\"$or\":[{\"productRanges\":{\"$like\":\"%,5007186,%\"}},{\"productRanges\":{\"$like\":\"%,5007190,%\"}},{\"productRanges\":{\"$like\":\"%,5007194,%\"}}]}]}"
defaultLanguage: "de"
) {
edges {
node {
articleNumber
ean
bezeichnung
}
}
}
}
Now I want to integrate this query into M / PowerQuery. I tried this code, but there seems to be a problem with the masking, probably at $ and the reserved words "and" and "or". Or somewhere else entirely?
let
Source = Web.Contents(
"https://www.domain.tld/endpoint",
[
Headers=[
#"Method"="POST",
#"Content-Type"="application/json"
],
Content=Text.ToBinary("{""query"": ""query { getProductListing( filter: \""{\\""$and\\"":[{\\""articleNumber\\"":{\\""$not\\"":null}},{\\""$or\\"":[{\\""productRanges\\"":{\\""$like\\"":\\""%,5007186,%\\""}},{\\""productRanges\\"":{\\""$like\\"":\\""%,5007190,%\\""}},{\\""productRanges\\"":{\\""$like\\"":\\""%,5007194,%\\""}}]}]}\"" defaultLanguage: \""de\"" ) { edges { node { articleNumber ean bezeichnung } } }}""}")
]
),
#"JSON" = Json.Document(Source)
in
#"JSON"
If I omit the filter section, it works in PowerQuery, but unfortunately I need the filter. Does anyone see a solution that I don't?
Related
New to Open Search and couldn't really find an answer that worked for this use case. Essentially, my query uses scripts to access field document values within a multi_term search, then aggregates them into buckets reflecting certain metrics. The bucket key is an array of strings in the format of ['val1', 'val2', 'val3'] with an associated key_as_string of 'val1|val2|val3'
My goal is to be able to sort these buckets after aggregation based on any of these 3 values. Problem is, I can't seem to get sorting to work outside of a root "order" entry that sorts by the entire key (I think). Query is here:
aggregations: {
plans: {
multi_terms: {
size: 10000,
terms: [
{
script: "doc['plan.title.keyword'].value"
},
{
script: "doc['plan.type.keyword'].value"
},
{
script: "doc['plan.id.keyword'].value"
}
],
order: { _key: order } // This orders buckets by entire key?
},
aggregations: {
completed: {
filter: {
term: { 'status.keyword': 'Completed' }
}
},
in_progress: {
filter: {
term: { 'status.keyword': 'Started' }
}
},
stopped: {
filter: {
term: { 'status.keyword': 'Stopped' }
}
},
assigned: {
filter: {
term: { 'status.keyword': 'Assigned' }
}
},
my_bucket: {
bucket_sort: {
sort: [{_key: {order: 'asc'}}] // Breaks sort
}
}
}
}
},
The output of the query is correct, but the order of buckets output is not and I can't seem to get it right. I've attempted various ways of implementing bucket_sort to no avail. Feels like there is an easy solution to this and I'm just not finding it. My end goal is to be able to sort the buckets returned by a specified index of the key.
Can anyone tell me what I'm doing wrong here?
Note: Using Open Search v2.3
I am in the process of attempting to create a method that will compose a query using Spring Data and I have a couple of questions. I am trying to perform a query using top level attributes of a document (i.e. the id field) as well as attributes of an subarray.
To do so I am using a query similar to this:
db.getCollection("journeys").find({ "_id._id": "0104", "journeyDates": { $elemMatch: { "period": { $in: [ 1,2 ] } } } })
As you can see I would also like to filter using $in for the values of the subarray. Running the above query though result in wrong results, as if the $elemMatch is ignored completely.
Running a similiar but slightly different query like this:
db.getCollection("journeys").find({ "_id._id": { $in: [ "0104" ] } }, { journeyDates: { $elemMatch: { period: { $in: [ 1, 2 ] } } } })
does seem to yield better results but it returns the only first found element matching the $in of the subarray filter.
Now my question is, how can I query using both top level attributes as well subarrays using $in. Preferably I would like to avoid aggregations. Secondly, how can I translate this native Mongo query to a Spring data Query object?
I'm using Contentful's GraphQL API. What I want to do is to query all the events that haven't past yet.
I tried using lt, but that doesn't seem to be working. I also found out that the date is a string, so what options do I have?
eventCollection(where: {eventEndDate: {lt: "2022-10-27T00:00:00.000-06:00"}}){
items {
slug
eventEndDate
}
}
A normal query (without the where condition) gives you:
"eventCollection": {
"items": [
{
"slug": "black-friday",
"eventEndDate": "2022-11-27T12:00:00.000-07:00"
}
]
}
You should have an eventEndDate_gte filter available. On every field, there will be type dependent filter available. It's best to use GraphiQL or the GraphQL Playground to discover available filter options.
The following filter works fine for my space.
query {
tilPostCollection(where: {date_gte: "2022-09-05T00:00:00.000+02:00"}) {
items {
title
date
}
}
}
If I have a Documents defined as
[
{
"title": "title",
"tags": [ "cool", "amazing", "funny" ]
},
{
"title": "another title",
"tags": [ "nice", "amazing", "funny" ]
}
]
I'd like to be able to query with MongoTemplate in order to pass a list of values like ["cool","amazing"] and have in return the first collection above, not the second.
For what I mean to achieve, the $in condition doesn't seems enough.
I tried with the $all condition and from the Mongo console it works as I need, but in my code something isn't working and it takes forever for my query to elaborate. With the $in operator my code goes fast, instead.
My method in my repository (for other reasons, I have to do an aggregation as below):
public Page<MyDocument> findByProperties(String title, List<ObjectId> tags, Pageable page) {
final List<Criteria> criteria = new ArrayList<>();
Aggregation aggregation = null;
if (title != null && !title.isEmpty()) {
criteria.add(Criteria.where("title").is(title));
}
if (tags != null && !tags.isEmpty()) {
criteria.add(Criteria.where("MyBook.tags").all(tags));
}
if (!criteria.isEmpty()) {
aggregation = Aggregation.newAggregation(
Aggregation.lookup("from_collection", "_id", "idParent", "MyBook"),
Aggregation.unwind("MyBook"),
Aggregation.match(new Criteria().andOperator(criteria.toArray(new Criteria[0]))),
Aggregation.skip(page.getOffset()),
Aggregation.limit(page.getPageSize()),
Aggregation.sort(page.getSort())
);
} else {
aggregation = Aggregation.newAggregation(
Aggregation.lookup("from_collection", "_id", "idParent", "MyBook"),
Aggregation.unwind("MyBook"),
Aggregation.skip(page.getOffset()),
Aggregation.limit(page.getPageSize()),
Aggregation.sort(page.getSort())
);
}
List<MyDocument> results = mongoTemplate.aggregate(aggregation, "my_document", MyDocument.class).getMappedResults();
return PageableExecutionUtils.getPage(results, page,
() -> (long)results.size());
}
Looking at this answer, I tried with
criteria.add(Criteria.where("MyBook.tags").in(tags).all(tags));
But nothing changed, the query takes forever and not with the expected output.
Any ideas please? Thank you!
To find if tags array field contains all the members of youe array, your can use the bellow, if you need it to be in aggregation pipeline use the second.
In your query you have more stages, and more code, if you want to ask more if you can give example data and expected output in JSON, and what you want to do, so people can help.
Query1
find using $all operator
Test code here
db.collection.find({"tags": {"$all": ["cool","amazing"]}})
Query2
aggregation solution with set difference
Test code here
aggregate(
[{"$match":
{"$expr":
{"$eq": [{"$setDifference": [["cool", "amazing"], "$tags"]}, []]}}}])
I'm querying for states using the state code as the query string, and "in" and "or" (Indiana and Oregon) are failing, presumably because they're reserved words.
I can confirm that the data exists in the index correctly, because when I run:
curl -XGET 'localhost:9200/state/_search?size=200&pretty=true' -d '{"query" : {"match_all" : {}}}' > out.txt
I can see the data there for both the working states and the non-working states. Plus, if I change the state code of a non-working state in CouchDB to something like XYZ, I can verify that the change makes it to ES by running the above command and searching for XYZ. So I know I'm looking at the right data and it's indexing fine.
The problem is the query. Right now, here's what my entire query object looks like:
var q = {
size: 0,
query: {
filtered: {
query: { term: { postcode: 'tn' } },
filter: { term: { version: 2 } }
}
},
facets: {
version: { terms: { field: "version" } },
count : { statistical : { field : "latestValues.enroll" } }
}
};
If I run that query, I get no results. If I change the "or" out with "tn" or "tx" or "sc" etc., then it works fine.
I looked for a way to escape reserved words and found this link but it doesn't seem to work for me, when running the following query:
var q = {
size: 0,
query: {
filtered: {
query: { match_all: { } },
filter: { term: { version: 2, postcode: 'or' } }
}
},
facets: {
version: { terms: { field: "version" } },
count : { statistical : { field : "latestValues.enroll" } }
}
};
(Note that that query also works when changing out "or" with a non-reserved-word-state so I know it's not a problem with the query itself).
Any ideas?
This is not about "reserved" words, its about stop words. You are using an analyzer which removes stop words (the default analyzer up to a more recent version of Elasticsearch).
You'll need to change the analyzer for the field, see here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis.html
This will change require reindexing, though