I am writing an application using graphql, spring and mongodb. Here i want to perform some filter on db document using graphql.
Below is the representation of single document from mongo collection:
{
"data" : {
"attribute1" : [
{
"key":"val1"
"region":"US"
},{
"key":"val2"
"region":"UK"
}
]
"attribute2" : [
{
"nestedAttribute1": [
{
"key":"val3"
"region":"US"
},{
"key":"val4"
"region":"UK"
}
]
}
]
}
}
I want to filter the document, using GraphQL, with all the data where 'region' is 'US'.
Expected output:
{
"data" : {
"attribute1" : [
{
"key":"val1"
"region":"US"
}
],
"attribute2" : [
{
"nestedAttribute1": [
{
"key":"val3"
"region":"US"
}
]
}
]
}
}
Is it possible to write a generic query across all the attributes?
If not, are there any alternatives to GraphQL which support this filtering.
Here is the schema that is exact representation for mongo collection
https://pastebin.com/JxEPEGhp
Related
Mongodb states that it is possible to put multiple conditions inside cond of filter. Like this:
{
$filter: {
input: [ 1, "a", 2, null, 3.1, NumberLong(4), "5" ],
as: "num",
cond: { $and: [
{ $gte: [ "$$num", NumberLong("-9223372036854775807") ] },
{ $lte: [ "$$num", NumberLong("9223372036854775807") ] }
] }
}
}
Source: - https://docs.mongodb.com/manual/reference/operator/aggregation/filter/
I would like to do the same inside spring data mongodb. I have this aggregation:
val aggregation = Aggregation.newAggregation(
project()
.and(
filter("some")
.`as`("some")
.by(
ComparisonOperators.Lte.valueOf("some.field").lessThanEqualToValue(2000)
/* here i would like to put "and" like .and(valueOf("some.field").greaterThanValue(1000) but it does not exists on returned value of valueOf*/
)
)
)
How can i do this ? I was looking for some "and" function on the level of "by" or "filter" but it does not exists. Adding another and will override my previous value (if for example i would like to filter using range of some.field or by multiple fields). How do i do this for multiple fields ?
I'm new in mongodb and spring, i need to convert the following query to java code but i didn't find how to do it.
db.collection.aggregate([
{
$project: {
members: {
$concatArrays: [
[
{
"userID": "$userID",
"userType": "$userType"
}
],
{
$reduce: {
input: "$clients",
initialValue: [],
in: {
$concatArrays: [
"$$value",
[
{
userID: "$$this.userID",
userType: "$$this.userType"
}
],
"$$this.members"
]
}
}
}
]
}
}
},
{
$unwind: "$members"
},
{
$replaceRoot: {
newRoot: "$members"
}
}
])
I m stack in the $project part, i didn't find how to implement it in spring.
Can someone help me?
For use cases, where you find it hard to write a query in spring MongoDB java format, You can use the JSON/JavaScript code directly like this:
String jsonExpression = "{\"members\":{\"$concatArrays\":[[{\"userID\":\"$userID\",\"userType\":\"$userType\"}],{\"$reduce\":{\"input\":\"$clients\",\"initialValue\":[],\"in\":{\"$concatArrays\":[\"$$value\",[{\"userID\":\"$$this.userID\",\"userType\":\"$$this.userType\"}],\"$$this.members\"]}}}]}}";
AggregationOperation project = Aggregation.project().and(context -> context.getMappedObject(Document.parse(jsonExpression))).as("difference");
You can refer to my other answer here : Difference between "now" and a given date
I am developing a webapp, for which I am pushing my search templates to ES during startup and using them to form the elastic search queries at runtime. I have a requirement wherein, I don't know the number of filters to be applied. Created a search template like -
{
"filters" : {
{{#toJson}}
clauses
{{/toJson}}"
}
}
And search will be made like this -
GET _search/template
{
"id": "template-id",
"params": {
"clauses": {
"filters" : {
{ "match": { "user" : "foo" } },
{ "match": { "user" : "bar" } }
}
}
}
which will render result as -
{
"filters":{
"filters":{
"match" : {
"user" : "foo"
}
},
{
"match" : {
"user" : "bar"
}
}
}
}
as suggested by ES documentation-
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html
But, since it's an invalid JSON, it doesn't allow me to push the template to ES.
My template works well when I use it as stored template in elastic-home/config/scripts. But I want to manage my templates with JAVA and push all templates during startup only.
Can I get any help?
I need a query that makes partial match on a string and filter outside documents that have a specific value for a field.
I tried this payload for es:
payload = {
search_request: {
_source: [ 'name', 'source','pg_id' ],
query: {
match: { name: query_string }
bool: {
must_not: {
term: { "source.source": source_value }
}
}
},
size: 100
},
query_hint: query,
algorithm: algorithm,
field_mapping: { title: ["_source.name", "_source.source"]}
}
But ES trows this error:
{
:error=> {
:root_cause=> [
{
:type=>"parse_exception",
:reason=>
"failed to parse search source. expected field name but got [
START_OBJECT
] "}],
:type=>" search_phase_execution_exception",
:reason=>"all shards failed",
:phase=>"query",
:grouped=>true,
:failed_shards=> [
{
:shard=>0,
:index=>"articles",
:node=>"3BUP3eN_TB2-zExigd_k2g",
:reason=> {
:type=>"parse_exception",
:reason=>
"failed to parse search source. expected field name but got [
START_OBJECT
] "
}
}
]
},
:status=>400
}
I am using Elasticsearch 2.4
First of all your json format is not valid. Check for a commas and quotes.
Also if you need just to filtrate documents - filters are much faster than queries. Check documentation
After asking question to understand a bit more of the aggregation framework in MongoDB I finally found the way to do aggregation for my need (thanks to a StackExchange user)
So basically here is a document from my collection:
{
"_id" : ObjectId("s4dcsd5s4d6c54s6d"),
"items" : [
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_3",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
}
]
}
The idea was to be able to filter only some elements of my collections (avoiding Type 2 and 3). In fact I have more than 30 types and 6 are not allowed but for simplicity I made this example.
So the aggregation command in command line is this one:
db.history.aggregate([{
$match: {
_id: ObjectId("s4dcsd5s4d6c54s6d")
}
}, {
$unwind: '$items'
}, {
$match: {
'items.type': { '$nin': [ "TYPE_2" , "TYPE_3"] }
}
},
{ $limit: 10 }
]);
With this I am able to retrieve the 10 elements items of this document which do not match TYPE_2 and TYPE_3
However when I am using spring data there is no output. I looked a bit at the example to build mine but its still not working.
So I did:
Aggregation aggregation = newAggregation(
match(Criteria.where("id").is(myID)),
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
AggregationResults<PersonnalHistory> results = mongAccess.getOperation().aggregate(query,
"items", PersonnalHistory.class);
PersonnalHistory is marked with annotation #Document(collection = "history") and id with the #id annotation
ignoreditemstype is a list containing TYPE_2 and TYPE_3
Here is what I have in the toString method of aggregation:
{
"aggregate" : "__collection__" ,
"pipeline" : [
{ "$match": { "id" : "s4dcsd5s4d6c54s6d"} },
{ "$unwind": "$items"},
{ "$match": { "items.type": { "$nin" : [ "TYPE_2" , "TYPE_3" ] } } },
{ "$limit" : 3},
{ "$skip" : 0 }
]
}
I tried a lot of stuff (to have at least an answer :) ) like removing id or the nin:
aggregation = newAggregation(
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
aggregation = newAggregation(
match(Criteria.where("id").is(myid)),
unwind("items")
);
For information when I do a simple query like:
query.addCriteria(Criteria.where("id").is(myID));
My document is returned. However I have thousands of items. So I just want to have the 15 first (in fact the 15 first are the 15 last added)
Do you maybe see what I am doing wrong?
Yeah looks like you are passing simple String while it is expecting ObjectId
Aggregation aggregation = newAggregation(
match(Criteria.where("_id").is(new ObjectId(myID))),
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
Now the question is why it works with simple query, my answer would be because spring-data driver is not that mature at least not with aggregation pipeline.