Set hint for update to use indexes - mongodb-java

As per documentation it is possible to provide a hint to an update.
Now I'm using the java mongo client and mongo collection to do an update.
For this update I cannot find any way to provide a hint which index to use.
I see for the update I'm doing a COLSCAN in the logs, so wanting to provide the hint.
this.collection.updateOne(
or(eq("_id", "someId"), eq("array1.id", "someId")),
and(
addToSet("array1", new Document()),
addToSet("array2", new Document())
)
);
Indexes are available for both _id and array1.id
I found out in the logs the query for this update is using a COLSCAN to find the document.
Anyone who can point me in the right direction?
Using AWS DocumentDB, which is MongoDB v3.6

Lets consider a document with an array of embedded documents:
{ _id: 1, arr: [ { fld1: "x", fld2: 43 }, { fld1: "r", fld2: 80 } ] }
I created an index on arr.fld1; this is a Multikey index (indexes on arrays are called as so). The _id field already has the default unique index.
The following query uses the indexes on both fields - arr.fld1 and the _id. The query plan generated using explain() on the query showed an index scan (IXSCAN) for both fields.
db.test.find( { $or: [ { _id: 2 }, { "arr.fld1": "m" } ] } )
Now the same query filter is used for the update operation also. So, the update where we add two sub-documents to the array:
db.test.update(
{ $or: [ { _id: 1 }, { "arr.fld1": "m" } ] },
{ $addToSet: { arr: { $each: [ { "fld1": "xx" }, { "fld1": "zz" } ] } } }
)
Again, the query plan showed that both the indexes are used for the update operation. Note, I have not used the hint for the find or the update query.
I cannot come to conclusion about what the issue is with your code or indexes (see point Notes: 1, below).
NOTES:
The above observations are based on queries run on a MongoDB server
version 4.0 (valid for version 3.6 also, as I know).
The
explain
method is used as follows for find and update:
db.collection.explain().find( ... ) and
db.collection.explain().update( ... ).
Note that you cannot generate a query plan using explain() for
updateOne method; it is only available for findAndModify() and
update() methods. You can get a list of methods that can generate a
query plan by using the command at mongo shell:
db.collection.explain().help().
Note on Java Code:
The Java code to update an array field with multiple sub-document add, is as follows:
collection.updateOne(
or(eq("_id", new Integer(1)), eq("arr.fld1", "m")),
addEachToSet("arr", Arrays.asList(new Document("fld1", "value-1"), new Document("fld1", "value-2"))
);

Related

$elemMatch with $in SpringData Mongo Query

I am in the process of attempting to create a method that will compose a query using Spring Data and I have a couple of questions. I am trying to perform a query using top level attributes of a document (i.e. the id field) as well as attributes of an subarray.
To do so I am using a query similar to this:
db.getCollection("journeys").find({ "_id._id": "0104", "journeyDates": { $elemMatch: { "period": { $in: [ 1,2 ] } } } })
As you can see I would also like to filter using $in for the values of the subarray. Running the above query though result in wrong results, as if the $elemMatch is ignored completely.
Running a similiar but slightly different query like this:
db.getCollection("journeys").find({ "_id._id": { $in: [ "0104" ] } }, { journeyDates: { $elemMatch: { period: { $in: [ 1, 2 ] } } } })
does seem to yield better results but it returns the only first found element matching the $in of the subarray filter.
Now my question is, how can I query using both top level attributes as well subarrays using $in. Preferably I would like to avoid aggregations. Secondly, how can I translate this native Mongo query to a Spring data Query object?

how to use Elastic Search nested queries by object key instead of object property

Following the Elastic Search example in this article for a nested query, I noticed that it assumes the nested objects are inside an ARRAY and that queries are based on some object PROPERTY:
{
nested_objects: [ <== array
{ name: "x", value: 123 },
{ name: "y", value: 456 } <== "name" property searchable
]
}
But what if I want nested objects to be arranged in key-value structure that gets updated with new objects, and I want to search by the KEY? example:
{
nested_objects: { <== key-value, not array
"x": { value: 123 },
"y": { value: 456 } <== how can I search by "x" and "y" keys?
"..." <=== more arbitrary keys are added now and then
]
}
Thank you!
You can try to do this using the query_string query, like this:
GET my_index/_search
{
"query": {
"query_string": {
"query":"nested_objects.\\*.value:123"
}
}
}
It will try to match the value field of any sub-field of nested_objects.
Ok, so my final solution after some ES insights is as follows:
1. The fact that my object keys "x", "y", ... are arbitrary causes a mess in my index mapping. So generally speaking, it's not a good ES practice to plan this kind of structure... So for the sake of mappings, I resort to the structure described in the "Weighted tags" article:
{ "name":"x", "value":123 },
{ "name":"y", "value":456 },
...
This means that, when it's time to update the value of the sub-object named "x", I'm having a harder (and slower) time finding it: I first need to query the entire top-level object, traverse the sub objects until I find one named "x" and then update its value. Then I update the entire sub-object array back into ES.
The above approach also causes concurrency issues in case I have multiple processes updating the same index. ES has optimistic locking I can use to retry when needed, or, I can queue updates and handle them serially

Increasing 'view' counter of a document in an index everytime it gets queried explicitly using _id via _search endpoint

Say, I have an index called blog which has 10 documents called article. The article is a JSON with one of the property being views which is initialized to 0.
I was wondering if there's a good way of updating the views counter everytime the document gets explicitly called via _search endpoint using document id, so that I can sort it by view on my other queries.
Or would that be something that will have to be taken care of at the application layer?
My feeble attempt query dsl so far:
let options = {
index: 'blog',
body: {
query: {
function_score: {
query: {
match: { _id: req.params.articleID }
},
"weight" : 2
,
score_mode: "sum"
,
script_score : {
script : {
inline: "(2 + doc['view'].value)"
}
}
}
},
}
};
I have been trying inline script but that would require me to send two separate request. First search & then update if found. I was wondering if I could do it on a single query i.e trigger the views counter to increase by one automatically everytime I query via _search.

Mongo with Spring Data - Use Distinct and Orderby in the same query

I am working on some dataset whose json format for one object is given below.
{
_id: ............,
code : G12220,
type : etf,
volume : 13,
modified_time:..................
.
.
.
}
This dataset gets updated very frequently (every 1 minute) and there are few thousands unique codes. I want to write a query to fetch the set of documents for each of the most latest distinct "codes" available. Eg: If there are two documents each having same code the result should be the most latest. I am using Spring Data.
I started writing my query and the given below is a sample.
#Query("{type : ?0}......")
public List<ProductEntities> getLatestProductsSet(String type);
I am not very sure how to write a complex query on this. Would be grateful if you can help me.
Thanks in advance,
You might want to have a look at the following mongo query to achieve this.
db.collection.aggregate([
{
"$group": {
"_id": {
"code": "$code",
"modifiedTime": "$modified_time"
},
"docs": {
"$push": "$$ROOT"
}
}
},
{
"$sort": {
"_id.modifiedTime": -1
}
},
{
"$limit": 1
}
]);
Following are the links which will help you understand the above query.
http://docs.mongodb.org/manual/reference/operator/aggregation/group/
http://docs.mongodb.org/manual/reference/operator/aggregation/sort/
http://docs.mongodb.org/manual/reference/operator/aggregation/push/
http://docs.mongodb.org/manual/reference/operator/aggregation/limit/
Spring part follows.
http://www.mkyong.com/mongodb/spring-data-mongodb-aggregation-grouping-example/
Let me know, if this helps.

NEST: How to query against multiple indices and handle different subclasses (document types)?

I’m playing around with ElasticSearch in combination with NEST in my C# project. My use case includes several indices with different document types which I query separately so far. Now I wanna implement a global search function which queries against all existing indices, document types and score the result properly.
So my question: How do I accomplish that by using NEST?
Currently I’m using the function SetDefaultIndex but how can I define multiple indices?
Maybe for a better understanding, this is the query I wanna realize with NEST:
{
"query": {
"indices": {
"indices": [
"INDEX_A",
"INDEX_B"
],
"query": {
"term": {
"FIELD": "VALUE"
}
},
"no_match_query": {
"term": {
"FIELD": "VALUE"
}
}
}
}
}
TIA
You can explicitly tell NEST to use multiple indices:
client.Search<MyObject>(s=>s
.Indices(new [] {"Index_A", "Index_B"})
...
)
If you want to search across all indices
client.Search<MyObject>(s=>s
.AllIndices()
...
)
Or if you want to search one index (thats not the default index)
client.Search<MyObject>(s=>s.
.Index("Index_A")
...
)
Remember since elasticsearch 19.8 you can also specify wildcards on index names
client.Search<MyObject>(s=>s
.Index("Index_*")
...
)
As for your indices_query
client.Search<MyObject>(s=>s
.AllIndices()
.Query(q=>q
.Indices(i=>i
.Indices(new [] { "INDEX_A", "INDEX_B"})
.Query(iq=>iq.Term("FIELD","VALUE"))
.NoMatchQuery(iq=>iq.Term("FIELD", "VALUE"))
)
)
);
UPDATE
These tests show off how you can make C#'s covariance work for you:
https://github.com/Mpdreamz/NEST/blob/master/src/Nest.Tests.Integration/Search/SubClassSupport/SubClassSupportTests.cs
In your case if all the types are not subclasses of a shared base you can still use 'object'
i.e:
.Search<object>(s=>s
.Types(typeof(Product),typeof(Category),typeof(Manufacturer))
.Query(...)
);
This will search on /yourdefaultindex/products,categories,manufacturers/_search and setup a default ConcreteTypeSelector that understands what type each returned document is.
Using ConcreteTypeSelector(Func<dynamic, Hit<dynamic>, Type>) you can manually return a type based on some json value (on dynamic) or on the hit metadata.

Resources