MongoDB - group inside group && group and project - laravel

I'm a newer and studying MongoDB with Laravel.
This is my problem.
I need to get records in 60 days and group them by type.
In each type group, I have to group them by range time:
60 - 30 days ago
30 days ago - now
I researched in 2 days and these are my codes so far.
$collection->aggregate([
[
'$match' => $match,
],
[
'$project' => [
'range' => [
'$concat' => [
[
'$cond' => [
[
'$and' => [
['$gte' => ['$created_at', $ago_60_days]],
['$lte' => ['$created_at', $ago_30_days]],
],
],
'before',
'',
],
],
[
'$cond' => [
[
'$and' => [
['$gt' => ['$created_at', $ago_30_days]],
['$lte' => ['$created_at', $now]],
],
],
'current',
'',
],
],
],
],
],
],
[
'$group' => [
'_id' => '$type,
'total' => ['$sum' => 1],
],
],
... anything after ???
]);
Please let me know anything, any clues could help me.
I can handle with mongo shell.
Thank you guys so much.

The query:
db.test.aggregate( [
{
$match: {
$expr: {
$gte: [ "$dt", { $subtract: [ ISODate(), 60*24*60*60*1000 ] } ]
}
}
},
{
$addFields: {
rangeTime: {
$cond: {
if: { $gte: [ "$dt", { $subtract: [ ISODate(), 30*24*60*60*1000 ] } ] },
then: "range_30_now",
else: "range_60_30"
}
}
}
},
{
$group: {
_id: {
t: "$type", r: "$rangeTime"
},
total: { $sum: 1 }
}
},
{
$project: {
type: "$_id.t",
rangeTime: "$_id.r",
total: 1,
_id: 0
}
},
{
$sort: {
type: 1,
rangeTime: -1
}
}
] )
The Output:
{ "total" : 2, "type" : "A", "rangeTime" : "range_60_30" }
{ "total" : 1, "type" : "B", "rangeTime" : "range_60_30" }
{ "total" : 2, "type" : "B", "rangeTime" : "range_30_now" }
The Input Documents:
{ "_id" : 1, "dt" : ISODate("2019-10-15T00:00:00Z"), "type" : "A" }
{ "_id" : 2, "dt" : ISODate("2019-10-20T00:00:00Z"), "type" : "B" }
{ "_id" : 3, "dt" : ISODate("2019-11-08T00:00:00Z"), "type" : "A" }
{ "_id" : 4, "dt" : ISODate("2019-11-29T00:00:00Z"), "type" : "B" }
{ "_id" : 5, "dt" : ISODate("2019-11-15T00:00:00Z"), "type" : "A" }
{ "_id" : 9, "dt" : ISODate("2019-12-15T00:00:00Z"), "type" : "B" }

Related

How to fetch field from array of objects in Elasticsearch Index as CSV file to Google Cloud Storage Using Logstash

I am using ElasticSearch to index data and wanted to export few fields from index created every day to Google cloud storage, How to get fields from array of objects in elastic search index and send them as csv file to GCS bucket using Logstash
Tried below conf to fetch nested fields from index:
input {
elasticsearch {
hosts => "host:443"
user => "user"
ssl => true
connect_timeout_seconds => 600
request_timeout_seconds => 600
password => "pwd"
ca_file => "ca.crt"
index => "test"
query => '
{
"_source": ["obj1.Name","obj1.addr","obj1.obj2.location", "Hierarchy.categoryUrl"],
"query": {
"match_all": {}
}
}
'
}
}
filter {
mutate {
rename => {
"[obj1][Name]" => "col1"
"[obj1][addr]" => "col2"
"[obj1][obj2][location]" => "col3"
"[Hierarchy][0][categoryUrl]" => "col4"
}
}
}
output {
google_cloud_storage {
codec => csv {
include_headers => true
columns => [ "col1", "col2","col3"]
}
bucket => "bucket"
json_key_file => "creds.json"
temp_directory => "/tmp"
log_file_prefix => "log_gcs"
max_file_size_kbytes => 1024
date_pattern => "%Y-%m-%dT%H:00"
flush_interval_secs => 600
gzip => false
uploader_interval_secs => 600
include_uuid => true
include_hostname => true
}
}
How to get field populated to above csv from array of objects, in below example wanted to fetch categoryUrl from the first object of an array and populate to csv table and send it to GCS Bucket:
have tried below approaches :
"_source": ["obj1.Name","obj1.addr","obj1.obj2.location", "Hierarchy.categoryUrl"]
and
"_source": ["obj1.Name","obj1.addr","obj1.obj2.location", "Hierarchy[0].categoryUrl"]
with
mutate {
rename => {
"[obj1][Name]" => "col1"
"[obj1][addr]" => "col2"
"[obj1][obj2][location]" => "col3"
"[Hierarchy][0][categoryUrl]" => "col4"
}
for input sample :
"Hierarchy" : [
{
"level" : "1",
"category" : "test",
"categoryUrl" : "testurl1"
},
{
"level" : "2",
"category" : "test2",
"categoryUrl" : "testurl2"
}}
Attaching sample document where I am trying to fetch merchandisingHierarchy[0].categoryUrl and pricingInfo[0].basePrice :
{
"_index" : "amulya-test",
"_type" : "_doc",
"_id" : "ldZPJoYBFi8LOEDK_M2f",
"_score" : 1.0,
"_ignored" : [
"itemDetails.description.keyword"
],
"_source" : {
"itemDetails" : {
"compSku" : "202726",
"compName" : "abc.com",
"compWebsite" : "abc.com",
"title" : "Monteray 38.25 in. x 73.375 in. Frameless Hinged Corner Shower Enclosure in Brushed Nickel",
"description" : "Create the modthroom of your dreams with the clean lines of the VIGO Monteray Frameless Shower Enclosure. Solid 3/8 in. tempered glass combined with stainless steel and solid brass construction makes this enclosure strong and long-lasting. The sleek, reversible, outward-opening door features a convenient towel bar. This versatile enclosure can be installed on a tile floor or with a VIGO Shower Base. With a single water deflector along the bottom seal strip, water is redirected back into the shower to keep your bathroom dry, clean, and pristine.",
"modelNumber" : "VG6011BNCL40",
"upc" : "8137756684",
"hasVariations" : false,
"productDetailsBulletPoints" : [ ],
"itemUrls" : {
"productPageUrl" : "https://.abc.com/p/VIGO-Monteray-38-in-x-73-375-in-Frameless-Hinged-Corner-Shower-Enclosure-in-Brushed-Nickel-VG6011BNCL40/202722616",
"primaryImageUrl" : "https://images.thdstatic.com/productImages/d77d9e8b-1ea1-4811-a470-8364c8e47402/svn/vigo-shower-enclosures-vg6011bncl40-64_600.jpg",
"secondaryImageUrls" : [
"https://images.thdstatic.com/productImages/d77d9e8b-1e1-4811-a470-8364c8e47402/svn/vigo-shower-enclosures-vg6011bncl40-64_1000.jpg",
"https://images.thdstatic.com/productImages/db539ff9-6df-48c2-897a-18dd1e1794e3/svn/vigo-shower-enclosures-vg6011bncl40-e1_1000.jpg",
"https://images.thdstatic.com/productImages/47c5090b-49a-46bc-a36d-921ddae5e1ab/svn/vigo-shower-enclosures-vg6011bncl40-40_1000.jpg",
"https://images.thdstatic.com/productImages/add6691c-a02-466d-9a1a-47200b05685e/svn/vigo-shower-enclosures-vg6011bncl40-a0_1000.jpg",
"https://images.thdstatic.com/productImages/d638230e-0d9-40c9-be93-7f7bf24f0732/svn/vigo-shower-enclosures-vg6011bncl40-1d_1000.jpg"
]
}
},
"merchandisingHierarchy" : [
{
"level" : "1",
"category" : "Home",
"categoryUrl" : "host/"
},
{
"level" : "2",
"category" : "Bath",
"categoryUrl" : "host/b/Bath/N-5yc1vZbzb3"
},
{
"level" : "3",
"category" : "Showers",
"categoryUrl" : "host/b/Bath-Showers/N-5yc1vZbzcd"
},
{
"level" : "4",
"category" : "Shower Doors",
"categoryUrl" : "host/b/Bath-Showers-Shower-Doors/N-5yc1vZbzcg"
},
{
"level" : "5",
"category" : "Shower Enclosures",
"categoryUrl" : "host/b/Bath-Showers-Shower-Doors-Shower-Enclosures/N-5yc1vZcbn2"
}
],
"reviewsAndRatings" : {
"pdtReviewCount" : 105
},
"additionalAttributes" : {
"isAddon" : false
},
"productSpecifications" : {
"Warranties" : { },
"Details" : { },
"Dimensions" : { }
},
"promoDetails" : [
{
"promoName" : "Save $150.00 (15%)",
"promoPrice" : 849.9
}
],
"locationDetails" : { },
"storePickupDetails" : {
"deliveryText" : "Get it by Mon, Feb 20",
"toEddDate" : "Mon, Feb 20",
"isBackordered" : false,
"selectedEddZipcode" : "20147",
"shipToStoreEnabled" : true,
"homeDeliveryEnabled" : true,
"scheduledDeliveryEnabled" : false
},
"recommendedProducts" : [ ],
"pricingInfo" : [
{
"type" : "SAS",
"offerPrice" : 849.9,
"sellerName" : "abc.com",
"onClearance" : false,
"inStock" : true,
"isBuyBoxWinner" : true,
"promo" : [
{
"onPromo" : true,
"promoName" : "Save $150.00 (15%)",
"promoPrice" : 849.9
}
],
"basePrice" : 999.9,
"priceVariants" : [
{
"basePrice" : 999.9,
"offerPrice" : 849.9
}
],
"inventoryDetails" : {
"stockInStore" : false,
"stockOnline" : true
}
}
]
}
}
You can do it like this:
input {
elasticsearch {
...
query => '
{
"_source": ["merchandisingHierarchy.categoryUrl"],
"query": {
"match_all": {}
}
}
'
}
}
filter {
mutate {
add_field => {
"col1" => "%{[merchandisingHierarchy][0][categoryUrl]}"
"col2" => "%{[pricingInfo][0][basePrice]}"
}
}
}
output {
stdout {
codec => csv {
include_headers => true
columns => [ "col1"]
}
}
}
I've tested with your sample document and I get the output below, which looks like is working per your expectation:
col1,col2
host/,999.9

Step function cloud formation issue with Fn::Sub when passing list as second parameter

I am trying to create step function using cloud formation. I want to pass the lambda arns as second argument to Fn::Sub function. It works if I pass just one Arn but fails when I pass multiple. (with Fn::Get). I checked the template with Yml validator and did not see any issues.
Cloud formation template definition for Step:
---
Resources:
ContractDraftStateMachine:
Type: "AWS::StepFunctions::StateMachine"
Properties:
RoleArn:
Fn::GetAtt: [ StepFunctionExecutionRole, Arn ]
DefinitionString:
Fn::Sub:
- |-
{
"Comment" : "Sample draft process",
"StartAt" : "AdvanceWorkflowToDraftInProgress",
"States" : {
"AdvanceWorkflowToDraftInProgress" : {
"Type" : "Task",
"Resource": "${WorkflowStateChangeLambdaArn}",
"InputPath":"$.contractId",
"OutputPath":"$",
"ResultPath":null,
"Next" : "CheckQuestionnaireType",
"Retry" : [
{
"ErrorEquals" : ["States.TaskTimeout"],
"MaxAttempts": 5,
"IntervalSeconds": 1
},
{
"ErrorEquals" : ["CustomErrorA"],
"MaxAttempts": 5
}
],
"Catch": [
{
"ErrorEquals": [ "States.ALL" ],
"Next": "FailureNotifier"
}
]
},
"CheckQuestionnaireType" : {
"Type" : "Choice",
"Choices" : [
{
"Variable" : "$.questionnaireType",
"StringEquals" : "CE",
"Next" : "PublishQuestionnaireAnswersToCE"
},
{
"Variable" : "$.questionnaireType",
"StringEquals" : "LEAF",
"Next" : "PublishQuestionnaireAnswersToLeaf"
}
]
},
"PublishQuestionnaireAnswersToCE" : {
"Type" : "Task",
"Resource": "${WorkflowStateChangeLambdaArn}",
"Next" : "UpdateCEMetadataAndGenerateDocuments",
"ResultPath" : null,
"OutputPath" : "$",
"Retry" : [
{
"ErrorEquals" : ["States.TaskTimeout"],
"MaxAttempts": 5,
"IntervalSeconds": 1
},
{
"ErrorEquals" : ["CustomErrorA"],
"MaxAttempts": 5
}
],
"Catch": [
{
"ErrorEquals": [ "States.ALL" ],
"Next": "FailureNotifier"
}
]
},
"PublishQuestionnaireAnswersToLeaflet" : {
"Type" : "Task",
"Resource": "${WorkflowStateChangeLambdaArn}",
"End" : true,
"Retry" : [
{
"ErrorEquals" : ["States.TaskTimeout"],
"MaxAttempts": 5,
"IntervalSeconds": 1
},
{
"ErrorEquals" : ["CustomErrorA"],
"MaxAttempts": 5
}
],
"Catch": [
{
"ErrorEquals": [ "States.ALL" ],
"Next": "FailureNotifier"
}
]
},
"UpdateCEMetadataAndGenerateDocuments" : {
"Type" : "Task",
"Resource": "${WorkflowStateChangeLambdaArn}",
"End" : true,
"Retry" : [
{
"ErrorEquals" : ["States.TaskTimeout"],
"MaxAttempts": 5,
"IntervalSeconds": 1
},
{
"ErrorEquals" : ["CustomErrorA"],
"MaxAttempts": 5
}
],
"Catch": [
{
"ErrorEquals": [ "States.ALL" ],
"Next": "FailureNotifier"
}
]
},
"FailureNotifier" : {
"Type" : "Task",
"Resource": "${FailureNotifierLambdaArn}",
"End" : true,
"Retry" : [
{
"ErrorEquals" : ["States.TaskTimeout"],
"MaxAttempts": 5,
"IntervalSeconds": 1
},
{
"ErrorEquals" : ["CustomErrorA"],
"MaxAttempts": 5
}
]
}
}
}
- WorkflowStateChangeLambdaArn:
Fn::GetAtt: [ CreateContractFromQuestionnaireFunction, Arn ]
- FailureNotifierLambdaArn:
Fn::GetAtt: [ CreateContractFromQuestionnaireFunction, Arn ]
Error - Template error: One or more Fn::Sub intrinsic functions don't specify expected arguments. Specify a string as first argument, and an optional second argument to specify a mapping of values to replace in the string
This is just a sample with same lambda used multiple times but the problem is in passing list/map to Fn::Sub.
Could anyone help me resolve this issue or provide an alternate solution to achieve the same?
Thanks,
Fn::Sub takes either a single string as a parameter or a list. When using the list method there should be just two elemenets in the list. The first element is a string (the template) and the second is a map.
From the Fn::Sub documentation
Fn::Sub:
- String
- { Var1Name: Var1Value, Var2Name: Var2Value }
Note: since you are just using Fn::Get attribute to build the substitution value you can just use ${CreateContractFromQuestionnaireFunction.Arn} and use the single string version of Fn::Sub.
E.g. (I've shortened the step function for clarity.
Fn::Sub:|-
{
"Comment" : "Sample draft process",
"StartAt" : "AdvanceWorkflowToDraftInProgress",
"States" : {
"AdvanceWorkflowToDraftInProgress" : {
"Type" : "Task",
"Resource": "${CreateContractFromQuestionnaireFunction.Arn}",
"InputPath":"$.contractId",
"OutputPath":"$",
"ResultPath":null,
"Next" : "CheckQuestionnaireType",
"Retry" : [
...

Spring Data MongoDB building dynamic query

Need help to build dynamic MongoDB query.
everything inside the "$or" Array is dynamic.
db.group.find({
"version" : NumberLong(0),
"$or" : [{
"$and" : [
{
"object_type" : "D"
},
{
"type" : "R"
},
{
"name" : "1"
}
]
},{
"$and" : [
{
"object_type" : "D"
},
{
"type" : "E"
},
{
"name" : "2"
}
]
]
});
Did the below spring data query but doesn't work
Criteria criteria = Criteria.where("version").is("123");
List<Criteria> docCriterias = new ArrayList<Criteria>();
groups.stream().forEach(grp -> {
docCriterias.add(Criteria.where("type").is(grp.get("type").toString())
.andOperator(Criteria.where("object_type").is(grp.get("objectType").toString()))
.andOperator(Criteria.where("name").is(grp.get("name").toString())));
});
criteria.orOperator((Criteria[]) docCriterias.toArray());
Query q = new Query(criteria);
Thanks for the help
You should pay attention to how you combine the operators.
The ff code should work for you (note this is groovy remember to change the closure into to java lambda expression):
List<Criteria> docCriterias = new ArrayList<Criteria>();
List groups = [
[
type: "type1",
object_type: "object_type1",
name: "name1"
],
[
type: "type2",
object_type: "object_type2",
name: "name2"
],
[
type: "type3",
object_type: "object_type3",
name: "name3"
],
]
groups.stream().each {grp ->
docCriterias.add(new Criteria().andOperator(
Criteria.where("type").is(grp.get("type")),
Criteria.where("object_type").is(grp.get("object_type")),
Criteria.where("name").is(grp.get("name"))
))
};
Criteria criteria = new Criteria().andOperator(
Criteria.where("version").is("123"),
new Criteria().orOperator(docCriterias.toArray(new Criteria[docCriterias.size()]))
);
Query q = new Query(criteria);
Which will give you this query:
{
"$and":[
{
"version":"123"
},
{
"$or":[
{
"$and":[
{
"type":"type1"
},
{
"object_type":"object_type1"
},
{
"name":"name1"
}
]
},
{
"$and":[
{
"type":"type2"
},
{
"object_type":"object_type2"
},
{
"name":"name2"
}
]
},
{
"$and":[
{
"type":"type3"
},
{
"object_type":"object_type3"
},
{
"name":"name3"
}
]
}
]
}
]
},
Fields:{
},
Sort:{
}
You could reach this using MongoDB Aggregation Pipeline in Json and Apache Velocity to customize more the Query, then execute this using db.runCommand using Spring MongoTemplate.
Example:
monodb_client_dynamic_query.vm
{
"aggregate": "client",
"pipeline": [
{
"$match" : {
"$and" : [
{
"is_removed" : {
"$ne" : [
true
]
}
},
{
"errors" : {
"$size" : 0.0
}
},
{
"client_id": "$velocityMap.client_id"
}
]
}
},
{
"$project" : {
"_id" : -1.0,
"account" : "$_id.account",
"person_id" : "$_id.person_id",
"begin_date": { $dateToString: { format: "%Y-%m-%d", date: "$value.begin_date" } },
"end_date": { $dateToString: { format: "%Y-%m-%d", date: "$value.end_date" } }
}
}
]
}
Then execute using MondoTemplate:
String script = ...load from file the script monodb_client_dynamic_query.vm
Map parameters = ... put all variables to replace in the mongodb script
String scriptNoSql = VelocityUtil.loadTemplateVM(script, parameters);
DBObject dbObject = (BasicDBObject) JSON.parse(scriptNoSql);
if (null == dbObject) {
return;
}
DB db = mongoTemplate.getDb();
CommandResult result = db.command(dbObject);
if(!result.ok()) {
throw result.getException();
}

MongoDB scans entire index when using $all and $elemMatch

I have a collection of user documents, where each user can have an arbitrary set of properties. Each user is associated to an app document. Here is an example user:
{
"appId": "XXXXXXX",
"properties": [
{ "name": "age", "value": 30 },
{ "name": "gender", "value": "female" },
{ "name": "alive", "value": true }
]
}
I would like to be able to find/count users based on the values of their properties. For example, find me all users for app X that have property Y > 10 and Z equals true.
I have a compound, multikey index on this collection db.users.ensureIndex({ "appId": 1, "properties.name": 1, "properties.value": 1}). This index is working well for single condition queries, ex:
db.users.find({
appId: 'XXXXXX',
properties: {
$elemMatch: {
name: 'age',
value: {
$gt: 10
}
}
}
})
The above query completes in < 300ms with a collection of 1M users. However, when I try and add a second condition, the performance degrades considerably (7-8s), and the explain() output indicates that the whole index is being scanned to fulfill the query ("nscanned" : 2752228).
Query
db.users.find({
appId: 'XXXXXX',
properties: {
$all: [
{
$elemMatch: {
name: 'age',
value: {
$gt: 10
}
}
},
{
$elemMatch: {
name: 'alive',
value: true
}
}
]
}
})
Explain
{
"cursor" : "BtreeCursor appId_1_properties.name_1_properties.value_1",
"isMultiKey" : true,
"n" : 256,
"nscannedObjects" : 1000000,
"nscanned" : 2752228,
"nscannedObjectsAllPlans" : 1018802,
"nscannedAllPlans" : 2771030,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 21648,
"nChunkSkips" : 0,
"millis" : 7425,
"indexBounds" : {
"appId" : [
[
"XXXXX",
"XXXXX"
]
],
"properties.name" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
],
"properties.value" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"filterSet" : false
}
I assume this is because Mongo is unable to create suitable bounds since I am looking for both boolean and integer values.
My question is this: Is there a better way to structure my data, or modify my query to improve performance and take better advantage of my index? Is it possible to instruct mongo to treat each condition separately, generate appropriate bounds, and then perform the intersection of the results, instead of scanning all documents? Or is mongo just not suited for this type of use case?
I know this is an old question, but I think it would be much better to structure your data without the "name" and "value" tags:
{
"appId": "XXXXXXX",
"properties": [
{ "age": 30 },
{ "gender: "female" },
{ "alive": true }
]
}

Why is my mongo query not using index only?

Please, observe:
MongoDB shell version: 2.4.1
connecting to: test
> use dummy
switched to db dummy
> db.invoices.find({'items.nameTags': /^z/}, {_id: 1}).explain()
{
"cursor" : "BtreeCursor items.nameTags_1_created_1_special_1__id_1_items.qty_1_items.total_1 multi",
"isMultiKey" : true,
"n" : 55849,
"nscannedObjects" : 223568,
"nscanned" : 223568,
"nscannedObjectsAllPlans" : 223568,
"nscannedAllPlans" : 223568,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 86,
"nChunkSkips" : 0,
"millis" : 88864,
"indexBounds" : {
"items.nameTags" : [
[
"z",
"{"
],
[
/^z/,
/^z/
]
],
"created" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
],
"special" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
],
"_id" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
],
"items.qty" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
],
"items.total" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : "IL-Mark-LT:27017"
}
>
Here is the definition of the index:
> db.system.indexes.find({name : 'items.nameTags_1_created_1_special_1__id_1_items.qty_1_items.total_1'}).pretty()
{
"v" : 1,
"key" : {
"items.nameTags" : 1,
"created" : 1,
"special" : 1,
"_id" : 1,
"items.qty" : 1,
"items.total" : 1
},
"ns" : "dummy.invoices",
"name" : "items.nameTags_1_created_1_special_1__id_1_items.qty_1_items.total_1"
}
>
Finally, here is an example invoice document (with just 2 items):
> db.invoices.findOne({itemCount: 2})
{
"_id" : "85923",
"customer" : "Wgtd Fm 91",
"businessNo" : "314227928",
"billTo_name" : "Wgtd Fm 91",
"billTo_addressLine1" : "3839 Ross Street",
"billTo_addressLine2" : "Kingston, ON",
"billTo_postalCode" : "K7L 4V4",
"purchaseOrderNo" : "boi",
"terms" : "COD",
"shipDate" : "2013-07-10",
"shipVia" : "Moses Transportation Inc.",
"rep" : "Snowhite",
"items" : [
{
"qty" : 4,
"name" : "CA 7789",
"desc" : "3 pc. Coffee Table set (Silver)",
"price" : 222.3,
"total" : 889.2,
"nameTags" : [
"ca 7789",
"a 7789",
" 7789",
"7789",
"789",
"89",
"9"
],
"descTags" : [
"3",
"pc",
"c",
"coffee",
"offee",
"ffee",
"fee",
"ee",
"e",
"table",
"able",
"ble",
"le",
"e",
"set",
"et",
"t",
"silver",
"ilver",
"lver",
"ver",
"er",
"r"
]
},
{
"qty" : 4,
"name" : "QP 8681",
"desc" : "Ottoman Bed",
"price" : 1179.1,
"total" : 4716.4,
"nameTags" : [
"qp 8681",
"p 8681",
" 8681",
"8681",
"681",
"81",
"1"
],
"descTags" : [
"ottoman",
"ttoman",
"toman",
"oman",
"man",
"an",
"n",
"bed",
"ed",
"d"
]
}
],
"itemCount" : 2,
"discount" : "10%",
"delivery" : 250,
"hstPercents" : 13,
"subTotal" : 5605.6,
"totalBeforeHST" : 5295.04,
"total" : 5983.4,
"totalDiscount" : 560.56,
"hst" : 688.36,
"modified" : "2012-10-08",
"created" : "2014-06-25",
"version" : 0
}
>
My problem is that mongodb does not use index only according to the aforementioned explain() output. Why? After all I only request the _id field, which is part of the index.
In general, I feel that I am doing something very wrong. My invoices collection has 65,000 invoices with the total of 3,291,092 items. It took almost 89 seconds to explain() the query.
What am I doing wrong?
You are using arrays and subdocuments. Covered Indexes dont work with either of these.
From the mongo docs:
An index cannot cover a query if:
any of the indexed fields in any of the documents in the collection includes an array. If an indexed field is an array, the index becomes a multi-key index index and cannot support a covered query.
any of the indexed fields are fields in subdocuments. To index fields in subdocuments, use dot notation. For example, consider a collection users with documents of the following form:
http://docs.mongodb.org/manual/tutorial/create-indexes-to-support-queries/

Resources