I am very new to Mongo but I have SQL experience so I am trying to wrap my head around this concept. I am attempting to remove a whole document based on the result of a subdocument.
The document/row looks close to the following:
{
"_id" : ObjectId("5a7e04e3809303035bf6437a"),
"receivedTime" : ISODate("2018-02-09T20:30:27.118Z"),
"status" : "NORMALIZED",
"originalHeaders" : {
"name" : "My Alert Name",
"description" : null,
"version" : 0,
"severity" : 3
},
"partOfIncident" : false
}
I want to remove all documents that have the name = "My Alert Name". I have been trying something like the following by calling it from a bash script. This is the command after variable substitution has been performed:
++ mongo admin -u admin -p password --eval 'db.getSiblingDB("database_name").collection.deleteMany({originalHeaders: {name: "I ALERT EVERYTHING"} })'
After calling it, nothing is removed. Any pointers on how to accomplish my end goal would be greatly appreciated. I suppose it is possible to run through a find and save all of the node _id to run for deletion but that sounds terribly inefficient.
When accessing a nested field you need to use dot notation.
db.collection_name.deleteMany( { "originalHeaders.name" : "My Alert Name" } )
This will delete all documents where originalHeaders.name = "My Alert Name"
Related
I have the following data in a MongoDB collection named users:
{
"_id" : ObjectID("5a3903562cdc59fad5fdc098"),
"name" : "Ana",
"hobbies" : [
{
"title" : "kissing",
"with" : "pets"
},
{
"title" : "playing",
"with" : "pets"
},
{
"title" : "sleeping",
"with" : "pets"
}
]
}
{
"_id" : ObjectID("5a3903a32cdc59fad5fdc099"),
"name" : "Bart",
"hobbies" : [
{
"title" : "hitting",
"with" : "pets"
},
{
"title" : "beating",
"with" : "pets"
},
{
"title" : "eating",
"with" : "pets"
}
]
}
I need to replace the pets value of all the with keys with a new value like legos.
MongoDB's documentation for version 3.6 states the following:
The $[] operator can be used for queries which traverse more than one array and nested arrays.
As each of the with keys lives inside of two separate arrays, using $[] should accomplish what I need to do. And on a Mac, it works perfectly but on a Windows machine, I get this error:
cannot use the part (hobbies of hobbies.$[].with) to traverse the element
Both machines are running MongoDB shell version 3.6.0. The operating system for the Mac is macOS Sierra 10.12.6 and for the Windows machine, it is Windows 10.
SO has many questions related to the positional operator and to the error I am getting specifically. But none of them address why identical operations executed on identical collections fail on Windows but are successful on Mac.
I have tried the following two commands to achieve the result I need. Both work on Mac and both fail on Windows with the same error given above.
db.users.updateMany({}, {$set: {"hobbies.$[].with": "legos"}});
and
db.users.update({}, {$set: {"hobbies.$[].with": "legos"}}, {multi: true});
You can see screen recordings of the difference here. My apologies in advance that the text in the recording on the Windows machine is on the smaller side.
Any help to understand how to resolve this on Windows is greatly appreciated.
Check your feature compatibility version.
By default it may be 3.4. Documentation.
You need to set feature compatibility version to 3.6
db.adminCommand( { setFeatureCompatibilityVersion: "3.6" } )
I can't tell but on windows it looks like for the query you have () instead of {}
The same issue exists on Linux too.
I'm using Ubuntu 16.04, MongoDB 3.6.1
Following is the output after the $[] is executed
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 0 }
P.S: I don't have enough reputation to add a comment, hence posting it here
I check the mongodb reference, the least version is 3.6 which contains $[].
I could have been in a struggle that to derive a facet search in the mongodb with c# driver. I have verified many tutorials but didn't get the suitable solution.
My document/collection will be as follows.
db.products.insert([
{"product_name": "Product 1", "year":2014,"Manufacturer":"manufacturer1"},
{"product_name": "Product 2", "year":2015,"Manufacturer":"manufacturer2"},
{"product_name": "Product 3", "year":2014,"Manufacturer":"manufacturer1"},
{"product_name": "Product 4", "year":2015,"Manufacturer":"manufacturer2"},
{"product_name": "Product 5", "year":2014,"Manufacturer":"manufacturer1"}
])
I want the output like
Year:
2014 : 3
2015 : 2
Manufacturer
Manufacturer1:3
Manufacturer1:2
Could any one please help me to solve the above problem using c# driver.
Using mongodb shell this can be done using $group in two phases:
db.products.aggregate([{$group:{_id:"$year",count:{$sum:1}}}])
db.products.aggregate([{$group:{_id:"$Manufacturer",count:{$sum:1}}}])
You can put multiple pipelines inside a faceted query. But remember that you cannot pass the output of one pipe to another pipe.
Each sub-pipeline within $facet is passed the exact same set of input documents. These sub-pipelines are completely independent of one another and the document array output by each is stored in separate fields in the output document. The output of one sub-pipeline can not be used as the input for a different sub-pipeline within the same $facet stage. If further aggregations are required, add additional stages after $facet and specify the field name, , of the desired sub-pipeline output.
You can try out the following query to get the desired result.
db.products.aggregate([
{
$facet : {
year : {
$group : {
_id : '$year',
count : {
$sum : 1
}
}
},
manufacturer : {
$group : {
_id : '$Manufacturer',
count : {
$sum : 1
}
}
}
}
}
])
here is elasticsearch official website about terms:
https://www.elastic.co/guide/en/elasticsearch/reference/2.1/query-dsl-terms-query.html
As we can see, if we want to do terms lookup mechanism query, we should use command like this:
curl -XGET localhost:9200/tweets/_search -d '{
"query" : {
"terms" : {
"user" : {
"index" : "users",
"type" : "user",
"id" : "2",
"path" : "followers"
}
}
}
}'
But what if i want to do query by other field of users.
Assume that users has some other fields such as name and can i use terms lookup mechanism finding the tweets by giving users name but not id.
I have tried to use command like this:
curl -XGET localhost:9200/tweets/_search -d '{
"query" : {
"terms" : {
"user" : {
"index" : "users",
"type" : "user",
"name" : "Jane",
"path" : "followers"
}
}
}
}'
but it occurs error.
Looking forward to your help. Thank you!
The terms lookup mechanism is basically a built-in optimization to not have to make two queries to JOIN two indices, i.e. one in index A to get the ids to lookup and a second to fetch the documents with those ids in index B.
In contrary to SQL, such a JOIN can only work on the id field since this is the only way to uniquely retrieve a document from Elasticsearch via a GET call, which is exactly what Elasticsearch will do in the terms lookup.
So to answer your question, the terms lookup mechanism will not work on any other field than the id field since the first document to be retrieved must be unique. In your case, ES would not know how to fetch the document for the user with name Jane since name is just a field present in the user document, but in no way a unique identifier for user Jane.
I think you did not understand exactly how this works. Terms lookup query works by reading values from a field of a document with the given id. In this case, you are trying to match the value of field user in tweets index with values of field followers in document with id "2" present in users index and user type.
If you want to read from any other field then simply mention that in "path".
What you mainly need to understand is that the lookup values are all fetched from a field of a single document and not multiple documents.
I need to grab the top 3 results for each of the 8 users. Currently I am looping through for each user and making 8 calls the the db. Is there a way to structure the query to pull the same 8X3 dataset in a single db pull?
selected_users = users.sample(8)
cur = 0
while cur <= selected_users .count-1
cursor = status_store.find({'user' => selected_users[cur]},{:fields =>params}).sort('score', -1).limit(3)
*do something*
cur+=1
end
The collection I am pulling from looks like the below. Each user can have an unbound number of tweets so I have not embedded them within within a user document.
{
"_id" : ObjectId("51e92cc8e1ce7219e40003eb"),
"id_str" : "57915476419948544",
"score" : 904,
"text" : "Yesterday we had a bald eagle on the show. Oddly enough, he was in the country illegally.",
"timestamp" : "19/07/2013 08:10",
"user" : {
"id_str" : "115485051",
"name" : "Conan O'Brien",
"screen_name" : "ConanOBrien",
"description" : "The voice of the people. Sorry, people.",
}
}
Thanks in advance.
Yes you can do this using the aggregation framework.
Another way would be to keep track of the top 3 scores for in the user documents. If this is faster or not depends on how often you write to scores vs read to top scores by users.
I have a near query I perform using Spring Data's NearQuery operation. Everything works fine for the most part. However, I have my code on several test machines and a production machine. The query works on some of the test machines, yet it does not return results for newly created objects on the production machine and on one of my test machines. When I drop the mongodb collection on the machines that do not work, then use the same code to insert a new document and recreate the collection, the query begins to work again. My question is; what can cause this type of behavior? Can adding new variables to a class cause mongodb near queries to stop performing? If there are documents added to a collection with different variables than what already exist can that cause problems? In production, I can not simply drop collections to fix this. Is there something I am missing about keeping data in mongodb collections consistent so that my spring mongodb code continues to work?
The mongoTemplate code:
Point point = new Point(locationAsDoubleArray[0],locationAsDoubleArray[1]);
NearQuery query = NearQuery.near(point.getX(),point.getY()).spherical(true).maxDistance(maxDistance,Metrics.MILES).distanceMultiplier(Metrics.MILES).query(regularQuery);//maxDistance(new Distance(radius,Metrics.MILES));
GeoResults<CalendarEvent> results = ((MongoOperations)mongoTemplate).geoNear(query, CalendarEvent.class);
The document that should be returned in JSON format:
{ "_class" : "com.eatmyfish.services.custom.CalendarEvent" , "_id" : { "$oid" : "5011c5cf51527fce6c4d2a00"} , "_keywords" : [ "test" , "search" , "function" , "test" , ""] , "address1" : "221 East 5th Street" , "address2" : "" , "allDay" : false , "categories" : [ 14] , "city" : "Saint Paul" , "clientId" : 109 , "clientProductId" : 962 , "color" : "#003666" , "createUser" : "peterson.dean" , "description" : "test" , "end" : "2012-07-26 14:00:00" , "endDate" : { "$date" : "2012-07-26T19:00:00.000Z"} , "externalLink" : "<a href='http://'>More Info</a>" , "geoLocation" : [ -93.0875195 , 44.9490055] , "latitude" : 0.0 , "location" : "221 East 5th Street Saint Paul,MN 55101 " , "locationManuallyEntered" : false , "locationName" : "My Cubicle" , "longitude" : 0.0 , "moreInfoLink" : "<a href='http://localhost:8080/posts/list/3150.page'>More Info</a>" , "note" : "" , "privateEventIn" : "N" , "restFormattedAddress" : "221+East+5th+Street+Saint+Paul,+MN+55101" , "start" : "2012-07-26 04:00:00" , "startDate" : { "$date" : "2012-07-26T09:00:00.000Z"} , "state" : "MN" , "title" : "Test Search Function" , "topicId" : 3150 , "url" : "http://localhost:8080/posts/list/3150.page" , "zip" : "55101"}
The code works differently depending on the machine it is being run. I have ensured my jar files, etc. are identical on each machine. The only thing that will make the query work once it begins to fail, is to drop the collection and start over. I am not sure when or what causes the query to stop working however. I do not think the code is the problem. There may be some administrative task I do not know about that will clean the data. I have used the repair command already without any luck.
I had some old entries that had the long/lat order reversed. That caused all my near queries to fail. It is odd that having a few long/lat values in reverse order would cause this. Still, that is the cause. When I fixed the order of the long/lats for the entries in reverse, the queries are working again. To find this out I had to build and use direct mongodb commands in java rather than use Spring's succinct approach. By viewing the command's return value while debugging, I could actually see the error message about having incorrect values for latitude. No such errors were returned using Spring's near query operation. Spring's inadequate error messaging made this bug very hard to track down.