I've been playing around storing tweets inside mongodb, each object looks like this:
{
"_id" : ObjectId("4c02c58de500fe1be1000005"),
"contributors" : null,
"text" : "Hello world",
"user" : {
"following" : null,
"followers_count" : 5,
"utc_offset" : null,
"location" : "",
"profile_text_color" : "000000",
"friends_count" : 11,
"profile_link_color" : "0000ff",
"verified" : false,
"protected" : false,
"url" : null,
"contributors_enabled" : false,
"created_at" : "Sun May 30 18:47:06 +0000 2010",
"geo_enabled" : false,
"profile_sidebar_border_color" : "87bc44",
"statuses_count" : 13,
"favourites_count" : 0,
"description" : "",
"notifications" : null,
"profile_background_tile" : false,
"lang" : "en",
"id" : 149978111,
"time_zone" : null,
"profile_sidebar_fill_color" : "e0ff92"
},
"geo" : null,
"coordinates" : null,
"in_reply_to_user_id" : 149183152,
"place" : null,
"created_at" : "Sun May 30 20:07:35 +0000 2010",
"source" : "web",
"in_reply_to_status_id" : {
"floatApprox" : 15061797850
},
"truncated" : false,
"favorited" : false,
"id" : {
"floatApprox" : 15061838001
}
How would I write a query which checks the created_at and finds all objects between 18:47 and 19:00? Do I need to update my documents so the dates are stored in a specific format?
Querying for a Date Range (Specific Month or Day) in the MongoDB Cookbook has a very good explanation on the matter, but below is something I tried out myself and it seems to work.
items.save({
name: "example",
created_at: ISODate("2010-04-30T00:00:00.000Z")
})
items.find({
created_at: {
$gte: ISODate("2010-04-29T00:00:00.000Z"),
$lt: ISODate("2010-05-01T00:00:00.000Z")
}
})
=> { "_id" : ObjectId("4c0791e2b9ec877893f3363b"), "name" : "example", "created_at" : "Sun May 30 2010 00:00:00 GMT+0300 (EEST)" }
Based on my experiments you will need to serialize your dates into a format that MongoDB supports, because the following gave undesired search results.
items.save({
name: "example",
created_at: "Sun May 30 18.49:00 +0000 2010"
})
items.find({
created_at: {
$gte:"Mon May 30 18:47:00 +0000 2015",
$lt: "Sun May 30 20:40:36 +0000 2010"
}
})
=> { "_id" : ObjectId("4c079123b9ec877893f33638"), "name" : "example", "created_at" : "Sun May 30 18.49:00 +0000 2010" }
In the second example no results were expected, but there was still one gotten. This is because a basic string comparison is done.
To clarify. What is important to know is that:
Yes, you have to pass a Javascript Date object.
Yes, it has to be ISODate friendly
Yes, from my experience getting this to work, you need to manipulate the date to ISO
Yes, working with dates is generally always a tedious process, and mongo is no exception
Here is a working snippet of code, where we do a little bit of date manipulation to ensure Mongo (here i am using mongoose module and want results for rows whose date attribute is less than (before) the date given as myDate param) can handle it correctly:
var inputDate = new Date(myDate.toISOString());
MyModel.find({
'date': { $lte: inputDate }
})
Python and pymongo
Finding objects between two dates in Python with pymongo in collection posts (based on the tutorial):
from_date = datetime.datetime(2010, 12, 31, 12, 30, 30, 125000)
to_date = datetime.datetime(2011, 12, 31, 12, 30, 30, 125000)
for post in posts.find({"date": {"$gte": from_date, "$lt": to_date}}):
print(post)
Where {"$gte": from_date, "$lt": to_date} specifies the range in terms of datetime.datetime types.
db.collection.find({"createdDate":{$gte:new ISODate("2017-04-14T23:59:59Z"),$lte:new ISODate("2017-04-15T23:59:59Z")}}).count();
Replace collection with name of collection you want to execute query
MongoDB actually stores the millis of a date as an int(64), as prescribed by http://bsonspec.org/#/specification
However, it can get pretty confusing when you retrieve dates as the client driver will instantiate a date object with its own local timezone. The JavaScript driver in the mongo console will certainly do this.
So, if you care about your timezones, then make sure you know what it's supposed to be when you get it back. This shouldn't matter so much for the queries, as it will still equate to the same int(64), regardless of what timezone your date object is in (I hope). But I'd definitely make queries with actual date objects (not strings) and let the driver do its thing.
Use this code to find the record between two dates using $gte and $lt:
db.CollectionName.find({"whenCreated": {
'$gte': ISODate("2018-03-06T13:10:40.294Z"),
'$lt': ISODate("2018-05-06T13:10:40.294Z")
}});
Using with Moment.js and Comparison Query Operators
var today = moment().startOf('day');
// "2018-12-05T00:00:00.00
var tomorrow = moment(today).endOf('day');
// ("2018-12-05T23:59:59.999
Example.find(
{
// find in today
created: { '$gte': today, '$lte': tomorrow }
// Or greater than 5 days
// created: { $lt: moment().add(-5, 'days') },
}), function (err, docs) { ... });
db.collection.find({$and:
[
{date_time:{$gt:ISODate("2020-06-01T00:00:00.000Z")}},
{date_time:{$lt:ISODate("2020-06-30T00:00:00.000Z")}}
]
})
##In case you are making the query directly from your application ##
db.collection.find({$and:
[
{date_time:{$gt:"2020-06-01T00:00:00.000Z"}},
{date_time:{$lt:"2020-06-30T00:00:00.000Z"}}
]
})
You can also check this out. If you are using this method, then use the parse function to get values from Mongo Database:
db.getCollection('user').find({
createdOn: {
$gt: ISODate("2020-01-01T00:00:00.000Z"),
$lt: ISODate("2020-03-01T00:00:00.000Z")
}
})
Save created_at date in ISO Date Format then use $gte and $lte.
db.connection.find({
created_at: {
$gte: ISODate("2010-05-30T18:47:00.000Z"),
$lte: ISODate("2010-05-30T19:00:00.000Z")
}
})
use $gte and $lte to find between date data's in mongodb
var tomorrowDate = moment(new Date()).add(1, 'days').format("YYYY-MM-DD");
db.collection.find({"plannedDeliveryDate":{ $gte: new Date(tomorrowDate +"T00:00:00.000Z"),$lte: new Date(tomorrowDate + "T23:59:59.999Z")}})
mongoose.model('ModelName').aggregate([
{
$match: {
userId: mongoose.Types.ObjectId(userId)
}
},
{
$project: {
dataList: {
$filter: {
input: "$dataList",
as: "item",
cond: {
$and: [
{
$gte: [ "$$item.dateTime", new Date(`2017-01-01T00:00:00.000Z`) ]
},
{
$lte: [ "$$item.dateTime", new Date(`2019-12-01T00:00:00.000Z`) ]
},
]
}
}
}
}
}
])
For those using Make (formerly Integromat) and MongoDB:
I was struggling to find the right way to query all records between two dates. In the end, all I had to do was to remove ISODate as suggested in some of the solutions here.
So the full code would be:
"created": {
"$gte": "2016-01-01T00:00:00.000Z",
"$lt": "2017-01-01T00:00:00.000Z"
}
This article helped me achieve my goal.
UPDATE
Another way to achieve the above code in Make (formerly Integromat) would be to use the parseDate function. So the code below will return the same result as the one above :
"created": {
"$gte": "{{parseDate("2016-01-01"; "YYYY-MM-DD")}}",
"$lt": "{{parseDate("2017-01-01"; "YYYY-MM-DD")}}"
}
⚠️ Be sure to wrap {{parseDate("2017-01-01"; "YYYY-MM-DD")}} between quotation marks.
Convert your dates to GMT timezone as you're stuffing them into Mongo. That way there's never a timezone issue. Then just do the math on the twitter/timezone field when you pull the data back out for presentation.
Why not convert the string to an integer of the form YYYYMMDDHHMMSS? Each increment of time would then create a larger integer, and you can filter on the integers instead of worrying about converting to ISO time.
Scala:
With joda DateTime and BSON syntax (reactivemongo):
val queryDateRangeForOneField = (start: DateTime, end: DateTime) =>
BSONDocument(
"created_at" -> BSONDocument(
"$gte" -> BSONDateTime(start.millisOfDay().withMinimumValue().getMillis),
"$lte" -> BSONDateTime(end.millisOfDay().withMaximumValue().getMillis)),
)
where millisOfDay().withMinimumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T00:00:00.000Z"
and
where millisOfDay(). withMaximumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T23:59:99.999Z"
i tried in this model as per my requirements i need to store a date when ever a object is created later i want to retrieve all the records (documents ) between two dates
in my html file
i was using the following format mm/dd/yyyy
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<script>
//jquery
$(document).ready(function(){
$("#select_date").click(function() {
$.ajax({
type: "post",
url: "xxx",
datatype: "html",
data: $("#period").serialize(),
success: function(data){
alert(data);
} ,//success
}); //event triggered
});//ajax
});//jquery
</script>
<title></title>
</head>
<body>
<form id="period" name='period'>
from <input id="selecteddate" name="selecteddate1" type="text"> to
<input id="select_date" type="button" value="selected">
</form>
</body>
</html>
in my py (python) file i converted it into "iso fomate"
in following way
date_str1 = request.POST["SelectedDate1"]
SelectedDate1 = datetime.datetime.strptime(date_str1, '%m/%d/%Y').isoformat()
and saved in my dbmongo collection with "SelectedDate" as field in my collection
to retrieve data or documents between to 2 dates i used following query
db.collection.find( "SelectedDate": {'$gte': SelectedDate1,'$lt': SelectedDate2}})
Related
I'm querying a cloudant DB from my nodejs App.
I am now trying to sort results from a view query.
My index (keys) are like this:
[ "FR000001", 1577189089166 ]
[ "FR000001", 1577189089165 ]
etc
from the following view:
function(doc) {
emit([doc.siteId, doc.creationDate],{"id" :doc._id, "rev": doc._rev, "siteId": doc.siteId, "creationDate": doc.creationDate, "scores": doc.scores, locationId: doc.locationId});
}
I managed to make that work on a real index using the syntax "sort: "-creationDate" " using syntax found in the bugs sections of cloudant github.
var ddoc = {
q: "site:\"" + id + "\"",
include_docs: false,
sort: "-creationDate",
};
const tmp = await cloudant.use('alarms').search('alarmSearch', 'IndexBySite', ddoc);
I can't make it work on my view with an array of query parameters. I have tried different variation around:
var ddoc_view = {
startkey: ["siteid1",0000000000000],
endkey: ["siteid1",9999999999999],
include_docs: true,
sort: "creationDate"
};
Can anyone help finding the right syntax, or pointing me to where I can find good "cloudant API for nodejs" documentation? for instance there is nothing on how to use sort" on the github... Thanks...
ok after another day of searching:
- best documentation I found is directly the couchdb doc: https://docs.couchdb.org/en/stable/ddocs/views/intro.html
- I ended up modifying the view:
emit([doc.creationDate, doc.siteId], {"id" :doc._id, "rev": doc._rev, "siteId": doc.siteId, "locationTag":doc.locationTag});
and the request:
var ddoc_view = {
endkey: [0000000000000, siteid],
startkey: [9999999999999, siteid],
include_docs: false,
descending: true,
limit: docsReturned,
};
To get a sorted response.
I'm starting to use Request objects to validate incoming post data, and I've seen examples of how others are using date_format, but I can't seem to get dates to pass even though I'm using a format that passes when you use PHP's date_parse_from_format:
print_r(date_parse_from_format('Y-m-d','2015-07-27'));
Output
UPDATE: their is a warning in date_parse_from_format, but I don't understand why as it reflects the format.
{
"year": 2015,
"month": 2,
"day": 30,
"hour": false,
"minute": false,
"second": false,
"fraction": false,
"warning_count": 1,
"warnings": {
"10": "The parsed date was invalid"
},
"error_count": 0,
"errors": [],
"is_localtime": false
}
Validator
public function rules()
{
return [
'rental_id' => 'required|exists:rentals,id',
'start_at' => 'required|date_format:Y-m-d',
'end_at' => 'required|date_format:Y-m-d'
];
}
Using PostMan I'm sending in a payload of:
rental_id = 1 as text
start_at = 2015-02-30 as text
end_at = 2015-02-30 as text
And it throws a NotFoundHttpException, but if I comment out start_at and end_at in the validator request it passes, and enters my controllers action with the proper payload:
{
"rental_id": "1",
"start_at": "2015-02-30",
"end_at": "2015-02-30"
}
Apparently, the date_format validation failed as I randomly chose 2015-02-30 to test my API, but that day doesn't exist as that would be February 30... oh the shame and the wasted time. Thanks to #ceejayoz for all the help!!!
I have a collection name projects and I am trying to retrieve everything except its url like this query
db.projects.find({name:"arisha"},{url:0}).pretty()
This query is working perfectly and returning everything except url but my question is how to achieve this in
Node module for MongoDB name monk.
I am using this code but its not working and returning every field:
var projs = db.get('projects');
projs.find({creator : req.session.user._id},{url:0}, function (err,data) {
console.log(data);
if(!err) {
res.locals.projs = data;
console.log(data);
res.render("projects.ejs",{title: "Projects | Bridge"});
}
});
I did not get where the problem is, please help and thanks in advance :)
Sample document:
{
"name" : "arisha",
"date" : {
"day" : 18,
"month" : 4,
"year" : 2015
},
"creator" : "552edb6f8617322203701ad1",
"url" : "EyjPdYoW",
"members" : [
"552edb6f8617322203701ad1"
],
"_id" : ObjectId("5532994ba8ffdca31258bd1a")
}
To exclude the url field in monk, try the following syntax:
var db = require('monk')('localhost/mydb');
var projs = db.get('projects');
projs.find({ creator : req.session.user._id }, "-url", function (err, data) {
// exclude url field
});
EDIT:
To exclude multiple fields, use the following projection syntax:
projs.find({ creator : req.session.user._id }, { fields: { url: 0, creator: 0 } }, function(err, data) {
// exclude the fields url and creator
});
Alternatively (as you had discovered), you could also do:
projs.find({ creator : req.session.user._id }, "-url -creator", function (err, data) {
// exclude url and creator fields
});
I' trying to use aggregation framework (with ruby) and project the date like this:
db['requests'].aggregate([
{"$project" => {
_id: 0,
method: '$method',
user: '$user',
year: {'$year' => '$timestamp'}
}}])
the document is like this one:
{
_id: ObjectId("5177d7d7df26358289da7dfd"),
timestamp: ISODate("2013-04-12T03:58:05+00:00"),
method: "POST",
status: "200",
inputsize: "874",
outputsize: "4981",
user: "131"
}
but i get the following error:
Mongo::OperationFailure: Database command 'aggregate' failed: (errmsg: 'exception: can't convert from BSON type EOO to Date'; code: '16006'; ok: '0.0').
This is strange because it works correctly if I run this on the exactly same db which is imported with mongorestore.
The problem was that I was saving some documents without the timestamp field.
If you needed to have some documents without this timestamp field, you could try this (I'm using Javascript/Mongoose notation):
year: { $cond: [{ $ifNull: ['$timestamp', 0] }, { $year: '$deliveryDateEnd' }, -1] }
In this case, every document without the timestamp field would return -1. All the other documents would return the year as expected.
I'm trying to bring back a list of year/month combinations with counts for describing blog posts. The idea is that they will be displayed like so:
January 2010 (1 post)
December 2009 (2 posts)
...
I have managed to get this to work using the MongoDB JS shell, and it returns results in a useful format:
db.posts.group({
keyf: function(x){
return {
month: x.datetime.getMonth(),
year:x.datetime.getFullYear()
};
},
reduce: function(x,y){ y.count++ },
initial:{count:0}
})
Results:
[ { "month" : 0, "year" : 2010, "count" : 3 },
{ "month" : 0, "year" : 1970, "count" : 1 } ]
This is great, exactly what I'm after. However, trying to convert this into code appropriate for the ruby driver, I can't get it to work. I have looked at the documentation and from my understanding, the following should yield the same results (I'm using MongoMapper, hence the Post.collection):
#archive = Post.collection.group(
"function(x) { return { month: x.datetime.getMonth(), year:x.datetime.getFullYear() }; }",
nil, { :count => 0 }, 'function(x,y){y.count++}', true)
Instead of giving back the nice array of useful data, I'm getting this mess:
{
"function(x) { return { month: x.datetime.getMonth(), year:x.datetime.getFullYear() }; }" => nil,
"count" => 4.0
}
It seems that either it is completely defying its own documentation (and my understanding of the source code!) or I am missing something fundamental here. I'm almost pulling my hair out, any help gratefully accepted.
That's pretty strange behavior. I just ran your code locally, and everything worked. Can you verify that you're using the driver version 0.18.2? If so, make sure that that's the only version installed (just as a sanity check).
I don't think it should make any difference, but I wasn't running #group from MongoMapper -- I was using the gem alone. You might try that, too. Here's the code I ran:
require 'rubygems'
require 'mongo'
d = Mongo::Connection.new.db('blog')
c = d['post']
p c.group("function(x) { return { month: x.date.getMonth(), year:x.date.getFullYear() }; }",
nil,
{ :count => 0 },
"function(x,y){y.count++}",
true)