I have been trying to do this for a long time. I have tried every thing that I know, from finding how to freeze all objects to trying to find the instance id, I have try looking on the internet.
Related
Short question is on the title: I work with my mongo Shell wich is in safe mode by default, and I want to gain better performance by deactivating this behaviour.
Long Question for those willing to know the context:
I am working on a huge set of data like
{
_id:ObjectId("azertyuiopqsdfghjkl"),
stringdate:"2008-03-08 06:36:00"
}
and some other fields and there are about 250M documents like that (whole database with the indexes weights 36Go). I want to convert the date in a real ISODATE field. I searched a bit how I could make an update query like
db.data.update({},{$set:{date:new Date("$stringdate")}},{multi:true})
but did not find how to make this work and resolved myself to make a script that take the documents one after the other and make an update to set a new field which takes the new Date(stringdate) as its value. The query use the _id so the default index is used.
Problem is that it takes a very long time. I already figured out that if only I had inserted empty dates object when I created the database I would now get better performances since there is the problem of data relocation when a new field is added. I also set an index on a relevant field to process the database chunk by chunk. Finally I ran several concurrent mongo clients on both the server and my workstation to ensure that the limitant factor is the database lock availability and not any other factor like cpu or network costs.
I monitored the whole thing with mongotop, mongostats and the web monitoring interfaces which confirmed that write lock is taken 70% of the time. I am a bit disappointed mongodb does not have a more precise granularity on its write lock, why not allowing concurrent write operations on the same collection as long as there is no risk of interference? Now that I think about it I should have sharded the collection on a dozen shards even while staying on the same server, because there would have been individual locks on each shard.
But since I can't do a thing right now to the current database structure, I searched how to improve performance to at least spend 90% of my time writing in mongo (from 70% currently), and I figured out that since I ran my script in the default mongo shell, every time I make an update, there is also a getLastError() which is called afterwards and I don't want it because there is a 99.99% chance of success and even in case of failure I can still make an aggregation request after the end of the big process to retrieve the single exceptions.
I don't think I would gain so much performance by deactivating the getLastError calls, but I think itis worth trying.
I took a look at the documentation and found confirmation of the default behavior, but not the procedure for changing it. Any suggestion?
I work with my mongo Shell wich is in safe mode by default, and I want to gain better performance by deactivating this behaviour.
You can use db.getLastError({w:0}) ( http://docs.mongodb.org/manual/reference/method/db.getLastError/ ) to do what you want but it won't help.
This is because for one:
make a script that take the documents one after the other and make an update to set a new field which takes the new Date(stringdate) as its value.
When using the shell in a non-interactive mode like within a loop it doesn't actually call getLastError(). As such downing your write concern to 0 will do nothing.
I already figured out that if only I had inserted empty dates object when I created the database I would now get better performances since there is the problem of data relocation when a new field is added.
I did tell people when they asked about this stuff to add those fields incase of movement but instead they listened to the guy who said "leave them out! They use space!".
I shouldn't feel smug but I do. That's an unfortunately side effect of being right when you were told you were wrong.
mongostats and the web monitoring interfaces which confirmed that write lock is taken 70% of the time
That's because of all the movement in your documents, kinda hard to fix that.
I am a bit disappointed mongodb does not have a more precise granularity on its write lock
The write lock doesn't actually denote the concurrency of MongoDB, this is another common misconception that stems from the transactional SQL technologies.
Write locks in MongoDB are mutexs for one.
Not only that but there are numerous rules which dictate that operations will subside to queued operations under certain circumstances, one being how many operations waiting, another being whether the data is in RAM or not, and more.
Unfortunately I believe you have got yourself stuck in between a rock and hard place and there is no easy way out. This does happen.
I received an abandoned student LabView project I need to adapt. There is one VI which contains an array of two controls with length 30. I need to change the strings in several places within that array which is possible as long as I do not close the vi. If I close and reload the vi the previous values are there again, saving the changed values as default does not help. Cutting the connection to the type def did not help either.
There is also no databinding and I could not find any mechanism that might lock the values. It must be something obvious to experienced LabView developers, but I could not find anything on the net.
What can be the reason that the values cannot be changed permanently?
Edit:
After playing around for a long time, i found out that I have to set as default value on the array, and not on the individual positions in the array. That solved it, simple problem, several hours spent...
After playing around for a long time, i found out that I have to set as default value on the array, and not on the individual positions in the array. That solved it, simple problem, several hours spent...
In the iphone game "Tiny Tower", I'm guessing it uses some kind of simulation based on the time spent between the last play and the current time, because you can set the current time forward and you will get the benefit from the fake elapsed time span.
Is there an algorithm that I can use to prevent this sort of thing? (Or at least make it difficult enough for the average user to pull off!)
Edit: thanks, I understand that, despite my wording, there's no way to prevent things you store on the client side, but I want to make it at least more difficult than "changing the time" to hack it!
The gamecube had a way to do this so it must be possible.
Is there an event triggered when the iphone time is set ? In that case you can react that.
Another solution is to require to be online when the game is launched, this way you can check time on a remote server.
You could has well check if you got an event on the phone login or wake up react to it, saving the time at that moment in your DB. You would have the last non modified time.
A last possible trick is to check for a file you know is going to be modified by an action prior to time change (such as login), and check the 'last modification' date.
You can investigate in the GPS direction as well. A GPS need to be synchronised with the satellite it contact, so it must keep track of time in some way, and maybe there is an API for that.
Unfortunatly you are on an iphone, which mean your possibilities are limited since applications got very few rights and are sandboxed.
EDIT:
Just though about it but, can you create event in the iphone calendar ? And check if it has been trigered ? Cause you could set a fake meeting or something for every day. Not clean, but creative.
EDIT 2: can you set a timer as a code for IOS to execute in 60 minutes ? If you can, set this timer, pass the time expected to be when this code run, then when the code run, compare and inform your program.
One way to prevent it is to monitor time passing by checking timestamps for their logins in a database. It doesn't matter if the client's iPhone's time is off; the database on your end will still know how long it's been since the last login.
I think if you have internet access you can take the time from a server.
A second solution : You can record the "datetime" and every time you see a "BIG" difference between the record datetime and the running datetime you know there might be a problem.
but this is not elegant, i know.
You can also record a small ammount of datetimes that the application started and check the diffrence with the running datetime.
Also you can use "Activity"->"Datetime" so the "Updates" (levels etc) can't be retaken.
Because the system Datetime can be changed by user, there is potential for "hack".
call a web service to get the time, rather than rely on the phone. There are several places you could get time from, google is your friend i'm sure, or create one yourself, and use the local time of the machine the service runs on for the time.
You could also use the Network Time Protocol (NTP) servers to get a consistent time
I am writing a Scheduling type application in C#, and allowing the user to store tasks that they want to run at certain times. Right now I give them the option of specifying how often to run it (Daily/Weekly/Monthly) as well as specify a time, which is then stored in a Database.
I am having a little bit of trouble just wrapping my head around the pseudo code behind this, and am looking for some suggestions about how to implement it. I am running a repeating timer every 60 seconds to check each task to see if it needs to run, but always seem to hit road blocks when I need to work with Date/Time and adding Recurring Day (daily/weekly/etc) has complicated it even more.
Does midiOutPrepareHeader, midiInPrepareHeader just setup some data fields, or does it do something that is more time intensive?
I am trying to decide whether to build and destroy the MIDIHDR's as needed, or to maintain a pool of them.
You really have only two ways to tell (without the Windows source):
1) Profile it. Depending on your findings for how long it takes, have a debug-only scoped timer that logs when it suddenly takes longer than what you think is acceptable for your application, or do your pool solution. Though the docs say not to modify the buffer once you call the prepare function, and it seems if you wanted to re-use it you may have to modify it. I'm not familiar enough with the docs to say one way or the other if your proposed solution would work.
2) Step through the assembly and see. Don't be afraid. Get the MSFT public symbols and see if it looks like it's just filling out fields or if it's doing something complicated.