Say I want to run a custom Node script on my DocPad server once a day (like a cron job), where would I put it? I can build a Node script that does stuff after an interval, I'm more curious about where to reference / run the script in the DocPad server.
A plugin is possible, though I've seen that you can require Node libraries within the DocPad configuration file so it could go in there.
Is there a suggested way to approach this?
If you're wanting something purely cron-like, probably using the docpadReady event would be the way to go, doing something like:
docpadReady: ->
require('schedule').every('2 minutes').do ->
require('safeps').spawn('your cron job')
Alternatively, maybe DocPad's regenerateEvery configuration option is suitable. This tells DocPad to regenerate every X millseconds, which will naturally call the generate events that you could hook into.
Alternatively, is there a need for these crons to run on the same server as DocPad? If not, you could do them completely separately.
A final option, is to see if your server you are deploying to supports spawning multiple files. So DocPad's Server is spawned, and so is cron, with DocPad not knowing about the cron task at all.
Related
I found this similar question How to check If the current app process is running within a queue environment in Laravel
But actually this is the opposite of what I want. I want to be able to distinguish between code being executed manually from an artisan command launched on the CLI, and when a job is being run as a result of a POST trigger via a controller, or a scheduled run
Basically I want to distinguish between when a job is being run via the SYNC driver, manually triggered by the developer with eyes on the CLI output, and otherwise
app()->runningInConsole() returns true in both cases so it is not useful to me
Is there another way to detect this? For example is there a way to detect the currently used queue connection? Keeping in mind that it's possible to change the queue connection at runtime so just checking the value of the env file is not enough
Using Codeigniter 2.2.0 for my project. Is $this->input->is_cli_request() validation is necessary for a cron job?
It is recommended to protect your cronjob not to execute when someone type the URL in their browser. However if you don't have any problem running your cronjob invoked by anyone then can avoid this check.
Refer https://ellislab.com/codeigniter/user-guide/general/cli.html for more details.
It recommented to run cron jobs with command line.
There are many reasons for running CodeIgniter from the command-line, but they are not always obvious.
Run your cron-jobs without needing to use wget or curl
Make your cron-jobs inaccessible from being loaded in the URL by checking for $this->input->is_cli_request()
Make interactive "tasks" that can do things like set permissions, prune cache folders, run backups, etc.
Integrate with other applications in other languages. For example, a random C++ script could call one command and run code in your models!
More info read here
But you also can prevent calling from URL in your server.
We're using the Ruby gem whenever to manage large batches of import jobs. But what if a file is still being imported when the next cron job occurs?
For example:
12am: whenever starts an import cron job for import.csv
2am: import.csv is still being imported, but the next cron job is scheduled in whenever.
Would whenever skip that file or try to run it again? Any suggestions to make sure it doesn't try to process the same file twice?
Whenever is merely a frontend for the crontab. Whenever doesn't actually launch any of the processes, it writes a crontab that handles the actual scheduling and launching. Whenever cannot do what you're asking.
The crontab cannot do what you want either. It launches the process and that's it.
You need to implement the checking yourself in the process launched by cron. A common way of doing this could be a lockfile, and I'm sure there are libraries for this (ie http://rubygems.org/gems/lockfile).
Depending on your situation you might be able to create other checks before launching the import.
Well, this isn't really an issue of whenever
However, you could rename the file you want to import when you start processing (12am to 2am is a reasonable amount of time to do that) and move it to an archive directory once you are done processing so there is no confusion.
The next time the task runs it should look for all files that do not match a naming pattern (as already suggested in one of the comments)
And you might want to add an additional task that checks for imports that might have failed (e.g. a file has a naming pattern including the exact time but after a whole day it is still not archived) and either create some kind of notification or just trigger the task again/rename the task so it is picked up again (depending on how well your rollback works)
So I've been meaning to create a cron job on my prototype Flask app running on Heroku. Searching the web I found that the best way is by using Flask-Script but I fail to see the point of using it. Do I get easier access to my app logic and storage info? And if I do use Flask-Script, how do I organize it around my app? I'm using it right now to start my server without really knowing the benefits. My folder structure is like this:
/app
/manage.py
/flask_prototype
all my Flask code
Should I put the 'script.py' to be run by the Heroku Scheduler on app folder, the same level as manage.py? If so, do I get access to the models defined within flask_prototype?
Thank you for any info
Flask-Script just provides a framework under which you can create your script(s). It does not give you any better access to the application than what you can obtain when you write a standalone script. But it handles a few mundane tasks for you, like command line arguments and help output. It also folds all of your scripts into a single, consistent command line master script (this is manage.py, in case it isn't clear).
As far as where to put the script, it does not really matter. As long as manage.py can import it and register it with Flask-Script, and that your script can import what it needs from the application you should be fine.
I have a script that pulls some data from a web service and populates a mysql database. The idea is that this runs every minute, so I added a cron job to execute the script.
However, I would like the ability to occasionally suspend and re-start the job without modifying my crontab.
What is the best practice for achieving this? Or should I not really be using crontab to schedule something that I want to occasionally suspend?
I am considering an implementation where a global variable is set, and checked inside the script. But I thought I would canvas for more apt solutions first. The simpler the better - I am new to both scripting and ruby.
If I were you my script would look at a static switch, like you said with your global variable, but test for a file existence instead of a global variable. This seems clean to me.
Another solution is to have a service not using crontab but calling your script every minute. This service would be like other services in /etc/init.d or (/etc/rc.d depending on your distribution) and have start, stop and restart commands as other services.
These 2 solutions can be mixed:
the service only create or delete the switching file, and the crontab line is always active.
Or your service directly edits the crontab like this, but
I prefer not editing the crontab via a script and the described technique in the article is not atomic (if you change your crontab between the reading and the writting by the script your change is lost).
So at your place I would go for 1.