https://github.com/jmettraux/rufus-scheduler states that:
rufus-scheduler is a Ruby gem for scheduling pieces of code (jobs). It understands running a job AT a certain time, IN a certain time, EVERY x time or simply via a CRON statement.
rufus-scheduler is no replacement for cron/at since it runs inside of Ruby.
so what if it runs inside ruby? can't i access cron using the system command in ruby?
rufus-scheduler is a "in-ruby-process" scheduler. It is not meant as a cron/at replacement at all.
rufus-scheduler was not meant for people not comfortable with cron/at on the command line, it was meant for people willing to schedule stuff directly inside their ruby process (and understanding what it implies).
If rufus-scheduler was meant as a replacement for cron/at, it would provide some kind of persistence for the jobs, but it does not.
Another take on that : http://adam.heroku.com/past/2010/6/30/replace_cron_with_clockwork/
I think rufus-scheduler is for those people who aren't comfortable using the system's crontab, at or batch.
cron does repeating/periodic jobs and at and batch are for one-time jobs because those two commands don't support automatically repeating commands.
So rufus-scheduler is creating the functionality of the other commands, but if you're comfortable at the command-line and with the other commands, it doesn't buy you much in my opinion.
I haven't used it, but did look through the source, and my concern is that it appears rufus-scheduler relies on threads, which mean Ruby will keep your app running in the background, waiting for the appropriate time or interval to run. If the process gets killed, or the machine restarts it looks like the job won't run, which is a major difference compared to the system's commands which will persist across reboots or the app not being in memory.
We use cron a lot at work for jobs; It's an industry standard tool, and every Linux and Mac computer is running cron-scheduled jobs all through the day, though most users don't know it.
Related
Does anyone know of a way to configure the built in SLURM GUI sview so that it will show recent job history? I'd like to have it show something like the last 6 hours of jobs or at least display the jobs a little longer before they disappear after completion.
The motivation for this is that user of the cluster often run array jobs wth thousands of sub-jobs. Some of these sub-jobs often fail due to something in the data that needs to be investigated. Having a GUI for this has been an imense help. We used to have a custom scheduler and GUI for this but have moved to more opensource software.
I have an external tool (written in perl) that does work through a C++ object interface. The launching and tear-down take time so I'd like to have an NSOperationQueue and 4 NSOperation threads that run throughout the life of my app but only do work when I feed them a file to work on. I can round-robin feed them as every file will take roughly the same amount of work.
How can I best do this, or is this asking too much of NSOperation in a way it was not designed to do.
I have tried a normal NSOperation task for each file and the launch & tear-down of the perl tool slows things down and sometimes hangs (not sure why - but seems related to the launch process happening too quickly in sequence).
I'm looking to launch the perl tool once on each of 4 threads and then keep them around for the life of the app as the perl tool will stay open waiting for commands... but those commands have to come from the thread that were launched from.
I have a program written in C++11. On the current input it takes too long to run. Luckily, the data can be safely split into chunks for parallel processing, which makes it a good candidate for, say, a Map/Reduce service.
AWS EMR could be a possible solution. However, since my code uses many modern libraries, it's quite a pain to compile it on the instances that are assigned for Apache Hadoop clusters. For example, I want to use soci (not available at all), boost 1.58+ (1.53 is there), etc etc. I also need a modern C++ compiler.
Obviously, all libraries and compilers can be manually upgraded (and the process scripted), but this sounds like a lot of manual work. And what about slave nodes - will they get all the libraries? Somehow I'm not sure. And the whole process of initializing the environment can now take very long time - thus killing a lot of performance advantage that distributing the jobs was supposed to bring in to begin with.
On the other hand, I don't really need all the advanced functionality that Apache Hadoop provides. And I don't want to set up a personal permanent cluster with my own installation of Hadoop or similar, because I will need to run the tasks only periodically and most of the time the servers will be idle, wasting money.
So, what would be the best product (or overall strategy) that could do the following:
Grab the given binaries + set of input files
Run the binaries on a predefined number of instances, using a recent Linux, ideally Ubuntu 15.10
Put the resulting files in a predefined location (S3 bucket?)
Shut everything down
I am sure I could write a number of scripts using the aws tool to achieve that manually, but I really don't want to reinvent the wheel. Any thoughts?
Thanks in advance!
Honestly that would be pretty easy to script, and you'll need to probably use scripting to grab the latest code on the servers when they start up anyway. I would suggest looking into defining an AutoScaling group with scheduled scaling policies. Alternatively you could have a Lambda function scheduled to run and issue the API command to create your instances.
You could either have a startup script on the server AMI, or simply pass a user-data script when you create the instances, that pulls down the binaries and input files and runs the command. The final step of the script could be to copy results to S3 and shutdown the server.
The (relatively new) AWS Batch is made for this purpose specifically.
I'm looking for some comparison between Quartz.NET and Windows Scheduled Tasks?
How different are they? What are the pros and cons of each one? How do I choose which one to use?
TIA,
With Quartz.NET I could contrast some of the earlier points:
Code to write - You can express your intent in .NET language, write unit tests and debug the logic
Integration with event log, you have Common.Logging that allows to write even to db..
Robust and reliable too
Even richer API
It's mostly a question about what you need. Windows Scheduled tasks might give you all you need. But if you need clustering (distributed workers), fine-grained control over triggering or misfire handling rules, you might like to check what Quartz.NET has to offer on these areas.
Take the simplest that fills your requirements, but abstract enough to allow change.
My gut reaction would be to try and get the integral WinScheduler to work with your needs first before installing yet another scheduler - reasoning:
no installation required - installed and enabled by default
no code to write - jobs expressed as metadata
integration with event log etc.
robust and reliable - good enough for MSFT, Google etc.
reasonably rich API - create jobs, check status etc.
integrated with remote management tools
security integration - run jobs in different credentials
monitoring tooling
Then reach for Quartz if it doesn't meet your needs. Quartz certainly has many of these features too, but resist adding yet another service to own and manage if you can.
One important distinction, for me, that is not included in the other answers is what gets executed by the scheduler.
Windows Task Scheduler can only run executable programs and scripts. The code written for use within Quartz can directly interact with your project's .NET components.
With Task Scheduler, you'll have to write a shell executable or script. Inside of that shell, you can interact with your project's components. While writing this shell code is not a difficult process, you do have to consider deploying the extra files.
If you anticipate adding more scheduled tasks over the lifetime of the project, you may end up needing to create additional executable shells or script files, which requires updates to the deployment process. With Quartz, you don't need these files, which reduces the total effort needed to create and deploy additional tasks.
Unfortunately, Quartz.NET job assemblies can't be updated without restarting the process/host/service. That's a pretty big one for some folks (including myself).
It's entirely possible to build a framework for jobs running under Task Scheduler. MEF-based assemblies can be called by a single console app, with everything managed via a configuration UI. Here's a popular managed wrapper:
https://github.com/dahall/taskscheduler
https://www.nuget.org/packages/TaskScheduler
I did enjoy my brief time of working with Quart.NET, but the restart requirement was too big a problem to overcome. Marko has done a great job with it over the years, and he's always been helpful and responsive. Perhaps someday the project will get multiple AppDomain support, which would address this. (That said, it promises to be a lot of work. Kudos to he and his contributors if they decide to take it on.)
To paraphrase Marko, if you need:
Clustering (distributed workers)
Fine-grained control over triggering or misfire handling rules
...then Quartz.NET will be your requirement.
Is there a standard(ish) POSIX way of determining if my process (I’m writing this as a Ruby script right now; but I’m curious for multiple environments, including Node.js and ISO C command-line applications) is being run in an interactive terminal, as opposed to, say, cron, or execution from another tool, or… so on and so forth.
Specifically, I need to acquire user input in certain situations, and I need to fail fatally if that is determinably not possible (i.e. being run by cron.) I can do this with an environment variable, but I’d prefer something more standard-ish, if I can.
I've always used $stdout.isatty to check for this. Other approaches might include checking the value of ENV['TERM'] or utilizing the ruby-terminfo gem.
The closest you'll get, AFAIK, is isatty(). But as the page itself says, nothing guarantees that a human is controlling the terminal.
As #Mitch wrote it in a comment, which can also be useful to some people:
For anyone who came here looking for windows, try System.Environment.UserInteractive for .Net or GetUserObjectInformation for win32, which will fail for non-interactive processes