I would like to run a batch application (that is a short lived process that should not be restarted) on Pivotal CloudFoundry.
I am not sure how to do that. My current batch app is restarted repeatedly by Pivotal CF.
It seems there's a new CF primitive called a task - as opposed to a long-running process. Tasks are supposed to be available on CF 1.7 (see https://stackoverflow.com/a/35512113/536299).
I was neither able to find relevant information in the CF documentation nor to figure out which version of the Pivotal CF is currently being run...
Can someone please help?
I just got relevant information regarding short-lived/one-off processes on CF. It currently seems to be very difficult to run short-lived/one-off processes on CF.
This will change when CF v3's tasks becomes generally available.
Here is the information I was given:
Batch jobs are a little tricky on PWS and PCF because at the moment
the platform expects your application to continue running forever.
Even if the app exits successfully, the platform considers it to have
crashed and will restart it. There is support in v3 of the platform
for one-off tasks like batch jobs, so this will get easier in the
future. For now, what you need to do is to make the app run forever.
One option is to add a loop to the main method in the app, the loop
would essentially run the batch job, pause for some set amount of time
and repeat indefinitely.
So bottom-line is wait for CF v3's tasks.
See here for documentation about tasks: http://v3-apidocs.cloudfoundry.org/version/release-candidate/index.html#tasks
Related
We are implementing a utility that will apply the DDLs to the database. The utility is built using spring boot java and it has a main program that will run just once on startup. Can some one share what kind of K8s recipe file. Here are my considerations, the pod is expected to be short lived and after the program executes I want the POD to get killed.
Kubernetes Jobs are what you want for that.
Here is a great example.
Once you start running jobs you'll also want to think of an automated way of cleaning up the old jobs. There are custom controllers written to clean up jobs, so you could look at those, but there is first-class support being built-in for job clean-up that I believe is still in alpha state, but you can already use this of course.
It works by simply adding a TTL to your job manifests. Here is more info on the job clean-up mechanism with TTL.
I'm using System.Runtime.Caching.MemoryCache to simulate a repeated task on a running .NET MVC application deployed on AppHarbor.
Entries in the cache are added using a CacheItemPolicy which contains an AbsoluteExpiration offset and a RemovedCallback that calls a method and retriggers the adding of the item in the cache (as described here)
MemoryCache is populated first time in Application_Start. It works fine locally, but doesn't seem to work once deployed on AppHarbor (tried also with HttpRuntime.Cache, same result).
My application is running under a CANOE (free) account on AppHarbor that only has one worker. Does this mean that I won't be able to simulate the background task until I upgrade to some paid plan?
Thanks!
Your application has to have visitors every once in a while for this to work. Other than StillAlive, Pingdom is also a good bet for generating requests to your app. You should also take a look at MomentApp. We expect to have background tasks ready shortly.
I don't think upgrading will help, they are working on adding background jobs to AppHarbor but to my knowledge they available yet.
What about using a service like https://stillalive.com/ to periodically hit a page on your site that then spins up a new thread and starts running your background task? Its available as a free add-on.
I was thinking of doing something like this while waiting for the background task functionality to be available.
I am wondering if there is a way to monitor these automatically. Right now, in our production/QA/Dev environments - we have bunch of services running that are critical to the application. We also have automatic ETLs running on windows task scheduler at a set time of the day. Currently, I have to log into each server and see if all the services are running fine or not, or check event logs for any errors, or check task scheduler to see if ETLs ran well etc etc... I have to do all the manually... I am wondering if there is a tool out there that will do the monitoring for me and send emails only in case something needs attention (like ETLs fail to run, or service get stopped for whatever reason or errors in event log etc). Thanks for the help.
Paessler PRTG Network Monitor can do all that. we have very good experience with it.
http://www.paessler.com/prtg/features
Nagios is the best tool for monitoring. It checks for the server status as well the defined services in it and if any service goes down or system goes down, sends the mail to specified mail id.
Refer the : http://nagios.org/
Thanks for the above information. I looked at the above options but they have a price.. what I did is an inexpensive way to address my concerns..
For my windows task scheduler jobs that run every night - I installed this tool/service from codeplex that is working great.
http://motash.codeplex.com/documentation#CommentsAnchor
For Windows services - I am just setting the "Recovery" Tab in each service "property" with actions to do when it fails. (like restart, reboot, or run a program which could be an email that will notify)
I built a simple tool (https://cronitor.io) for monitoring periodic/scheduled tasks. The name is a play on "cron" from the unix world, but it is system/task agnostic. All you have to do is make an http request to a unique tracking URL whenever your job runs. If your job doesn't check-in according to the rules you define then it will send you an email/sms message.
It also allows you to track the duration of your jobs by making calls at the beginning and end of your task. This can be really useful for long running jobs since you can be alerted if they start taking too long to run. For example, I once had a backup task that was scheduled every hour. About six months after I set it up it started taking longer than an hour to run!
There is https://eyewitness.io - which is for monitoring server cron tasks, queues and websites. It makes sure each of your cron jobs run when they are supposed to, and alerts you if they failed to be run.
Suppose I include a rather long-running startup task into my Azure role - running something like up to several minutes. What happens if the startup task runs "too long".
I'm currently testing on Compute Emulator and observe the following.
I have a 450 megabytes .zip file together with Info-Zip unzip. The startup task unzips the archive. Deployment starts and I look into Task Manager. Numerous service processes start, then unzip.exe is run. After about two minutes all those processes stop and then start anew and unzip.exe starts again.
So it looks like a deployment is allowed to run for about two minutes, then is forcefully reset and started again.
Is this the expected behavior? Does it persist on real cloud? Are there any hard limits on how long a role startup can take? How do I address this situation except moving the unpacking into RoleEntryPoint.OnStart()?
I had the same question, so tried an experiment. I ran a Startup Task - taskType="simple" so that it would block the Roles from beginning to execute - and let it run for 50 hours. The Fabric Controller did not complain and the portal did not show any error. It finished its long "do nothing" loop after the 50 hours was up, then this Startup Task exited, and my Web Role started up fine.
So my emperical test says Startup Tasks can take a long time! At least 50 hours.
This should inform the load balancer that your process is still busy:
http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.serviceruntime.roleinstancestatuscheckeventargs.setbusy.aspx
I have run startup tasks that run for a pretty long time (think 20-30 mins) and the role is simply in a 'Busy' state. I don't think there is a hard limit for how long the role will stay in that state as long as the Startup task is still executing and did not exit with a non-zero return code (in fact, this is a gotcha for most first time startup task creators when they pop a prompt). The FC is technically still running just fine, so there would be no reason to 'recover' the role (i.e. heartbeats are still going).
The dev emulator just notices when the role hasn't started and warns you. If you click the 'keep waiting' option, it will continue to run the Startup task to completion. The cloud does not do this of course (warn you).
Never tried a task that ran super long, so there might be a very long limit. I seem to recall 3 hrs was a magic number in some timeout cases like role recycles, but I have never tried...
There are some heartbeats that the Azure Fabric Agent will do against the role. If these are not acknowledged (say a long-running blocking process), this could cause the role to be flagged as unavailable.
You might try putting your startup process into a background thread that runs independently. This should help you keep the role from being recycled while the process is starting up. Just keep in mind you may need to make some adjustments if you get requests before the role fully starts up. There's also a way (that I can't seem to recall ATM) to flag the role and take it out of the load balancer temporarially while your process completes.
A consultant setup a windows service to run a application. The application is supposed to run every 15 minutes. The application is not running at all and the service appears to be running fine.
I am not familiar with how an application will run at a standard interval when running as a service.
The service uses the SRVANY.EXE tool.
Any 'consultant' that sets up a service to run using SRVANY.EXE should be fired. SRVANY is an unfortunate hack that should have been retired a decade ago; it should never be used in a production environment.
If the only purpose for the service is to run the app on a schedule then it shouldn't exist at all. Run the app as a Scheduled Task. If it has other functionality then rewrite it as a real service. If it is reasonably well written it should be a fairly easy conversion.
There are many potential issues with your application.
SRVANY.EXE turns any application into a Windows Service. If that application ever asks for user input, it will hang. You will want to confirm that the application running as a service does nothing more than start the other application.
You should also be able to run the "starter" application manually, outside the Windows Service. If it still doesn't work as it should you know it's not related to being run as a service.
To add to the other answers: see KB137890 on what SRVANY.EXE actually does and how to find out what application it is running.
It seems to me that you would be better off (if you can) setting up a scheduled task that runs every the application every 15 minutes if you can.
I'm not sure if this is correct, but I believe one way of a serivce running an application is merely to have a thread within OnStart and set it to run the application on an invertal of 15 minutes.