I've uploaded a single file to Heroku that crawls a website and responds the edited content in JSON format on an http request. I now want to update the content regularly so the content stays up to date. I tried using the Heroku Scheduler however I am failing to schedule the process so that it runs correctly.
I have specified the following process in the Heroku Scheduler:
run phantomjs phantom.js //Using 1X Dyno, every hour.
//phantom.js is the file that contains my source code and that runs the server.
However if I enter
heroku ps
into the terminal, I only see one web dyne running and no scheduler task. Also if I type
heroku logs --ps scheduler.1
as described in the Scheduler documentation, there is no output.
What am I doing wrong? Any help would be appreciated.
For what it sounds like you want to accomplish, you need to be constantly running
1 Web Dyno
1 Background Worker
When your scheduled task executes, it will be run by the background worker. Which, since you haven't provisioned it, isn't executing.
Found it: I only had to write
phantomjs phantom.js
in order to get it working. It was the "run" that made the expression invalid
Related
I recently installed Heroku Redis. Until then, the app worked just fine. I am using Bull for queuing and ioredis as the redis library. I had connection issues initially but I have resolved that as I no longer get the error. However, this new Error described shows up.
Please check these details below;
Package.json Start Script
"scripts": {
"start": "sh ./run.sh"
}
run.sh file
node ./app/services/queues/process.js &&
node server.js
From the logs on the heroku console, I see this.
Processing UPDATE_USER_BOOKING... Press [ctrl C] to Cancel
{"level":"info","message":"mongodb connected"}
1 is my log in the process script. This tells me that the consumer is running and ready to process any data it receives.
2 Tells me that mongo is connected. It can be found in my server.js(entry file).
My challenge is after those 2 lines, it then shows this;
Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
Stopping process with SIGKILL
Error waiting for process to terminate: No child processes
Process exited with status 22
State changed from starting to crashed
So, I don't know why this is happening even when I have the PORT sorted out already as described in their docs. See this for clarity:
app.listen(process.env.PORT || 4900, ()=>{})
Note: It was working before until I introduced the Redis bit just a day ago.
Could there be an issue with the way I am running both server and the Queue process in the package.json file? I have been reading answers similar to this, but they are usually focused on the PORT fix which is not my own issue as far as I know.
TroubleShooting : I removed the queue process from the start script and the issue was gone. I had this instead
"scripts": {
"start": "node server.js -p $PORT"
}
So it becomes clear that this line below;
node ./app/services/queues/process.js was the issue
Now, How then do I run this queue process script? I need it to run to listen to any subscription and then run the processor script. It works fine locally with the former start script.
Please Note: I am using Bull for the Queue. I followed this guide to implement it and it worked fine locally.
Bull Redis Implementation Nodejs Guide
I will appreciate any help on this as I am currently blocked on my development.
So I decided to go another way. I read up on how to run background jobs on heroku with Bull and I got a guide which I implemented. The idea is to utilize Node's concurrency API. For the guide a wrapper was used called throng to implement this.
I removed the process file and just wrapped my consumer script inside the start function and passed that to throng.
Link to heroku guide on enabling concurrency in your app
Result: I started getting EADDR in use Error which was because that app.listen() is being run twice..
Solution: I had to wrap the app.listen function inside a worker and pass it to throng and it worked fine.
Link to the solution to the EADDR in use Error
On my local Machine, I was able to push to the Queue and consume from it. After deploying to heroku, I am not getting any errors so far.
I have tested the update on heroku and it works fine too.
I am trying to execute an api in laravel every minute.
The api's method is GET. However I could not specify the method in the cron.yaml file. Could I use DELETE method here and how? The code should be deployed on google cloud.
I have created a cron.yaml file that has the following format:
cron:
- description: "every minutes job"
url: /deletestories
schedule: every 1 mins
retry_parameters:
min_backoff_seconds: 2.5
max_doublings: 5
I also created the api deletestories that delete rows under specific conditions.
However this isn't working, and when I open google cloud console I could not found any error or any cron job executed.
This cron.yaml file appears to be a Google App Engine cron configuration. If this is correct then only the GET method is supported, you cannot use DELETE.
The GAE cron service itself consists simply of scheduled GET requests that your app needs to handle. From Scheduling Tasks With Cron for Python (the same applies to other languages and to the flexible environment cron as well):
A cron job makes an HTTP GET request to a URL as scheduled. The
handler for that URL executes the logic when it is called.
You also need to deploy your cron.yaml file for it to be effective. You should be able to see the deployed cron configuration in the developer console's Cron Jobs tab under the Task Queues Menu (where you can also manually trigger any of the cron jobs). The performed GET requests for the respective cron jobs should appear in your app's request logs as well, when executed.
I'm really struggling with this issue, which is a little frustrating to me because it seems to be so simple in a Linux server. I have a Windows Azure Web App and I want to run "php artisan queue:listen" on the server continuously to take care of dispatched jobs. From what I read from the documentation, in Linux you just use Supervisor to run the command constantly and to revive it, in case it dies. From what I found online, Azure has a similar functionality called WebJobs where you can serve them a script to be ran and then decide whether it should run on a schedule or continuously (kinda like the Scheduler in Laravel). With this I have 2 questions.
1 - Is this the right solution? Place a script to run the command on a WebJob and have the WebJob run continuously?
2 - I'm not experienced in writing php scripts to run command lines, so all I can do is something like this:
echo shell_exec('php artisan queue:work');
Problem is this does not give me the output of the command (I don't see anything like the "processed" result that I see when I run the command by hand on my command console and a job is processed). It is important to me to be able to read the output of the command, because I want to be able to check the logs for errors in case something happens when a job isn't able to be processed. From the documentation shell_exec returns null in case an error is thrown so I'm completely clueless on how to deal with this.
Thank you so much in advance!
Instead of using shell_exec() you can directly upload .cmd file which includes your command php artisan queue:work, and then you can find the output log in WebJob Details page.
About how to do that, please check Ernesto's answer out.
For Azure you can make a new webjob to your web app, and upload a .cmd
file including a command like this.
php %HOME%\site\wwwroot\artisan queue:work --daemon
and defining that as a triguered and 0 * * * * * frecuency cron.
that way work for me.
For more information, please refer to Run Background tasks with WebJobs.
I'm having a PHP file which has some JS in it. I run a script on this file which will check if the date on a firebase database matches with the current date and if not update a few values. So I want this PHP file to run 24x7. I am completely new to heroku. Is it possible to do this on heroku or is there any other simpler solution ? Any help would be appreciated.
I don't see why not. You can set a worker process in your Procfile that will run a non web based process. In that PHP script you can set a timer to perform this action daily, and if were to crash for whatever reason, Heroku would restart the process.
Here's an explanation of how they work: https://devcenter.heroku.com/articles/background-jobs-queueing
I'm trying to start a service via a script that I run through cron. Basically it just does this
/local/services/servicename status
/local/services/servicename stop
/local/services/servicename start
The service starts fine if I run the commands myself, but whenever I run it via the script, and I check for the service status manually, its response is always
Servicename Service is not running.
I am truly confuse right now. Any particular reason why a bash script wouldn't be able to start the services?
Not really an answer to your specific question, but definitely a useful tip for debugging cron behavior in general. Cron sends email messages to the user it runs as. If you run that in the root crontab, run the mail command in a terminal as root and you'll see the cron messages in your inbox. Check them by typing the message number (or man mail to learn how to use it).