Node Manager Stop - Hadoop 2.2.0 - hadoop

On invoking ./stop-yarn.sh, I get
"nodemanager did not stop gracefully after 5 seconds: killing with kill -9".
Is this something I should be concerned about? What should I do to avoid this error?

There is not way to do a graceful stop till yet. you can follow this link https://issues.apache.org/jira/browse/YARN-914 for tracking and feature changes.

The reason is because the Resource Manager (RM) is stopped before Node Manager (NM). The NM needs to unregister itself from the RM. So when NM tries to connect with RM (to unregister itself) it fails as RM is already stopped. And so a timeout occurs (after waiting 5 sec - mention in start-yarn.sh) causing NM to be killed and not allowing to stop gracefully.

You could use "hadoop job" to see if you have jobs still running and use the option "-kill " http://hadoop.apache.org/docs/r1.0.4/commands_manual.html#job

Related

Boot hangs: A start job is running

In a virtual box I have a Debian that I sometimes want to run without X. So I edited /etc/grub.d/10_linux and added another menu item with a kernel option "nox" appended. Then I added a line to /lib/systemd/system/lightdm.service, Section [Unit]:
ConditionKernelCommandLine=!nox
However, when starting this, it hangs with the message:
A start job is running for Hold until boot process finishes up (56min / no limit)
Thank you, systemd for informing me about that. I wouldn't have noticed. Yet, I would like to know, which job it is that's hanging.
The system allows me to connect via SSH, but none of the systemctl or journalctl commands I tried did tell me the name of the service causing the problem. lightdm.service itself seems to be satisfied.
I known it's a but late, but I just found out that one can use:
systemctl list-jobs
to find out what units are waiting or running at any given moment.
By adding systemd.debug-shell=1 to the kernel command line, a root shell will be available on TTY9 (crlt+alt+F9) to run the command above.
I first tried "systemd-analyze", and that gave me the message about "systemctl list-jobs".
hope this helps someone with similar problems.

Laravel horizon supervisor does not restart after horizon::terminate with forge daemon running

Horizon runs fine but only recently, after a deploy, supervisor and queue workers do not start back up again with Horizon GUI showing "Inactive"
To get them running again I can:
restart the daemon worker from within forge
restart the supervisor /etc/init.d/supervisor restart
My deploy script has php artisan horizon:terminate within it. I have also tried reset/purge and a combination thereof.
When I run terminate in the command with an inactive horizon, it seems to do nothing. When I run the same command with horizon active, it shuts it down but the daemon does not reboot supervisor.
The daemon runs without any errors throughout all of this.
Should terminate take down and bring up the service or is it the daemon itself?
Running horizon:terminate will kill the daemon, when the daemon is killed supervisor will realize this and boot up a new daemon. You can clearly see this if you monitor your server with htop while running terminate command.
If a long running job is running, it will run the current job until it finishes. Terminate in general is to reboot the process, to be certain the new code is loaded into horizon, this should be done after the last step in envoyer or similar deployment tool.
This seems like there is something wrong in your setup. Does the horizon process run before you call terminate, again check htop?. Or what happens when the command is called manually?

After jenkins job complete, the service also down

I have met a problem.
I used Jenkins to install haproxy and start the service, but after the job complete, the executor is free, and the haproxy daemon also disappear.
if I use sleep 30s after the service start, and the haproxy service will also alive at the 30s, after that, the haproxy daemon will down.
This behaviour is by design, as explained in ProcessTreeKiller. To avoid daemons spawned by the Jenkins build being terminated, add
export BUILD_ID=dontKillMe
to the beginning of your shell step.

Ubuntu run service in foreground

I've made a (docker) container for ddclient.
The problem is that I'm having problems in running that service in the foreground so that the docker container keeps running.
I've managed to keep the docker running by adding a bashat the end of the script but this is hackish, since the actual process it should be whatching is the ddclient.
Another way I found was to tail -f the log file, but if the service stops, the container will keep running instead of stoping.
Q: So is there any (easy) way to keep a service running in the foreground?
The problem with the process (any process) running in a container is signal management: you need to make sure the SIGKILL and other signals are properly communicated to the right process(es) in order to successfully stop/remove a container (and not leave zombie processes: see "PID 1 zombie reaping issue")
One option is at least to make your service at least write in a log file
ENTRYPOINT ["/bin/sh" "-c" ]
CMD yourProcess > log
That should keep it in foreground, as suggested in "How do I bring a daemon process to foreground?".
For a service, try and use as a base image phusion/baseimage-docker which manages other services properly.

Apache 2 - reload config on Windows

I have a PHP script that modifies my httpd.conf file, so I need to automatically reload it in Apache.
On Linux, there is graceful restart, but on Windows (I use the restart command) it terminates all the current connections. Is there a command as graceful restart on Windows? Is there a workaround on this?
Yes, you should use the -k switch.
httpd.exe -k restart or apache.exe -k restart
More info here has well. http://www.zrinity.com/developers/apache/usage.cfm
Edit:
It shouldn't that is the point of Graceful. Notice I used the -k. That is not the same as a normal restart. It let's the current sessions complete their task while the config is being reread, so that it will start taking new requests immediately.
From the documentation:
The USR1 or graceful signal causes the parent process to advise the children to exit after their current request (or to exit immediately if they're not serving anything). The parent re-reads its configuration files and re-opens its log files. As each child dies off the parent replaces it with a child from the new generation of the configuration, which begins serving new requests immediately.
http://httpd.apache.org/docs/2.2/stopping.html#graceful
It's doing what you are asking for.
Edit 2:
Adding this link and gave both possible versions because some people think you there is only one specific way to do something instead of search themselves.
http://httpd.apache.org/docs/2.4/platform/windows.html#wincons
I think I'm just going to delete this answer because either people can't read or if it doesn't work for someone it gets a DV. There are different windows versions made by different developers. If it doesn't work look for the answer from them. Even Linux has different commands depending on the distro. geez
In the newest Apache 2.4.20 VC10 the "httpd -k restart" command actually DOES do a graceful restart. It won't drop any connections, for example if somebody is downloading something from your server, it WILL NOT interrupt this process. One more proof is that "-k restart" will not reset your server statistics that mod_status provides, won't even alter the "Restart Time" value.
Although "httpd -k graceful" and "httpd -k graceful-stop" commands are available in Windows, but they will not work giving an error "couldn't make a socket".

Resources