When a machine gets rebooted. it fails on chef-client run(run from rc.local) but if i run it manually it gets success and same error return on next auto chef-run.when i restart the chef-client the error stop coming .
in fact when chef-client was failing on auto run ,ohai was giving the correct output.
root#something:~# ohai | grep 'fqdn'
"fqdn": "something.someone",
root#something:~# chef-client -v
Chef: 12.20.3
root#something:~# ohai -v
Ohai: 8.23.0
root#something:~# hostname -f
something.someone
Output of stacktrace
root#something:~# cat /var/chef/cache/chef-stacktrace.out Generated at
2018-04-02 15:36:30 +0000 Chef::Exceptions::CannotDetermineNodeName:
Unable to determine node name: configure node_name or configure the
system's hostname and fqdn
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:299:in
node_name'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:313:in
register'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:416:in
do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:213:in
block in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:207:in
fork'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/client.rb:207:in
run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application.rb:237:in
run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/client.rb:338:in
block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/client.rb:327:in
loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application/client.rb:327:in
run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/lib/chef/application.rb:55:in
run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.14.2/bin/chef-client:26:in
' /usr/bin/chef-client:23:in load'
/usr/bin/chef-client:23:in'root#web1.cst.webpod1-cph3:~#
You've clearly got a mismatch in what you are running. Your CLI says Chef 12, but the stack trace says Chef 11. I'm guessing you installed one via our Omnibus packages and the other via Rubygems? No matter the cause, fix that conflict and you'll probably be happier.
Related
I am trying passenger on Ubuntu 18.04, pretty much generic install and the passenger-status call crashes. Not super clear how to set the path that it uses for local socket communication.
Version : 6.0.12
Date : 2021-11-17 23:20:19 +0000
Instance: PdoRpeL5 (Apache/2.4.29 (Ubuntu) Phusion_Passenger/6.0.12)
Traceback (most recent call last):
6: from /usr/sbin/passenger-status:349:in `<main>'
5: from /usr/sbin/passenger-status:346:in `start'
4: from /usr/sbin/passenger-status:62:in `command_show_status'
3: from /usr/sbin/passenger-status:124:in `show_status'
2: from /usr/lib/ruby/vendor_ruby/phusion_passenger/admin_tools/instance.rb:94:in `http_request'
1: from /usr/lib/ruby/vendor_ruby/phusion_passenger/admin_tools/instance.rb:94:in `new'
/usr/lib/ruby/vendor_ruby/phusion_passenger/admin_tools/instance.rb:94:in `initialize': too long unix socket path (116bytes given but 108bytes max) (ArgumentError)
Looking at that code, #path is /tmp/systemd-private-af50a27a57d04fec9366f72dd251a3be-apache2.service-Dy9bBV/tmp/passenger.PPuC2RN which makes the socket name /tmp/systemd-private-af50a27a57d04fec9366f72dd251a3be-apache2.service-Dy9bBV/tmp/passenger.PPuC2RN/agents.s/core_api
Seems like a passenger internal issue but I'm looking for a workaround...
Have you tried editing your web server service file, nginx.service or apache2.service? I had this problem and solved it by removing the line:
PrivateTmp=true
from /etc/systemd/system/multi-user.target.wants/apache2.service.
Then:
sudo systemctl daemon-reload
sudo systemctl restart apache2
https://github.com/phusion/passenger/issues/2397
"this is because the path generated by the systemd private tmp feature is 80 chars long, which only leaves 27 chars for an application to use, and passenger uses 35 chars in its path to its socket."
I'm running a provisioning setup using vagrant and chef-solo to install the gitlab cookbook (however, this problem does not seem to be specific to the gitlab cookbook).
The run-list is simple:
{
"run_list": [
"recipe[ohai::default]"
]
}
The chef-solo run (chef-solo -c solo.rb -j dna.json) results in the following error:
Error executing action create on resource 'cookbook_file[/etc/chef/ohai_plugins/README]'
The file IS there:
# ls -l /tmp/vagrant-chef/e939c8a8cabcf9cdd72f5d7c3a98d728/cookbooks/ohai-2.0.1/files/default/plugins/README
-rwxrwxrwx. 1 vagrant vagrant 49 Oct 21 13:16 /tmp/vagrant-chef/e939c8a8cabcf9cdd72f5d7c3a98d728/cookbooks/ohai-2.0.1/files/default/plugins/README
When I check the process with 'strace' it looks like I can see the error source:
10444 0.000210 stat("/tmp/vagrant-chef/cookbooks/cookbooks/ohai/files/default/plugins/README", 0x7ffff0570bb0) = -1 ENOENT (No such file or directory)
If you notice, the cookbook is named 'ohai-2.0.1' however the process is trying to access the cookbook 'ohai' (i.e., without the version number).
Has anyone else encountered this before? I've seen one other post related to the issue that suggested putting ohai::default first in the run-list, which I've done (see dna.json above)
This happened to me while using Berkshelf. It names all of the cookbooks name-version instead of just name. To solve this, I did berks vendor and added that directory to my cookbooks path. Now everything works!
I have tried to troubleshoot an UnknownError returned by the CodeDeploy Agent on an Amazon Linux EC2 instance. I push files from my local git repo on my Mac to a S3 bucket, and deploy from there to the EC2 instance.
I have edited the /etc/codedeploy-agent/conf/codedeployagent.yml and set :verbose: to true on a EC2 instance. I tried to deploy again and open codedeploy-agent.log at /var/log/aws/codedeploy-agent.
The last lines in log before failing shows that the agent installs ELB scripts provided by AWS.
2015-06-19 08:09:13 DEBUG [codedeploy-agent(7637)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandBuilder: Copying /opt/codedeploy-agent/deployment-root/c6b25857-8911-4f61-8b30-b39f0c34c395/d-6UZ0VFID9/deployment-archive/scripts/stop_server.sh to /var/www/html/scripts/stop_server.sh
2015-06-19 08:09:13 DEBUG [codedeploy-agent(7637)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandBuilder: Copying /opt/codedeploy-agent/deployment-root/c6b25857-8911-4f61-8b30-b39f0c34c395/d-6UZ0VFID9/deployment-archive/scripts/register_with_elb.sh to /var/www/html/scripts/register_with_elb.sh
2015-06-19 08:09:13 DEBUG [codedeploy-agent(7637)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Calling PutHostCommandComplete: "Code Error"
2015-06-19 08:09:13 INFO [codedeploy-agent(7637)]: [Aws::CodeDeployCommand::Client 200 0.084236 0 retries] put_host_command_complete(command_status:"Failed",diagnostics:{format:"JSON",payload:"{\"error_code\":5,\"script_name\":\"\",\"message\":\"\\\"\\\\xCC\\\" from ASCII-8BIT to UTF-8\",\"log\":\"\"}"},host_command_identifier:"WyJjb20uYW1hem9uLmFwb2xsby5kZXBsb3ljb250cm9sLmRvbWFpbi5Ib3N0Q29tbWFuZElkZW50aWZpZXIiLHsiZGVwbG95bWVudElkIjoiQ29kZURlcGxveS9ldS13ZXN0LTEvUHJvZC9hcm46YXdzOnNkczpldS13ZXN0LTE6NzMzNTE3MzY5MDAyOmRlcGxveW1lbnQvZC02VVowVkZJRDkiLCJob3N0SWQiOiJhcm46YXdzOmVjMjpldS13ZXN0LTE6NzMzNTE3MzY5MDAyOmluc3RhbmNlL2ktZWRjYTkzNDciLCJjb21tYW5kTmFtZSI6Ikluc3RhbGwiLCJjb21tYW5kUG9zaXRpb24iOjQsImNvbW1hbmRBdHRlbXB0IjoxfV0=")
The last files installed are not related. Since I also have tried to remove the scripts I mention above. The error seems to be upstream and Ruby related.
2015-06-19 08:09:13 ERROR [codedeploy-agent(7637)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform: Encoding::UndefinedConversionError - "\xCC" from ASCII-8BIT to UTF-8 - /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/install_instruction.rb:165:in `encode'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/install_instruction.rb:165:in `to_json'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/install_instruction.rb:165:in `to_json'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/installer.rb:38:in `block in install'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/installer.rb:37:in `open'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/installer.rb:37:in `install'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:103:in `block in <class:CommandExecutor>'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:51:in `execute_command'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:123:in `process_command'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:56:in `perform'
/opt/codedeploy-agent/lib/instance_agent/agent/base.rb:28:in `run'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:38:in `block in run'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:37:in `run'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:70:in `block in run_with_error_handling'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:69:in `run_with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:33:in `block in start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `loop'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:198:in `block in spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:196:in `fork'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:196:in `spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:188:in `block in spawn_children'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:187:in `times'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:187:in `spawn_children'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:133:in `start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:36:in `block in start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:35:in `fork'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:35:in `start'
/opt/codedeploy-agent/bin/codedeploy-agent:37:in `block (2 levels) in <main>'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `call'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `execute'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:262:in `block in call_command'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call_command'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:69:in `run'
/opt/codedeploy-agent/bin/codedeploy-agent:84:in `<main>'
I have tried $ grep --color -in -r '\xCC' /my/folder/ which returns something like
Binary file /my/folder/a.png matches
But also returned /my/folder/wp-content/plugins/wordpress-seo/admin/class-metabox.php:348:'Ì'
So I removed the plugin and tried to convert the filenames from ASCII to UTF-8 with convmv (it didn't convert anything). But without success.
I have also updated several gems according to a couple of StackOverflow threads.
How could I mitigate this error? I find the error message from Amazon CodeDeploy vaugue and hard to troubleshoot and I've run out of ideas.
Issue looks to be LANG env has not been set when Installing / Starting CodeDeploy.
I found that if I were to restart CodeDeploy via the CLI then the app would deploy without issue.
You can check the LANG env of the codedploy process by running
ps auxe |grep codede
We deploy CodeDeploy via puppet and have to set (environment => 'LANG=en_US.UTF-8') when running the install script.
Try updating codedeploy-agent. It worked for me.
SSH into your instance and run this command:
sudo /opt/codedeploy-agent/bin/install auto
The LANG environment variable must be defined when the deployment starts or else the CodeDeploy install event fails.
You can ensure that CodeDeploy has it by editing the CodeDeploy agent service configuration:
sudo vim /etc/init.d/codedeploy-agent
You can export the variable when the service first starts with this line: export LANG="en_US.UTF-8".
So the start block will look like this:
start() {
echo -n $"Starting $prog:"
export LANG="en_US.UTF-8"
cd $AGENT_ROOT
nohup $BIN start >/dev/null </dev/null 2>&1 # Try to start the server
exit $?
}
Some additional notes
For me the problem was only occurring during an auto scale event. In other words I could manually launch the instance and then deploy just fine, but not right when a new instance was launched. This indicates a race condition in the sense that the environment is not ready as soon as the service starts. I tried changing the upstart timing by running:
update-rc.d -f codedeploy-agent remove
update-rc.d codedeploy-agent start 99 2 3 4 5 . stop 20 0 1 6 .
but this did not work. Setting the variable in the start block guaranteed that it was available in time.
To ensure you have all required environment variables available to CodeDeploy as soon as it starts it's probably wise to follow Amazon's advice:
we recommend that you keep the agent service in a stopped state and use the launch scripts to start the agent service
When I try to start my sensu service it fails and I get the following error in the log. It was working before and then when I did a restart I continued to get the following error. I am not sure how to debug it. Appreciate any help in the right direction.
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3-x86-mingw32/lib/eventmachine.rb:859:in
open_udp_socket': no datagram socket (RuntimeError) from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3-x86-mingw32/lib/eventmachine.rb:859:in
open_datagram_socket' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/client.rb:257:in
setup_sockets' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/client.rb:293:in
start' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/client.rb:13:in
block in run' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3-x86-mingw32/lib/eventmachine.rb:187:in
call' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3-x86-mingw32/lib/eventmachine.rb:187:in
run_machine' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3-x86-mingw32/lib/eventmachine.rb:187:in
run' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/client.rb:12:in
run' from
C:/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/bin/sensu-client:10:in
' from C:/opt/sensu/embedded/bin/sensu-client:23:in
load' from C:/opt/sensu/embedded/bin/sensu-client:23:in'
I don't know what kind of Windows version do you have but netstat -a -n -o does the job on most of versions.
I am trying to setup redmine on my server which is Redhat 6.2
I intend it to run with Nginx using thin ruby gem.
I was following http://www.redmine.org/projects/redmine/wiki/HowTo_configure_Nginx_to_run_Redmine
I did following things
gem install thin
thin install
this gives me init script under /etc/rc.d/thin The YML file config is as follows:
---
chdir: /app/redmine-root/
environment: development
address: 0.0.0.0
port: 5000
timeout: 30
log: log/thin.log
pid: tmp/pids/thin.pid
max_conns: 1024
max_persistent_conns: 100
require: []
wait: 30
servers: 4
daemonize: true
Now when I do /etc/rc.d/thin start it shows
Starting server on 0.0.0.0:5000 ...
Starting server on 0.0.0.0:5001 ...
Starting server on 0.0.0.0:5002 ...
Starting server on 0.0.0.0:5003 ...
but when I see pids under /app/redmine-root/tmp/pids there are no Pids.
therefore, I can't see any service running. This is issue number 1
Second thing I'd like to ask that, in Nginx conf as suggested by the above link the upstream block is like as follows:
upstream thin_cluster {
server unix:/tmp/thin.0.sock;
server unix:/tmp/thin.1.sock;
server unix:/tmp/thin.2.sock;
server unix:/tmp/thin.3.sock;
}
But the pid file is in /app/redmine-root/tmp/pids should this work?
Third, At the time of Install I marked the env as production
RAILS_ENV=production rake db:migrate
But whenever I do thin config -C /etc/thin/redmine.yml it changes to development.
Please note that I have RVM also. And the user & owner of /app/redmine-root/ is apache.
My nginx runs with apache and I am trying to run thin also as apache.
I have no background in Ruby. Any help is much appreciated.
EDIT
After suggestions, I found this in log.
/usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/backends/tcp_server.rb:16:in `connect': cannot load such file -- thin/connection (LoadError)
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/backends/base.rb:55:in `block in start'
from /usr/local/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `call'
from /usr/local/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'
from /usr/local/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/backends/base.rb:63:in `start'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/server.rb:159:in `start'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/controllers/controller.rb:86:in `start'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/runner.rb:187:in `run_command'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/lib/thin/runner.rb:152:in `run!'
from /usr/local/lib/ruby/gems/2.0.0/gems/thin-1.5.1/bin/thin:6:in `<top (required)>'
from /usr/local/bin/thin:23:in `load'
from /usr/local/bin/thin:23:in `<main>'
is it because, I am trying to configure on UNIX socket or its something else??
You should add gem thin to your Gemfile
PS: see at https://github.com/macournoyer/thin/issues/115 for example.