I wanted to set vm.swappiness value for my Amazon EC2 instance runnign RedHat Enterprise 7. I am able to set it manually after machine starts using
sudo sysctl -w vm.swappiness=10
But vm.swappiness gets reset to default 30 once I stop and start my instance. I googled for it and found couple of solutions like
Adding vm.swappiness = 10 in /etc/sysctl.conf. But this didn't work.
Adding vm.swappiness = 10 in /etc/sysctl.conf and adding sysctl -p in /etc/rc.local. This also didn't work. When I run sysctl -p manually, it works correctly.
Nevermind. Found solution after consulting AWS support for same. Need to change the value in /usr/lib/tuned/virtual-guest/tuned.conf.
Related
I want to install Pseudo-Distributed HBase environment on my Mac OS Sierra (10.12.4), and it requires ssh installed and can log with ssh localhost without password. But sometimes I came across with error when I use ssh to log in. Above all are question background, and the actual question is where can I find debug logs of sshd so I could know why logging is failed in further?
As I know, Mac OS already have sshd installed and use launchd to manage it, and I know one way to output debug logs by sshd -E /var/log/sshd.log, but when I reviewed /etc/ssh/sshd_config configuration and there are two lines:
#SyslogFacility AUTH
#LogLevel INFO
I guess these two lines are used to config debug mode, then I removed # before them and set LogLevel to DEBUG3 and then restarted sshd:
$ launchctl unload -w /System/Library/LaunchDaemons/ssh.plist
$ launchctl load -w /System/Library/LaunchDaemons/ssh.plist
And then I set log path in /etc/syslog.conf:
auth.*<tab>/var/log/sshd.log
<tab> means tab character here, and reloaded the config:
$ killall -HUP syslogd
But sshd.log file can not be found in /var/log folder when I executed ssh localhost. I also tried config the /etc/asl.log:
> /var/log/sshd.log format=raw
? [= Facility auth] file sshd.log
And the result was the same, can someone help me?
Apple, as usual, decided to re-invent the wheel.
In super-user window
# log config --mode "level:debug" --subsystem com.openssh.sshd
# log stream --level debug 2>&1 | tee /tmp/logs.out
In another window
$ ssh localhost
$ exit
Back in Super-user window
^C (interrupt)
# grep sshd /tmp/logs.out
2019-01-11 08:53:38.991639-0500 0x17faa85 Debug 0x0 37284 sshd: (libsystem_network.dylib) sa_dst_compare_internal <private>#0 < <private>#0
2019-01-11 08:53:38.992451-0500 0xb47b5b Debug 0x0 57066 socketfilterfw: (Security) [com.apple.securityd:unixio] open(/usr/sbin/sshd,0x0,0x1b6) = 12
...
...
In super-user window, restore default sshd logging
# log config --mode "level:default" --subsystem com.openssh.sshd
You can find it in /var/log/system.log. Better if you filter by "sshd":
cat /var/log/system.log | grep sshd
Try this
cp /System/Library/LaunchDaemons/ssh.plist /Library/LaunchDaemons/ssh.plist
Then
vi /Library/LaunchDaemons/ssh.plist
And add your -E as shown below
<array>
<string>/usr/sbin/sshd</string>
<string>-i</string>
<string>-E</string>
<string>/var/log/system.log</string>
</array>
And lastly restart sshd now you will see sshd logs in /var/log/system.log
launchctl unload /System/Library/LaunchDaemons/ssh.plist && launchctl load -w /Library/LaunchDaemons/ssh.plist
I also had an ssh issue that I wanted to debug further and was not able to figure out how to get the sshd debug logs to appear in any of the usual places. I resorted to editing the /System/Library/LaunchDaemons/ssh.plist file to add a -E <log file location> parameter (/tmp/sshd.log, for example). I also edited /etc/ssh/sshd_config to change the LogLevel. With these changes, I was able to view the more verbose logs in the specified log file.
I don't have much experience with MacOS so I'm sure there is a more correct way to configure this, but for lack of a better approach this got the logs I was looking for.
According to Apple's developer website, logging behavior has changed in macOS 10.12 and up:
Important:
Unified logging is available in iOS 10.0 and later, macOS 10.12 and later, tvOS 10.0 and later, and watchOS 3.0 and later, and supersedes ASL (Apple System Logger) and the Syslog APIs. Historically, log messages were written to specific locations on disk, such as /etc/system.log. The unified logging system stores messages in memory and in a data store, rather than writing to text-based log files.
Unfortunately, unless someone comes up with a pretty clever way to extract the log entries from memory or this mysterious "data store", I think we're SOL :/
There is some sshd log in
/var/log/system.log
for example
Apr 26 19:00:11 mac-de-mamie com.apple.xpc.launchd[1] (com.openssh.sshd.7AAF2A76-3475-4D2A-9EEC-B9624143F2C2[535]): Service exited with abnormal code: 1
Not very instructive. I doubt if more can be obtained. LogLevel VERBOSE and LogLevel DEBUG3 in sshd_config do not help.
According to man sshd_config :
"Logging with a DEBUG level violates the privacy of users and is not recommended."
By the way, I relaunched sshd not with launchctl but with System preference Sharing, ticking Remote login.
There, I noticed the option : Allow access for ...
I suspect this settings to be OUTSIDE /etc/ssh/sshd_config
(easy to check but I have no time).
Beware that Mac OS X is not Unix : Apple developpers can do many strange things behind the scene without any care for us command line users.
I've been trying to figure this out for quite some time and I don't seem to be able to find any information on this issue.
diving into the issue:
I am running an application on Ubuntu 14.04 using Monit V5.6
The deployment of the application and Monit is done by using Chef scripts with AWS Opsworks which works excellent.
The problem is that once done, Monit starts the application using the following syntax:
start program = "/bin/sh -c 'ulimit -n 150000; <some more commands here which are not intersting>'" as uid <user> and gid <user_group>
This indeed starts the application using the correct user but the problem is that max open files for the process is showing 4096 instead of the number set in limits.conf
Just to be clear, I have set the following in /etc/security/limits.conf
root hard nofile 150000
root soft nofile 150000
* hard nofile 150000
* soft nofile 150000
Further more, if I stop the application then do a service monit restart and then start the application, the max open files values is received correctly and I am seeing 150000.
If I then redeploy the application without rebooting the instance then this happens again and I have to manually restart monit again.
Also if I run the application using the following syntax in order to mimic Monit:
sudo -H -u <user> /bin/sh -c 'ulimit -n 150000; <more commands here>'
Then everything is working and the process is receiving the correct value of max open files.
I try to script this manual service monit restart with stopping and starting the application via Chef scripts then this also fails and I receive 4096 as the max open files value thus my only option is to manually do this each time I deploy which is not very convenient.
Any help on this or thoughts would be greatly appreciated.
Thanks!
P.S. I also reviewed the following articles:
https://serverfault.com/questions/797650/centos-monit-ulimit-not-working
https://lists.nongnu.org/archive/html/monit-general/2010-04/msg00018.html
but as manually restarting Monit causes this to work then I am looking for a solution without changing init scripts.
After SSH'ing into an EC2 instance running the Amazon Linux AMI, I tried:
ulimit -n 20000
...and got the following error:
-bash: ulimit: open files: cannot modify limit: Operation not permitted
However, the shell allows me to decrease this number, for the current session only.
Is there anyway to increase the ulimit on an EC2 instance (permanently)?
In fact, changing values through the ulimit command only applies to the current shell session. If you want to permanently set a new limit, you must edit the /etc/security/limits.conf file and set your hard and soft limits. Here's an example:
# <domain> <type> <item> <value>
* soft nofile 20000
* hard nofile 20000
Save the file, log-out, log-in again and test the configuration through the ulimit -n command. Hope it helps.
P.S. 1: Keep the following in mind:
Soft limit: value that the kernel enforces for the corresponding resource.
Hard limit: works as a ceiling for the soft limit.
P.S. 2: Additional files in /etc/security/limits.d/ might affect what is configured in limits.conf.
Thank you for the answer. For me just updating /etc/security/limits.conf wasn't enough. Only the 'open files' ulimit -n was getting updated and nproc was not getting updated. After updating /etc/security/limits.d/whateverfile, nproc "ulimit -u" also got updated.
Steps:
sudo vi /etc/security/limits.d/whateverfile
Update limits set for nproc/ nofile
sudo vi /etc/security/limits.conf
* soft nproc 65535
* hard nproc 65535
* soft nofile 65535
* hard nofile 65535
Reboot the machine sudo reboot
P.S. I was not able to add it as a comment, so had to post as an answer.
I don't have enough rep points to comment...sorry for the fresh reply, but maybe this will keep someone from wasting an hour.
Viccari's answer finally solved this headache for me. Every other source tells you to edit the limits.conf file, and if that doesn't work, to add
session required pam_limits.so
to the /etc/pam.d/common-session file
DO NOT DO THIS!
I'm running an Ubuntu 18.04.5 EC2 instance, and this locked me out of SSH entirely. I could log in, but as soon as it was about to drop me into a prompt, it dropped my connection (I even saw all the welcome messages and stuff). Verbose showed this as the last error:
fd 1 is not O_NONBLOCK
and I couldn't find an answer to what that meant. So, after shutting down the instance, waiting about an hour to snapshot the volume, and then mounting it to another running instance, I removed the edit to the common-session file and bam, SSH login worked again.
The fix that worked for me was looking for files in the /etc/security/limits.d/ folder, and editing those.
(and no, I did not need to reboot to get the new limits, just log out and back in)
I've been trying to figure out how to use postfix on my Mac and something has gone horribly wrong and I can't seem to fix it.
I believe the problem is related to starting Postfix.
Basically, the Mac seems to refuse to change its hostname. In bash, the user appears as "admin#(null)", if I type 'hostname' I'm given "(null)" also.
Changing the hostname in Sharing from System Preferences causes the second example to change (where it says, for example, "Other users can access shared folders on this computer, and administrators all volumes, at afp://null/ or “lion2”.") but the first stays as null.
I've even tried /etc/hostconfig manually setting hostname but nothing works.
Is there somewhere else the hostname is trying to be set but is perhaps corrupt? Or contains an invalid character or something?
This is causing Postfix not to work and report:
postfix: warning: valid_hostname: invalid character 40(decimal): (null)
postfix: fatal: unable to use my own hostname
Please, I really hope someone can help me fix this. I've been trying for hours now.
Cheers,
Scott
Have you tried scutil?
sudo scutil --get pref will show the current value and sudo scutil --set pref name will set the value to name. pref can be one of these:
ComputerName The user-friendly name for the system.
LocalHostName The local (Bonjour) host name.
HostName The name associated with hostname(1) and gethostname(3).
Here's what I get on my machine:
$ sudo scutil --get ComputerName
SteveBook2
$ sudo scutil --get LocalHostName
SteveBook2
$ sudo scutil --get HostName
HostName: not set
You should have tried running /bin/hostname directly to set the hostname on the unix/bsd layer, the values from scutil are SystemConfiguration settings which is a higher layer that unix is oblivious to.
It is perfectly normal for sudo scutil --get HostName to return not set even if running /bin/hostname shows you a hostname.
To set the hostname run sudo hostname Foo.bar (this is basically identical to sethostname() BSD call in the code given by another answer)
Optionally you could then run sudo scutil --set HostName Foo.bar to keep the SystemConfiguration settings in sync
NOTE:
The HostName in SystemConfiguration can be different from the LocalHostName and ComputerName , it can also be different from what /bin/hostname returns but it is best that they are all in sync, so you could also do :
sudo scutil --set LocalHostName Foo
sudo scutil --set ComputerName Foo
All other answers and help was much appreciated, however after much investigation, the problem appears to lie with my router and iMac: router not allowing the iMac to change its hostname client-side OR possibly sending a weird hostname to the iMac for it to use.
If you run this tiny program before starting postfix, does it work?
#include <unistd.h>
int main(int argc, char* argv[]) {
char host[] = "newhostname";
sethostname(host, sizeof(host));
return 0;
}
I don't presume to know what else might depend upon your old hostname -- do some testing on all your services after running this.
Details at: http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/gethostname.3.html
Update
To compile and run this little program, save its contents to a file (/tmp/newhostname.c will do) then run:
cd /tmp
make newhostname
sudo ./newhostname
At least I assume your make(1) knows how to compile from C sources to runnable binaries with default rules.
If the compiler isn't already installed maybe someone else will have a better idea.
I'm running a couple of standard Fedora instances on EC2. I feel the public hostnames of the instances assigned by Amazon are too weird and hard to remember. I'd like to change them to something short (like red/blue/green/etc).
Is there any draw back in doing this? And how do I set it up such that it persists after reboots?
Thanks.
Before you get started, try running hostname and hostname --fqdn and take note of what the responses are.
You can edit /etc/hostname and set a hostname, which will stick around after rebooting. You can force the hostname to be "reloaded" by using hostname -F /etc/hostname to read that value into the hostname. The bash prompt will change after you logout and login.
warning / note:
Yes, it is nice to have the hostname in the bash prompt set to something more useful than ip-123-123-123-123 but I've decided to leave mine (at least for now) because it seems like a lot of things really count on having the hostname on ec2 instances set in a standard way. After editing /etc/hostname and changing the hostname to webserver a lot of the services seems to fail because the hostname would not resolve, and apache wouldn't start. Next I edited /etc/hosts and added in
127.0.0.1 webserver
as the second line. Apache would then start but complained that it couldn't find the FQDN. I confirmed that running hostname --fqdn no longer worked.
Next I consulted man hostname and learned that while you can set the hostname it appears that the FQDN is what is returned via a DNS lookup.
THE FQDN
You can't change the FQDN (as returned by hostname --fqdn) or the DNS domain name (as returned by dnsdomainname) with this command. The FQDN of the system is the name that the resolver(3) returns for the host name.
Technically: The FQDN is the name getaddrinfo(3) returns for the host name returned by gethostname(2). The DNS domain name is the part after the first dot.
Therefore it depends on the configuration (usually in /etc/host.conf) how you can change it. Usually (if the hosts file is parsed before DNS or NIS) you can change it in /etc/hosts.
I think it might be possible to set the system / fool the system into return the FQDN, something like ip-123-123-123-123.ec2.internal even though the hostname is webserver but at this point it started to seem like more trouble than it was worth, and that for me to have a nicer bash prompt might cause a lot software and configuration problems down the road and so I decided to give up.
I also learned that a lot of amazon ec2 instances use something called cloud-init:
cloud-init is the Ubuntu package that handles early initialization of a cloud instance. It is installed in the Ubuntu Cloud Images and also in the official Ubuntu images available on EC2.
Some of the things it configures are:
setting a default locale
setting hostname
generate ssh private keys
adding ssh keys to user's .ssh/authorized_keys so they can log in
setting up ephemeral mount points
cloud-init's behavior can be configured via user-data. User-data can be given by the user at instance launch time. This is done via the --user-data or --user-data-file argument to ec2-run-instances
I also found this which talks about how the hostname is configured with cloud-init:
On EBS instances, a shutdown and later start would end up with a different IP address.
In the case where the user has not modified /etc/hostname from its original value (seeded by metadata's 'local-hostname'), then cloud-init will again set the hostname and update /etc/hostname.
In the case where the user has modified /etc/hostname, it will remain user managed.
Additionally, if /etc/cloud/cloud.cfg contains 'preserve_hostname' value set to a True value, then /etc/hostname will not ever be touched.
The interesting takeaway is that if you don't change the hostname the cloud-init package will keep it up to date for you.
If someone else has a workaround or can address some of the issues mentioned and help reassure that nothing will break on ec2 instances because of changing the hostname I would be happy to hear it.
Another way is to simply edit ~/.bashrc and prepend PS1 with the nickname of the machine.
Edit: perhaps more correctly, machine-wide, e.g. on the AWS Linux AMI (an example) (paste this into console or add to your arbitrary install .sh):
cat << EOF | sudo tee /etc/profile.d/ps1.sh
if [ "$PS1" ]; then
PS1="[\u#myinst1:\l \t \! \W]\\$ "
fi
EOF
Edit /etc/sysconfig/network as root.
Replace
HOSTNAME=localhost.localdomain
with
HOSTNAME=hostname.DOMAIN_NAME
Then, either reboot or run /etc/init.d/network restart
The server then should report its name as a FQDN.
From this site:
Change the hostname on a running system
On any Linux system you can change its hostname with the command hostname (surprised?)…
Here are some quick usages of the command line hostname:
$> hostname
without any parameter it will output the current hostname of the system.
$> hostname --fqd
it will output the fully qualified domain name (or FQDN) of the system.
$> hostname NEW_NAME
will set the hostname of the system to NEW_NAME.
You can also edit /etc/hostname (at least on Ubuntu).
To make sure it stays after a reboot in AWS, either add the command in /etc/rc.local so it runs when the machine starts.
There's also a way to set the hostname dynamically via USER_DATA:
USER_DATA=`/usr/bin/curl -s http://169.254.169.254/latest/user-data`
HOSTNAME=`echo $USER_DATA`
IPV4=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/public-ipv4`
hostname $HOSTNAME
echo $HOSTNAME > /etc/hostname
To change the system hostname to a public DNS name
Follow this procedure if you already have a public DNS name registered
Open the /etc/sysconfig/network configuration file in your favorite text editor and change the HOSTNAME entry to reflect the fully qualified domain name (such as webserver.mydomain.com).
HOSTNAME=webserver.mydomain.com
Reboot the instance to pick up the new hostname.
[ec2-user ~]$ sudo reboot
Log into your instance and verify that the hostname has been updated. Your prompt should show the new hostname (up to the first ".") and the hostname command should show the fully qualified domain name.
[ec2-user#webserver ~]$ hostname
webserver.mydomain.com
To change the system hostname without a public DNS name
Open the /etc/sysconfig/network configuration file in your favorite text editor and change the HOSTNAME entry to reflect the desired system hostname (such as webserver).
HOSTNAME=webserver.localdomain
Open the /etc/hosts file in your favorite text editor and add an entry beginning with 127.0.1.1 (on DHCP systems) or eth0's address (on static IP systems) to match the example below, substituting your own hostname. (127.0.0.1 should be left as the localhost line.)
127.0.0.1 localhost localhost.localdomain
127.0.1.1 webserver.example.com webserver
Reboot the instance to pick up the new hostname.
[ec2-user ~]$ sudo reboot
Log into your instance and verify that the hostname has been updated. Your prompt should show the new hostname (up to the first ".") and the hostname command should show the fully qualified domain name.
[ec2-user#webserver ~]$ hostname
webserver.localdomain
Note: You can also change the shell prompt without affecting the hostname. Refer to this AWS documentation.
Sure, you can do that if you have your own domain (setup a CNAME to point to the Amazon hostname). Otherwise, you're pretty much stuck with the one they give you (or an Elastic IP, if you set one of those up).
The /etc/rc.local solution worked for me for a basic hostname but does not give me a FQDN.
In my Linux AMI (a snapshot of other instance).. none of the above formula worked. Then, I simply changed HOSTNAME field in file: /etc/init.d/modifyhostname and did a normal reboot.
You will need to do multiple things to set the hostname:
hostname newname - sets the hostname, but is volatile
edit /etc/hostname - sets the hostname for the next reboot
edit /etc/hosts - to keep sudo from complaining
I put these together into a script and uploaded it as a gist:
https://gist.github.com/mnebuerquo/5443532036af8b48995547e2817dba85
sudo hostname *yourdesiredhostnamehere*
sudo /etc/init.d/networking restart
then the hostname is changed. On my server all other services like apache and postfix works. Server is Ubuntu 12.04 LTS