What can I do to redirect the output of a command such as ls to the input of another command such as grep.
In other words:
what would be in the blank in the following?
$ ls /etc ____ grep host #
You can use the pipe | character.
Your command would become: ls /etc | grep host #
This is called piping and you simply use | in that spot.
ls /etc | grep host
Returns:
hosts
hosts.equiv
hosts-orig
Another example:
cat /etc/hosts | grep 127
Returns (for me):
#127.0.0.1 google.ca
127.0.0.1 localhost
Related
I'm trying to understand what's happening here out of curiosity, even though I can just copy and paste the output of the terminal to do what I need to do. The following command does not print anything.
ls /opt/local/var/macports/registry/portfiles -1 | sed 's/-.*//g' | sort -u | parallel "sudo port -N install" {} 2>&1 | grep -Po "Use '\K.*(?=')" | parallel "{}"
The directory I call ls on contains a bunch of filenames starting with the string I want to extract that ends at the first dash (so stringexample-4.2009 pipes stringexample into parallel (like xargs but to run each line separately). After running the command sudo port install <stringexample>, I get error outputs like so:
Unable to activate port <stringexample>. Use 'port -f activate <stringexample>' to force the activation.
Now, I wish to run port -f activate <stringexample>. However, I cannot seem to do anything with the output port -f activate gettext that I get to the terminal.
I cannot even do ... | grep -Po "Use '\K.*(?=')" | xargs echo or ... | grep -Po "Use '\K.*(?=')" >> commands_to_run.txt (the output stream to file only creates an empty file), despite the shorter part of the command:
ls /opt/local/var/macports/registry/portfiles -1 | sed 's/-.*//g' | sort -u | parallel "sudo port -N install {}" 2>&1 | grep -Po "Use '\K.*(?=')"
printing the commands to the terminal. Why does the pipe operator not work here? If the commands I wish to run are outputting to the terminal, surely there's got to be a way to capture them.
I would like to extract the IP v4 address only from my /etc/hosts file with shell sed command.
I managed to isolate those line with localhost at the end of the line with the following command:
$ sed -E '/localhost$/!d ' host_1 | sed -n 1p
which gave me the following output :
# 127.0.0.1 localhost
How can I only extract the IP v4 address alone form the above result?
The test data:
$ cat file
127.0.0.1 localhost
# 127.0.0.2 localhost
127 127.0.0.3 localhost
The sed:
$ sed -n 's/\(.*[^0-9]\|\)\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\).*/\2/p' file
The output:
127.0.0.1
127.0.0.2
127.0.0.3
The sed with the -E switch:
$ sed -nE 's/(.*[^0-9]|)([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+).*/\2/p' file
I tried this command and it works fine.
$ sed -E '/localhost$/!d' host_1 | sed -n 1p |grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"
output: 127.0.0.1
Nonetheless I am still curious about how can I substitute the grep by a sed command.
sed -E '/localhost$/!d ' /etc/hosts | awk '{print $1}' | grep -v :
Late but you can try this :
sed -E '/localhost$/!d ' host_1 | sed -n 1p | sed -r 's/\localhost//g'
The last part replaces the grep used by 4everLRG
If you need to extract localhost IP from /etc/hosts make sure to replace host_1 with /etc/hosts
I'm trying to create script to be run by cron to create multiple folders with subfolders.
DATE=`date +%Y-%m-%d`
IP_ADDR=`ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p'`
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
If i run this script manually everything is created as expected. When script is ran by cron subdirectory $IP_ADDR is not created and there is no errors.
I suspect that /sbin is not part of the PATH for the environment that the cron job runs under. You should specify the full path for the ifconfig command:
IP_ADDR=$(/sbin/ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p')
It's also better practice (in general) to use $() for command substitution.
Try to use debug mode :
set -x
DATE=`date +%Y-%m-%d`
IP_ADDR=`ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p'`
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
set +x
Then, redirect the output of your cron to a file and have a look, you should find useful information in it.
You are not far off, but there are several ordering caveats that could cause problems. Many systems have different formats for the ifconfig output line. Some with inet xxx.xxx.xxx.xxx, others with inet addr:xxx.xxx.xxx.xxx. (those are the two most common). You may also need to handle the case where there are multiple wired inet interfaces (2+ NICs in the box). However, if you have only 1 NIC, you could try the following to handle the common ifconfig formats:
DATE=`date +%Y-%m-%d`
IP_ADDR=$(ifconfig |
grep -v '127.0.0.1' |
grep -E 'inet[ ](addr:)*[0-9]{1,3}([.][0-9]{1,3}){3}' |
sed -e 's/^.*inet \(addr:\)*//' -e 's/ .*$//')
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
or with IP_ADDR written as one line:
IP_ADDR=$(ifconfig | grep -v '127.0.0.1' | grep -E 'inet[ ](addr:)*[0-9]{1,3}([.][0-9]{1,3}){3}' | sed -e 's/^.*inet \(addr:\)*//' -e 's/ .*$//')
I sent a batch of files to a remote server via SFTP. If it were a local directory I could do something like this ls -l | wc -l to get the total number of files. However, with SFTP, I get an error Can't ls: "/|" not found.
echo ls -l | sftp server | grep -v '^sftp' | wc -l
If you want to count the files in a directory the directory path should be put after the ls -l command like
echo ls -l /my/directory/ | sftp server | grep -v '^sftp' | wc -l
Use a batch file to run commands remotely and get the data back to work with in bash:
Make your batch file called mybatch.txt with these sftp commands:
cd your_directory/your_sub_directory
ls -l
Save it out and give it 777 permissions.
chmod 777 mybatch.txt
Then run it like this:
sftp your_username#your_server.com < mybatch.txt
It will prompt you for the password, enter it.
Then you get the output dumped to bash terminal. So you can pipe that to wc -l like this:
sftp your_user#your_server.com < mybatch.txt | wc -l
Connecting to your_server.com...
your_user#your_server.com's password:
8842
The 8842 is the number of lines returned by ls -l in that directory.
Instead of piping it to wc, you could dump it to a file for parsing to determine how many files/folders.
I would use sftp batch file.
Create a file called batchfile and enter "ls -l" in it.
Then run
sftp -b batchfile user#sftpHost | wc -l
The easiest way I have found is to use the lftp client which supports a shell-like syntax to transfer the output of remote ftp commands to local processes.
For example using the pipe character:
lftp -c 'connect sftp://user_name:password#host_name/directory; ls -l | wc -l'
This will make lftp spawn a local wc -l and give it the output of the remote ls -l ftp command on its stdin.
Shell redirection syntax is also supported and will write directly to local files:
lftp -c 'connect sftp://user_name:password#host_name/directory; ls -l >list.txt'
Thus a file named list.txt containing the remote file listing will be created in the current folder on the local machine. Use >> to append instead.
Works perfectly for me.
In a shell script how to validate if a given host name is localhost
for example :-
localhost
127.0.0.1
myhostname.com
::1
all belong to same machine name, Is there any way to identify that a given host name belongs to localhostname family
Usually all local host names are in /etc/hosts file:
grep -c machine_name /etc/hosts
if the machine name is among the localhost the command above returns 1 or greater, otherwise is 0.
for example:
grep -c myhostname.com /etc/hosts
1
grep -c google.com /etc/hosts
0
Not sure if this is exactly you're looking for but I hope it will help.
Beware of partial match, for example if you have 'myhost' in /etc/hosts grep -c host will return 1 as well. In this case you'll need to use regular expressions or parse /etc/hosts file with cut, awk and similar tools. Or use the following command:
grep -c '\bmachine name\b'
To skip the comments use the command below:
grep -v '^#.*' /etc/hosts | grep -c machine_name
so full command is
grep -v '^#.*' /etc/hosts | grep -c '\bmachine_name\b'
You could check
sysctl kernel.hostname
i.e
sysctl kernel.hostname | grep -c "my_hostname"
I use the following to check whether a supplied hostname is the same as localhost:
hostname_ip(){
host "$1" | sed -e 's/.* \([^ ]*[^ .]\)\.*$/\1/'
}
normalize_hostname(){
local normalized="$1"
grep -q "^\(\([0-9]{1,3}\)\.\)\{3\}\([0-9]{1,3}\)$" <<< "$normalized" || normalized="$(hostname_ip "$normalized")"
normalized="$(hostname_ip "$normalized")"
echo "$normalized"
}
myname="$(normalize_hostname "$(hostname)")"
argname="$(normalize_hostname "$1")"
if [[ "$myname" == "$argname" || "$argname" == "localhost" ]]; then
...
First, normalize the supplied parameter into format set by host utility by running it twice. If IP address is supplied -- checked by regex -- run it only once.
Then compare the value to normalized value of hostname utility or to string "localhost".