shell for in loop with multible variable [duplicate] - bash

This question already has answers here:
Iterate over two arrays simultaneously in bash
(6 answers)
Closed 6 years ago.
in my question i learn to use loops and with you help I final a installation script for multible instance of a software. thank you very much :)
now I try to automatic setup the configuration files by using sed. For this i need multible variables in a loop.
I read from the system the IP-Adresses and the hostnames for the IP's (PTR)
IPADDR=`ifconfig | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+'|grep -v 127.0.0.1`
for ipaddr in ${IPADDR[#]}; do echo $ipaddr; done
for iphost in ${IPADDR[#]}; do host $iphost |grep pointer | awk '{print $NF}' RS='.\n'; done
my Script know, there ar 3 IP's, know the IP-Addresses and the Hostnames.
the numbers of IP (3) are now my 001 002 003. this running well.
if I like to edit the config files with sed, I need the 3 variable to do this.
command anyname-001 -some -parameter in my case is a copy to a path. my path is now
/etc/anyname-001, /etc/anyname-003 and /etc/anyname-003
by using sed I need also the 3 IP-Addresses and the 3 hostnames.
sed -i 's/IPADDR/'${ipaddr}'/g' /etc/anyname-${number}/config.cfg
sed -i 's/HOSTNAME/'${hostname}'/g' /etc/anyname-${number}/config.cfg
how can I bring my loop to this with all variables on same time. T try many things. I found nested loops but it not work
001 >> IP:a.a.a.a >> hostname aaa.aaa.aa
002 >> IP:b.b.b.b >> hostname bbb.bbb.bb
003 >> IP:c.c.c.c >> hostname ccc.ccc.cc
Thank you

Assuming the length of both of your arrays HOSTNAME and IPADDR are the same, then you can loop over their elements via indexes in one run.
The length of an array is calculated by using the '#' in the array variable, for example as:
echo ${#HOSTNAME[#]}
So, overall your code would look like something:
count=${#HOSTNAME[#]}
for (( i=0; i<${count}; i++ ));
do
echo ${HOSTNAME[$i]};
echo ${IPADDR[$i]};
((j=i+1));
sed -i 's/IPADDR/'${IPADDR[$i]}'/g' /etc/anyname-${j}/config.cfg
sed -i 's/HOSTNAME/'${HOSTNAME[$i]}'/g' /etc/anyname-${j}/config.cfg
done

Related

linux bash insert text at a variable line number in a file

I'm trying to temporarily disable dhcp on all connections in a computer using bash, so I need the process to be reversible. My approach is to comment out lines that contain BOOTPROTO=dhcp, and then insert a line below it with BOOTPROTO=none. I'm not sure of the correct syntax to make sed understand the line number stored in the $insertLine variable.
fileList=$(ls /etc/sysconfig/network-scripts | grep ^ifcfg)
path="/etc/sysconfig/network-scripts/"
for file in $fileList
do
echo "looking for dhcp entry in $file"
if [ $(cat $path$file | grep ^BOOTPROTO=dhcp) ]; then
echo "disabling dhcp in $file"
editLine=$(grep -n ^BOOTPROTO=dhcp /$path$file | cut -d : -f 1 )
#comment out the original dhcp value
sed -i "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" $path$file
#insert a line below it with value of none.
((insertLine=$editLine+1))
sed "$($insertLine)iBOOTPROTO=none" $path$file
fi
done
Any help using sed or other stream editor greatly appreciated. I'm using RHEL 6.
The sed editor should be able to do the job, without having to to be combine bash, grep, cat, etc. Easier to test, and more reliable.
The whole scripts can be simplified to the below. It performs all operations (substitution and the insert) with a single pass using multiple sed scriptlets.
#! /bin/sh
for file in $(grep -l "^BOOTPROTO=dhcp" /etc/sysconfig/network-scripts/ifcfg*) ; do
sed -i -e "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" -e "/BOOTPROTO=dhcp/i BOOTPROTO=none" $file
done
As side note consider NOT using path as variable to avoid possible confusion with the 'PATH` environment variable.
Writing it up, your attempt with the following fails:
sed "$($insertLine)iBOOTPROTO=none" $path$file
because:
$($insertLine) encloses $insertLIne in a command substitution which when $insertLIne is evaluated it returns a number which is not a command generating an error.
your call to sed does not include the -i option to edit the file $path$file in place.
You can correct the issues with:
sed -i "${insertLine}i BOOTPROTO=none" $path$file
Which is just sed - i (edit in place) and Ni where N is the number of the line to insert followed by the content to insert and finally what file to insert it in. You add ${..} to insertLine to protect the variable name from the i that follows and then the expression is double-quoted to allow variable expansion.
Let me know if you have any further questions.
(and see dash-o's answer for refactoring the whole thing to simply use sed to make the change without spawning 10 other subshells)

for loop iteration on a string which contains * [duplicate]

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 4 years ago.
I have a file which was created by another scripts with file names. Only thing is that it contains * for similar file names. now i want to iterate a for loop on this file to execute a few operation but i want to do it on different servers for that i simply need exact same name from that file. For this, i used grep and passed it to 'for' loop for one by one execution. But instead before going for the loop it actually executes it on current server itself and gives wrong output. I don't know what i'm missing.
# node_name=auxiliary;grep -F $node_name /opt/log_automation/ME_log_sheet.csv|awk -F "," '{print $2}'
/opt/tpa/logs/install_SPS.*.log
/opt/tpa/logs/localization.log
/opt/tpa/logs/SPI.*.log
/opt/tpa/logs/TPA_system.log
#
This seems to be working as expected as i want * to be present in the name. Now if i put it in a variable or pass it to the loop it expands and gives actual present file which i don't want. I want it to return simply the name with * on it.
# node_name=auxiliary;myvar=$(grep -F $node_name /opt/log_automation/ME_log_sheet.csv|awk -F "," '{print $2}');echo $myvar
/opt/tpa/logs/install_SPS.2019.02.09-12:12.log /opt/tpa/logs/localization.log /opt/tpa/logs/SPI.2019.02.09-12:10.log /opt/tpa/logs/TPA_system.log
# node_name=auxiliary;for i in $(grep -F $node_name /opt/log_automation/ME_log_sheet.csv|awk -F "," '{print $2}');do echo $i;done
/opt/tpa/logs/install_SPS.2019.02.09-12:12.log
/opt/tpa/logs/localization.log
/opt/tpa/logs/SPI.2019.02.09-12:10.log
/opt/tpa/logs/TPA_system.log
Please help me resolve this. What am i doing wrong?
P.S.: '-F' with grep i added after a few changes. It wasn't working either way.
Quote all variables:
node_name=auxiliary
for i in "$(grep -F $node_name /opt/log_automation/ME_log_sheet.csv|awk -F "," '{print $2}')"
do
echo "$i"
done
I think you should be using set -f to disable filename expansion : https://unix.stackexchange.com/questions/333867/what-does-set-f-in-korn-shell-do
However as triplee suggest, using such characters (eg asterisk) in filenames is a very bad practice..

Assign nmap result to an array in bash

I made a bash script to insert the result of nmap command to an array. The script is working on bash 4.3.30, but it does not work when I try to run it on bash 4.4.12. It looks like the array is empty or it just have the first value.
Here is my code:
#!/bin/bash
declare -a IP_ARRAY
NMAP_OUTPUT=`nmap -sL $1 | grep "Nmap scan report" | awk '{print $NF}'`
read -a IP_ARRAY <<< $NMAP_OUTPUT
printf '%s\n' "${IP_ARRAY[#]}"
With bash 4.3, the values of the string NMAP_OUTPUT are well copied to the array IP_ARRAY. The the other version not and I don't find the error.
The string NMAP_OUTPUT looks like:
10.0.0.0 10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9 10.0.0.10
Instead of using my code above, this code works:
IP_ARRAY=(${NMAP_OUTPUT})
I would like to understand with my previous code is working on one version and not in the other one.
Thank you very much!!!
Your script has multiple issues which could be fixed. It could be done very simply minimizing a number of steps.
You are using NMAP_OUTPUT as a variable. The bash shell does support arrays which you can use to store a list. Also independent entries present in a variable's context undergo Word-Splitting done by the shell. The consequence of that is, if a entry has spaces in-between, it will be tough to identify if it is a separate word or part of a whole word.
Storing the command output to a variable and later parsing to an array is round about way. You can directly pass the output to an array
Using grep and awk together is not needed, awk can do whatever grep can
Always quote the shell variable and array expansions. Never use unquoted expansion in your results (like in <<< $NMAP_OUTPUT). It could have adverse affects in case of words containing spaces.
Always use lower case variable names for user-defined functions/variables and array names.
Use mapfile built-in
Version of bash v4.0 on-wards provides options mapfile/readarray to directly read from a file or output of command.
All your script needs is
mapfile -t nmapOutput < <(nmap -sL "$1" | awk '/Nmap scan report/{print $NF}')
printf '%s\n' "${nmapOutput[#]}"
There is nothing I could infer why your script didn't work between the versions of bash you've indicated. I was able to run your script on the given input on bash 4.4.12
But the crux of the problem seems to be using variables and arrays interchangeably in the wrong way.
it seems you're trying to do this the hard way.
why not simply:
IP_ARRAY=( `nmap -sL 127.1/29 | grep "Nmap scan report" | awk '{print $NF}'` )

Unable to use shell pipe via Makefile? [duplicate]

This question already has answers here:
I just assigned a variable, but echo $variable shows something else
(7 answers)
Closed 7 years ago.
I have a shell command spurious ports which returns the following data...
Service Host Port Browser link
spurious-elasticache-docker 192.168.59.103 32794 -
spurious-sqs sqs.spurious.localhost 32789 http://sqs.spurious.localhost:32789
spurious-s3 s3.spurious.localhost 32790 http://s3.spurious.localhost:32790
spurious-elasticache 192.168.59.103 32793 -
spurious-dynamo dynamodb.spurious.localhost 32791 http://dynamodb.spurious.localhost:32791
spurious-browser browser.spurious.localhost 32795 http://browser.spurious.localhost:32795
spurious-memcached 192.168.59.103 32792 -
If I want to get the port number for the "dynamo" service then the following shell script works...
lines=`spurious ports`; echo $lines | grep "dynamo" | awk '{print $3}'
This retuns 32791
But if I try and run this from inside a Makefile, I instead get the entire spurious ports output (all on one line?):
foobar:
$(eval port:=$(shell lines=`spurious ports`; echo $$lines | grep 'dynamo' | awk '{print $3}'))
#echo $(port)
I've also tried moving the commands into a shell script file:
#!/bin/sh
lines=`spurious ports`; echo $lines | grep "dynamo" | awk '{print $3}'
And inside the Makefile used:
PORT:=$(shell ./scripts/spurious-ports.sh)
bing:
#echo $(PORT)
But this doesn't work either.
This seems quotes issue. When you variable with multiple lines you need to put that variable in double quotes to retain the /n at end of line.
for E.g.
VAR="1
2
3
4"
echo $VAR ## here it will print "1 2 3 4"
but echo "$VAR" will print
1
2
3
4
I Think this will solve your problem. For more details read about quotes in shell.

removing leading zeros from IP addresses: converting ipfilter.dat to bluetack.co.uk ipfilter with sed

I had a need to convert uTorrent-style ipfilter.dat into a bluetack-style ipfilter file, and wrote this shell script to achieve this:
#!/bin/bash
# read ipfilter.dat-formatted file line by line
# (example: 000.000.000.000-008.008.003.255,000,Badnet
# - ***here, input file's lines/fields are always the same length***)
# and convert into a bluetack.co.uk-formatted output
# (example: Badnet:0.0.0.0-8.8.3.255
# - fields moved around, leading zeros removed)
while read record
do
start=`echo ${record:0:15} | awk -F '.' '{for(i=1;i<=NF;i++)$i=$i+0;}1' OFS='.'`
end=`echo ${record:16:15} | awk -F '.' '{for(i=1;i<=NF;i++)$i=$i+0;}1' OFS='.'`
echo ${record:36:7}:${start}-${end}
done < $1
However, on a 2000-line input file this script takes on average 10(!) seconds to complete - a mere 200 lines/sec.
I'm sure this same result can be achieved with sed, and sed-version is likely to be much faster.
Is there a sed-guru around to suggest a solution for this kind of fixed-positions replacements?
Feel free to suggest a solution in other languages as well - I would enjoy testing a Python or a C version, for example. A more efficient shell/bash version would be welcome as well.
You could try this.
sed -r 's/^0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)-0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+)\.0*([0-9]+),...,(.*)$/\9:\1.\2.\3.\4-\5.\6.\7.\8/' inputfile
I didn't test the performance but I guess it could be faster than 200 lines/sec.
You will be sacrificing performance using the shell's while read loop on a big file. It is empirically proven that tools such as awk/sed (and some languages eg Perl/Python/Ruby) are better at iterating big files and processing the lines than the shell's while read loop. Moreover, in your script, while iterating over the lines, you are also piping a few calls to awk. This is extra overheads.
Ruby(1.9+)
$ cat file
000.000.000.000-008.008.003.255,000,Badnet
001.010.110.111-002.020.220.222,111,Badnet
$ ruby -F"," -ane 'puts "#{$F[-1].chomp}:" + $F[0].gsub(/(00|0)([0-9]+)([.-])/,"\\2\\3")' file
Badnet:0.0.0.0-8.8.3.255
Badnet:1.10.110.111-2.20.220.222
I really wanted to get this to work in a single sed command, but I wasn't able to figure it out. Surely this will still be faster than 200 lines/s though.
sed 's/\.0\{1,2\}/\./g' | sed 's/^0\{1,2\}//'
#!/bin/tclsh
#Regsub TCL script to remove the leading zeros from the ip address.
#Author : Shoeb Masood , Bangalore
puts "Enter the ip address"
set ip [gets stdin]
set list_ip [split $ip .]
foreach index $list_ip {
regsub {^0|^00} $index {\1} index
lappend list_ip2 $index
}
set list_ip2 [join $list_ip2 "."]
puts $list_ip2

Resources