I'm trying to temporarily disable dhcp on all connections in a computer using bash, so I need the process to be reversible. My approach is to comment out lines that contain BOOTPROTO=dhcp, and then insert a line below it with BOOTPROTO=none. I'm not sure of the correct syntax to make sed understand the line number stored in the $insertLine variable.
fileList=$(ls /etc/sysconfig/network-scripts | grep ^ifcfg)
path="/etc/sysconfig/network-scripts/"
for file in $fileList
do
echo "looking for dhcp entry in $file"
if [ $(cat $path$file | grep ^BOOTPROTO=dhcp) ]; then
echo "disabling dhcp in $file"
editLine=$(grep -n ^BOOTPROTO=dhcp /$path$file | cut -d : -f 1 )
#comment out the original dhcp value
sed -i "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" $path$file
#insert a line below it with value of none.
((insertLine=$editLine+1))
sed "$($insertLine)iBOOTPROTO=none" $path$file
fi
done
Any help using sed or other stream editor greatly appreciated. I'm using RHEL 6.
The sed editor should be able to do the job, without having to to be combine bash, grep, cat, etc. Easier to test, and more reliable.
The whole scripts can be simplified to the below. It performs all operations (substitution and the insert) with a single pass using multiple sed scriptlets.
#! /bin/sh
for file in $(grep -l "^BOOTPROTO=dhcp" /etc/sysconfig/network-scripts/ifcfg*) ; do
sed -i -e "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" -e "/BOOTPROTO=dhcp/i BOOTPROTO=none" $file
done
As side note consider NOT using path as variable to avoid possible confusion with the 'PATH` environment variable.
Writing it up, your attempt with the following fails:
sed "$($insertLine)iBOOTPROTO=none" $path$file
because:
$($insertLine) encloses $insertLIne in a command substitution which when $insertLIne is evaluated it returns a number which is not a command generating an error.
your call to sed does not include the -i option to edit the file $path$file in place.
You can correct the issues with:
sed -i "${insertLine}i BOOTPROTO=none" $path$file
Which is just sed - i (edit in place) and Ni where N is the number of the line to insert followed by the content to insert and finally what file to insert it in. You add ${..} to insertLine to protect the variable name from the i that follows and then the expression is double-quoted to allow variable expansion.
Let me know if you have any further questions.
(and see dash-o's answer for refactoring the whole thing to simply use sed to make the change without spawning 10 other subshells)
Related
I have a small script which basically generates a menu of all the scripts in my ~/scripts folder and next to each of them displays a sentence describing it, that sentence being the third line within the script commented out. I then plan to pipe this into fzf or dmenu to select it and start editing it or whatever.
1 #!/bin/bash
2
3 # a script to do
So it would look something like this
foo.sh a script to do X
bar.sh a script to do Y
Currently I have it run a for loop over all the files in the scripts folder and then run sed -n 3p on all of them.
for i in $(ls -1 ~/scripts); do
echo -n "$i"
sed -n 3p "~/scripts/$i"
echo
done | column -t -s '#' | ...
I was wondering if there is a more efficient way of doing this that did not involve a for loop and only used sed. Any help will be appreciated. Thanks!
Instead of a loop that is parsing ls output + sed, you may try this awk command:
awk 'FNR == 3 {
f = FILENAME; sub(/^.*\//, "", f); print f, $0; nextfile
}' ~/scripts/* | column -t -s '#' | ...
Yes there is a more efficient way, but no, it doesn't only use sed. This is probably a silly optimization for your use case though, but it may be worthwhile nonetheless.
The inefficiency is that you're using ls to read the directory and then parse its output. For large directories, that causes lots of overhead for keeping that list in memory even though you only traverse it once. Also, it's not done correctly, consider filenames with special characters that the shell interprets.
The more efficient way is to use find in combination with its -exec option, which starts a second program with each found file in turn.
BTW: If you didn't rely on line numbers but maybe a tag to mark the description, you could also use grep -r, which avoids an additional process per file altogether.
This might work for you (GNU sed):
sed -sn '1h;3{H;g;s/\n/ /p}' ~/scripts/*
Use the -s option to reset the line number addresses for each file.
Copy line 1 to the hold space.
Append line 3 to the hold space.
Swap the hold space for the pattern space.
Replace the newline with a space and print the result.
All files in the directory ~/scripts will be processed.
N.B. You may wish to replace the space delimiter by a tab or pipe the results to the column command.
I am trying to script a way of removing hosts from the hostgroup file in Nagios Core.
The format of the hostgroup file is:
server1,server2,server3,server4
When removing a server, I need to be able to not only remove the server, but also the comma that follows it. So in my example above, if I am removing server2, the file would result as follows
server1,server3,server4
So I have googled and tested the following which works to remove server2 and a comma after it (I don't know what the b is used for exactly)
sed -i 's/\bserver2\b,//g' myfile
What I want to be able to do is to feed a list of hostnames to a small script to remove a bunch of hosts (and their following comma) with something similar to the following. The problem lies in that placing a variable like $x breaks the script so that nothing happens.
#!/bin/ksh
for x in `cat /tmp/list`
do
sed -i 's/\b${x}\b,//g' myfile
done
I think I am very close on a solution here, but could use a little help. Thanks much in advance for your kind assistance.
Using single quotes tells the shell not to replace the ${x} - it turns off variable interpolation if you want to google for it.
https://www.tldp.org/LDP/abs/html/quotingvar.html. So use double quotes around the sed replacement string instead:
while read -r x; do sed -i "s/\b${x},\b//g" myfile; done < /tmp/list
But since the last field won't have a comma after it, might be a good idea to run two sed commands, one looking for \bword,\b and the other for ,word$ - where \b is a word boundary and $ is the end of line.
while read -r x; do sed -i "s/\b${x},\b//g" myfile; sed -i "s/,${x}$//" myfile ; done < /tmp/list
One other possible boundary condition - what if you have just server2 on a line by itself and that's what you're trying to delete? Perhaps add a third sed, but this one will leave a blank line behind which you might want to remove:
while read -r x
do
sed -i "s/\b${x},\b//g" myfile # find and delete word,
sed -i "s/,${x}$//" myfile # find and delete ,word
sed -i "s/^${x}$//" myfile # find word on a line by itself
done < t
This works quite nicely:
#!/bin/bash
IN_FILE=$1
shift; sed -i "s/\bserver[$#],*\b//g" $IN_FILE; sed -i "s/,$//g" $IN_FILE
if you invoke it like ./remove_server.sh myfile "1 4" for your example file containing server1,server2,server3,server4, you get the following output:
server2,server3
A quick explanation of what it does:
shift shifts the arguments down by one (making sure that "myfile" isn't fed into the regex)
First sed removes the server with the numbers supplied as arguments in the string (e.g. "1 4")
Second sed looks for a trailing comma and removes it
The \b matches a word boundary
This is a great resource for learning about and testing regex: https://regex101.com/r/FxmjO5/1. I would recommend you check it out and use it each time you have a regex problem. It's helped me on so many occasions!
An example of this script working in a more general sense:
I tried it out on this file:
# This is some file containing server info:
# Here are some servers:
server2,server3
# And here are more servers:
server7,server9
with ./remove_server.sh myfile "2 9" and got this:
# This is some file containing info:
# Here are some servers:
server3
# And here are more servers:
server7
Pretty sure there is a pure sed solution for this but here is a script.
#!/usr/bin/env bash
hosts=()
while read -r host; do
hosts+=("s/\b$host,\{,1\}\b//g")
done < /tmp/list
opt=$(IFS=';' ; printf '%s' "${hosts[*]};s/,$//")
sed "$opt" myfile
It does not run sed line-by-line, but only one sed invocation. Just in case, say you have to remove 20+ pattern then sed will not run 20+ times too.
Add the -i if you think the output is ok.
Using perl and regex by setting the servers to a regex group in a shell variable:
$ remove="(server1|server4)"
$ perl -p -e "s/(^|,)$remove(?=(,|$))//g;s/^,//" file
server2,server3
Explained:
remove="(server1|server4)" or "server1" or even "server."
"s/(^|,)$remove(?=(,|$))//g" double-quoted to allow shell vars, remove leading comma, expected to be followed by a comma or the end of string
s/^,// file remove leading comma if the first entry was deleted
Use the -i switch for infile editing.
bash script that reads the servers to remove from standard input, one per line, and uses perl to remove them from the hostfile (Passed as the first argument to the script):
#!/usr/bin/env bash
# Usage: removehost.sh hostgroupfile < listfile
mapfile -t -u 0 servers
IFS="|"
export removals="${servers[*]}"
perl -pi -e 's/,?(?:$ENV{removals})\b//g; s/^,//' "$1"
It reads the servers to remove into an array, joins that into a pipe-separated string, and then uses that in the perl regular expression to remove all the servers in a single pass through the file. Slashes and other funky characters (As long as they're not RE metacharacters) won't mess up the parsing of the perl, because it uses the environment variable instead of embedding the string directly. It also uses a word boundry so that removing server2 won't remove that part of server22.
I'm trying to undergo pdbqt-flexible files merge into one pdb using following script:
http://prosciens.com/prosciens/oldproscienssarl/files/flexrigidpdbqt2pdb_template.sh
Problematic fragment:
Let's merge the files
First we clean up the model PDB
grep ATOM ${FLEXPDBQT}_${MODEL}.pdb > ${FLEXPDBQT}_${MODEL}.pdb.tmp
Next we create a list of residues
cut -c 18-27 ${FLEXPDBQT}_${MODEL}.pdb.tmp > residuelistraw.tmp`
cat residuelistraw.tmp | uniq > residuelist.tmp
Then we split the model file into residues
while read r
do
rns= echo $r | sed 's/ //g'
egrep "[ \t]$r[ \t]" ${FLEXPDBQT}_${MODEL}.pdb.tmp > $rns.pdb.tmp
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' $rns.pdb.tmp
Currently it fails at #3 step yielding following error:
/flexrigidpdbqt2pdb_template.sh: line 133: $ rns.pdb.tmp: ambiguous redirect
sed: -e expression # 1, character 9: unknown option for the `s' command
I tried fix the error using some sed substitution:
rns=`echo "${r/ /}"`
echo $rns
egrep "[ \t]$r[ \t]" ${FLEXPDBQT}_${MODEL}.pdb.tmp > $rns.pdb.tmp
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' $rns.pdb.tmp
But so far nothing changed.
My sed version is 4.4
On the "ambiguous redirection" error
An "ambiguous redirection" error doesn't come from sed at all, it comes from your shell.
An "ambiguous redirection" error means your shell can't start the command you gave it at all, because it wasn't able to perform the redirections requested as part of that command's environment.
In that case, the variable rns is empty.
That's because rns= echo $r | sed 's/ //g' doesn't assign rns to be the output of sed at all. Instead, it assigns a transient environment variable named rns to be an empty string, only for the duration of execution of echo $r (the output of which is sent to sed, and from there to the script's stdout).
Instead, use:
rns=${r//[[:space:]]/}
...or, less efficiently:
rns=$(sed -e 's/ //g' <<<"$r")
To avoid the same error in cases where the variable isn't empty, be sure you quote!
That is, instead of running ... >$file, always run ... >"$file", to ensure that unwanted string-splitting or glob expansion can't make an otherwise-valid redirection unworkable. (This doesn't happen with all shell versions, but that means that failing to quote causes your code's behavior to be unpredictable unless you know which shell release it's going to be run with!).
On the "unknown option" error
If you use / as the sigil separating parts of your sed command, then you must not have any /s in the data being substituted. If you cannot guarantee this, use a different sigil instead of /; for example, s#/foo/bar#/baz/qux# works properly.
It looks like that the essential part to repair that script was changing rns as you proposed to:
rns=${r//[[:space:]]/}
and subsequent quoting of the output:
sed -i 's/'$FLEXPDBQT'_'$MODEL'.pdb.tmp://' "$rns.pdb.tmp"
However, further problems come out with next step, not previously shown:
sed '/'"HN ${r}"'/ r '${rns}'.pdb.tmp' ${RIGIDPDBQT}_${LIGNAME}_${MODEL}_apo.pdb > "${RIGIDPDBQT}_${LIGNAME}_${MODEL}_apo.pdb.tmp"
It yields no output (and no error), propably due to another problem with sed (which I really can't understand).
It is quite hard to give here a working example - a lot of preformatted txt files. Despite that, the problem about which I asked is solved, thank you!
might be an easy question, I'm new in bash and haven't been able to find the solution to my question.
I'm writing the following script:
for file in `ls *.map`; do
ID=${file%.map}
convertf -p ${ID}_par #this is a program that I use, no problem
NAME=head -n 1 ${ID}.ind | cut -f1 -d":" #Now: This step is the problem: don't seem to be able to make a proper NAME function. I just want to take the first column of the first line of the file ${ID}.ind
It gives me the return
line 5: bad substitution
any help?
Thanks!
There are a couple of issues in your code:
for file in `ls *.map` does not do what you want. It will fail e.g. if any of the filenames contains a space or *, but there's more. See http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29 for details.
You should just use for file in *.map instead.
ALL_UPPERCASE names are generally used for system variables and built-in shell variables. Use lowercase for your own names.
That said,
for file in *.map; do
id="${file%.map}"
convertf -p "${id}_par"
name="$(head -n 1 "${id}.ind" | cut -f1 -d":")"
...
looks like it would work. We just use $( cmd ) to capture the output of a command in a string.
I am developing a deployment script "that I want to be able to run over again without double entries"
I am trying to add a sed command that will look for "address" field, if it doesn't exist, create it, if it does exist modify it to the correct IP Address.
This is what I have so far...
#!/bin/bash
ipaddress=192.168.1.1
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i 's/^address.*/"address $ipaddress"/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
It will create the correct entry if no entry exists but I have all kinds of problems if the entry exists or is correct.
Any help would be greatly appreciated!
Final Answer based on response from "1stSi Dave" Below
The final working script that creates the entry if it doesn't exist or alters any existing address entry is:
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
First, I think you need the '-e' command line option to your sed command:
sed -i -e 's/...'
Maybe that's a typo, because the rest of your command line indicates you want to append to the file, not edit in place. Next, single quotes are hiding the variable expansion you're trying to achieve in the sed command script. They are also preserving the double quotes in the output, which I don't think is what you want. Try this:
sed -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces
Third, you may want to include the possibility of white space preceding the "address" token. Finally, you probably do want to edit-in-place (with sed -i), because tacking on the edited line at the end of the file is probably not going to work.