I have a docker-compose file, in which I have a docker images with its versioning tag, example myrepo.com/myimage:1.0.1.0. I want to write a bash script that receives the version number as a parameter and updates the version tag of the relevant image.
Currently I am using the below command to replace the version tag with the help of grep and sed, but I believe there should be a more efficient way for this. Any suggestions?
sedval='1.1.1.1' # <-- new value
imageval=$(grep -i 'image: myrepo.com/myimage:' docker-compose.yml | cut -d ':' -f2,3)
imagename=$(grep -i 'image: myrepo.com/myimage:' docker-compose.yml | cut -d ':' -f2)
imageversion=$(grep -i 'image: myrepo.com/myimage:' docker-compose.yml | cut -d ':' -f3)
sed -i -e "s+$imageval+$imagename:$sedval+g" docker-compose.yml
docker-compose supports variable substitution. You could do the following:
version: "3"
services:
myrepo:
image: myrepo.com/myimage:${VERSION}
docker-compose looks for the VERSION environment variable and substitutes its value in the file. Run with:
export VERSION=1.1.1.1; docker-compose up
Manipulating YAML with line-oriented regex tools is generally not advisable. You should probably switch to a YAML-aware tool like yq for this.
There doesn't seem to be a reason you extract imageversion; you assign this variable, but never use it. In fact, if you forgo the requirement to match case-insensitively, all you really need is
sed -i "s+\\(image: myrepo\\.com/myimage:\\)[.0-9]*+\\1$1+" docker-compose.yml
assuming (like you say in your question) the new version number is in $1, which is the first command-line argument to your script.
I don't think Docker allows you to use upper case; and if it does, you can replace i with [Ii], m with [Mm], etc. Notice also how the literal dot needs to be backslash-escaped, strictly speaking.
Generally speaking, you can often replace grep 'x' | sed 'y' with sed '/x/y'; see also useless use of grep. Also, the repeated grep could easily be refacored into a single grep with something like
imageval=$(grep -i 'image: myrepo.com/myimage:' docker-compose.yml)
imagever=${imageval##*:}
imagename=${imageval%:$imagever}
where the shell parameter substitutions are both simpler and more elegant as well as more efficient than repeatedly spawning cut in a separate process for this very simple string extraction task.
Related
I'm trying to temporarily disable dhcp on all connections in a computer using bash, so I need the process to be reversible. My approach is to comment out lines that contain BOOTPROTO=dhcp, and then insert a line below it with BOOTPROTO=none. I'm not sure of the correct syntax to make sed understand the line number stored in the $insertLine variable.
fileList=$(ls /etc/sysconfig/network-scripts | grep ^ifcfg)
path="/etc/sysconfig/network-scripts/"
for file in $fileList
do
echo "looking for dhcp entry in $file"
if [ $(cat $path$file | grep ^BOOTPROTO=dhcp) ]; then
echo "disabling dhcp in $file"
editLine=$(grep -n ^BOOTPROTO=dhcp /$path$file | cut -d : -f 1 )
#comment out the original dhcp value
sed -i "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" $path$file
#insert a line below it with value of none.
((insertLine=$editLine+1))
sed "$($insertLine)iBOOTPROTO=none" $path$file
fi
done
Any help using sed or other stream editor greatly appreciated. I'm using RHEL 6.
The sed editor should be able to do the job, without having to to be combine bash, grep, cat, etc. Easier to test, and more reliable.
The whole scripts can be simplified to the below. It performs all operations (substitution and the insert) with a single pass using multiple sed scriptlets.
#! /bin/sh
for file in $(grep -l "^BOOTPROTO=dhcp" /etc/sysconfig/network-scripts/ifcfg*) ; do
sed -i -e "s/BOOTPROTO=dhcp/#BOOTPROTO=dhcp/g" -e "/BOOTPROTO=dhcp/i BOOTPROTO=none" $file
done
As side note consider NOT using path as variable to avoid possible confusion with the 'PATH` environment variable.
Writing it up, your attempt with the following fails:
sed "$($insertLine)iBOOTPROTO=none" $path$file
because:
$($insertLine) encloses $insertLIne in a command substitution which when $insertLIne is evaluated it returns a number which is not a command generating an error.
your call to sed does not include the -i option to edit the file $path$file in place.
You can correct the issues with:
sed -i "${insertLine}i BOOTPROTO=none" $path$file
Which is just sed - i (edit in place) and Ni where N is the number of the line to insert followed by the content to insert and finally what file to insert it in. You add ${..} to insertLine to protect the variable name from the i that follows and then the expression is double-quoted to allow variable expansion.
Let me know if you have any further questions.
(and see dash-o's answer for refactoring the whole thing to simply use sed to make the change without spawning 10 other subshells)
Hello I am trying to insert the output of command output > sed replace after a particular string in a file as part of user data on machine boot up
[centos#ip-192-168-2-22 scylla]$ sudo sed -i.bak 's/broadcast_rpc_address: : /broadcast_rpc_address:\/$hostname -i | awk '{print $2}'/' /etc/scylla/scylla.yaml
file is currently
broadcast_rpc_address:
replace with
broadcast_rpc_address: (the ip of the machine)
Something like this should work:
sed -i.bak "s/broadcast_rpc_address:/broadcast_rpc_address: $(hostname -i)/" /etc/scylla/scylla.yaml
This will replace broadcast_rpc_address: by broadcast_rpc_address: $(hostname -i). Now, because this string is in double quotes - not single quotes - this tells the shell to interpret some magic sequences inside the string. In particular $(somecommand) means to run somecommand and insert its output into the string. Of course, change "hostname -i" in the command I gave above to anything else you want (it can even be an entire pipeline.
Your original attempt used something that started with $hostname. This syntax, $hostname, doesn't run the command hostname, but rather looks for a variable called hostname, which isn't what you wanted. You need the $(...) syntax instead. Your original attempt also had problems with nested quotes, which don't work.
I made a bash script to insert the result of nmap command to an array. The script is working on bash 4.3.30, but it does not work when I try to run it on bash 4.4.12. It looks like the array is empty or it just have the first value.
Here is my code:
#!/bin/bash
declare -a IP_ARRAY
NMAP_OUTPUT=`nmap -sL $1 | grep "Nmap scan report" | awk '{print $NF}'`
read -a IP_ARRAY <<< $NMAP_OUTPUT
printf '%s\n' "${IP_ARRAY[#]}"
With bash 4.3, the values of the string NMAP_OUTPUT are well copied to the array IP_ARRAY. The the other version not and I don't find the error.
The string NMAP_OUTPUT looks like:
10.0.0.0 10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9 10.0.0.10
Instead of using my code above, this code works:
IP_ARRAY=(${NMAP_OUTPUT})
I would like to understand with my previous code is working on one version and not in the other one.
Thank you very much!!!
Your script has multiple issues which could be fixed. It could be done very simply minimizing a number of steps.
You are using NMAP_OUTPUT as a variable. The bash shell does support arrays which you can use to store a list. Also independent entries present in a variable's context undergo Word-Splitting done by the shell. The consequence of that is, if a entry has spaces in-between, it will be tough to identify if it is a separate word or part of a whole word.
Storing the command output to a variable and later parsing to an array is round about way. You can directly pass the output to an array
Using grep and awk together is not needed, awk can do whatever grep can
Always quote the shell variable and array expansions. Never use unquoted expansion in your results (like in <<< $NMAP_OUTPUT). It could have adverse affects in case of words containing spaces.
Always use lower case variable names for user-defined functions/variables and array names.
Use mapfile built-in
Version of bash v4.0 on-wards provides options mapfile/readarray to directly read from a file or output of command.
All your script needs is
mapfile -t nmapOutput < <(nmap -sL "$1" | awk '/Nmap scan report/{print $NF}')
printf '%s\n' "${nmapOutput[#]}"
There is nothing I could infer why your script didn't work between the versions of bash you've indicated. I was able to run your script on the given input on bash 4.4.12
But the crux of the problem seems to be using variables and arrays interchangeably in the wrong way.
it seems you're trying to do this the hard way.
why not simply:
IP_ARRAY=( `nmap -sL 127.1/29 | grep "Nmap scan report" | awk '{print $NF}'` )
might be an easy question, I'm new in bash and haven't been able to find the solution to my question.
I'm writing the following script:
for file in `ls *.map`; do
ID=${file%.map}
convertf -p ${ID}_par #this is a program that I use, no problem
NAME=head -n 1 ${ID}.ind | cut -f1 -d":" #Now: This step is the problem: don't seem to be able to make a proper NAME function. I just want to take the first column of the first line of the file ${ID}.ind
It gives me the return
line 5: bad substitution
any help?
Thanks!
There are a couple of issues in your code:
for file in `ls *.map` does not do what you want. It will fail e.g. if any of the filenames contains a space or *, but there's more. See http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29 for details.
You should just use for file in *.map instead.
ALL_UPPERCASE names are generally used for system variables and built-in shell variables. Use lowercase for your own names.
That said,
for file in *.map; do
id="${file%.map}"
convertf -p "${id}_par"
name="$(head -n 1 "${id}.ind" | cut -f1 -d":")"
...
looks like it would work. We just use $( cmd ) to capture the output of a command in a string.
I am trying to do something like this on my OSX terminal
> grep -i "((\D*)ful)" ./Myfile.rtf
The above statement fails however when I do this
> grep -i "\D*ful" ./Myfile.rtf
it passes - does grep have an issue with regex groups
Since basic grep uses BRE, you need to use \(..\) for capturing group.
grep -i "\(\(\D*\)ful\)" ./Myfile.rtf
The most likely problem when this sort of thing happens is that the special characters are or are not special. In this case, I think the brackets are not special unless you quote them, so:
> grep -i "\(\(\D*\)ful\)" ./Myfile.rtf
would probably work better.
[One of the irritations of regex is the variation that has developed in exactly how they are written...]