when I try to edit a yaml file, and replace the string after image :
I am doing this sed -i "s/\<image :\>/& bar/" test.yaml
where my test.yaml file has this
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2019-11-19T22:42:10Z"
generation: 1
....
image: foo
want replace foo with bar
getting error
sed: -e expression #1, char 3: unterminated `s' command
this is the error I am getting
In sed, you can't put a forwardslash / before the substitution command s. The correct syntax is s/regexp/replacement/ or simply s/old/new/.
When you execute sed like you did with sed -i "/s/\<image :\>/& bar/", it will expect a builtin command after the second / such as sed -i /regexp/command or sed -i /string/d to delete the line with the matched string. However, it find the \ character and stops because it isn't a sed command.
what I want to achieve is to find an string in file and go on finding first occurrence of another match and replace that with some value.
Ex: string = {Name: name;Address:someadd;var1:var1;var2:var2},{Name: differntName;Address:someadd;var1:var1;var2:var2}
Now what i need to do is to find Name: name and then find first occurrence of "var2:var2" after "Name: name" and replace it with "var2:newvarvalue"
Please note that i need to complete this task with sed in bash scripting.
Thanks in advance.
Edit : i am trying to modify .yaml docker compose file
Here is a terrible solution using sed :
split on ,, one part per line :
sed 's/,/\n/g'
replace var2:var2 by var2:newvarvalue if on the same line as Name: name :
sed '/Name: name/s/var2:var2/var2:newvarvalue/'
or
sed -E 's/(Name: name.*)var2:var2/\1var2:newvarvalue/'
It's terrible because any extra comma or linefeed might break the whole thing.
var='{Name: name;Address:somea,dd;var1:var1;var2:var2},{Name: differntName;Address:someadd;var1:var1;var2:var2}'
NEW_VAL='new_value'
IFS=$'\n'
OBJECTS=("$(echo "${var}" | sed -nE 's/[^{]*(\{[^}]+\})/\1\n/gp')")
for obj in "${OBJECTS[#]}"; do
echo "${obj}" | sed -E 's/(.*var2:)(var2)(.*)/\1'"${NEW_VAL}"'\3/'
done
Output:
{Name: name;Address:somea,dd;var1:var1;var2:new_value}
{Name: differntName;Address:someadd;var1:var1;var2:new_value}
This solution accounts for a comma in the object (as exemplified in the first object) by extracting each and setting the delimiter to a newline.
I'm trying to get a multiline output from a CSV into one line in Bash.
My CSV file looks like this:
hi,bye
hello,goodbye
The end goal is for it to look like this:
"hi/bye", "hello/goodbye"
This is currently where I'm at:
INPUT=mycsvfile.csv
while IFS=, read col1 col2 || [ -n "$col1" ]
do
source=$(awk '{print;}' | sed -e 's/,/\//g' )
echo "$source";
done < $INPUT
The output is on every line and I'm able to change the , to a / but I'm not sure how to put the output on one line with quotes around it.
I've tried BEGIN:
source=$(awk 'BEGIN { ORS=", " }; {print;}'| sed -e 's/,/\//g' )
But this only outputs the last line, and omits the first hi/bye:
hello/goodbye
Would anyone be able to help me?
Just do the whole thing (mostly) in awk. The final sed is just here to trim some trailing cruft and inject a newline at the end:
< mycsvfile.csv awk '{print "\""$1, $2"\""}' FS=, OFS=/ ORS=", " | sed 's/, $//'
If you're willing to install trl, a utility of mine, the command can be simplified as follows:
input=mycsvfile.csv
trl -R '| ' < "$input" | tr ',|' '/,'
trl transforms multiline input into double-quoted single-line output separated by ,<space> by default.
-R '| ' (temporarily) uses |<space> as the separator instead; this assumes that your data doesn't contain | instances, but you can choose any char. that you know not be part of your data.
tr ',|' '/,' then translates all , instances (field-internal to the input lines) into / instances, and all | instances (the temporary separator) into , instances, yielding the overall result as desired.
Installation of trl from the npm registry (Linux and macOS)
Note: Even if you don't use Node.js, npm, its package manager, works across platforms and is easy to install; try
curl -L https://git.io/n-install | bash
With Node.js installed, install as follows:
[sudo] npm install trl -g
Note:
Whether you need sudo depends on how you installed Node.js and whether you've changed permissions later; if you get an EACCES error, try again with sudo.
The -g ensures global installation and is needed to put trl in your system's $PATH.
Manual installation (any Unix platform with bash)
Download this bash script as trl.
Make it executable with chmod +x trl.
Move it or symlink it to a folder in your $PATH, such as /usr/local/bin (macOS) or /usr/bin (Linux).
$ awk -F, -v OFS='/' -v ORS='"' '{$1=s ORS $1; s=", "; print} END{printf RS}' file
"hi/bye", "hello/goodbye"
There is no need for a bash loop, which is invariably slow.
sed and tr can do this more efficiently:
input=mycsvfile.csv
sed 's/,/\//g; s/.*/"&", /; $s/, $//' "$input" | tr -d '\n'
s/,/\//g uses replaces all (g) , instances with / instances (escaped as \/ here).
s/.*/"&", / encloses the resulting line in "...", followed by ,<space>:
regex .* matches the entire pattern space (the potentially modified input line)
& in the replacement string represent that match.
$s/, $// removes the undesired trailing ,<space> from the final line ($)
tr -d '\n' then simply removes the newlines (\n) from the result, because sed invariably outputs each line with a trailing newline.
Note that the above command's single-line output will not have a trailing newline; simply append ; printf '\n' if it is needed.
In awk:
$ awk '{sub(/,/,"/");gsub(/^|$/,"\"");b=b (NR==1?"":", ")$0}END{print b}' file
"hi/bye", "hello/goodbye"
Explained:
$ awk '
{
sub(/,/,"/") # replace comma
gsub(/^|$/,"\"") # add quotes
b=b (NR==1?"":", ") $0 # buffer to add delimiters
}
END { print b } # output
' file
I'm assuming you just have 2 lines in your file? If you have alternating 2 line pairs, let me know in comments and I will expand for that general case. Here is a one-line awk conversion for you:
# NOTE: I am using the octal ascii code for the
# double quote char (\42=") in my printf statement
$ awk '{gsub(/,/,"/")}NR==1{printf("\42%s\42, ",$0)}NR==2{printf("\42%s\42\n",$0)}' file
output:
"hi/bye", "hello/goodbye"
Here is my attempt in awk:
awk 'BEGIN{ ORS = " " }{ a++; gsub(/,/, "/"); gsub(/[a-z]+\/[a-z]+/, "\"&\""); print $0; if (a == 1){ print "," }}{ if (a==2){ printf "\n"; a = 0 } }'
Works also if your Input has more than two lines.If you need some explanation feel free to ask :)
I'm trying to use an array to define the lines to replace using sed; I can delete the lines using a variable for the line number but I can't get sed to use the variable to define the line number to write to. The problem seems to reside in the insert line. How do you pass the value of an array as a line number to sed?
#!/bin/bash
lineNum=$(sed -n '/max_allowed_packet/=' /etc/mysql/my.cnf)
IFS= #There's a space as the delimiter#
ary=($lineNum)
#for key in "${!ary[#]}";
# do
# sed -i '$ary[$key]'d /etc/mysql/my.cnf;
# #The folllowing line errors#
# sed -i "'$ary[$key]'imax_allowed_packet = 32M" /etc/mysql/my.cnf;
# #The above line errors#
#done
#for val in "${ary[#]}";
# do
# sed -i "${val}d" /etc/mysql/my.cnf;
# sed -i "${val}imax_allowed_packet = 32M" /etc/mysql/my.cnf;
# done
for val in "${ary[#]}";
do
sed -i "${val}s/.*/imax_allowed_packet = 32M" /etc/mysql/my.cnf";
done
For the first stanza of script I get the following output:
Error: sed: -e expression #1, char 1: unknown command: `''
For the second Stanza I get the following output:
sed: -e expression #1, char 3: unknown command:
'
sed: -e expression #1, char 3: unknown command:
'
For the third Stanza I get the following output:
./test.sh: line 22: unexpected EOF while looking for matching `"'
./test.sh: line 24: syntax error: unexpected end of file
Edit, rewriting the sed commands as sed -i "${ary[$key]}" generates the following error output: sed: -e expression #1, char 3: unknown command: `
I think you're over-complicating the issue. Your script can be reduced to this:
sed 's/\(max_allowed_packet\).*/\1 = 32M/' /etc/mysql.cnf
This performs a substitution on every occurrence of max_allowed_packet, setting the rest of the line to = 32M. Add the -i switch to overwrite the file when you're happy with the result.
Problems with your attempt
Shell parameters are not expanded within single quotes, so you would need to use double quotes, e.g. sed -i "${ary[$key]}d". You can use set -x to see what is happening here - at the moment, you will see the literal string $ary[$key], rather than the array value.
If I understand your intention correctly (you want to substitute the entire line), there's no need to call sed twice:
for val in "${ary[#]}"; do
sed -i.bak "${val}s/.*/imax_allowed_packet = 32M" /etc/mysql/my.cnf
done
I have chosen to loop through the values of the array, instead of the keys, in order to simplify things a little. When using the -i option, it is always a good idea to specify a backup file, as I have done.
I have a custom made tool that parses XML files using SAX and that reproduces them into pipe separated values, an example of a line would be:
name|lastname|address|telephone|age|other info|other info
I would like to rewrite each line, i will do this using a bash script but i'm having some difficulties.
Basically i would like to set each word between quotes, an example for the above line would be:
"name"|"lastname"|"address"|"telephone"|"age"|"other info"|"other info"
i'm trying to do this using sed, and i'm partially successful with this sed line
sed 's:|:"|":g'
as i get the output:
name"|"lastname"|"address"|"telephone"|"age"|"other info"|"other info
but i dont know how to set quotes for the first char and the last char,
Any advice?
You were definitely on the right track, Here's how to cover the special cases of beginning of line, end of line:
sed 's:|:"|":g;s/^/"/;s/$/"/'
^ char anchors search to beginning of line, the $ char anchors search at the end of the line.
IHTH
line='name|lastname|address|telephone|age|other info|other info'
echo $line| sed -e 's/|/"|"/g' -e 's/^/"/' -e 's/$/"/'
gives:
"name"|"lastname"|"address"|"telephone"|"age"|"other info"|"other info"
One way using GNU awk:
awk 'BEGIN { FS=OFS="|"; Q="\"" } { for (i=1; i<=NF; i++) $i = Q $i Q }1'
Results:
"name"|"lastname"|"address"|"telephone"|"age"|"other info"|"other info"