Use sed to remove string, add whitespace and insert "" - bash

I have a file with the following entry:
export TF_VAR_environment_name=dev
export TF_VAR_project_name=hello-world
I would like to do 3 things with these enteries:
Remove the export TF_VAR_ string
Add whitespace to both sides of =
Wrap the string right of = in " "
So my file would end up looking like:
environment_name = "dev"
project_name = "hello-world"
I'm able to remove the string with s/"export TF_VAR_"//, but haven't been able to wrap the = in whitespace, or wrap the final string in quotes. Any help would be greatly appriciated.
Is this possible to do in sed?

input.txt is your textfile.
output.txt is the wanted result.
sed 's/export TF_VAR_// ; s/=\ (.*\ )$/ = "\1"/ ' < input.txt > output.txt
there is no blank between \ and (
and no blank between \ and )

input.txt is your textfile. output.txt is the wanted result.
sed 's/export TF_VAR_// ; s/=\(.*\)$/ = "\1"/ ' < input.txt > output.txt
its the same as above. i just tried to post it here in a correct way.

Related

Trying to comment a source file

I am trying to comment lines of source so that something like
LANG = 'ENG';
becomes
// LANG = 'ENG';
There are over a thousand lines in the source file and 'ENG' is not unique but the entire line IS.
I gave up on wild carding the spaces and just tried the entire extant line 'as-is' but no joy.
Something like (commented shell)--
enter code here
#!/bin/bash
#if [ -n "$5" ] ; then
#if [ "$5" == "ENG" ] ; then
sed -i "s/' LANG = '\''ENG'\''/\/\/' LANG'
= '\''ENG'\''/" vc.pas > vc.out
#fi
#fi
So it reduces down to a single line. No joy whatever I try.
TIA !
Howie
This works for me, passing any whitespace through to the output:
$ echo "LANG='ENG';" | sed "s#^\(\s*LANG\s*=\s*'ENG'\s*;\)#// \1#"
// LANG='ENG';
$ echo " LANG = 'ENG' ; " | sed "s#^\(\s*LANG\s*=\s*'ENG'\s*;\)#// \1#"
// LANG = 'ENG' ;
Technically it needs double backslashes inside the double-quoted string, but because none of the sequences form a valid escape sequence, bash doesn't mind.
With GNU sed, use
sed -i "s,.*LANG *= *'ENG';.*,//&," vc.pas
where
-i - enables inline file modification
s - substitution command
, - delimiter
.*LANG *= *'ENG';.* - text containing LANG = 'ENG'; with any amount of spaces around =
//& - replaces the matched line with // and the line itself
Use a regular expression for the line selection, followed by a substitution. Put the replacement commands in a file:
$ cat dt.sed
/^LANG[[:space:]]*=[[:space:]]*'[^']*'/s;^;// ;
$
Then run sed(1) with that script:
$ echo "LANG='FOO'" | sed -f dt.sed
// LANG='FOO'
This works on a Fedora 34 system, but should work on any Linux.

bash-replacing string in file, that contains special chars

as i said in the title im trying to replace a string in a file, that contains special characters , now the idea is to loop on every line of a "infofile" contains many lines of: whatiwantotreplace,replacer.
once I have this i want to do sed to a certain file to replace all the occurrences of string-> "whatiwantotreplace" with ->"replacer".
my code:
infofile="inforfilepath"
replacefile="replacefilepath"
while IFS= read -r line
do
what2replace="a" #$(echo "$line" | cut -d"," -f1);
replacer="b\\" #$(echo "$line" | cut -d"," -f2 );
sed -i -e "s/$what2replace/$replacer/g" "$replacefile"
#sed -i -e "s/'$what2replace'/'$replacer'/g" "$replacefile"
#sed -i -e "s#$what2replace#$replacer#g" "$replacefile"
#sed -i -e s/$what2replace/$replacer/g' "$replacefile"
#sed -i -e "s/${what2replace}/${replacer}/g" "$replacefile"
#${replacefile//what2replace/replacer}
done < "$infofile"
As you can see, the string that want to replace and the string that i want to replace with,may contain special characters , all the commented lines are the things I tried (things I saw online) but still clueless.
for some i got this error:
"sed: -e expression #1, char 8: unterminated `s' command"
and for some just nothing happend.
really need your help
Edit: inputs and outputs:
It's hard to give inputs and output, because all of the variations I tried had the same thing , didn't changed anything, the only one gave the above error is the variation with #.
thanks for your effort.
You're barking up the wrong tree - you're trying to do literal string replacements using a tool, sed, that doesn't have functionality to handle literal strings. See Is it possible to escape regex metacharacters reliably with sed for the convoluted mess required to try to force sed to do what you want and also https://unix.stackexchange.com/q/169716/133219 for why to avoid shell loops for manipulating text.
Just use awk instead since it has literal string functions and loops implicitly itself:
awk '
NR==FNR{map[$1]=$2; next}
{
for (old in map) {
new = map[old]
head = ""
tail = $0
while ( s = index(tail,old) ) {
head = head substr(tail,1,s-1) new
tail = substr(tail,s+length(old))
}
$0 = head tail
}
}
' "$infofile" "$replacefile"
The above is untested of course since you didn't provide any sample input/output.
You can try this way
what='a';to='b\\\\';echo 'sdev adfc xdae' | sed "s/${what}/${to}/g"
output
sdev b\\dfc xdb\\e

"unterminated address regex" using variable in sed

I'm trying to use a variable in a sed append and hitting an issue.
The following command works as expected:
sed -i "\:#file = /mnt/var/log/hadoop-yarn/containers/application_1495965866386_0001/container_1495965866386_0001_01_000002/stderr:a file = /path/to/other/file" /etc/conf/service.conf
However if I replace the pattern with a variable I'm hitting an error:
$ echo $item
#file = /mnt/var/log/hadoop-yarn/containers/application_1495965866386_0001/container_1495965866386_0001_01_000002/stdout
$ sed -i "\:$item:a file = /path/to/other/file" /etc/conf/service.conf
sed: -e expression #1, char 122: unterminated address regex
EDIT for more info: So the 'item' variable is being populated from an array. That array is created from a readarray and grep:
$readarray LINES < <(grep "#file = /mnt/var/" /etc/conf/service.conf)
$item=${LINES[1]}
$echo $item
#file = /mnt/var/log/hadoop-yarn/containers/application_1495965866386_0001/container_1495965866386_0001_01_000002/stdout
However I've found if i populate 'item' manually it then works e.g:
$item="#file = /mnt/var/log/hadoop-yarn/containers/application_1495965866386_0001/container_1495965866386_0001_01_000002/stdout"
$sed -i "\:$item:a file = /path/to/other/file" /etc/conf/service.conf
$
So something strange seems to be happening with the readarray/grep
So the problem here turned out to be newline characters that were being pulled in as part of the grep.
This is why populating $item manually worked - no '\n'
Thanks to Ed Morton for pointing me in the right direction. While
echo "$item" | cat -v
did not show anything I added '-t' to the readarray command to trim newline characters:
$readarray -t LINES < <(grep "#file = /mnt/var/" /etc/conf/service.conf)
After that things worked as expected.

Sed fails to update long text

Consider test file csf.conf:
CC_DENY = ""
Running the command:
sed -i -E 's/(CC_DENY *= *")[^"]+/\1AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN,AE,AF,AL,AS,AZ,BA,BD,BF,BH,BJ,BN,CI,DJ,EG,EH,ER,ET,GM,GN,GW,IQ,IR,IS,JO,KG,KM,KW,KZ,LB,LY,MC,MK,ML,MR,MV,MY,NE,NG,OM,PK,PS,QA,SA,SD,SL,SN,SO,SY,TD,TJ,TM,TN,TR,UZ,XK,YE,YT/g' csf.conf
Does not replace the match inside the file. Output should look like this:
CC_DENY="AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN,AE,AF,AL..."
Sed v4.2.2, same result on Debian 8, and Centos 7
This has nothing to do with long text, your regexp just doesn't match the content of your file. Change [^"]+ to [^"]* so it'll match even when there's nothing between the double quotes "". Look:
$ cat csf.conf
CC_DENY = ""
$ sed -E 's/(CC_DENY *= *")[^"]+/\1foo/' csf.conf
CC_DENY = ""
$ sed -E 's/(CC_DENY *= *")[^"]*/\1foo/' csf.conf
CC_DENY = "foo"
wrt the comment below from the OP that this sed command works:
$ cat file
LF_SPI = ""
$ sed -E 's/(LF_SPI *= *\")[^\"]+/\1blah/g' file
LF_SPI = ""
Clearly and predictably, no it does not. It simply can't because the regexp metacharacter + means 1 or more so [^\"]+ states there must be at least one non-" after the " and that just does not exist in the input file. There is no reason to escape the double quotes btw.
Suppose the current variable value in the file is empty. Then your regular expression doesn't match because [^"]+ means "any character, except double quote repeated one or more times".
You might fix it by replacing + quantifier with * (zero or more times). But suppose the value contains a double quote:
CC_DENY = "\""
Then the [^"]* will match everything until it gets to the double quote within the value.
Thus, I suggest the following command:
# Put the variable value here
value='AR,BE\\" ... YE,YT';
sed -i -r 's/^( *CC_DENY *= *").*"/\1'"$value"'"/' csf.conf
Also note, that the expression above uses an anchor for the beginning of the line. Otherwise, it will fail to match as expected, if such a CC_DENY = "... exists in the variable value in the configuration file: CC_DENY = "SOMETHING_CC_DENY = \"value\"".
Sed is certainly the wrong tool for this:
#!/usr/bin/awk -f
BEGIN {
FS = OFS = "\42"
}
$2 = "AR,BE,CL,CN,CO,CS,ES,FR,GR,HK,IT,KO,PA,PE,PH,PL,RS,RU,SG,SK,TH,UA,VN," \
"AE,AF,AL,AS,AZ,BA,BD,BF,BH,BJ,BN,CI,DJ,EG,EH,ER,ET,GM,GN,GW,IQ,IR,IS,JO,KG," \
"KM,KW,KZ,LB,LY,MC,MK,ML,MR,MV,MY,NE,NG,OM,PK,PS,QA,SA,SD,SL,SN,SO,SY,TD,TJ," \
"TM,TN,TR,UZ,XK,YE,YT"

Creating a comma separated array in bash

I have a file with contents as the following .
line1
line2
line3
I need to create array like this
('line2','line2','line3')
How can I do that ?
The should help you :
sed -r "s/^|$/'/g" file | echo "(`paste -d, -s`)"
I'm using sed to add ' in start & end of each line, then concatenate the content using paste and enclosing it with parenthesis using echo.
You can use the following:
while read line; do printf "'$line',"; done < file | sed 's/^/(/;s/,$/)\n/'
The while loop gets the content and enclose it between brackets.
s/^/(/ is adding a ( at the beginning of the string.
s/,$/)\n/ is replacing the last , by a ) and a cariage return.

Resources