This function is part of a shell script:
function json_ccp {
local PP=$(one_line_pem $4)
local CP=$(one_line_pem $5)
sed -e "s/\${ORG}/$1/" \
-e "s/\${P0PORT}/$2/" \
-e "s/\${P1PORT}/$6/" \
-e "s/\${CAPORT}/$3/" \
-e "s#\${PEERPEM}#$PP#" \
-e "s#\${CAPEM}#$CP#" \
organizations/ccp-template.json
}
I get the following error when I execute the script:
sed: -e expression #5, char 841: unterminated `s' command
Everything looks correct to me, in the last two sed expressions I have used # instead of / as a delimiter.
$ sed --version
sed (GNU sed) 4.8
Related
While installing ncurses package using Cygwin64 Terminal on Windows, some errors are showing during the make process as follow:
Kindly note that i am using sed -x command at the beginning of current file named MKlib_gen.sh, one of the downloaded source files, to print out the commands and errors at the same time in order to help debug the code.
The 1st error:
+ preprocessor='gcc -DNCURSES_WATTR_MACROS -DNCURSES_INTERNALS -I../include'
+ AWK=-E
+ USE=-DHAVE_CONFIG_H
++ echo gcc
++ -E '{ sub(/^[ ]*/,""); sub(/[ ].*$/, ""); print; }'
./base/MKlib_gen.sh: line 87: -E: command not found
The 2nd error:
+ sed -n -f sed1_1576.sed
+ sed -e 's/NCURSES_EXPORT(\(.*\)) \(.*\) (\(.*\))/\1 \2(\3)/'
+ sed -f sed2_1576.sed
+ -E -f awk1_1576.awk using=-DHAVE_CONFIG_H
./base/MKlib_gen.sh: line 509: -E: command not found
+ sed -e 's/ [ ]*$//g' -e 's/^\([a-zA-Z_][a-zA-Z_]*[ *]*\)/\1 gen_/' -e 's/gen_$//' -e 's/ / /g'
The 3rd error:
+ gcc -DNCURSES_WATTR_MACROS -DNCURSES_INTERNALS -I../include gen__1576.c
+ sed -e 's/ / /g' -e 's/^ //' -e s/_Bool/NCURSES_BOOL/g
+ -E -f awk2_1576.awk
./base/MKlib_gen.sh: line 521: -E: command not found
Kindly advise how to solve these errors.
I am using the sed command on Ubuntu to replace content.
This initial command comes from here.
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
However, as you can see, I have a slash in the replacement. The slash causes the command to throw:
sed: -e expression #1, char 9: unknown option to `s'
Moreover, my replacement is stored in a variable.
So the following will not work because of the slash:
sed -i "$ s/$/ $1/" "$DIR./result/doc.md"
As stated here and in duplicate, I should use another delimiter. If I try with #:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
It gives the error:
sed: -e expression #1, char 42: unterminated `s' command
My question is:
How can I use a variable in this command as well as other delimiter than / ?
Don't use sed here; perl and awk allow more robust approaches.
sed doesn't allow variables to be passed out-of-band from code, so they always need to be escaped. Use a language without that limitation, and you have code that always works, no matter what characters your data contains.
The Short Answer: Using perl
The below is taken from BashFAQ #21:
inplace_replace() {
local search=$1; shift; local replace=$1; shift
in="$search" out="$replace" perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' "$#"
}
inplace_replace '#' "replacement" "$DIR/result/doc.md"
The Longer Answer: Using awk
...or, using awk to do a streaming replacement, and a shell function to make that file replacement instead:
# usage as in: echo "in should instead be out" | gsub_literal "in" "out"
gsub_literal() {
local search=$1 replace=$2
awk -v s="${search//\\/\\\\}" -v r="${rep//\\/\\\\}" 'BEGIN {l=length(s)} {o="";while (i=index($0, s)) {o=o substr($0,1,i-1) r; $0=substr($0,i+l)} print o $0}'
}
# usage as in: inplace_replace "in" "out" /path/to/file1 /path/to/file2 ...
inplace_replace() {
local search=$1 replace=$2 retval=0; shift; shift
for file; do
tempfile=$(mktemp "$file.XXXXXX") || { retval |= $?; continue; }
if gsub_literal "$search" "$replace" <"$file" >"$tempfile"; then
mv -- "$tempfile" "$file" || (( retval |= $? ))
else
rm -f -- "$tempfile" || (( retval |= $? ))
fi
done
}
TL;DR:
Try:
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Long version:
Let's start with your original code:
sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"
And let's compare it to the code you referenced:
sed -i '$ s/$/abc/' file.txt
We can see that they don't exactly match up. I see that you've correctly made this substitution:
file.txt --> "$DIR./result/doc.md"
That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:
abc --> /replacement
You actually introduced another delimeter /. However, if we replace the delimiters with '#' we get this:
sed -i '$ s#$# /replacement#' "$DIR./result/doc.md"
I think that the above is perfectly valid in sed/bash. The $# will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.
Looking at one of your attempts:
sed -i "$ s#$# $1#" "$DIR./result/doc.md"
You will have problems due to the shell interpolation of $# in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):
sed -i '$ s#$# '"$1"'#' "$DIR./result/doc.md"
Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.
Use shell parameter expansion to add escapes to the slashes in the variable:
$ cat file
foo
bar
baz
$ set -- ' /repl'
$ sed "s/$/$1/" file
sed: 1: "s/$/ /repl/": bad flag in substitute command: 'r'
$ sed "s/$/${1//\//\\\/}/" file
foo /repl
bar /repl
baz /repl
That is a monstrosity of leaning toothpicks, but it serves to transform this:
sed "s/$/ /repl/"
into
sed "s/$/ \/repl/"
The same technique can be used for whatever you choose as the sed s/// delimiter.
I wrote this code:
cat /etc/passwd | cut -d : -f1 | sed -n "${FT_LINE1}, ${FT_LINE2} p"
Output:
sed: -e expression #1, char 1: unknown command: `,'
But I have a problem with variables $FT_LINE1, $FT_LINE2.
When I use constants instead of a variables, this code works correctly
cat /etc/passwd | cut -d : -f1 | sed -n "3, 5 p"
I tried to use these constructions:
sed -n -e "${FT_LINE1}, ${FT_LINE2} p"
sed -n "{$FT_LINE1}, {$FT_LINE2} p"
sed -n "${FT_LINE1},${FT_LINE2} p"
sed -n "${FT_LINE1}, ${FT_LINE2}" p
sed -n "$FT_LINE1, $FT_LINE2" p
but the error remained.
As noted in melpomene and PesaThe's comments, sed address ranges can't be blank, both shell variables ${FT_LINE1}, and ${FT_LINE2}, must be set to some appropriate value.
This simplest way to reproduce the error is:
sed ,
Which outputs:
sed: -e expression #1, char 1: unknown command: `,'
Because , is not a sed command, it's just a delimiter that separates range addresses.
It might help to look at some other related errors. Let's add a starting address of 1:
sed 1,
Output:
sed: -e expression #1, char 2: unexpected `,'
Which seems unhelpful, since it should be expecting an address after the ,. Now let's add a second address of 1:
sed 1,1
Output:
sed: -e expression #1, char 3: missing command
A little better, but really it's char 4 that's missing a command, or rather there's a missing command after char 3.
Now let's add a command, and a bit of input and it works:
echo foo | sed 1,1p
Output:
foo
So i have an array with filenames.
for i in "${!array_FILE[#]}"; do
printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > array_FILE[$i].js
done
The printf works and gives me
0 app_calander
1 app_contacts
2 app_search
3 app_index
but the files created are:
array_FILE[0].js
array_FILE[1].js
array_FILE[2].js
array_FILE[3].js
instead of
app_calander.js
app_contacts.js
app_search.js
app_index.js
If you can help me i appreciate it, it has to be changed by index because i have two array and i need to change values at the same index.
My temporary solution is:
filename="${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $filename.js
but i was wondering if there is a better way!
REAL ISSUE
Now the real issue is when i try to pass a URL
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" -e "s/\${URL}/'$url'/" template > $filename.js
done
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sample value of url is URL:https://example.com/app/index.html
EDIT FOR CLARIFICATION
data.txt
USERNAME_VAR:input_username
USERNAME:user01
PASSWORD_VAR:input_password
PASSWORD:password1
SUBMIT:submit
AUTH:cas
URL:https://example.com/app/calander.html
FILE:app_calander
URL:https://example.com/app/contacts.html
FILE:app_contacts
URL:https://example.com/app/search.html
FILE:app_search
URL:https://example.com/app/index.html
FILE:app_index
template
${USERNAME_VAR} = ${USERNAME}
${SUBMIT} IS TRUE
${PASSWORD_VAR} = ${PASSWORD}
${AUTH} = AUTH IS
URL TO HIT IS ${URL}
inject.sh
#!/bin/bash
USERNAME_VAR=($(grep -o 'USERNAME_VAR.*' data.txt | cut -f2- -d':'))
USERNAME=($(grep -o 'USERNAME.*' data.txt | grep -v 'VAR.*' | cut -f2- -d':'))
echo $USERNAME_VAR
echo $USERNAME
array_URL=($(grep -o 'URL.*' data.txt | cut -f2- -d':'))
array_FILE=($(grep -o 'FILE.*' data.txt | cut -f2- -d':'))
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
FILENAME="${array_FILE[$i]}"
URL="${array_URL[$i]}"
echo $URL
sed -e "s/\${URL}/$URL/" -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $FILENAME.js
done
Continuing from the comment, you could do something like:
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s#\${USERNAME_VAR}#$USERNAME_VAR#" \
-e "s#\${USERNAME}#$USERNAME#" \
-e "s#\${URL}#$url#" template > $filename.js
done
The answer to the first question is staring right at you: "${array_FILE[$i]}" is obviously different from "array_FILE[$i]"
To understand the "REAL ISSUE", just look at the error messages. They are telling you the problem is with the third sed expression, which assumes that $url does not have a "/" in it.
Unless you are certain that $USERNAME, $USERNAME_VAR and $url do not have "/" in them, then those sed commands will not work in the way you seem to expect.
Why does this code return an error?
#!/bin/bash
ARG=$1
MYVAR="TEST"
FILE="/path/to/file/$ARG"
AAA="/path/test/$MYVAR"
BBB="foo bar $AAA bar foo $AAA"
sed -i -e "s/TEXT/$BBB/g" $FILE
sed: -e expression #1, char 58: unknown option to `s'
Your replacement ($BBB) has slashes in it, the same delimiter you are using for sed. Use another:
sed -i -e "s|TEXT|$BBB|g" $FILE
Using the debug mode (use #!/bin/bash -x as a shebang):
+ ARG=foo
+ MYVAR=TEST
+ FILE=/path/to/file/foo
+ AAA=/path/test/TEST
+ BBB='foo bar /path/test/TEST bar foo /path/test/TEST'
+ sed -i -e 's/TEXT/foo bar /path/test/TEST bar foo /path/test/TEST/g' /path/to/file/foo
sed: -e expression #1, char 18: unknown option to `s'
Other people gave an explanation.