Simple question.
When I use sed to add \r\n into the variable
it fails.
how to add \r\n?
dateRecent=$(sed 's| 年| 年'"\r\n"'|g' <<< $newsDate)
dateRecent=$(sed 's| 年| 年\r\n|g' <<< $newsDate)
sed: -e expression #1, char 146: unterminated `s' command
The whole code is here:
cp /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.html /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.test.html
echo "Please input Date in this format(eg.2011 年 7 月 8 日):"
read -e newsDate
echo "Please input Title:"
read -e title
echo "Please input Description:"
read -e desc
echo "Please input ID(eg.d071101):"
read -e id
echo "Please input reference website:"
read -e web
echo "Confirm? Have to do it all over again if wrong (Yes:y, No:n)"
read -e confirm
dateRecent=$newsDate
if [[ "$mail" == "y" ]]; then
dateRecent=$(sed -e 's/ 年/ 年\r\n/g' <<< $newsDate)
fi
#Add Phishing attack in recent.html
sed -i '0,/<li>/ { s/<li>/<li><a href="'"$web"'" target="_blank">'"$dateRecent"' - '"$title"'<\/a><\/li>\r\n <li>/ }' /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.test.html
Ppl can't re-create. So it might depends on sed version.
New gnused should handle \r\n like some in comments reports.
Older gnused and other sed might need the original newline and return re-produced. Hence you can use echo to get it and neglect sed impplementions, but this brings another dependency on your shell.
# ksh is my suggested standard style:
sed "s/ 年/ 年`echo -e \\\r\\\n`/g"
# zsh is like ksh and you can omit the -e for echo
# Old bash?:
sed 's/ 年'"/ 年`echo \\\r\\\n`/g"
Windows is easy, just double quote GNUSed is assumed.
sed "s/ 年/ 年\r\n/g"
See how life is a pain under AIX...
Related
So i have an array with filenames.
for i in "${!array_FILE[#]}"; do
printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > array_FILE[$i].js
done
The printf works and gives me
0 app_calander
1 app_contacts
2 app_search
3 app_index
but the files created are:
array_FILE[0].js
array_FILE[1].js
array_FILE[2].js
array_FILE[3].js
instead of
app_calander.js
app_contacts.js
app_search.js
app_index.js
If you can help me i appreciate it, it has to be changed by index because i have two array and i need to change values at the same index.
My temporary solution is:
filename="${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $filename.js
but i was wondering if there is a better way!
REAL ISSUE
Now the real issue is when i try to pass a URL
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" -e "s/\${URL}/'$url'/" template > $filename.js
done
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sample value of url is URL:https://example.com/app/index.html
EDIT FOR CLARIFICATION
data.txt
USERNAME_VAR:input_username
USERNAME:user01
PASSWORD_VAR:input_password
PASSWORD:password1
SUBMIT:submit
AUTH:cas
URL:https://example.com/app/calander.html
FILE:app_calander
URL:https://example.com/app/contacts.html
FILE:app_contacts
URL:https://example.com/app/search.html
FILE:app_search
URL:https://example.com/app/index.html
FILE:app_index
template
${USERNAME_VAR} = ${USERNAME}
${SUBMIT} IS TRUE
${PASSWORD_VAR} = ${PASSWORD}
${AUTH} = AUTH IS
URL TO HIT IS ${URL}
inject.sh
#!/bin/bash
USERNAME_VAR=($(grep -o 'USERNAME_VAR.*' data.txt | cut -f2- -d':'))
USERNAME=($(grep -o 'USERNAME.*' data.txt | grep -v 'VAR.*' | cut -f2- -d':'))
echo $USERNAME_VAR
echo $USERNAME
array_URL=($(grep -o 'URL.*' data.txt | cut -f2- -d':'))
array_FILE=($(grep -o 'FILE.*' data.txt | cut -f2- -d':'))
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
FILENAME="${array_FILE[$i]}"
URL="${array_URL[$i]}"
echo $URL
sed -e "s/\${URL}/$URL/" -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $FILENAME.js
done
Continuing from the comment, you could do something like:
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s#\${USERNAME_VAR}#$USERNAME_VAR#" \
-e "s#\${USERNAME}#$USERNAME#" \
-e "s#\${URL}#$url#" template > $filename.js
done
The answer to the first question is staring right at you: "${array_FILE[$i]}" is obviously different from "array_FILE[$i]"
To understand the "REAL ISSUE", just look at the error messages. They are telling you the problem is with the third sed expression, which assumes that $url does not have a "/" in it.
Unless you are certain that $USERNAME, $USERNAME_VAR and $url do not have "/" in them, then those sed commands will not work in the way you seem to expect.
I need you help on this;
I am currently trying to apply a sed command to lines from a file.
2014-08-05T09:29:13+01:00 (INFO:3824.87075728): [27219] [ <email#domain.com>] A message from <user1#domain.com> source <asdfg> this is a test.
I need to apply this sed cmd to this line but keep this others that does not have 'this is a test'
pattern="this\ is\ a test"
while IFS='' read -r line; do
if [[ $line = *"${pattern}"* ]]; then
sed 's/\[ .*\(source\)/\1/g' ${line}
else
echo "${line}"
fi
done < ${INPUT} > ${OUPUT}
I have set the input and output; however ideally keeping the same file would be ideal.
Thank you for your input.
You don't need a loop for this. Use this sed:
sed -i.bak '/this is a test/s/\[ .*\(source\)/\1/g' "${INPUT}"
This question already has an answer here:
sed delete option change of delimiter
(1 answer)
Closed 8 years ago.
#!/bin/bash
# Let's say now, we are working in my $HOME directory
# Content of testfile (originally)
# 123456
# ABCDEF
# /home/superman
string="ABCDEF"
myfile="$HOME/testfile"
# test-1, this is okay
sed -i "/$string/d" $myfile
echo $string >> $myfile
# test-2, this fails
# ERROR (sed: -e expression #1, char 4: extra characters after command)
sed -i "/$PWD/d" $myfile
echo $PWD >> $myfile
# Not working either
sed -i ":$PWD:d" $myfile
echo $PWD >> $myfile
My question: How to handle the $PWD situation?
To use the alternate delimiters for addresses, you need to use backslash - \
sed "\:$PWD:d" < $myfile
Should work.
Of course for this exact example, grep -v is probably easier.
I'm processing some data from a text file using a bash script (Ubuntu 12.10).
The basic idea is that I select a certain line from a file using grep. Next, I process the line to get the number with sed. Both the grep and sed command are working. I can echo the number.
But the concatenation of the result with a string goes wrong.
I get different results when combining string when I do a grep command from a variable or a file. The concatenation goes wrong when I grep a file. It works as expected when I grep a variable with the same text as in the file.
What am I doing wrong with the grep from a file?
Contents of test.pdb
REMARK overall = 324.88
REMARK bon = 24.1918
REMARK coup = 0
My script
#!/bin/bash
#Correct function
echo "Working code"
TEXT="REMARK overall = 324.88\nREMARK bon = 24.1918\nREMARK coup = 0\n"
DATA=$(echo -e $TEXT | grep 'overall' | sed -n -e "s/^.*= //p" )
echo "Data: $DATA"
DATA="$DATA;0"
echo $DATA
#Not working
echo ""
echo "Not working code"
DATA=$(grep 'overall' test.pdb | sed -n -e "s/^.*= //p")
echo "Data: $DATA"
DATA="$DATA;0"
echo $DATA
Output
Working code
Data: 324.88
324.88;0
Not working code
Data: 324.88
;04.88
I went crazy with the same issue.
The real problem is that your "test.pdb" has probably a wrong EOL (end of line) character.
Linux EOL: LF (aka \n)
Windows EOL: CR LF (aka \r \n)
This mean that echo and grep will have problem with this extra character (\r), luckily tr, sed and awk manage it correctly.
So you can try also with:
DATA=$(grep 'overall' test.pdb | sed -n -e "s/^.*= //p" | sed -e 2s/\r$//")
or
DATA=$(grep 'overall' test.pdb | sed -n -e "s/^.*= //p" | tr -d '\r')
With awk, it will be more reliable and cleaner I guess :
$ awk '$2=="overall"{print "Working code\nData: " $4 "\n" $4 ";0"}' file.txt
Working code
Data: 324.88
324.88;0
Try this:
SUFFIX=";0"
DATA="${DATA}${SUFFIX}"
my question seems to be general, but i can't find any answers.
In sed command, how can you replace the substitution pattern by a value returned by a simple bash function.
For instance, I created the following function :
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
and the folowing sed command :
myCatFile=`sed -e "s/[0-3][0-9]\/[0-1][0-9]\/[0-9][0-9]/& parseDates &\}/p" myfile`
I found that the caracter '&' represents the current pattern found, i'd like it to be passed to my bash function and the whole pattern to be substituted by the pattern found +dateParsed.
Does anybody have an idea ?
Thanks
you can use the "e" option in sed command like this:
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/e" <<END
ni
END
you can see the result without "e":
cat t.sh
myecho() {
echo ">>hello,$1<<"
}
export -f myecho
sed -e "s/.*/myecho &/" <<END
ni
END
Agree with Glenn Jackman.
If you want to use bash function in sed, something like this :
sed -rn 's/^([[:digit:].]+)/`date -d #&`/p' file |
while read -r line; do
eval echo "$line"
done
My file here begins with a unix timestamp (e.g. 1362407133.936).
Bash function inside sed (maybe for other purposes):
multi_stdin(){ #Makes function accepet variable or stdin (via pipe)
[[ -n "$1" ]] && echo "$*" || cat -
}
sans_accent(){
multi_stdin "$#" | sed '
y/àáâãäåèéêëìíîïòóôõöùúûü/aaaaaaeeeeiiiiooooouuuu/
y/ÀÁÂÃÄÅÈÉÊËÌÍÎÏÒÓÔÕÖÙÚÛÜ/AAAAAAEEEEIIIIOOOOOUUUU/
y/çÇñÑߢÐð£Øø§µÝý¥¹²³ªº/cCnNBcDdLOoSuYyY123ao/
'
}
eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | sans_accent#p')
or
eval $(echo "Rogério Madureira" | sed -n 's#.*#sans_accent &#p')
Rogerio
And if you need to keep the output into a variable:
VAR=$( eval $(echo "Rogério Madureira" | sed -n 's#.*#echo & | desacentua#p') )
echo "$VAR"
do it step by step. (also you could use an alternate delimiter , such as "|" instead of "/"
function parseDates(){
#Some process here with $1 (the pattern found)
return "dateParsed;
}
value=$(parseDates)
sed -n "s|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& $value &|p" myfile
Note the use of double quotes instead of single quotes, so that $value can be interpolated
I'd like to know if there's a way to do this too. However, for this particular problem you don't need it. If you surround the different components of the date with ()s, you can back reference them with \1 \2 etc and reformat however you want.
For instance, let's reverse 03/04/1973:
echo 03/04/1973 | sed -e 's/\([0-9][0-9]\)\/\([0-9][0-9]\)\/\([0-9][0-9][0-9][0-9]\)/\3\/\2\/\1/g'
sed -e 's#[0-3][0-9]/[0-1][0-9]/[0-9][0-9]#& $(parseDates &)#' myfile |
while read -r line; do
eval echo "$line"
done
You can glue together a sed-command by ending a single-quoted section, and reopening it again.
sed -n 's|[0-3][0-9]/[0-1][0-9]/[0-9][0-9]|& '$(parseDates)' &|p' datefile
However, in contrast to other examples, a function in bash can't return strings, only put them out:
function parseDates(){
# Some process here with $1 (the pattern found)
echo dateParsed
}