BASH sed does not evaluate array - bash

So i have an array with filenames.
for i in "${!array_FILE[#]}"; do
printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > array_FILE[$i].js
done
The printf works and gives me
0 app_calander
1 app_contacts
2 app_search
3 app_index
but the files created are:
array_FILE[0].js
array_FILE[1].js
array_FILE[2].js
array_FILE[3].js
instead of
app_calander.js
app_contacts.js
app_search.js
app_index.js
If you can help me i appreciate it, it has to be changed by index because i have two array and i need to change values at the same index.
My temporary solution is:
filename="${array_FILE[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $filename.js
but i was wondering if there is a better way!
REAL ISSUE
Now the real issue is when i try to pass a URL
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" -e "s/\${URL}/'$url'/" template > $filename.js
done
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sed: -e expression #3, char 18: unknown option to `s'
sample value of url is URL:https://example.com/app/index.html
EDIT FOR CLARIFICATION
data.txt
USERNAME_VAR:input_username
USERNAME:user01
PASSWORD_VAR:input_password
PASSWORD:password1
SUBMIT:submit
AUTH:cas
URL:https://example.com/app/calander.html
FILE:app_calander
URL:https://example.com/app/contacts.html
FILE:app_contacts
URL:https://example.com/app/search.html
FILE:app_search
URL:https://example.com/app/index.html
FILE:app_index
template
${USERNAME_VAR} = ${USERNAME}
${SUBMIT} IS TRUE
${PASSWORD_VAR} = ${PASSWORD}
${AUTH} = AUTH IS
URL TO HIT IS ${URL}
inject.sh
#!/bin/bash
USERNAME_VAR=($(grep -o 'USERNAME_VAR.*' data.txt | cut -f2- -d':'))
USERNAME=($(grep -o 'USERNAME.*' data.txt | grep -v 'VAR.*' | cut -f2- -d':'))
echo $USERNAME_VAR
echo $USERNAME
array_URL=($(grep -o 'URL.*' data.txt | cut -f2- -d':'))
array_FILE=($(grep -o 'FILE.*' data.txt | cut -f2- -d':'))
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
FILENAME="${array_FILE[$i]}"
URL="${array_URL[$i]}"
echo $URL
sed -e "s/\${URL}/$URL/" -e "s/\${USERNAME_VAR}/$USERNAME_VAR/" -e "s/\${USERNAME}/$USERNAME/" template > $FILENAME.js
done

Continuing from the comment, you could do something like:
for i in "${!array_FILE[#]}"; do
#printf "%s\t%s\n" "$i" "${array_FILE[$i]}"
filename="${array_FILE[$i]}"
url="${array_URL[$i]}"
sed -e "s#\${USERNAME_VAR}#$USERNAME_VAR#" \
-e "s#\${USERNAME}#$USERNAME#" \
-e "s#\${URL}#$url#" template > $filename.js
done

The answer to the first question is staring right at you: "${array_FILE[$i]}" is obviously different from "array_FILE[$i]"
To understand the "REAL ISSUE", just look at the error messages. They are telling you the problem is with the third sed expression, which assumes that $url does not have a "/" in it.
Unless you are certain that $USERNAME, $USERNAME_VAR and $url do not have "/" in them, then those sed commands will not work in the way you seem to expect.

Related

Using sed to remove prefix with slash from string

I'm extracting a Jira issue from a string using sed. I want to get rid of prefixes too.
My possible prefixes are:
FEATURE_PREFIX="feature/"
BUGFIX_PREFIX="bugfix/"
I've tried three different ways to use sed despite the slash in the prefix but nothing seems to work.
First try:
export UNFIXED_ID=$(echo ${CI_MERGE_REQUEST_TITLE} | sed -e "s~^$BUGFIX_PREFIX~/" | sed -e "s~^$FEATURE_PREFIX~/")
export MERGE_REQUEST_JIRA_ID=$(echo ${UNFIXED_ID} | sed -r "s/^([A-Za-z][A-Za-z0-9]+-[0-9]+).*/\1/")
echo ${MERGE_REQUEST_JIRA_ID}
gives the error sed: unmatched '~'
Second try:
export UNFIXED_ID=$(echo ${CI_MERGE_REQUEST_TITLE} | sed -e "s~^$BUGFIX_PREFIX/~" | sed -e "s~^$FEATURE_PREFIX/~")
export MERGE_REQUEST_JIRA_ID=$(echo ${UNFIXED_ID} | sed -r "s/^([A-Za-z][A-Za-z0-9]+-[0-9]+).*/\1/")
echo ${MERGE_REQUEST_JIRA_ID}
gives the error sed: unmatched '~'
Third try:
export UNFIXED_ID=$(echo ${CI_MERGE_REQUEST_TITLE} | sed -e "s~^$BUGFIX_PREFIX~" | sed -e "s~^$FEATURE_PREFIX~")
export MERGE_REQUEST_JIRA_ID=$(echo ${UNFIXED_ID} | sed -r "s/^([A-Za-z][A-Za-z0-9]+-[0-9]+).*/\1/")
echo ${MERGE_REQUEST_JIRA_ID}
gives the error sed: unmatched '~'
As per this question Sed error : bad option in substitution expression I thought it was just a matter of replacing the / by ~
What am I failing to do here with the delimiter?
I apologise if I have misunderstood the question, but if the two variables are on their own lines, then you can probably just search for and print the entire line:-
sed -n '/FEATURE_PREFIX/p' file | \
cut -d'=' -f2 | \
head -c -3 | \
sed s'#$#\"#'

What is the correct syntax to combine multiple parameter expansions?

My current code:
while read -r rbv_line || [[ -n "$rbv_line" ]]; do
if [[ "${rbv_line}" =~ ${rbv_reg} ]]; then
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
((reg_matches="${reg_matches}"+1))
printf "\n"
printf "Setting Ruby version: %s\n" "${ruby_version}"
break
fi
done < "${1}"
It does what I want. But I would love to know if I can simplify this code even more, hoping someone can help me understand the syntax.
If you see these two lines:
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
Initially I tried to combine those into one using something like this:
ruby_version="${BASH_REMATCH[0],,//[^0-9a-z\.\-]/}"
That does not work.
Is there a way to combine those two parameter expansions (,, and the //[^0-9a-z\.\-]/) or is passing it through an intermediary variable the right approach?
You can view the latest version of the code here:
https://github.com/octopusnz/scripts
You cannot combine multiple parameter expansions, but...
... you can simplify this code!
The biggest gain is by using already available tools.
Instead of looping, let's use grep. It's supposed to do something when RegEx pattern is occurred, so:
grep -E "$rbv_reg" "$1" # -E is for extended RegEx
I guess your pattern isn't case sensitive, so let's disable it with -i flag.
The loop breaks after match, so let's pass -m 1 to stop processing file after first match.
You want to convert uppercase to lowercase, so let's pipe it through tr:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]'
You then replace some characters with //[^0-9a-z\.\-]/, piping it to sed will do the trick:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g'
And at the very end, let's grab the output to variable:
ruby_version="$( grep -m 1 -E -i '$rbv_reg' '$1' | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g' )"
Since you are printing new line anyway, let's use simple echo instead of printf
All what's left is if [ -n "$ruby_version" ] to increment reg_matches
At the end, we got:
ruby_version="$(
grep -m 1 -E -i '$rbv_reg' '$1' |
tr '[:upper:]' '[:lower:]' |
sed 's/[^0-9a-z\.\-]//g'
)"
if [ -n "$ruby_version" ]; then
reg_matches="$((reg_matches+1))"
echo
echo "Setting Ruby version: $ruby_version"
fi
The advantage of above code is the fact it isn't really Bash dependent and should work in any POSIX Bourne compatible shell.

Using variables in terminal

I wrote this code:
cat /etc/passwd | cut -d : -f1 | sed -n "${FT_LINE1}, ${FT_LINE2} p"
Output:
sed: -e expression #1, char 1: unknown command: `,'
But I have a problem with variables $FT_LINE1, $FT_LINE2.
When I use constants instead of a variables, this code works correctly
cat /etc/passwd | cut -d : -f1 | sed -n "3, 5 p"
I tried to use these constructions:
sed -n -e "${FT_LINE1}, ${FT_LINE2} p"
sed -n "{$FT_LINE1}, {$FT_LINE2} p"
sed -n "${FT_LINE1},${FT_LINE2} p"
sed -n "${FT_LINE1}, ${FT_LINE2}" p
sed -n "$FT_LINE1, $FT_LINE2" p
but the error remained.
As noted in melpomene and PesaThe's comments, sed address ranges can't be blank, both shell variables ${FT_LINE1}, and ${FT_LINE2}, must be set to some appropriate value.
This simplest way to reproduce the error is:
sed ,
Which outputs:
sed: -e expression #1, char 1: unknown command: `,'
Because , is not a sed command, it's just a delimiter that separates range addresses.
It might help to look at some other related errors. Let's add a starting address of 1:
sed 1,
Output:
sed: -e expression #1, char 2: unexpected `,'
Which seems unhelpful, since it should be expecting an address after the ,. Now let's add a second address of 1:
sed 1,1
Output:
sed: -e expression #1, char 3: missing command
A little better, but really it's char 4 that's missing a command, or rather there's a missing command after char 3.
Now let's add a command, and a bit of input and it works:
echo foo | sed 1,1p
Output:
foo

how to assign echo value to a variable in shell

Im tring to assign echo value which to a variable but im getting error
Var='(echo $2 | sed -e 's/,/: chararray /g'| sed -e 's/$/: chararray/')'
echo $var
Input : sh load.sh file 1,2,3,4
Error load.sh: line 1: chararray: command not found
Var=$(echo "$2" | sed -e 's/,/: chararray /g' | sed -e 's/$/: chararray/')
echo "$Var"
OR
Var=`echo "$2" | sed -e 's/,/: chararray /g' | sed -e 's/$/: chararray/'`
echo "$Var"
Use either $(…) or perhaps `…` backtick notation. However, the backtick notation is deprecated and should be avoided. Also, check the comments by mmgross, Etan Reisner and svlasov to your question. They are all correct.

how to add \r\n in a variable in sed?

Simple question.
When I use sed to add \r\n into the variable
it fails.
how to add \r\n?
dateRecent=$(sed 's| 年| 年'"\r\n"'|g' <<< $newsDate)
dateRecent=$(sed 's| 年| 年\r\n|g' <<< $newsDate)
sed: -e expression #1, char 146: unterminated `s' command
The whole code is here:
cp /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.html /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.test.html
echo "Please input Date in this format(eg.2011 年 7 月 8 日):"
read -e newsDate
echo "Please input Title:"
read -e title
echo "Please input Description:"
read -e desc
echo "Please input ID(eg.d071101):"
read -e id
echo "Please input reference website:"
read -e web
echo "Confirm? Have to do it all over again if wrong (Yes:y, No:n)"
read -e confirm
dateRecent=$newsDate
if [[ "$mail" == "y" ]]; then
dateRecent=$(sed -e 's/ 年/ 年\r\n/g' <<< $newsDate)
fi
#Add Phishing attack in recent.html
sed -i '0,/<li>/ { s/<li>/<li><a href="'"$web"'" target="_blank">'"$dateRecent"' - '"$title"'<\/a><\/li>\r\n <li>/ }' /var/www/html/INFOSEC/textonly/sc_chi/anti/recent.test.html
Ppl can't re-create. So it might depends on sed version.
New gnused should handle \r\n like some in comments reports.
Older gnused and other sed might need the original newline and return re-produced. Hence you can use echo to get it and neglect sed impplementions, but this brings another dependency on your shell.
# ksh is my suggested standard style:
sed "s/ 年/ 年`echo -e \\\r\\\n`/g"
# zsh is like ksh and you can omit the -e for echo
# Old bash?:
sed 's/ 年'"/ 年`echo \\\r\\\n`/g"
Windows is easy, just double quote GNUSed is assumed.
sed "s/ 年/ 年\r\n/g"
See how life is a pain under AIX...

Resources