I am trying to write a bash script(script.sh) to search and replace some variables in input.sh file. But I need to modify only the variables which are present in variable_list file and leave others as it is.
variable_list
${user}
${dbname}
input.sh
username=${user}
password=${password}
dbname=${dbname}
Expected output file
username=oracle
password=${password} > This line won't be changed as this variable(${password}) is not in variable_list file
dbname=oracle
Following is the script I am trying to use but I am not able to find the correct sed expression
script.sh
export user=oracle
export password=oracle123
export dbname=oracle
variable='variable_list'
while read line ;
do
if [[ -n $line ]]
then
sed -i 's/$line/$line/g' input.sh > output.sh
fi
done < "$variable"
This could work:
#!/bin/bash
export user=oracle
export password=oracle123
export dbname=oracle
variable='variable_list'
while read line ;
do
if [[ -n $line ]]
then
exp=$(sed -e 's/\$/\\&/g' <<< "$line")
var=$(sed -e 's/\${\([^}]\+\)}/\1/' <<< "$line")
sed -i "s/$exp/${!var}/g" input.sh
fi
done < "$variable"
The first sed expression escapes the $ which is a regex metacharacter. The second extracts just the variable name, then we use indirection to get the value in our current shell and use it in the sed expression.
Edit
Rather than rewriting the file so many times, it's probably more efficient to do it like this, building the arguments list for sed:
#!/bin/bash
export user=oracle
export password=oracle123
export dbname=oracle
while read var
do
exp=$(sed -e 's/\$/\\&/g' <<< "$var")
var=$(sed -e 's/\${\([^}]\+\)}/\1/' <<< "$var")
args+=("-e s/$exp/${!var}/g")
done < "variable_list"
sed "${args[#]}" input.sh > output.sh
user=oracle
password=oracle123
dbname=oracle
variable_list=( '${user}' '${dbname}' )
while IFS="=$IFS" read variable value; do
for subst_var in "${variable_list[#]}"; do
if [[ $subst_var = $value ]]; then
eval "value=$subst_var"
break
fi
done
printf "%s=%s\n" "$variable" "$value"
done < input.sh > output.sh
Here is a script.sh that works:
#!/bin/bash
user=oracle
password=oracle123
dbname=oracle
variable='variable_list'
text=$(cat input.sh)
while read line
do
value=$(eval echo $line)
text=$(sed "s/$line/$value/g" <<< "$text")
done < "$variable"
echo "$text" > output.sh
Note that your original version contains single quotes around the sed string, which doesn't insert the value of $line. It is trying to look for the literal line after the end of the line $ (which will never find anything).
Since you are looking for the value of the variable in $line, you need to do an eval to get this.
Also, since there are multiple variables you are looping over, the intermediate text variable stores the result as it loops.
The export keyword is also unnecessary in this script, unless it is being used in some sub-process not shown.
TXR solution. Build a filter dynamically. The filter is implemented internally as a trie data structure, which gives us a lex-like state machine which matches the entire dictionary at once as the input is scanned. For simplicity, we include the ${ and } as part of the variable name.
#(bind vars (("${user}" "oracle")
("${dbname}" "oracle")
("${password}" "letme1n")))
#(next "variable_list")
#(collect)
#entries
#(end)
#(deffilter subst . #(mapcar (op list #1 (second [find vars #1 equal first]))
entries))
#(next "input.sh")
#(collect)
#line
# (output :filter subst)
#line
# (end)
#(end)
Run:
$ txr subst.txr
username=oracle
password=${password}
dbname=oracle
input.sh: (as given)
username=${user}
password=${password}
dbname=${dbname}
variable_list: (as given)
${user}
${dbname}
Related
I have read the config file which has the below variable:
export BASE_DIR="\usr\usr1"
In the same script I read a file line by line and I wanted to substitute the ${BASE_DIR} with \usr\usr1.
In the script:
while read line; do
echo $line
done <file.txt
${BASE_DIR}\path1 should be printed as \usr\usr1\path1
Tried eval echo and $(( )).
Can use sed, This command will search and replace a value. The dollar sign is the separator.
sed -ie 's$\${BASE_DIR}$\\usr\\usr1$1' hello.txt
You need to set the variable when you read the line that contains the assignment. Then you can replace it later.
#!/bin/bash
while read line; do
if [[ $line =~ ^BASE_DIR= ]]
then basedir=${line#BASE_DIR=}
fi
line=${line/'${BASE_DIR}'/$basedir}
printf "%s\n" "$line"
done < file.txt > newfile.txt
I want to input multiple strings.
For example:
abc
xyz
pqr
and I want output like this (including quotes) in a file:
"abc","xyz","pqr"
I tried the following code, but it doesn't give the expected output.
NextEmail=","
until [ "a$NextEmail" = "a" ];do
echo "Enter next E-mail: "
read NextEmail
Emails="\"$Emails\",\"$NextEmail\""
done
echo -e $Emails
This seems to work:
#!/bin/bash
# via https://stackoverflow.com/questions/1527049/join-elements-of-an-array
function join_by { local IFS="$1"; shift; echo "$*"; }
emails=()
while read line
do
if [[ -z $line ]]; then break; fi
emails+=("$line")
done
join_by ',' "${emails[#]}"
$ bash vvuv.sh
my-email
another-email
third-email
my-email,another-email,third-email
$
With sed and paste:
sed 's/.*/"&"/' infile | paste -sd,
The sed command puts "" around each line; paste does serial pasting (-s) and uses , as the delimiter (-d,).
If input is from standard input (and not a file), you can just remove the input filename (infile) from the command; to store in a file, add a redirection at the end (> outfile).
If you can withstand a trailing comma, then printf can convert an array, with no loop required...
$ readarray -t a < <(printf 'abc\nxyx\npqr\n' )
$ declare -p a
declare -a a=([0]="abc" [1]="xyx" [2]="pqr")
$ printf '"%s",' "${a[#]}"; echo
"abc","xyx","pqr",
(To be fair, there's a loop running inside bash, to step through the array, but it's written in C, not bash. :) )
If you wanted, you could replace the final line with:
$ printf -v s '"%s",' "${a[#]}"
$ s="${s%,}"
$ echo "$s"
"abc","xyx","pqr"
This uses printf -v to store the imploded text into a variable, $s, which you can then strip the trailing comma off using Parameter Expansion.
I have been working in bash, and need to create a string argument. bash is a newish for me, to the point that I dont know how to build a string in bash from a list.
// foo.txt is a list of abs file names.
/foo/bar/a.txt
/foo/bar/b.txt
/delta/test/b.txt
should turn into: a.txt,b.txt,b.txt
OR: /foo/bar/a.txt,/foo/bar/b.txt,/delta/test/b.txt
code
s = ""
for file in $(cat foo.txt);
do
#what goes here? s += $file ?
done
myShellScript --script $s
I figure there was an easy way to do this.
with for loop:
for file in $(cat foo.txt);do echo -n "$file",;done|sed 's/,$/\n/g'
with tr:
cat foo.txt|tr '\n' ','|sed 's/,$/\n/g'
only sed:
sed ':a;N;$!ba;s/\n/,/g' foo.txt
This seems to work:
#!/bin/bash
input="foo.txt"
while IFS= read -r var
do
basename $var >> tmp
done < "$input"
paste -d, -s tmp > result.txt
output: a.txt,b.txt,b.txt
basename gets you the file names you need and paste will put them in the order you seem to need.
The input field separator can be used with set to create split/join functionality:
# split the lines of foo.txt into positional parameters
IFS=$'\n'
set $(< foo.txt)
# join with commas
IFS=,
echo "$*"
For just the file names, add some sed:
IFS=$'\n'; set $(sed 's|.*/||' foo.txt); IFS=,; echo "$*"
I have some variables in a bash script that may contain a file name or be unset. Their content should be passed as an additional argument to a program. But this leaves an empty argument when the variable is unset.
$ afile=/dev/null
$ anotherfile=/dev/null
$ unset empty
$ cat "$afile" "$empty" "$anotherfile"
cat: : No such file or directory
Without quotes, it works just fine as the additional argument is simply omitted. But as the variables may contain spaces, they have to be quoted here.
I understand that I could simply wrap the whole line in a test on emptiness.
if [ -z "$empty" ]; then
cat "$afile" "$anotherfile"
else
cat "$afile" "$empty" "$anotherfile"
fi
But one test for each variable would lead to a huge and convoluted decision tree.
Is there a more compact solution to this? Can bash made to omit a quoted empty variable?
You can use an alternate value parameter expansion (${var+altvalue}) to include the quoted variable IF it's set:
cat ${afile+"$afile"} ${empty+"$empty"} ${anotherfile+"$anotherfile"}
Since the double-quotes are in the alternate value string (not around the entire parameter expression), they only take effect if the variable is set. Note that you can use either + (which uses the alternate value if the variable is set) or :+ (which uses the alternate value if the variable is set AND not empty).
A pure bash solution is possible using arrays. While "$empty" will evaluate to an empty argument, "${empty[#]}" will expand to all the array fields, quoted, which are, in this case, none.
$ afile=(/dev/null)
$ unset empty
$ alsoempty=()
$ cat "${afile[#]}" "${empty[#]}" "${alsoempty[#]}"
In situations where arrays are not an option, refer to pasaba por aqui's more versatile answer.
Try with:
printf "%s\n%s\n%s\n" "$afile" "$empty" "$anotherfile" | egrep -v '^$' | tr '\n' '\0' | xargs -0 cat
In the case of a command like cat where you could replace an empty argument with an empty file, you can use the standard shell default replacement syntax:
cat "${file1:-/dev/null}" "${file2:-/dev/null}" "${file3:-/dev/null}"
Alternatively, you could create a concatenated output stream from the arguments which exist, either by piping (as shown below) or through process substitution:
{ [[ -n "$file1" ]] && cat "$file1";
[[ -n "$file2" ]] && cat "$file2";
[[ -n "$file3" ]] && cat "$file3"; } | awk ...
This could be simplified with a utility function:
cat_if_named() { [[ -n "$1" ]] && cat "$1"; }
In the particular case of cat to build up a new file, you could just do a series of appends:
# Start by emptying or creating the output file.
. > output_file
cat_if_named "$file1" >> output_file
cat_if_named "$file2" >> output_file
cat_if_named "$file3" >> output_file
If you need to retain the individual arguments -- for example, if you want to pass the list to grep, which will print the filename along with the matches -- you could build up an array of arguments, choosing only the arguments which exist:
args=()
[[ -n "$file1" ]] && args+=("$file1")
[[ -n "$file2" ]] && args+=("$file2")
[[ -n "$file3" ]] && args+=("$file3")
With bash 4.3 or better, you can use a nameref to make a utility function to do the above, which is almost certainly the most compact and general solution to the problem:
non_empty() {
declare -n _args="$1"
_args=()
shift
for arg; do [[ -n "$arg" ]] && _args+=("$arg"); done
}
eg:
non_empty my_args "$file1" "$file2" "$file3"
grep "$pattern" "${my_args[#]}"
I have a large config file that I use to define variables for a script to pull from it, each defined on a single line. It looks something like this:
var val
foo bar
foo1 bar1
foo2 bar2
I have gathered a list of out of date variables that I want to remove from the list. I could go through it manually, but I would like to do it with a script, which would be at least more stimulating. The file that contains the vlaues may contain multiple instances. The idea is to find the value, and if it's found, remove the entire line.
Does anyone know if this is possible? I know sed does this but I do not know how to make it use a file input.
#!/bin/bash
shopt -s extglob
REMOVE=(foo1 foo2)
IFS='|' eval 'PATTERN="#(${REMOVE[*]})"'
while read -r LINE; do
read A B <<< "$LINE"
[[ $A != $PATTERN ]] && echo "$LINE"
done < input_file.txt > output_file.txt
Or (Use with a copy first)
#!/bin/bash
shopt -s extglob
FILE=$1 REMOVE=("${#:2}")
IFS='|' eval 'PATTERN="#(${REMOVE[*]})"'
SAVE=()
while read -r LINE; do
read A B <<< "$LINE"
[[ $A != $PATTERN ]] && SAVE+=("$LINE")
done < "$FILE"
printf '%s\n' "${SAVE[#]}" > "$FILE"
Running with
bash script.sh your_config_file pattern1 pattern2 ...
Or
#!/bin/bash
shopt -s extglob
FILE=$1 PATTERNS_FILE=$2
readarray -t REMOVE < "$PATTERNS_FILE"
IFS='|' eval 'PATTERN="#(${REMOVE[*]})"'
SAVE=()
while read -r LINE; do
read A B <<< "$LINE"
[[ $A != $PATTERN ]] && SAVE+=("$LINE")
done < "$FILE"
printf '%s\n' "${SAVE[#]}" > "$FILE"
Running with
bash script.sh your_config_file patterns_file
Here's one with sed. Add words to the array. Then use
./script target_filename
(assuming you put the following in a file called script). (Not very efficient). I think it might be more efficient if we concat the words and put it in the regex like bbonev did
#!/bin/bash
declare -a array=("foo1" "foo2")
for i in "${array[#]}";
do
sed -i "/^${i}\s.*/d" $1
done
It's actually even simpler using file input
If you have a word file
word1
word2
word3
.....
then the following will do the job
#!/bin/bash
while read i;
do
sed -i "/^${i}\s.*/d" $2
done <$1
usage:
./script wordlist target_file