Getting Bash to parse variables from file input [duplicate] - bash

This question already has answers here:
Forcing bash to expand variables in a string loaded from a file
(13 answers)
Closed 7 years ago.
Let's say I have a file called path.txt containing the text $HOME/filedump/ on a single line. How can I then read the contents of path.txt into a variable, while having Bash parse said content?
Here's an example of what I'm trying to do:
#!/bin/bash
targetfile="path.txt"
target=$( [[ -f $targetfile ]] && echo $( < $targetfile ) || echo "Not set" )
echo $target
Desired output: /home/joe/filedump/
Actual output: $HOME/filedump/
I've tried using cat in place of <, wrapping it in quotes, and more. Nothing seems to get me anywhere.
I'm sure I'm missing something obvious, and there's probably a simple builtin command. All I can find on Google is pages about reading variables from ini/config files or splitting one string into multiple variables.

If you want to evaluate the contents of path.txt and assign that to target, then use:
target=$(eval echo $(<path.txt))
for example:
$ target=$(eval echo $(<path.txt)); echo "$target"
/home/david/filedump/

This might not necessarily suit your needs (depending on the context of the code you provided), but the following worked for me:
targetfile="path.txt"
target=$(cat $targetfile)
echo $target

Here's a safer alternative than eval. In general, you should not be using configuration files that require bash to evaluate their contents; that just opens a security risk in your script. Instead, detect if there is something that requires evaluation, and handle it explicitly. For example,
IFS= read -r path < path.txt
if [[ $path =~ '$HOME' ]]; then
target=$HOME/${path#\$HOME}
# more generally, target=${path/\$HOME/$HOME}, but
# when does $HOME ever appear in the *middle* of a path?
else
target=$path
fi
This requires you to know ahead of time what variables might appear in path.txt, but that's a good thing. You should not be evaluating unknown code.
Note that you can use any placeholder instead of a variable in this case; %h/filedump can be detected and processed just as easily as $HOME/filedump, without the presumption that the contents can or should be evaluated as shell code.

Related

Unexpected end of file in while loop in bash

I am trying to write a bash script that will do the following:
Take a directory or file as input (will always begin with /mnt/user/)
Search other mount points for same file or directory (will always begin with /mnt/diskx)
Return value
So, for example, the input will be "/mnt/user/my_files/file.txt". It will search if ""/mnt/disk1/my_files/file.txt" exists and will incrementally look for each disk (disk2, disk3, etc) until it finds it or disk20.
This is what I have so far:
#/user/bin/bash
var=$1
i=0
while [ -e $check_var = echo $var | sed 's:/mnt/user:/mnt/disk$i+1:']
do
final=$check_var
done
It's incomplete yes, but I am not that proficient in bash so I'm doing a little at a time. I'm sure my command won't work properly yet either but right now I am getting an "unexpected end of file" and I can't figure out why.
There are many issues here:
If this is the actual code you're getting "unexpected end of file" on, you should save the file in Unix format, not DOS format.
The shebang should be #!/usr/bin/bash or #!/bin/bash depending on your system
You have to assign check_var before running [ .. ] on it.
You have to use $(..) to expand a command
Variables like $i are not expanded in single quotes
sed can't add numbers
i is never incremented
the loop logic is inverted, it should loop until it matches and not while it matches.
You'd want to assign final after -- not in -- the loop.
Consider doing it in even smaller pieces, it's easier to debug e.g. the single statement sed 's:/mnt/user:/mnt/disk$i+1:' than your entire while loop.
Here's a more canonical way of doing it:
#!/bin/bash
var="${1#/mnt/user/}"
for file in /mnt/disk{1..20}/"$var"
do
[[ -e "$file" ]] && final="$file" && break
done
if [[ $final ]]
then
echo "It exists at $final"
else
echo "It doesn't exist anywhere"
fi

Batch editing files 'stuck at weird place'

I'm trying to learn how to batch edit files and extract information from them. I've begun with trying to create some trial files and editing their names. I tried to search but couldn't find the problem I'm in anywhere.
If it's already answered, I'd be happy to be directed to that link.
So, I wrote the following code:
#!/bin/bash
mkdir -p ./trialscript
echo $1
i=1
while [ $i -le $1 ]
do
touch ./trialscript/testfile$i.dat
i=$(($i+1))
done
for f in ./trialscript/*.dat
do
echo $f
mv "$f" "$fhello.dat"
done
This doesn't seem to work, and I think it's because the echo output is like:
4
./trialscript/testfile1.dat
./trialscript/testfile2.dat
./trialscript/testfile3.dat
./trialscript/testfile4.dat
I just need the filename in the 'f' and not the complete path and then just rename it.
Can someone suggest what is wrong in my code, and what's correct way to do what I'm doing.
If you want to move the file, you have to use the path, too, otherwise mv wouldn't be able to find it.
The target specification for the mv command is more problematic, though. You're using
"$fhello.dat"
which, in fact, means "content of the $fhello variable plus the string .dat". How should the poor shell know where the seam is? Use
"${f}hello.dat"
to disambiguate.
Also, to extract parts of strings, see Parameter expansion in man bash. You can use ${f%/*} to only get the path, or ${f##*/} to only get the filename.

unexplained string from bash loop

I have some file in the name of OBS_SURFACE1**, OBS_SURFACE101, OBS_SURFACE103. Yes, there indeed is a file named OBS_SURFACE1**, which I guess where the problem arise. I wrote a bash script which has:
for fil in ` ls OBS_DOMAIN1?? `
do
echo "appending" $fil
done
The first value of fil will be OBS_SURFACE1** OBS_SURFACE101 OBS_SURFACE103, the second OBS_SURFACE101. While I expect the first to be OBS_SURFACE1**. If there is no OBS_SURFACE1** file, there would be no problem. Why is that then?
Don't parse ls! It will only ever lead to problems. Use a glob instead:
for fil in OBS_DOMAIN1??
do
echo "appending $fil"
done
The problem that you are experiencing stems from the fact that the output of ls contains *, which are being expanded by bash. Note that I have also quoted the whole string to be echoed, which protects against word splitting inside the loop. See the links provided in the comments above for more details on that.
As pointed out in the comments (thanks Charles), you may also want to enable nullglob before your loop like this: shopt -s nullglob. This will mean that if there are no files that match the pattern, the loop will not run at all, rather than running once with $fil taking the literal value OBS_DOMAIN1??. Another option would be to check whether the file exists in within the loop, for example using:
if [[ -e "$fil" ]]; then
echo "appending $fil"
fi
or the more compact [[ -e "$fil" ]] && echo "appending $fil".
yet another way of doing this :
echo appending OBS_DOMAIN1??
this will list all files , no loop needed.

Bash script - loop through loaded vars from a config file excluding wildcards

I have a bash script that loads a config via . $1. Later in the script I would like to loop through these but exclude any starting with an underscore.
config file looks like
#---Comment---
myVar1="aa"
myVar2="bb"
_myVar3="cc"
... etc
Is this possible? what would the for loop look like. Any pointers to other threads would be appreciated as I could not find a close match.
thanks Art
You can read the variables in the file with something like
while read v; do
case $v in
'#'* | _*) continue;;
*) n=${v%%=*}
echo "Value of $n is ${!n}" ;;
esac
done <"$1"
The ${!var} indirect reference is a Bash v2 extension.
Once you have sourced a file, there's no way to later determine if a variable was defined in the sourced file or "locally". You'll have to process the config file manually, as tripleee suggests.
If the variables in the config file share a unique prefix, however, you can use a form of parameter expansion to iterate over the matching names. Suppose, for example, the config file only defines parameters starting with "myfoo" (or, of course, "_myfoo").
# Simple example: show values of all shell variables
# whose name starts with "myfoo". Anything starting with _
# is implicitly skipped
for var in "${!myfoo#}"; do
echo "$var=${!var}"
done
I would assume that you're parsing declaration of variables in the file? Many conservative old-school scripters would disagree on this but using eval is easiest and fastest for that approach. Even so if you want a safer one, you'd have to separate the two tokens and use printf:
while IFS== read A B; do
[[ $A != #* && $A == +([[:alpha:]_])*([[:alnum:]_]) ]] && printf -v "$A" "%s" "$B"
done < "$1"
Of course depending on the complexity of your config files, additional checks may be added, but I doubt more obvious unhelpful conservative approaches would be needed. Anyhow the concept of the above script would be helpful enough for a start.
Notes:
$A != #* checks if line doesn't start with #.
+([[:alpha:]_])*([[:alnum:]_]) checks if it's a valid parameter.

String contains in Bash that is a directory path

I am writing an SVN script that will export only changed files. In doing so I only want to export the files if they don't contain a specific file.
So, to start out I am modifying the script found here.
I found a way to check if a string contains using the functionality found here.
Now, when I try to run the following:
filename=`echo "$line" |sed "s|$repository||g"`
if [ ! -d $target_directory$filename ] && [[!"$filename" =~ *myfile* ]] ; then
fi
However I keep getting errors stating:
/home/home/myfile: "no such file or directory"
It appears that BASH is treating $filename as a literal. How do I get it so that it reads it as a string and not a path?
Thanks for your help!
You have some syntax issues (a shell script linter can weed those out):
You need a space after "[[", otherwise it'll be interpretted as a command (giving an error similar to what you posted).
You need a space after the "!", otherwise it'll be considered part of the operand.
You also need something in the then clause, but since you managed to run it, I'll assume you just left it out.
You combined two difference answers from the substring thing you posted, [[ $foo == *bar* ]] and [[ $foo =~ .*bar.* ]]. The first uses a glob, the second uses a regex. Just use [[ ! $filename == *myfile* ]]

Resources