I have a simple naive question, I've figured out how to make my script run but I'd like to know why it didn't work previously.
I was assigning a variable with a wildcard using syntax similar to:
var=$dir/$subj/name*text*text.nii.gz
I could call the proper filename with ls $file, but when I tried to substitute in $file as an input into a command line (using FSL for image processing), I got an error saying it couldn't find the file with wildcards in place.
However, when I assign the variable with parentheses:
var=($dir/$subj/name*text*text.nii.gz)
It runs just fine. I'm assuming there are other and probably better ways to do this, but I'm just wondering why the initial variable assignment didn't work, and what the optimal way to assign variables in this manner is.
Thanks!
Let's consider a directory with three files:
$ ls
file1 file2 file3
Now define a variable:
$ var=file*
We can see what is in var by using declare -p:
$ declare -p var
declare -- var="file*"
As you can see, var still has the * in it. This is because pathname expansion is not performed for variable assignments. Consequently, var will not always work as you may have wanted. For example:
$ ls "$var"
ls: cannot access file*: No such file or directory
Next, let's try creating an array:
$ var=(file*)
$ declare -p var
declare -a var='([0]="file1" [1]="file2" [2]="file3")'
As you can see, pathname expansion is performed on arrays. Consequently, the following does work:
$ ls "$var"
file1
But, note that, for an array, $var refers only to the first element. If you wanted to access all its entries, a more complex notation is needed:
$ ls "${var[#]}"
file1 file2 file3
Related
I want to read a list of file names stored in a file, and the top level directory is a macro, since this is for a script that may be run in several environments.
For example, there is a file file_list.txt holding the following fully qualified file paths:
$TOP_DIR/subdir_a/subdir_b/file_1
$TOP_DIR/subdir_x/subdir_y/subdir_z/file_2
In my script, I want to tar the files, but in order to do that, tar must know the actual path.
How can I get the string containing the file path to expand the macro to get the actual path?
In the code below the string value echoed is exactly as in the file above.
I tried using actual_file_path=`eval $file_path` and while eval does evaluate the macro, it returns a status, not the evaluated path.
for file_path in `cat $input_file_list`
do
echo "$file_path"
done
With the tag ksh I think you do not have the utility envsubst.
When the number of variables in $input_file_list is very limited, you can substitute vars with awk :
awk -v top_dir="${TOP_DIR}" '{ sub(/$TOP_DIR/, top_dir); print}' "${input_file_list}"
I was using eval incorrectly. The solution is to use an assignment on the right side of eval as follows:
for file_path in `cat $input_file_list`
do
eval myfile=$file_name
echo "myfile = $myfile"
done
$myfile now has the actual expansion of the macro.
I have this code snippet:
userjobs=$(grep -rw "$USER" /my/job/dir/|awk '{print $1}'|sort|uniq|rev|cut -c 2-|rev)
for job in "${userjobs[#]}"; do
cat "$job"
done
exit 0
When I run it as is, I get the following output:
cat: /my/job/dir/45
/my/job/dir/46: No such file or directory
However, if I unquote $job, I no longer receive this behavior, and it cats each of the files as expected.
I've done some reading up on globbingand splitting to see if this is occurring, but it seems like double-quoting should prevent that from happening. Can anyone explain why the behavior is different between "$job" and $job?
This happens because your variable looks like:
userjobs='/my/job/dir/45
/my/job/dir/46'
If you expand it as an array, with "${userjobs[#]}", that it acts as an array with exactly one element -- that string. Thus, behavior is identical to:
userjobs=( [0]='/my/job/dir/45
/my/job/dir/46' )
...still exactly one string with a literal newline in it.
Thus, cat "$job" looks for a file with a literal newline in its name.
To load your result into a real array you can iterate over with "${userjobs[#]}" expanding to a distinct element per line, use:
readarray -t userjobs < <(grep ...)
userjobs needs to be an array. Put parentheses around the value when assigning it:
userjobs=($(grep -rw "$USER" /my/job/dir/|awk '{print $1}'|sort|uniq|rev|cut -c 2-|rev))
I have a bash script which gets the arguments from external 'source.txt' file.
Sources file includes 10 rows of arguments for instance (mixed files and directories).
One function should use the source file entirely. I achieved this with $(<source.txt) and it works OK.
Whereas the second function should use the same 'source.txt' file partially, filtering the arguments with regex or something else.
Source file:
/etc/sysconfig/network-scripts/
/etc/ntp.conf
/etc/localtime
/etc/sysconfig/iptables-config
/etc/resolv.conf
/sbin/ifup-local
/sbin/ifdown-local
/usr/local/sbin
/var/spool/cron/
/boot
Second function must take only '^/etc/[a-z][A-Z]*' sources with all the content recursively.
How do i do it?
You can simply grep it, like this:
$(grep '^/etc/[a-z][A-Z]*'<source.txt)
Take a note through, that if your arguments happen to contain some spaces or quotes, the subshell approach (command substitution) might fail for you.
To workaround this you can use readarray (mapfile) instead:
readarray -t args < <(grep '^/etc/[a-z][A-Z]*' source.txt)
your_function "${args[#]}"
Sorry if its sounds dumb but I'm currently studying for an exam to apply for a job (administrative one) and I learnt about java, php, python but not much about unix scripts. I can see it's very similar to php in structure and I have many questions about it .
One of them is this loop, what does it do and why does it output b*?
for var in b*
echo $var
done
I tried this code and the terminal output was b*, Why does this code answer me with text (in this case, b*) or does something else happen?
A for loop in shell scripting takes each whitespace separated string from the input and assigns it to the variable given, and runs the code once for each value. In your case b* is the input and $var is the assigned variable.
The shell will also perform expansion on the input. Basically, the * acts like a wildcard for any part of a file name. So if you have files beginning with b in the current directory, those will get looped over:
$ ls
bean.java bean.class readme.txt
$ for var in b*; do echo $var; done
bean.java
bean.class
If however you don't have any files beginning with b (so that your pattern b* doesn't match anything) the pattern will remain unexpanded:
$ rm bean.*
$ ls
readme.txt
$ for var in b*; do echo $var; done
b*
Say if I have:
MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
then I want to construct another path AllHeaders/NetworkUtil/network.h
I actually need to get different components out fro the original path, is there a way to do it?
I found in:
Bash: remove first directory component from variable (path of file)
I can something like
${MYPATH#../Library}
to strip out the specified part, but that assumes I know the structure already, what if in my case I need the 3rd and last components in the original path?
Thanks
You can use bash arrays to access individual elements:
$ MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
$ OLD="$IFS"
$ IFS='/' a=($MYPATH)
$ IFS="$OLD"
$ NEWPATH="AllHeaders/${a[2]}/${a[-1]}"
$ echo $NEWPATH
AllHeaders/NetworkUtil/network.h
ADDENDUM: For completeness, another way to make MYPATH into an array is to use bash's pattern substitution: a=(${MYPATH//\// }):
$ MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
$ a=(${MYPATH//\// })
$ NEWPATH="AllHeaders/${a[2]}/${a[-1]}"
$ echo $NEWPATH
AllHeaders/NetworkUtil/network.h
This eliminates the need for messing with IFS but would break badly if MYPATH had spaces, tabs, or CR in it to begin with.