I have the following simple bash script:
#!/bin/bash -fx
ls *sh
The problem is that bash add a quote to the pattern and I get wrong output.
+ ls '*sh'
ls: cannot access *sh: No such file or directory
How can I change this behavior?
The output of ls *sh from the terminal is:
$ls *sh
a.bash a.sh b.sh
I tried to add quotes according to this post - "Bash variable containing file wildcard"
without success
That because you're disabling pathname expansion with the -f option.
#!/bin/bash -fx
From man:
-f
Disable filename expansion (globbing).
Related
I have a script, /home/user/me/my_script.sh that is supposed to loop over multiple directories and process files. My current working directory is /home/user/me. A call to ls -R yields:
./projects:
dir1 dir2 dir3
./projects/dir1:
image1.ntf points2.csv image1.img image1.hdr
./projects/dir2:
image2.ntf points2.csv image2.img image2.hdr
./projects/dir3:
image3.ntf points3.csv image3.img image3.hdr
I have this script:
#! /bin/bash -f
for $dir in $1*
do
echo $dir
set cmd = `/home/tools/tool.sh -i $dir/*.ntf -flag1 -flag2 -flag3 opt3`
$cmd
done
This is how it is run (from cwd /home/user/me) and the result:
bash-4.1$ ./myscript.sh projects/
projects/*
bash-4.1$
This is not the expected output.The expected output is:
bash-4.1$ ./myscript.sh projects/
projects/dir1
[output from tool.sh]
projects/dir2
[output from tool.sh]
projects/dir3
[output from tool.sh]
bash-4.1$
What should happen is the script should go into the first directory, find the *.ntf file and pass it to tool.sh. At that point I would start seeing output from that tool. I have run the tool on a single file:
bash-4.1$ /home/tools/tool.sh -i /home/user/me/projects/dir1/image1.ntf -flag1 -flag2 -flag3 opt3
[expected output from tool. lengthy.]
bash-4.1$
I have tried syntax found here: How to loop over directories in Linux? and here: Looping over directories in Bash
for $dir in /$1*/
do ...
Result:
bash-4.1$ ./myscript.sh projects/
/projects/*/
And:
for $dir in $1/*
do ...
Result:
bash-4.1$ ./myscript.sh projects
projects/*
I'm not sure how many other iterations of wildcard and slash I can come up with. What is the correct syntax?
First, you should remove flag -f in your shebang, because it utterly means:
$ man bash
[…]
-f Disable pathname expansion.
Second, there are some typical bug patterns: spaces missing around variables (write "$dir" to cope with directory names containing spaces), there is a spurious $ in your for line (write for dir in "$1"*) instead, the set line is incorrect (set is a shell builtin only used to change the configuration of the shell, e.g., set -x), according to your answer to #ghoti's question it seems that the $cmd line is unnecessary. Also, the backquotes syntax is deprecated and could have been replaced with cmd=$(/home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3).
This would lead to the following script:
#!/bin/bash
for dir in "$1"*
do
[[ -d "$dir" ]] || continue # only consider existing folders
printf "%s=%q\n" dir "$dir"
/home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3
done
As an aside, I would recommend to always run the ShellCheck static analyzer on your Bash scripts, in order to detect typical bugs and have feedback w.r.t. good practices. If you have a Linux distribution, it should be installable with your standard package manager.
I have an rsync command in my csh script like this:
#! /bin/csh -f
set source_dir = "blahDir/blahBlahDir"
set dest_dir = "foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
When I run this I get the following error:
rsync: No match.
If I remove the --exclude option it works. I wrote the equivalent script in bash and that works as expected
#/bin/bash -f
source_dir="blahDir/blahBlahDir"
dest_dir="foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
The problem is that this has to be done in csh only. Any ideas on how I can get his to work?
It's because csh is trying to expand --exclude=*.csv into a filename, and complaining because it cannot find a file matching that pattern.
You can get around this by enclosing the option in quotes:
rsynv -rv '--exclude=*.csv' ...
or escaping the asterisk:
rsynv -rv --exclude=\*.csv ...
This is a consequence of the way csh and bash differ in their default treatment of arguments with wildcards that don't match a file. csh will complain while bash will simply leave it alone.
You may think bash has chosen the better way but that's not necessarily so, as shown in the following transcript where you have a file matching the argument:
pax> touch -- '--file=xyzzy.csv' ; ls -- *.csv
--file=xyzzy.csv
pax> echo --file=*.csv
--file=xyzzy.csv
You can see there that the bash shell expands the argument rather than giving it to the program as is. Both sides have their pros and cons.
Running the command
cd \`echo -n "~"\`
I get the following error:
bash: cd: ~: No such file or directory
What's the problem if 'cd ~' works fine?
The issue is that bash does not do an additional expansion after command substitution. So while cd ~ is expanded the way you want, cd $(echo '~') does not.
There is a keyword called eval that was created for this sort of situation--it forces the command line to be expanded (evaluated) again. If you use eval on that line, it forces the ~ to be expanded into the user directory, even though the normal time for expansion has already passed. (Because the ~ does not exist until the echo command is run, and at that point, it's too late for expansion.)
eval cd `echo -n "~"`
If you do cd ~, the shell expands ~ to your home directory before executing the command. But if you use double quotes ("~"), then this is taken as a literal string and not expanded.
You can see the difference:
$ echo ~
/home/username
$ echo "~"
~
In order to have ~ expanded by the shell, you need to remove the double quotes.
The escaping behaviour of double quotes is described in the Bash manual: http://www.gnu.org/software/bash/manual/html_node/Double-Quotes.html
You will also get the same issue if you simply do cd "~":
$ cd "~"
bash: cd: ~: No such file or directory
cd doesn't understand that ~ is special. It tries, and fails, to find a directory literally called ~.
The reason that cd ~ works is that bash edits the command before running it. bash replaces cd ~ with cd $HOME, and then expands $HOME to get cd /home/YourUsername.
Therefore,
cd `echo -n "~"`
becomes
cd "~"
$0 expands to the name of the shell script.
$ cat ./sample-script
#!/bin/bash
echo $0
$ chmod 700 ./sample-script
$ ./sample-script
./sample-script
If the shell script is invoked via a symbolic link, $0 expands to its name:
$ ln -s ./sample-script symlinked-script
$ ./symlinked-script
./symlinked-script
How could I get the name of an alias? Here `$0' expands again to the filename:
$ alias aliased-script=./sample-script
$ aliased-script
./sample-script
Aliases are pretty dumb, according to the man page
...Aliases are expanded when a command is read, not when it is executed...
so since bash is basically just replacing a string with another string and then executing it, there's no way for the command to know what was expanded in the alias.
I imagine you already know this, but for the record the answer is: you need cooperation by the code implementing the alias.
alternate_name () {
MY_ALIAS_WAS=alternate_name real_name "$#"
}
or, if you really want to use the superseded alias syntax:
alias alternate_name="MY_ALIAS_WAS=alternate_name real_name"
...and then...
$ cat ~/bin/real_name
#!/bin/sh
echo $0, I was $MY_ALIAS_WAS, "$#"
bash does not make this available. This is why symlinks are used to invoke multiplex commands, and not aliases.
#!/bin/sh
files = 'ls /myDir/myDir2/myDir3/'
for file in $files do
echo $file
java myProg $file /another/directory/
done
What i'm trying to do is iterate through every file name under /myDir/myDir2/myDir3/, then use that file name as the first argument in calling a java program (second argument is "/another/directory")
When I run this script: . myScript.sh
I get this error:
-bash: files: command not found
What did I do wrong in my script? Thanks!
Per Neeaj's answer, strip off the whitespace from files =.
Better yet, use:
#!/bin/sh -f
dir=/myDir/MyDir2/MyDir3
for path in $dir/*; do
file=$(basename $path)
echo "$file"
java myProg "$file" arg2 arg3
done
Bash is perfectly capable of expanding the * wildcard itself, without spawning a copy of ls to do the job for it!
EDIT: changed to call basename rather than echo to meet OP's (previously unstated) requirement that the path echoed be relative and not absolute. If the cwd doesn't matter, then even better I'd go for:
#!/bin/sh -f
cd /myDir/MyDir2/MyDir3
for file in *; do
echo "$file"
java myProg "$file" arg2 arg3
done
and avoid the calls to basename altogether.
strip off the whitespace in and after files = as files=RHS of assignment
Remove the space surrounding the '=' : change
files = 'ls /myDir/myDir2/myDir3/'
into:
files='ls /myDir/myDir2/myDir3/'
and move the 'do' statement to its own line:
for file in $files
do
....
quote your variables and no need to use ls.
#!/bin/sh
for file in /myDir/myDir2/*
do
java myProg "$file" /another/directory/
done