At the command line, this cat works as expected:
cat /home/me/path\ with\ spaces/to/file
But if I put it in a script:
#!/bin/bash
FILE="$1"
cat $FILE # cat "$FILE" gives same result
and call the script with ./script.sh "/home/me/path\ with\ spaces/to/file", I get:
cat: /home/me/path\ with\ spaces/to/file: No such file or directory
Note the escape quotes, which should be in the right places.
What gives?
Simply use double quotes to prevent word splitting:
cat "$FILE"
As an aside, upper case variable names should be reserved for shell internal variables, so you should change FILE to file.
If you are quoting the argument to your script (which is a good idea), then the file name doesn't need backslashes:
./script.sh "/home/me/path with spaces/to/file"
Related
I would like to capture a directory that contains spaces in a bash variable and pass this to the ls command without surrounding in double quotes the variable deference. Following are two examples that illustrate the problem. Example 1 works but it involves typing double quotes. Example 2 does not work, but I wish it did because then I could avoid typing the double quotes.
Example 1, with quotes surrounding variable, as in the solution to How to add path with space in Bash variable, which does not solve the problem:
[user#machine]$ myfolder=/home/username/myfolder\ with\ spaces/
[user#machine]$ ls "$myfolder"
file1.txt file2.txt file3.txt
Example 2, with quotes part of variable, which also does not solve the problem. According to my understanding, in this example, the first quote character sent to the ls command before the error is thrown:
[user#machine]$ myfolder=\"/home/username/myfolder\ with\ spaces/\"
[user#machine]$ ls $myfolder
ls: cannot access '"/home/username/myfolder': No such file or directory
In example 2, the error message indicates that the first double quote was sent to the ls command, but I want these quotes to be interpreted by bash, not ls. Is there a way I can change the myfolder variable so that the second line behaves exactly as the following:
[user#machine]$ ls "/home/username/myfolder with spaces/"
The goal is to craft the myfolder variable in such a way that (1) it does not need to be surrounded by any characters and (2) the ls command will list the contents of the existing directory that it represents.
The motivation is to have an efficient shorthand to pass long directory paths containing spaces to executables on the command line with as few characters as possible - so without double quotes if that is possible.
Assuming some 'extra' characters prior to the ls command is acceptable:
$ mkdir /tmp/'myfolder with spaces'
$ touch /tmp/'myfolder with spaces'/myfile.txt
$ myfolder='/tmp/myfolder with spaces'
$ myfolder=${myfolder// /?} # replace spaces with literal '?'
$ typeset -p myfolder
declare -- myfolder="/tmp/myfolder?with?spaces"
$ set -xv
$ ls $myfolder
+ ls '/tmp/myfolder with spaces'
myfile.txt
Here's a fiddle
Granted, the ? is going to match on any single character but how likely is it that you'll have multiple directories/files with similar names where the only difference is a space vs a non-space?
I want to store the contents of a file in a bash shell variable. This works fine:
$ cat hello
Hello, world!
$ F=hello
$ Q=$(cat $F)
$ echo $Q
Hello, world!
However, if the file contains an asterisk, the asterisk is replaced by a list of all files in the current directory.
How can I quote the filename to protect the asterisk? Or otherwise load the file into the shell variable?
I am aware of this question, but it doesn't work for files that contain an asterisk.
Q contains the asterisk. It is the unquoted expansion of $Q that replaces the * with a list of files.
$ Q="*"
$ echo $Q
<list of files>
$ echo "$Q"
*
The right-hand side of an assignment is not subject to path name expansion, so Q=* would work as well, and the command substitution used to read from the file is also not affected. Q=$(cat hello) works fine: you just need to quote the expansion of Q.
I am using the bash shell and want to execute a command that takes filenames as arguments; say the cat command. I need to provide the arguments sorted by modification time (oldest first) and unfortunately the filenames can contain spaces and a few other difficult characters such as "-", "[", "]". The files to be provided as arguments are all the *.txt files in my directory. I cannot find the right syntax. Here are my efforts.
Of course, cat *.txt fails; it does not give the desired order of the arguments.
cat `ls -rt *.txt`
The `ls -rt *.txt` gives the desired order, but now the blanks in the filenames cause confusion; they are seen as filename separators by the cat command.
cat `ls -brt *.txt`
I tried -b to escape non-graphic characters, but the blanks are still seen as filename separators by cat.
cat `ls -Qrt *.txt`
I tried -Q to put entry names in double quotes.
cat `ls -rt --quoting-style=escape *.txt`
I tried this and other variants of the quoting style.
Nothing that I've tried works. Either the blanks are treated as filename separators by cat, or the entire list of filenames is treated as one (invalid) argument.
Please advise!
Using --quoting-style is a good start. The trick is in parsing the quoted file names. Backticks are simply not up to the job. We're going to have to be super explicit about parsing the escape sequences.
First, we need to pick a quoting style. Let's see how the various algorithms handle a crazy file name like "foo 'bar'\tbaz\nquux". That's a file name containing actual single and double quotes, plus a space, tab, and newline to boot. If you're wondering: yes, these are all legal, albeit unusual.
$ for style in literal shell shell-always shell-escape shell-escape-always c c-maybe escape locale clocale; do printf '%-20s <%s>\n' "$style" "$(ls --quoting-style="$style" '"foo '\''bar'\'''$'\t''baz '$'\n''quux"')"; done
literal <"foo 'bar' baz
quux">
shell <'"foo '\''bar'\'' baz
quux"'>
shell-always <'"foo '\''bar'\'' baz
quux"'>
shell-escape <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
shell-escape-always <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
c <"\"foo 'bar'\tbaz \nquux\"">
c-maybe <"\"foo 'bar'\tbaz \nquux\"">
escape <"foo\ 'bar'\tbaz\ \nquux">
locale <‘"foo 'bar'\tbaz \nquux"’>
clocale <‘"foo 'bar'\tbaz \nquux"’>
The ones that actually span two lines are no good, so literal, shell, and shell-always are out. Smart quotes aren't helpful, so locale and clocale are out. Here's what's left:
shell-escape <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
shell-escape-always <'"foo '\''bar'\'''$'\t''baz '$'\n''quux"'>
c <"\"foo 'bar'\tbaz \nquux\"">
c-maybe <"\"foo 'bar'\tbaz \nquux\"">
escape <"foo\ 'bar'\tbaz\ \nquux">
Which of these can we work with? Well, we're in a shell script. Let's use shell-escape.
There will be one file name per line. We can use a while read loop to read a line at a time. We'll also need IFS= and -r to disable any special character handling. A standard line processing loop looks like this:
while IFS= read -r line; do ... done < file
That "file" at the end is supposed to be a file name, but we don't want to read from a file, we want to read from the ls command. Let's use <(...) process substitution to swap in a command where a file name is expected.
while IFS= read -r line; do
# process each line
done < <(ls -rt --quoting-style=shell-escape *.txt)
Now we need to convert each line with all the quoted characters into a usable file name. We can use eval to have the shell interpret all the escape sequences. (I almost always warn against using eval but this is a rare situation where it's okay.)
while IFS= read -r line; do
eval "file=$line"
done < <(ls -rt --quoting-style=shell-escape *.txt)
If you wanted to work one file at a time we'd be done. But you want to pass all the file names at once to another command. To get to the finish line, the last step is to build an array with all the file names.
files=()
while IFS= read -r line; do
eval "files+=($line)"
done < <(ls -rt --quoting-style=shell-escape *.txt)
cat "${files[#]}"
There we go. It's not pretty. It's not elegant. But it's safe.
Does this do what you want?
for i in $(ls -rt *.txt); do echo "FILE: $i"; cat "$i"; done
I know about escaping, quoting and stuff, but still have a problem.
I you have a script containing "cd $1", and call it with an argument containing spaces, cd will always return an error message - it stops at the first space and can't find the directory. I tried protecting the arguments in every way :
ls -l
+-rwx... script
+drwx... dir with spaces/
cat script
+cd $1
script dir with spaces
+cd: dir: no such file or directory
script "dir with spaces"
+cd: dir: no such file or directory
script dir\ with\ spaces
+cd: dir\: no such file or directory
but none will work.
I feel like I'm missing the obvious, thanks for enlightening me.
You need to quote the expansion of "$1" to prevent it from being word split as well as quoting the string passed to the script to prevent it from being word-split.
So
$ cat script.sh
cd -- "$1"
$ ./script.sh "dir with spaces"
Edit: As gniourf_gniourf correctly pointed out using -- as the first argument to cd prevents problems should paths ever start with -.
Use double quotes on the variable
cd "$1"
This is pretty basic, I guess I'm missing something really obvious...
The following sequence should explain it:
$ cat read_file_names.sh
#!/bin/bash
for i in $# ; do
echo "$i"
done
$ touch "filename has many spaces"
$ ./read_file_names.sh filename\ has\ many\ spaces
filename
has
many
spaces
ideally, the command line will have quotes around the filename as in:
$ ./read_file_names.sh "filename\ has\ many\ spaces"
The problem is that when allowing bash to auto-complete the filename (by hitting tab), the file name is left unquoted. Instead, it has a backslash-space "\ " to signal a space. I understand I can add quotes manually, but that would be tedious and a poor user experience.
I'm looking for a solution which assigns the entire file name to the for-loop variable, so that the output looks something like this:
$ ./read_file_names.sh filename\ has\ many\ spaces
filename has many spaces
The backslashes are working. It's your debugging printer that's wrong:
for i in $# ; do
That needs to be:
for i in "$#"; do
Otherwise, the argument string is inserted unquoted into the for expression and then word-split.