Bash: variable not getting expanded properly [duplicate] - bash

This question already has answers here:
Filename not printing correctly with underscore "_" in Bash [duplicate]
(2 answers)
Closed 6 years ago.
I am trying to use variable in renaming a file. However, I when insert the variable to the beginning of the filename, things does not work as expected.
Here's the case, I have a file name test:
$ ls
test
and a variable
i=1
When adding the variable to the end or middle of filename, it works:
$ mv test test_$i
$ ls
test_1
When adding the variable to the beginning of filename, it doesn't work:
$mv test_1 test
$mv test $i_test
mv: missing destination file operand after 'test'
Try 'mv --help' for more information.
And even worse, when there is extension in my filename, the file will be removed.
$ touch test.try
$ ls
test.try
$ mv test.try $i_test.try
$ ls
(nothing!)
Can anyone explain this to me? Is it a bug or something I don't know?

You need to put {} around the variable name to disambiguate it from the rest of the literal (remember, _ is a valid character in an identifier):
mv test.try ${i}_test.try
or, use double quotes, which gives you protection against word splitting and globbing:
mv test.try "${i}"_test.try
In your code:
$i_test => shell treats "i_test" as the variable name
$i_test.try => shell treats "i_test" as the variable name ('.' is not a valid character in an identifier)
mv test.try $i_test.try => test.try got moved to .try as "$i_test" expanded to nothing. That is why ls didn't find that file. Use 'ls -a' to see it.
See this related post: When do we need curly braces in variables using Bash?

Related

Have bash interpret double quotes stored in variable containing file path

I would like to capture a directory that contains spaces in a bash variable and pass this to the ls command without surrounding in double quotes the variable deference. Following are two examples that illustrate the problem. Example 1 works but it involves typing double quotes. Example 2 does not work, but I wish it did because then I could avoid typing the double quotes.
Example 1, with quotes surrounding variable, as in the solution to How to add path with space in Bash variable, which does not solve the problem:
[user#machine]$ myfolder=/home/username/myfolder\ with\ spaces/
[user#machine]$ ls "$myfolder"
file1.txt file2.txt file3.txt
Example 2, with quotes part of variable, which also does not solve the problem. According to my understanding, in this example, the first quote character sent to the ls command before the error is thrown:
[user#machine]$ myfolder=\"/home/username/myfolder\ with\ spaces/\"
[user#machine]$ ls $myfolder
ls: cannot access '"/home/username/myfolder': No such file or directory
In example 2, the error message indicates that the first double quote was sent to the ls command, but I want these quotes to be interpreted by bash, not ls. Is there a way I can change the myfolder variable so that the second line behaves exactly as the following:
[user#machine]$ ls "/home/username/myfolder with spaces/"
The goal is to craft the myfolder variable in such a way that (1) it does not need to be surrounded by any characters and (2) the ls command will list the contents of the existing directory that it represents.
The motivation is to have an efficient shorthand to pass long directory paths containing spaces to executables on the command line with as few characters as possible - so without double quotes if that is possible.
Assuming some 'extra' characters prior to the ls command is acceptable:
$ mkdir /tmp/'myfolder with spaces'
$ touch /tmp/'myfolder with spaces'/myfile.txt
$ myfolder='/tmp/myfolder with spaces'
$ myfolder=${myfolder// /?} # replace spaces with literal '?'
$ typeset -p myfolder
declare -- myfolder="/tmp/myfolder?with?spaces"
$ set -xv
$ ls $myfolder
+ ls '/tmp/myfolder with spaces'
myfile.txt
Here's a fiddle
Granted, the ? is going to match on any single character but how likely is it that you'll have multiple directories/files with similar names where the only difference is a space vs a non-space?

Bash too many arguments error with `cd $VAR`, where $VAR is path name with escaped spaces [duplicate]

This question already has answers here:
How to cd into a directory with space in the name?
(18 answers)
Closed 2 years ago.
I could not cd to a directory, whose path name contains escaped spaces, that is represented by a variable.
The following command does not work.
~$ VAR="/mnt/e/documents/my\ files"
~$ cd $VAR
-bash: cd: too many arguments
However, when I use the path name as it is instead of assigning it to a variable, it works.
~$ cd /mnt/e/documents/my\ files
cd /mnt/e/documents/my files$
How to use cd with a variable as its argument, where the variable is a string with spaces that are escaped.
You have two problems:
Remove the backslash in the middle of my\ files. You only need to escape that space when you're not quoting it.
Add quotes around $VAR: cd "$VAR", otherwise it looks as though you are providing two parameters: "/mnt/e/documents/my" and "files"

Expanding an Index in a Bash Loop Script in a Combination with a Wildcard and an Underscore [duplicate]

This question already has answers here:
bash variable interpolation separate variables by a hyphen or underscore
(3 answers)
Closed 5 years ago.
I have 80 pairs of files of the following type:
170309-N701-S517_S1_L001_R1_001.fastq.gz
170309-N701-S517_S1_L001_R2_001.fastq.gz
170309-N701-S502_S2_L001_R1_001.fastq.gz
170309-N701-S502_S2_L001_R2_001.fastq.gz
170309-N701-S503_S3_L001_R1_001.fastq.gz
170309-N701-S503_S3_L001_R2_001.fastq.gz
..
170309-N710-S507_S79_L001_R1_001.fastq.gz
170309-N710-S507_S79_L001_R2_001.fastq.gz
170309-N710-S508_S80_L001_R1_001.fastq.gz
170309-N710-S508_S80_L001_R2_001.fastq.gz
Essentially, a pair consists of the following files:
170309-N701-S517_S<ID>_L001_R1_001.fastq.gz
170309-N701-S517_S<ID>_L001_R2_001.fastq.gz
where ID varies from 1 to 80.
I would like to create 80 subfolders named S1..S80 and put each pair in the corresponding folder, e.g.
170309-N701-S517_S1_L001_R1_001.fastq.gz
170309-N701-S517_S1_L001_R2_001.fastq.gz
go to subfolder S1
170309-N701-S502_S2_L001_R1_001.fastq.gz
170309-N701-S502_S2_L001_R2_001.fastq.gz
go to subfolder S2
and so on.
I wrote the following script:
#!/bin/bash
for i in {1..80}
do
mkdir S$i
mv "*_S"$i"_*" S$i
done
but it didn't work as expected. It created the subfolders S1..S80. However, it didn't move any of the files. It produced instead
mv: cannot stat `*_S1_*': No such file or directory
mv: cannot stat `*_S2_*': No such file or directory
and so on.
What am I doing wrong? Can you correct the script?
What you're trying to do with the mv "*_S"$i"_*" S$i line is called variable expansion, and if you as a part of that expansion wants to concatenate the variable with other characters, you need to let BASH know what is a variable and what is a plain character. You do this by enclosing the variable name in curly braces. E.g:
var=FOO
echo "BAR ${var} BAZ"
# BAR FOO BAZ
In the case of your loop:
touch \
170309-N701-S517_S1_L001_R1_001.fastq.gz\
170309-N701-S517_S1_L001_R2_001.fastq.gz\
170309-N701-S502_S2_L001_R1_001.fastq.gz\
170309-N701-S502_S2_L001_R2_001.fastq.gz\
170309-N701-S503_S3_L001_R1_001.fastq.gz\
170309-N701-S503_S3_L001_R2_001.fastq.gz
for i in {1..80}
do
if test -n "$(find . -maxdepth 1 -name "*_S${i}_*" -print -quit)"
then
mkdir "S${i}"
mv *"_S${i}_"* "S$i"
fi
done
That if-then-fi bit is there just to avoid making directories for non-existing files. Entirely optional.
Also note that globbing character * must be used unquoted in order to work with mv, because inside "..." or '...', * is treated as a literal.
An exception to this is however seen in the find command, where the content between the double quotes will be expanded as it is passed on to find. To avoid expansion in this case the argument can be enclosed in single quotes ('...')

Escape spaces in bash script [duplicate]

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 6 years ago.
I am trying to do something in a bash script whenever a file in a directory I am iterating over contains a string using grep. The problem comes in where a subset of the files in the directory contain spaces in the name. Therefore, I have tried to replace the spaces with escaped spaces in place using sed:
if grep -c "main" ${source} | sed 's/ /\\ /g'; then
# do something
fi
However, I still get the error:
grep: /Users/me/Desktop/theDir/nameWith: No such file or directory
grep: spaces.txt: No such file or directory
What am I doing wrong?
You should quote the name of the file being grep'ed:
if grep -c main "$source" ; then
# do something
fi
...assuming $source is the name of a file. If $source is the name of a directory, I'll need more information about what you're trying to do.

Passing bash variables not working [duplicate]

This question already has answers here:
"~/Desktop/test.txt: No such file or directory"
(2 answers)
Closed 7 years ago.
I have the following set in bash:
export chain_dir='~/.eris/chains/simplechain'
This works
echo $chain_dir
~/.eris/chains/simplechain
Then try I try to use it with ls and I get:
ls $chain_dir
ls: cannot access ~/.eris/chains/simplechain: No such file or directory
But cut and paste the same directory string as in $chain_dir and it works:
~/.eris/chains/simplechain
accounts.csv addresses.csv simplechain_full_000 simplechain_root_000
I believe that I am missing something:
~ isnt expanded into your home dir before giving it to ls.
Use $HOME or
leteval do the expansion for you
$var1='~/.config/'
$ls $(eval echo "$var1")
The ~ isn't expanded to your home directory path before being passed to ls.
I would suggest writing it out in full, /home/user/, or using $HOME:
export chain_dir="$HOME/.eris/chains/simplechain"
Note the use of double quotes around the string.
~ isn't being expanded because you quoted it. Leave it outside the quotes, and bash will expand it before performing the assignment. Notice that you can quote the rest of the path if necessary, as long as the ~ is outside the quotes.
$ chain_dir=~'/.eris/chains/simplechain'
$ echo "$chain_dir"
/home/tbrooke/.eris/chains/simplechain
(Assuming /home/tbrooke is your home directory.)

Resources