This question already has answers here:
Can ${var} parameter expansion expressions be nested in bash?
(15 answers)
Closed 4 years ago.
I have the follwing which creates a directory from a list of files:
for file in $(ls *.txt); do
folder=${file//.txt/ };
folder=${folder//./'/'};
folder=${folder//[[:space:]]/};
mkdir -p $folder;
done
Can I link the 3 string manipulation commands that assign folder into one line?
I've tried several things with no success. Is it possible to use the | operator somehow?
Bash cannot do this, but Z Shell (zsh) (which is very similar) can nest the replacements:
for file in *.txt; do
folder=${${${file//.txt/ }//./'/'}//[[:space:]]/};
mkdir -p $folder;
done
(You don't need to do $(ls *.txt) (parsing ls in this way is dangerous: you lose all spaces) since you can just give it *.txt, which properly handles spaces in filenames. This works in any POSIX shell.)
Related
This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 3 years ago.
I have a variable:
rules=L002,L003
This rules variable denotes the files to copy from /usr/lib/vera++/scrpts/rules directory. The extension for the files is .tcl.
I am doing something like:
cp -r /usr/lib/vera++/scripts/rules/{${rules}}.tcl .
What goes wrong is that ${rules} is treated completely as a string. But bash should translate that into:
cp -r /usr/lib/vera++/scripts/rules/{L002,L003}.tcl .
The easiest way to take an argument and make a command from this argument is to use eval command. In your case whole script will look like:
#!/bin/bash
rules=L002,L003
eval "cp -r /usr/lib/vera++/scripts/rules/{${rules}}.tcl ."
This question already has answers here:
bash variable interpolation separate variables by a hyphen or underscore
(3 answers)
Closed 5 years ago.
I have 80 pairs of files of the following type:
170309-N701-S517_S1_L001_R1_001.fastq.gz
170309-N701-S517_S1_L001_R2_001.fastq.gz
170309-N701-S502_S2_L001_R1_001.fastq.gz
170309-N701-S502_S2_L001_R2_001.fastq.gz
170309-N701-S503_S3_L001_R1_001.fastq.gz
170309-N701-S503_S3_L001_R2_001.fastq.gz
..
170309-N710-S507_S79_L001_R1_001.fastq.gz
170309-N710-S507_S79_L001_R2_001.fastq.gz
170309-N710-S508_S80_L001_R1_001.fastq.gz
170309-N710-S508_S80_L001_R2_001.fastq.gz
Essentially, a pair consists of the following files:
170309-N701-S517_S<ID>_L001_R1_001.fastq.gz
170309-N701-S517_S<ID>_L001_R2_001.fastq.gz
where ID varies from 1 to 80.
I would like to create 80 subfolders named S1..S80 and put each pair in the corresponding folder, e.g.
170309-N701-S517_S1_L001_R1_001.fastq.gz
170309-N701-S517_S1_L001_R2_001.fastq.gz
go to subfolder S1
170309-N701-S502_S2_L001_R1_001.fastq.gz
170309-N701-S502_S2_L001_R2_001.fastq.gz
go to subfolder S2
and so on.
I wrote the following script:
#!/bin/bash
for i in {1..80}
do
mkdir S$i
mv "*_S"$i"_*" S$i
done
but it didn't work as expected. It created the subfolders S1..S80. However, it didn't move any of the files. It produced instead
mv: cannot stat `*_S1_*': No such file or directory
mv: cannot stat `*_S2_*': No such file or directory
and so on.
What am I doing wrong? Can you correct the script?
What you're trying to do with the mv "*_S"$i"_*" S$i line is called variable expansion, and if you as a part of that expansion wants to concatenate the variable with other characters, you need to let BASH know what is a variable and what is a plain character. You do this by enclosing the variable name in curly braces. E.g:
var=FOO
echo "BAR ${var} BAZ"
# BAR FOO BAZ
In the case of your loop:
touch \
170309-N701-S517_S1_L001_R1_001.fastq.gz\
170309-N701-S517_S1_L001_R2_001.fastq.gz\
170309-N701-S502_S2_L001_R1_001.fastq.gz\
170309-N701-S502_S2_L001_R2_001.fastq.gz\
170309-N701-S503_S3_L001_R1_001.fastq.gz\
170309-N701-S503_S3_L001_R2_001.fastq.gz
for i in {1..80}
do
if test -n "$(find . -maxdepth 1 -name "*_S${i}_*" -print -quit)"
then
mkdir "S${i}"
mv *"_S${i}_"* "S$i"
fi
done
That if-then-fi bit is there just to avoid making directories for non-existing files. Entirely optional.
Also note that globbing character * must be used unquoted in order to work with mv, because inside "..." or '...', * is treated as a literal.
An exception to this is however seen in the find command, where the content between the double quotes will be expanded as it is passed on to find. To avoid expansion in this case the argument can be enclosed in single quotes ('...')
This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 6 years ago.
I am trying to do something in a bash script whenever a file in a directory I am iterating over contains a string using grep. The problem comes in where a subset of the files in the directory contain spaces in the name. Therefore, I have tried to replace the spaces with escaped spaces in place using sed:
if grep -c "main" ${source} | sed 's/ /\\ /g'; then
# do something
fi
However, I still get the error:
grep: /Users/me/Desktop/theDir/nameWith: No such file or directory
grep: spaces.txt: No such file or directory
What am I doing wrong?
You should quote the name of the file being grep'ed:
if grep -c main "$source" ; then
# do something
fi
...assuming $source is the name of a file. If $source is the name of a directory, I'll need more information about what you're trying to do.
This question already has answers here:
How to pass the value of a variable to the standard input of a command?
(9 answers)
Closed 1 year ago.
I have a shell script:
TOPDIR=`pwd`
FOLDER=$($TOPDIR | sed 's/\//\_/g')
if [[ condition ]];then
source ~/[$FOLDER]-build/build-env.sh
fi
the TOPDIR here is /home/uname/project, so the variable FOLDER is supposed to be _home_uname_project because sed is called to replace / with _.
But it goes wrong when executing, terminal tells that /home/uname/[]-build/build-env.sh: No such file or directory which, I guess, means that FOLDER is unexpected empty in the if-then statement. Can anybody help me with figuring this out?
If you look at the output of just
$TOPDIR | sed 's/\//\_/g'
you'll realize that it's empty; it's trying to execute a command equal to the contents of $TOPDIR and pipe the output of that into sed, but there is no output in the first place.
You could do
pwd | sed 's\//_/g'
instead (no need to escape _), which would work.
Or, instead of using an external tool, you could use parameter expansion
topdir="$(pwd)"
topdir="${topdir//\//_}"
with the same result.
Notice that uppercase variable names are discouraged, as they're more likely to clash with existing, reserved names.
This question already has answers here:
How to skip the for loop when there are no matching files?
(2 answers)
Closed 3 years ago.
I want to loop over all files matching extension jpg or txt. I use:
for file in myDir/*.{jpg,txt}
do
echo "$file"
done
Problem: If the directory contains no jpg file at all, the loop will have one iteration with output myDir/*.jpg. I thought * will be replaced by an arbitrary file (and if no file exists it cannot be expanded). How can I avoid the unwanted iteration?
Use this to avoid the unwanted iteration:
shopt -s nullglob
From man bash:
nullglob: If set, bash allows patterns which match no files (see Pathname Expansion above) to expand to a null string, rather than themselves.
See: help shopt and shopt
This and a duplicate question both were in context of not just pathname-expansion, but also brace-expansion, and a duplicate asked for POSIX.
The compgen -G does bash --posix compatible pathname-expansion (no brace-expansion) and... you guessed it: yields nothing if there are no matches.
Therefore write a bash --posix function to do it. Brief outline: temporarily use set -f to first do brace-expansion (without pathname-expansion) via an echo, then apply compgen -G to each result for pathname-expansion. Full function left as an exercise.