This question already has answers here:
Appending a string before file extension in Bash script
(3 answers)
Execute command on all files in a directory
(10 answers)
Closed 3 years ago.
I have a set of files in a directory
test1.in
test2.in
test3.in
test4.in
...
I have written a conversion tool to extract some data from those files and save it into another file.
I want to run that conversion tool for each file and save the results into different output files
./tool -i test1.in -i test1.out
./tool -i test2.in -o test2.out
I have seen a lot of answer to iterate on the files and run the desired command for each file, but I am not sure how to specify the output.
for file in dir/*.in; do
./tool -i file -o ???
done
How can I create a name for the output file?
for file in dir/*.in; do
./tool -i "$file" -o "${file%.*}".out
done
"${file%.*}" represents the value of $file before the last . (See https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html#Shell-Parameter-Expansion, look for ${parameter%word}).
Related
This question already has answers here:
How to find all files containing specific text (string) on Linux?
(54 answers)
Closed 3 years ago.
Using a script need to identify all files or folder which contains a specific keyword under given path. The path will have multiple folders.
Below is the script that is used.
#!/bin/sh
DIR=''
for FILE in ls "$DIR"*
do
# echo $FILE
grep -l $FILE "*FY*"
done
~
You have to give the "keyword" before the path to be searched. which it not the case in your script.
grep -l -r "keyword" path/to/folder
Try the above script.
-l, --files-with-matches
Suppress normal output; instead print the name of each input
file from which output would normally have been printed. The
scanning will stop on the first match. (-l is specified by
POSIX.)
-R, -r, --recursive
Read all files under each directory, recursively; this is
equivalent to the -d recurse option.
for more info refer
This question already has answers here:
Filename not printing correctly with underscore "_" in Bash [duplicate]
(2 answers)
Closed 3 years ago.
Hi I am trying to transfer a list of files from the server to my computer using a while loop, but it keeps giving me the error that there is no file or directory. I can't understand why.
So I have this file list_numbers.txt with over 500 lines
list_numbers.txt
1234
345
2135
2132
...
And I want to re-iterate through this list to transfer files from server to my computer with the corresponding foldername to my computer but replacing assembly_graph.txt with $line_for_assembly.txt
while read line; do scp -r user#server.com:/home/Documents/$line_assembly/assembly_graph.txt /Users/Documents/$line_for_assembly.txt; done < list_numbers.txt
So basically I want it to do the below code, but instead of me having to manually type it, it would re-iterate through the list.
scp -r user#server.com:/home/Documents/1234_assembly/assembly_graph.txt /Users/Documents/1234_for_assembly.txt
scp -r user#server.com:/home/Documents/345_assembly/assembly_graph.txt /Users/Documents/345_for_assembly.txt
scp -r user#server.com:/home/Documents/2135_assembly/assembly_graph.txt /Users/Documents/2135_for_assembly.txt
scp -r user#server.com:/home/Documents/2132_assembly/assembly_graph.txt /Users/Documents/2132_for_assembly.txt
But I get this error:
scp: /home/Documents//assembly_graph.txt: No such file or directory
So it is not picking up the $line_assembly, even when I do ${line}_assembly. I don't know what I'm doing wrong.
because you do not have a variable named line_assembly.
Perhaps you mean to the following:
while read line
do
line=`echo $line | tr -d '\r\n'`
scp -r user#server.com:/home/Documents/${line}_assembly/assembly_graph.txt /Users/Documents/${line}_for_assembly.txt
done < list_numbers.txt
this will expand as the value of line followed by the text "_assembly".
For example if "line" has the value "component" you will get
.../Documents/component_assembly/...
EDIT: OP had spurious characters in his/her input file (probably \r characters from windows). I've edited the answer accordingly to accommodate OP's specific issue.
This question already has answers here:
How to loop through file names returned by find?
(17 answers)
Closed 3 years ago.
I am dealing with a legacy codebase where we're trying to convert all jpeg/png files to webp format using the cwebp command. Unfortunately, a lot of the image files were saved with spaces in the name.
Example: i am poorly named.jpg
So when running the following bash script to find all jpegs in the directory and loop through and convert them the words separated by spaces are treated as another file so the image never gets converted.
We don't want to remove the whitespaces, but just create a webp file with the exact same name.
files=$(find ./ -type f -name "*.jpg")
for jpg in $files
do
webp="${jpg/%jpg/webp}";
if [ ! -f $webp ]; then
echo "The webp version does not exist";
cwebp -q 80 "$jpg" -o "$webp";
fi
done
I've tried placing jpg=$(printf '%q' "$jpg") immediately after the do in the above code as well as other things.
I expect i am poorly named.webp to be created if file i am poorly named.jpg exists.
But there is no real reason to store all filenames. So an alternative is:
find ./ -type f -name "*.jpg" | while read jpg
do
....
But this works only, if a filename contains no linefeed. For files with linesfeeds there are other solutions. Ask for it if needed.
This question already has answers here:
Extending a script to loop over multiple files and generate output names
(1 answer)
File name without extension in bash for loop
(3 answers)
Closed 5 years ago.
beginner here. I want to write a simple script for use in the bash terminal, but I don't know how to make it. The gist is that I have a folder filled with different files, some .foo, some .bar, etc. I want to create a script that takes all the .foo files and perform a command on them, but in the same line rename them so that the output file is named file.baz.
For example:
command -i file.foo -o file.baz for all .foo files in a directory.
Renaming
You can use the rename command:
$ rename .foo .baz *.foo
Note that on some systems rename points to prename which uses a different syntax:
$ prename 's/.foo$/.baz' *.foo
Use man rename to find out which one you have.
Looping over files and running a command on each of them
You can provide the file list directly on the command line, using a globbing pattern:
$ your_script *.foo
Your script can then iterate over the list like this (using your command's usage):
for file in "$#"; do
your_command -i "$file" -o "${file%.*}.baz"
done
${file%.*} resolves as the name of the file without its extension (file.foo -> file). More information on string manipulation is available here.
This question already has answers here:
How can I use a file in a command and redirect output to the same file without truncating it?
(14 answers)
Closed 7 years ago.
I have many directories with multiple files, each following strict naming convention. I tried this loop:
for s in /home/admin1/phd/results_Sample_*_hg38_hg19/Sample_*_bwa_LongSeed_sorted_hg38.bam; do
bedtools genomecov -ibam $s > $s
done
but get for every file:
Failed to open file
I just can't see the mistake
Note that if you are redirecting the output to the input file, bash will truncate the file before the process starts. Like this:
cat file > file
will result an empty file. Meaning you'll loose it's contents.
This is because bash will open all files which are subject of io redirection before it starts the command. If you are using > bash will additionally truncate the file to zero length. I don't know bedtools but I guess it expects non-empty files as input, that's why the message.
About the loop. Usually it is not a good idea to loop over the results of a glob expression like you did. This is because file names may contain the internal field separator of bash which would lead to inconsistent results.
I would suggest to use find instead:
find /home/admin1/phd/results_Sample_*_hg38_hg19 \
-maxdepth 1 \
-name 'Sample_*_bwa_LongSeed_sorted_hg38.bam' \
-exec bash -c 'bedtools genomecov -ibam "{}" > "{}.copy"' \;