I have written a shell script which picks all the files recursively inside all the directories and prepared a report with the file last modified, size.
The problem that I am facing, there are few files with name as "User Interface"(space in between). How to use there files in the for loop of the shell script and fetch the files and directories inside this.
Thanks in advance
Just put the file name variable between double quotes "$FILENAME"
You're probably trying to use something like for file in $(command). Instead, use a while read loop or a for loop with globbing. Make sure you quote variables that contain filenamess.
#!/bin/sh
command | while read -r file
do
something_with "$file"
done
or, in shells that support process substitution:
#!/bin/bash
while read -r file
do
something_with "$file"
done < <(command)
If you're simply iterating over a list of files:
for file in "$dir"/*
do
something_with "$file"
done
Google Search led me to this page
Related
Using a Bash script, I'd like to move a list of files by using a for-loop, not a while-loop (for testing purpose). Can anyone explain to me why mv always acts as file rename rather than file move under this for loop? How can I fix it to move the list of files?
The following works:
for file in "/Volumes/HDD1/001.jpg" "/Volumes/HDD1/002.jpg"
do
mv "$file" "/Volumes/HDD2/"
done
UPDATE#1:
However, suppose that I have a sample_pathname.txt
cat sample_pathname.txt
"/Volumes/HDD1/001.jpg" "/Volumes/HDD1/002.jpg"
Why the following for-loop will not work then?
array=$(cat sample_path2.txt)
for file in "${array[#]}"
do
mv "$file" "/Volumes/HDD2/"
done
Thanks.
System: OS X
Bash version: 3.2.53(1)
cat sample_pathname.txt
"/Volumes/HDD1/001.jpg" "/Volumes/HDD1/002.jpg"
The quotation marks here are the problem. Unless you need to cope with file names with newlines in them, the simple and standard way to do this is to list one file name per line, with no quotes or other metainformation.
vbvntv$ cat sample_pathname_fixed.txt
/Volumes/HDD1/001.jpg
/Volumes/HDD1/002.jpg
vbvntv$ while read -r file; do
> mv "$file" "/Volumes/HDD2/"
> done <sample_pathname_fixed.txt
In fact, you could even
xargs mv -t /Volumes/HDD2 <sample_pathname_fixed.txt
(somewhat depending on how braindead your xargs is).
The syntax used in your example will not create an array... It is just storing the file contents in a variable named array.
IFS=$'\n' array=$(cat sample_path2.txt)
If you have a text file containing filenames (each on separate line would be simplest), you can load it into an array and iterate over it as follows. Note the use of $(< file ) as a better alternative to cat and the parenthesis to initialize the contents into an array. Each line of the file corresponds to an index.
array=($(< file_list.txt ))
for file in "${array[#]}"; do
mv "$file" "/absolute/path"
done
Update: Your IFS was probably not set correctly if the command at the top of the post didn't work. I updated it to reflect that. Also, there are a couple of other reliable ways to initialize an array from a file. But like you mentioned, if you are just piping the file directly into a while loop, you may not need it.
This is a shell builtin in Bash 4+ and a synonym of mapfile. This works great if its available.
readarray -t array < file
The 'read' command can also initialize an array for you:
IFS=$'\n' read -d '' -r -a array < file
use this:
for file in "/Volumes/HDD1/001.jpg" "/Volumes/HDD1/002.jpg"
do
f=$(basename $file)
mv "$file" "/Volumes/HDD2/$f"
done
All,
I am running BASH in Solaris 10
I have the following shell script that loops in a directory depending on the presence of CSV files.
The problem is with this piece of code is that it still does one loop even if there is no CSV files in that directory and then calls SQL loader.
SQLLoader then produces a log file because there is no file to process and this is beginning to mess up my directory filling it with log files.
for file in *.csv ;
do
echo "SQLLoader is reading : " $file
sqlldr <User>/<Password>#<DBURL>:<PORT>/<SID> control=sqlloader.ctl log=$inbox/$file.log data=$inbox/$file
done
How do I stop it going into a loop if there is no CSV files in that directory of $inbox
Say:
shopt -s nullglob
before your for loop.
This is not the default, and saying for file in *.csv when you don't have any matching files expands it to *.csv.
Quoting from the documentation:
nullglob
If set, Bash allows filename patterns which match no files to expand to a null
string, rather than themselves.
Use find to search files
for file in `find -name "*.csv"` ;
First off, using nullglob is the correct answer if it is available. However, a POSIX-compliant option is available.
The pattern will be treated as literal text if there are no matches. You can catch this with a small hack:
for file in *.csv; do
[ -f "$file" ] || break
...
done
When there are no matches, file will be set to the literal string *.csv, which is not the name of a file, so -f "$file" will fail. Otherwise, file will be set in turn to the name of each file matching the pattern, and -f "$file" will succeed every time. Note this will work even if there is an file named *.csv. The drawback is that you have to make a redundant test for each existing file.
Hi I have a file that sorts some code and reformats it. I have over 200 files to apply this to with incremental names run001, run002 etc. Is there a quick way to write a shell script to execute this file over all the files? The executable creates a new file called run001an etc so just running over all files containing run doesnt work, how do i increment the file number?
Cheers
how about:
for i in ./run*; do
process_the_file $i
done
which is valid Bash/Ksh
To be more specific with run### files you can have
for file in dir/run[0-9][0-9][0-9]; do
do_something "$file"
done
dir could simply be just . or other directories. If they have spaces, quote them around "" but only the directory parts.
In bash, you can make use of extended patterns to generate all number matches not just 3 digits:
shopt -s extglob
for file in dir/run+([0-9]); do
do_something "$file"
done
I need to rename 45 files, and I don't want to do it one by one. These are the file names:
chr10.fasta chr13_random.fasta chr17.fasta chr1.fasta chr22_random.fasta chr4_random.fasta chr7_random.fasta chrX.fasta
chr10_random.fasta chr14.fasta chr17_random.fasta chr1_random.fasta chr2.fasta chr5.fasta chr8.fasta chrX_random.fasta
chr11.fasta chr15.fasta chr18.fasta chr20.fasta chr2_random.fasta chr5_random.fasta chr8_random.fasta chrY.fasta
chr11_random.fasta chr15_random.fasta chr18_random.fasta chr21.fasta chr3.fasta chr6.fasta chr9.fasta
chr12.fasta chr16.fasta chr19.fasta chr21_random.fasta chr3_random.fasta chr6_random.fasta chr9_random.fasta
chr13.fasta chr16_random.fasta chr19_random.fasta chr22.fasta chr4.fasta chr7.fasta chrM.fasta
I need to change the extension ".fasta" to ".fa". I'm trying to write a bash script to do it:
for i in $(ls chr*)
do
NEWNAME = `echo $i | sed 's/sta//g'`
mv $i $NEWNAME
done
But it doesn't work. Can you tell me why, or give another quick solution?
Thanks!
Several mistakes here:
NEWNAME = should be without space. Here bash is looking for a command named NEWNAME and that fails.
you parse the output of ls. this is bad if you had files with spaces. Bash can build itself a list of files with the glob operator *.
You don't escape "$i" and "$NEWNAME". If any of them contains a space it makes two arguments for mv.
If a file name begins with a dash mv will believe it is a switch. Use -- to stop argument processing.
Try:
for i in chr*
do
mv -- "$i" "${i/%.fasta/.fa}"
done
or
for i in chr*
do
NEWNAME="${i/%.fasta/.fa}"
mv -- "$i" "$NEWNAME"
done
The "%{var/%pat/replacement}" looks for pat only at the end of the variable and replaces it with replacement.
for f in chr*.fasta; do mv "$f" "${f/%.fasta/.fa}"; done
If you have the rename command, you can do:
rename .fasta .fa chr*.fasta
So I am going to post a question about shell scripting again.
Problem Definition: For all files under a dir, ex.:
A_anything.txt, B_anything.txt, ......
I want to execute a script, say 'CMD', on each of them, with the output files named like:
A_result.txt, B_result.txt, ......
In addition, at the first line of these output file, I want to have the file name of the original one.
The 'find -exec' util seems to me unable to extract part of the file name.
Does someone know a solution to this problem, by any means(shell, python, find,etc)? Thank you!
cd /directory
for file in *.txt ; do
newfilename=`echo "$file"|sed 's/\(.\+\)_.*/\1_result.txt/`
echo "$file" > "$newfilename"
your-command $file >> "$newfilename"
done
HTH
Well, there's more than one way to do it (including using Perl, where that's the motto), but probably I'd write it like this:
find . -name '[A-Z]_*.txt' -type f -print0 |
xargs -0 modify_rename.sh
And then I'd write the script modify_rename.sh like this:
#!/bin/sh
for file in "$#"
do
dirname=$(dirname "$file")
basename=$(basename "$file" .txt)
leadname=${file%_*}
outname="$dirname/${leadname}_result.txt"
# Optionally check for pre-existence of $outname
{
# Optionally echo "$basename.txt" instead of "$file"
echo "$file"
# Does this invocation of CMD write to standard output?
# If not, adjust invocation appropriately.
CMD "$file"
} > "$outname"
done
The advantage of this separation into separate scripting operations is that the rename/modify operation can be checked out separately from the search process - which runs less risk of zapping your entire directory structure with bad commands.
Bash has the tools to avoid invoking basename and dirname but the notation is moderatly excruciating; I find the clarity of the command names worth having. I'd be happy if bash implemented them as built-ins. There are plenty of other ways to get the prefix of the file; this should be safe, though, even in the presence of spaces (tabs, newlines) in file or directory names because of the careful use of double quotes.