I have been using groovy for the past two weeks. I have a script which I would normally run like this:
groovy script input
Where input is the name of an input file. However, now I would like to repeat the same for a whole directory of about 100 of those files - is there a way to run this in a single command, as opposed to one after another?
Sure, use a shell loop:
for file in *; do groovy script "$file" > file.out; done
This will iterate over all files (and directories) in your current directory. To make it recursive, meaning that it will also look inside subdirectories, use:
shopt -s globstar
for file in **/*; do groovy script "$file" > file.out; done
Finally, to avoid directories (if any), use:
for file in *; do [ -f "$file" ] && groovy script "$file" > file.out; done
Related
I have several thousand files with name like this:
PIN_PMN_PT_010_02_00331_0004_018edf
and need to rename them all something like this:
PIN_PMN_PT_010_02_00331_0004_018.edf
I have used simple mv scripts like this:
for f in *; do echo mv "$f" "`echo $f | tr 'edf' '.edf'`"; done
For some reason it creates names like this:
PIN_PMN_PT_010_02_00331_0004_018.ed
They are missing the last f. I am running the script using echo to dry run. Any ideas please?
Using MACBook Pro running Mohave 10.14.6 and Bash.
Use shell Parameter Expansion!
Simply under any shell:
For the test, echo to show what's will go:
for file in *edf ;do echo mv "$file" "${file%edf}.edf" ;done
Then, for doing the job:
for file in *edf ;do mv "$file" "${file%edf}.edf" ;done
(This must work same on MacOs, than under Linux.)
... And to prevent renaming of already correctly named files:
for file in *edf ;do test -f "${file##*.edf}" && mv "$file" "${file%edf}.edf" ;done
Syntax ${file##*.edf} will replace any string, terminated by .edf, by an empty string. So test -f "" will fail.
... Still: I don't have any Mac for doing the test, but as this is POSIX Standard, this must work on any shell. (Let my know, please comment!)
More infos?
Have a look at man sh or man bash and search for Parameter Expansion
man -P"less +'/Parameter Expansion'" bash
Using sed, this should work
for f in *; do echo mv "$f" "`echo $f | sed 's/.\{3\}$//`.edf"; done
You are just removing the last 3 characters of a string, and adding your file extension.
Personally, I find the rename command invaluable for this sort of thing:
rename 's/edf$/.edf/' *edf
If you want to do a dry-run, you can do:
rename --dry-run 's/edf$/.edf/' *edf
Sample Output
'PIN_PMN_PT_010_02_00331_0004_018edf' would be renamed to 'PIN_PMN_PT_010_02_00331_0004_018.edf'
The benefits of using rename are:
it can do a dry-run to test before you run for real
it will create all necessary directories with the -p option
it will not clobber (overwrite) files without warning
you have the full power of Perl available to you and can make your renaming as sophisticated as you wish.
As helpfully suggested by F. Hauri in the comments, you may have some files that have already had the dot inserted before the extension in your directory. To protect against insertion of a second dot, you could either be more specific in the files you select for renaming and only rename those ending in a digit followed by edf:
rename 's/edf$/.edf/' *[0-9]edf
Or, as F.Hauri suggested:
rename 's/([^.])edf$/$1.edf/' *edf
Note that you can install on macOS with homebrew:
brew install rename
I am relatively new to bash scripting.
I need to create a script that will loop through a series of directories, go into subdirectories with a certain name, and then move their file contents into a common folder for all of the files.
My code so far is this:
#!/bin/bash
#used to gather usable pdb files
mkdir -p usable_pdbFiles
#loop through directories in "pdb" folder
for pdbDirectory in */
do
#go into usable_* directory
for innerDirectory in usable_*/
do
if [ -d "$innerDirectory" ] ; then
for file in *.ent
do
mv $file ../../usable_pdbFiles
done < $file
fi
done < $innerDirectory
done
exit 0
Currently I get
usable_Gather.sh: line 7: $innerDirectory: ambiguous redirect
when I try and run the script.
Any help would be appreciated!
The redirections < $innerDirectory and < $file are invalid and this is causing the problem. You don't need to use a loop for this, you can instead rely on the shell's filename expansion and use mv directly:
mkdir -p usable_pdbFiles
mv */usable_*/*.ent usable_pdbFiles
Bear in mind that this solution, and the loop based one that you are working on, will overwrite files with the same name in the destination directory.
Here is the condition:
I have a file with all packages installed.
I have a folder with all kinds of other packages, but they include all of the ones in the list, plus more.
I need a bash script that will read the file and check a folder for packages that don't exist in the list then remove them, they are not needed, but keep the packages that are on the list in that folder.
Or perhaps the bash should read folder then if packages in the folder aren't on the list them rm -f that or those packages.
I am familiar with writing if then conditional statements, I just don't know how to do if making the items in the list a variable or variables (in a loop).
thanks!
I would move the packages on the list to a new folder, delete the original folder, and move the temporary folder back:
DIR=directory-name
mkdir "$DIR-tmp"
while read pkgname; do
if [[ -f "$DIR/$pkgname" ]]; then
mv "$DIR/$pkgname" "$DIR-tmp"
fi
done < package-list.txt
# Confirm $DIR-tmp has the files you want first!
rm -rf "$DIR"
mv "$DIR-tmp" "$DIR"
I think you want something like this:
for file in $(ls folder) ; do
grep -E "$file" install-list-file >/dev/null || \
echo $file
done > rm-list
vi rm-list # view file to ensure correct
rm $(<rm_list)
There are ways to make this faster (using parameter substitution to avoid fork/exec's), but I recommend avoiding fancy shell stuff [${file##*/}] until you've got the basics down. Also, this script basically translates the description into a script and is not intended to be much more than a guide on how to approach the problem.
I have many files' path, but I need to copy all files into other location /sample, and I want to copy files into different folders:
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_2.fq.gz
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/clean_111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info
I want to copy those files into AS34_59329 folder inside /sample
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59328/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59328/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_2.fq.gz
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59328/clean_111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info
I want to copy those file into AS34_59328 folder inside /sample
I write codes to scp all file into /sample folder, but I don't know how to put each files into different sub-directory, like:
/ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59328/clean_111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info
put into AS34_59328
#! /bin/bash
while read myline
do
for i in $myline
do
if [ -f $i]; then
#how to put different files into different sub-directory
scp -r $i xxx#191.168.174.43:/sample
fi
done
done < data.list
new changed part
#! /bin/bash
while read myline
do
for i in $myline
do
if [ -f $i ]
then
relname=$(echo $i | sed 's%\(/[^/][^/]*\)\{5\}/%%')
echo $relname
fi
done
done < /home/jesse/T11073_all_3254.fq.list
It appears you need to strip the leading 5 components of the pathname off the filename. Since you don't have spaces in your names (the way you're using for i in $myline precludes that possibility), you can use:
#! /bin/bash
while read myline
do
for i in $myline
do
if [ -f $i ]
then
relname=$(echo $i | sed 's%\(/[^/][^/]*\)\{5\}/%%')
scp -r $i xxx#191.168.174.43:/sample/$relname
fi
done
done < data.list
The regex is just a way of looking for a sequence of five sets of slash followed by one or more non-slashes plus one more slash and deleting them. Since slashes figure prominently in the search, I used % to mark the sections of the s/// operation instead.
For example, given the input:
/a/b/c/d/e/f/g
the output from the sed is:
f/g
Note that this code does not explicitly create directories on the remote machine; it just specifies where the file is to go. If you need to create them too, you will have to investigate ssh, probably, to run mkdir -p /sample/$(dirname $relname) on the remote machine (where the dirname operation can be run either locally or remotely).
Note that scp has a recursive copy mode (-r) which would simplify things considerably if you knew you needed to copy all the files from the local directory to the remote.
I have a droplet made with automator, which moves files when I drop them on application icon to certain folder.
now script looks like this:
for f in "$#"
do
cp "$f" "volumes/testdrive/testfolder/$(basename "$f")"
done
I was wondering if it possible to do command to detect if multiple files were input into script and then archive them with zip function and move to same folder, and if single file was dropped do regular copy of file to specified folder.
Use a conditional, something like (syntax might be way off):
if [ $# > 1 ]
then zip $# > /path/to/location/foo.zip # this line might need to be researched
else cp $# /path/to/location/`basename $#`
fi
You wouldn't want to do this with a for loop, because then it'd go through the rigmarole of creating a zip archive for each file in the selection. If what you were doing was moving each one, then sure, use a loop, but you're just taking them all and zipping them
$# contains the number of arguments passed to the script.