I have a directory structure like so
/dir01/dir02/files
I want to copy the first file in dir02 onto a separate drive and into a directory with the same name as dir01
I wrote the following script
while [ "${*}" != "" ] ; do
INPUT="${1}"
FOLDER="${INPUT}"/*DPX
TARGET_FOLDER="/Users/user/Desktop/folder"/$(basename "${INPUT}")
for file in "${FOLDER}"; do
echo cp "$file" "${TARGET_FOLDER}"
break 1
done
done
Here INPUT is dir01 , FOLDER is dir02 and TARGET_FOLDER is the new directory with the same name as dir02 I want the file to copy to.
When I run the script it looks for a folder named *DPX in the INPUT path, which doesn't exist. There are many folders in the INPUT directory named *DPX and I want it to pull the first file from all of them.
Try replacing your for with:
for file in "$INPUT"/*DPX/*
Notes:
Your version is looking for a file called *DPX because ${FOLDER} is quoted on the for line.
for f in "$dir" will execute the for loop once, with f=$dir. To look for files under $dir, you need another /*.
Also, you want a shift before the last done.
Related
I have a folder contains around 650 folders (source), I generated a list of the relevant folders i want (final.txt).
i am trying to use a "for loop" to copy only the relevant sub-folders to a new location (target).
i keep getting the original content of the "source" copied to the "target".
i run:
for var in `cat final.txt` ; do rsync -ah $var source/ target/ ; done
I tried different syntax but can't seem to get what I need.
what am i doing wrong?
I expect to copy only the folders which name is in the final.txt list copied to the target (all "names" in the file are a single word, matching to some of the folder names for exactly)
ok after messing around I should have ran this (it works)
for var in `cat final.txt` ; do rsync -ah source/$var target/ ; done
I'm new in Bash and I have a list of names of directories stored in an excel file. I'd like to find those directories (they are located in different location at the computer) and to copy from each directory specific files (list of 4 files that ends with specific endings) to a remote computer.
For examples:
For a name of directory at the excel sheet - "NA123", I'd like to find it and copy it's partial content to a remote computer, for example copy the files: samples-sheet.csv, toInfo.xml, newfiles.gz, todo.csv to the remote computer, under a folder name "NA123".
How do I begin to do that?
****Editing to give an example of how it needs to be*****
A short example of the csv is as below:
A
1 14RD00129_TS1_01
2 SD-2015-06_01
3 US-005
4 RA99
All the names at the csv are directories that can be found under /home/bella/samples under 3 different folders: some will be at /home/bella/samples/gruop_1, some at:/home/bella/samples/gruop_2, and some at:/home/bella/samples/gruop_3
So first I need to iterate through the csv file, to locate the match directory at my computer, then I need to copy 4 specific files to a remote computer with the same name of directory. Hope this is clearer...
I guess you CSV file should only consist of directory names then, since there's only one column. I assume there is no header line in the CSV (A in your example) and no line number. You can take this as a starting point:
samples='/home/bella/samples'
while IFS= read -r line; do
dir=$(find "$samples"/gruop_{1..3} -type d -name "$line")
scp "$dir"/{samples-sheet.csv,toInfo.xml,newfiles.gz,todo.csv} \
user#host.com:"/path/to/$line"
done < 'file.csv'
Basically, you could do something like:
# create the directory on the remote:
ssh remote-ip 'mkdir -p NA123'
# copy the files to the remote in the directory just created
for f in samples-sheet.csv toInfo.xml newfiles.gz todo.csv; do scp $f remote-ip:NA123/; done
I'm writing a unix shell script that sorts data in ten subdirectories (labelled 1-10) of the home directory. In each subdirectory, the script needs to rename the files hehd.output and fort.hehd.time, as well as copy the file hehd.data to a .data file with a new name.
What I'd like it to do is rename each of these files in the following format:
AA.BB.CC
Where
AA = a variable in the hehd.data file within the subdirectory containing the file
BB = the name of the subdirectory containing the file (1-10)
CC = the original file name
Each subdirectory contains an hehd.data file, and each hehd.data file contains the string ij0=AA, where AA represents the variable I want to use to rename the files in the same subdirectory.
For example: When run, the script should search /home/4/hehd.data for the string ij0=2, then move /home/4/hehd.output to /home/4/2.4.hehd.output.
I'm currently using the grep command to have the script search for the string ij0=* and copy it to a new text file within the subdirectory. Next, the string ij0= is deleted from the text file, and then its contents are used to rename all target files in the same subdirectory. The last line of the shell script deletes the text file.
I'm looking for a better way to accomplish this, preferably such that all ten subdirectories can be sorted at once by the same script. My script seems incredibly inefficient, and doesn't do everything that I want it to by itself.
How can I improve this?
Any advice or suggestions would be appreciated; I'm trying to become a better computer user and that means learning better ways of doing things.
Try this:
fromdir=/home
for i in {1..10};do
AA=$(sed 's/ij0=\([0-9]*\)/\1/' "$fromdir/$i/hehd.data")
BB="$i"
for f in "$fromdir/$i/"*;do
CC="${f##*/}"
if [[ "$CC" = "hehd.data" ]]; then
echo cp "$f" "$fromdir/$i/$AA.$BB.$CC"
else
echo mv "$f" "$fromdir/$i/$AA.$BB.$CC"
fi
done
done
It loops over directories using Bash sequence {1..10].
In each directory, with the sed command the ij0 value is assigned to AA variable, the directory name is assigned to BB.
In the file loop, if the file is hehd.data it's copied, else it's renamed with the new name.
You can remove the echo before cp and mv commands if the output meets your needs.
Im trying to move multiple files from one directory to another directory.
File name is with sequence and will be varying.
Example:
/global/userhome/usrsats/---------directory which has file names as below:
fl_cl_filename1
fl_cl_filename2
fl_cl_filename3
...
...
Now when moved to another directory, i need to get only the file name and delimit the fl_cl part.
Please help
Assuming you're using bash, I would do this with the remove the matching prefix pattern facility like this (with DEST_DIR set to the destination directory):
cd /global/userhome/usrsats
for f in *; do mv $f ${DEST_DIR}/${f#fl_cl_}; done
I'm not very good in shell scripting and would like to ask you some question about looping of files big dataset: in my example I have alot of files with the common .pdb extension in the work dir. I need to loop all of them and i) to print name (w.o pdb extension) of each looped file and make some operation after this. E.g I need to make new dir for EACH file outside of the workdir with the name of each file and copy this file to that dir. Below you can see example of my code which are not worked- it's didn't show me the name of the file and didn't create folder for each of them. Please correct it and show me where I was wrong
#!/bin/bash
# set the work dir
receptors=./Receptors
for pdb in $receptors
do
filename=$(basename "$pdb")
echo "Processing of $filename file"
cd ..
mkdir ./docking_$filename
done
Many thanks for help,
Gleb
If all your files are contained within the .Repectors folder, you can loop each of them like so:
#!/bin/bash
for pdb in ./Receptors/*.pdb ; do
filename=$(basename "$pdb")
filenamenoextention=${filename/.pdb/}
mkdir "../docking_${filenamenoextention}"
done
Btw:
filenamenoextention=${filename/.pdb/}
Does a search replace in the variable $pdb. The syntax is ${myvariable/FOO/BAR}, and replaces all "FOO" substrings in $myvariable with "BAR". In your case it replaces ".pdb" with nothing, effectively removing it.
Alternatively, and safer (in case $filename contains multiple ".pdb"-substrings) is to remove the last four characters, like so: filenamenoextention=${filename:0:-4}
The syntax here is ${myvariable:s:e} where s and e correspond to numbers for the start and end index (not inclusive). It also let's you use negative numbers, which are offsets from the end. In other words: ${filename:0:-4} says: extract the substring from $filename starting from index 0, until you reach fourth-to-the-last character.
A few problems you have had with your script:
for pdb in ./Receptors loops only "./Receptors", and not each of the files within the folder.
When you change to parent directory (cd ..), you do so for the current shell session. This means that you keep going to the parent directory each time. Instead, you can specify the parent directory in the mkdir call. E.g mkdir ../thedir
You're looping over a one-item list, I think what you wanted to get is the list of the content of ./Receptors:
...
for pdb in $receptors/*
...
to list only file with .pdb extension use $receptors/*.pdb
So instead of just giving the path in for loop, give this:
for pdb in $receptors/*.pdb
To remove the extension :
set the variable ext to the extension you want to remove and using shell expansion operator "%" remove the extension from your filename eg:
ext=.pdb
filename=${filename%${ext}}
You can create the new directory without changing your current directory:
So to create a directory outside your current directory use the following command
mkdir ../docking_$filename
And to copy the file in the new directory use cp command
After correction
Your script should look like:
receptors=./Receptors
ext=.pdb
for pdb in $receptors/*.pdb
do
filename=$(basename "$pdb")
filename=${filename%${ext}}
echo "Processing of $filename file"
mkdir ../docking_$filename
cp $pdb ../docking_$filename
done