Copy Folders Recursively: Shell - shell

My code takes two arguments from the user. My goal is to find the folder given by the user (the first argument) and make as many copies as they want of it (the second argument). As of right now, my code recursively prints out all subdirectories and files within the subdirectories... My code also copies files as many times as the user specifies. However, that's my problem, my copy function only works on files and not folders... I am wondering how to modify my copy function (or any other part of the code) so that it recursively copies the specified folder along with all of its original content
I know my cp function towards the bottom of the code needs to be changed to
-r ./sourceFolder ./destFolder but I do not know how to do that given the arguments passed in. After all, the destination folder name will just be the original folder with an incrementing number at the end of it.
#!/bin/bash
read -p "Your folder name is $1 and the number of copies is $2. Press Y for yes N for no " -n 1 -r
if ! [[ $REPLY =~ ^[Yy]$ ]]
then
echo
echo "Rerun script and pass in the folder and how many copies yo would like (e.g folder 4)"
exit
fi
echo
echo "-------------COPY FILE----------------"
FOLDER=$1
for f IN $folder
do
if [ ! -f "$FOLDER" ]
then
echo "folder "$FOLDER" does not exist"
exit 1
fi
done
DIR="."
function list_files()
{
cd $1
echo; echo "$(pwd)":; #Display Directory name
for i in *
do
if test -d "$i" #if dictionary
then
list_files "$i" #recursively list files
cd ..
else
echo "$i"; #Display File name
fi
done
}
j=$2
for i in "$*"
do
DIR=$1
list_files "$DIR"
for ((i=1; i<J; i++))
do
cp "$1" "$1$i"; #copies the file i amount of times, and creates new files with names that increment by 1
done
shift 1
done

If you want to copy recursive you can use cp -r <source folder> <destination folder>.
This will copy the content of the source file to the destination folder.
#!/bin/bash
read -p "Your folder name is $1 and the number of copies is $2. Press Y for yes N for no " -n 1 -r
if ! [[ $REPLY =~ ^[Yy]$ ]]
then
echo
echo "Rerun script and pass in the folder and how many copies yo would like (e.g folder 4)"
exit
fi
echo
echo "-------------COPY FILE----------------"
FOLDER=$1
if [ ! -d "$FOLDER" ]
then
echo "folder $FOLDER does not exist"
exit 1
fi
j=$2
for ((i=1; i<=j; i++))
do
cp -r "$1" "$1$i"; #copies the file i amount of times, and creates new files with names that increment by 1
echo "folder $1 copied to $1$i"
done
I changed the folder check from -f to -d, changed the variable nameing a bit (case), removed the list_files function and changed in the last for loop the i<$j to i<=j because the loop starts with 1. Then I added the -r and it works.

Related

Bash simple script copying files to specific folder + renaming to todays effective date

Good day,
I need your help in creating next script
Every day teacher uploading files in next format:
STUDENT_ACCOUNTS_20200217074343-20200217.xlsx
STUDENT_MARKS_20200217074343-20200217.xlsx
STUNDENT_HOMEWORKS_20200217074343-20200217.xlsx
STUDENT_PHYSICAL_20200217074343-20200217.xlsx
SUBSCRIBED_STUDENTS_20200217074343-20200217.xlsx
[file_name+todaydatetime-todaydate.xlsx]
But sometimes a teacher is not uploading these files and we need to do manual renaming the files received for the previous date and then copying every separate file to separate folder like:
cp STUDENT_ACCOUNTS_20200217074343-20200217.xlsx /incoming/A1/STUDENT_ACCOUNTS_20200318074343-20200318.xlsx
cp STUDENT_MARKS_20200217074343-20200217.xlsx /incoming/B1/STUDENT_ACCOUNTS_20200318074343-20200318.xlsx
.............
cp SUBSCRIBED_STUDENTS_20200217074343-20200217.xlsx /incoming/F1/SUBSCRIBED_STUDENTS_20200318074343-20200318.xlsx.
In two words - taking the files from previous date copying them to specific folder with a new timestamp.
#!/bin/bash
cd /home/incoming/
date=$(date '+%Y%m%d')
previousdate="$( date --date=yesterday '+%Y%m%d' )"
cp /home/incoming/SUBSCRIBED_STUDENTS_'$previousdate'.xlsx /incoming/F1/SUBSCRIBED_STUDENTS_'$date'.xlsx
and there could be case when teacher can upload one file and others not, how to do check for existing files?
Thanks for reading that, if you can help me i will ne really thankful - you will save plenty of manual work for me.
The process can be automated completely if your directory structure is known. If it follows some kind of pattern, do mention it here.
For the timing, this maybe helpful:
Filename "tscp"
#
# Stands for timestamped cp
#
tscp() {
local file1=$1 ; shift
local to_dir=$1 ; shift
local force_copy=$1 ; shift
local current_date="$(date '+%Y%m%d')"
if [ "${force_copy}" == "--force" ] ; then
cp "${file1}" "${to_dir}/$(basename ${file1%-*})-${current_date}.xlsx"
else
cp -n "${file1}" "${to_dir}/$( basename ${file1%-*})-${current_date}.xlsx"
fi
}
tscp "$#"
It's usage is as follows:
tscp source to_directory [-—force]
Basically the script takes 2 arguments and the 3rd one is optional.
First arg is source file path and second are is the directory path to where you want to copy (. if same directory).
By default this copy would be made if and only if destination file doesn't exist.
If you want to overwrite the destination file then pass a third arg —force.
Again, this can be refined much much more based on details provided.
Sample usage for now:
bash tscp SUBSCRIBED_STUDENTS_20200217074343-20200217.xlsx /incoming/F1/
will copy SUBSCRIBED_STUDENTS_20200217074343-20200217.xlsx to directory /incoming/F1/ with updated date if it doesn't exist yet.
UPDATE:
Give this a go:
#! /usr/bin/env bash
printf_err() {
ERR_COLOR='\033[0;31m'
NORMAL_COLOR='\033[0m'
printf "${ERR_COLOR}$1${NORMAL_COLOR}" ; shift
printf "${ERR_COLOR}%s${NORMAL_COLOR}\n" "$#" >&2
}
alias printf_err='printf_err "Line ${LINENO}: " '
shopt -s expand_aliases
usage() {
printf_err \
"" \
"usage: ${BASH_SOURCE##*/} " \
" -f copy_data_file" \
" -d days_before" \
" -m months_before" \
" -o" \
" -y years_before" \
" -r " \
" -t to_dir" \
>&2
exit 1
}
fullpath() {
local path="$1" ; shift
local abs_path
if [ -z "${path}" ] ; then
printf_err "${BASH_SOURCE}: Line ${LINENO}: param1(path) is empty"
return 1
fi
abs_path="$( cd "$( dirname "${path}" )" ; pwd )/$( basename ${path} )"
printf "${abs_path}"
}
OVERWRITE=0
REVIEW=0
COPYSCRIPT="$( mktemp "/tmp/copyscriptXXXXX" )"
while getopts 'f:d:m:y:t:or' option
do
case "${option}" in
d)
DAYS="${OPTARG}"
;;
f)
INPUT_FILE="${OPTARG}"
;;
m)
MONTHS="${OPTARG}"
;;
t)
TO_DIR="${OPTARG}"
;;
y)
YEARS="${OPTARG}"
;;
o)
OVERWRITE=1
;;
r)
REVIEW=1
COPYSCRIPT="copyscript"
;;
*)
usage
;;
esac
done
INPUT_FILE=${INPUT_FILE:-$1}
TO_DIR=${TO_DIR:-$2}
if [ ! -f "${INPUT_FILE}" ] ; then
printf_err "No such file ${INPUT_FILE}"
usage
fi
DAYS="${DAYS:-1}"
MONTHS="${MONTHS:-0}"
YEARS="${YEARS:-0}"
if date -v -1d > /dev/null 2>&1; then
# BSD date
previous_date="$( date -v -${DAYS}d -v -${MONTHS}m -v -${YEARS}y '+%Y%m%d' )"
else
# GNU date
previous_date="$( date --date="-${DAYS} days -${MONTHS} months -${YEARS} years" '+%Y%m%d' )"
fi
current_date="$( date '+%Y%m%d' )"
tmpfile="$( mktemp "/tmp/dstnamesXXXXX" )"
awk -v to_replace="${previous_date}" -v replaced="${current_date}" '{
gsub(to_replace, replaced, $0)
print
}' ${INPUT_FILE} > "${tmpfile}"
paste ${INPUT_FILE} "${tmpfile}" |
while IFS=$'\t' read -r -a arr
do
src=${arr[0]}
dst=${arr[1]}
opt=${arr[2]}
if [ -n "${opt}" ] ; then
if [ ! -d "${dst}" ] ;
then
printf_err "No such directory ${dst}"
usage
fi
dst="${dst}/$( basename "${opt}" )"
else
if [ ! -d "${TO_DIR}" ] ;
then
printf_err "No such directory ${TO_DIR}"
usage
fi
dst="${TO_DIR}/$( basename "${dst}" )"
fi
src=$( fullpath "${src}" )
dst=$( fullpath "${dst}" )
if [ -n "${OVERWRITE}" ] ; then
echo "cp ${src} ${dst}"
else
echo "cp -n ${src} ${dst}"
fi
done > "${COPYSCRIPT}"
if [ "${REVIEW}" -eq 0 ] ; then
${BASH} "${COPYSCRIPT}"
rm "${COPYSCRIPT}"
fi
rm "${tmpfile}"
Steps:
Store the above script in a file, say `tscp`.
Now you need to create the input file for it.
From you example, a sample input file can be like:
STUDENT_ACCOUNTS_20200217074343-20200217.xlsx /incoming/A1/
STUDENT_MARKS_20200217074343-20200217.xlsx /incoming/B1/
STUNDENT_HOMEWORKS_20200217074343-20200217.xlsx
STUDENT_PHYSICAL_20200217074343-20200217.xlsx
SUBSCRIBED_STUDENTS_20200217074343-20200217.xlsx /incoming/FI/
Where first part is the source file name and after a "tab" (it should be a tab for sure), you mention the destination directory. These paths should be either absolute or relative the the directory where you are executing the script. You may not mention destination directory if all are to be sent to same directory (discussed later).
Let's say you named this file `file`.
Also, you don't really have to type all that. If you have these files in the current directory, just do this:
ls -1 > file
(the above is ls "one", not "l".)
Now we have the `file` from above in which we didn't mention destination directory for all but only for some.
Let's say we want to move all other directories to `/incoming/x` and it exists.
Now script is to be executed like:
bash tscp -f file -t /incoming/x -r
Where `/incoming/x` is the default directory i.e. when none other directory is mentioned in `file`, your files are moved to this directory.
Now in the current directory a script named `copyscript` will be generated which will contain `cp` commands to copy all files. You can open a review `copyscript` and if the copying seems right, go ahead and:
bash copyscript
which will copy all the files and then you can:
rm copyscript
You need not generate to `copyscript` and can straight away go for a copy like:
bash tscp -f file -t /incoming/x
which won't generate any copyscript and copy straight away.
Previously `-r` caused the generation of `copyscript`.
I would recomment to use version with `-r` because that is a little safer and you will be sure that right copies are being made.
By default it would check for the previous day and rename to current date, but you can override that behaviour as:
bash tscp -f file -t /incoming/x -d 3
`-d 3` would look for 3 days back files in `file`.
By default copies won't overwrite i.e. if file at the destination already exists, copies won't be made.
If you want to overwrite, add flag `-o`.
As a conclusion I would advice to use:
bash tscp -f file -r
where file contains tab separated values like above for all.
Also, adding tscp to path would be a good idea after you are sure it works ok.
Also the scipt is made on mac and there is always a change of version clash of tools used. I would suggest to try the script on some sample data first to make sure script works right on your machine.

Nested for loop to enter and exit multiple directories Bash script

As an example, I have 7 directories each containing 4 files. The 4 files follow the following naming convention name_S#_L001_R1_001.fastq.gz. The sed command is to partially keep the unique file name.
I have a nested for loop in order to enter a directory and perform a command, exit the directory and proceed to the next directory. Everything seems to be working beautifully, however the code gets stuck on the last directory looping 4 times.
for f in /completepath/*
do
[ -d $f ] && cd "$f" && echo Entering into $f
for y in `ls *.fastq.gz | sed 's/_L00[1234]_R1_001.fastq.gz//g' | sort -u`
do
echo ${y}
done
done
Example output-
Entering into /completepath/m_i_cast_avpv_1
iavpvcast1_S6
Entering into /completepath/m_i_cast_avpv_2
iavpvcast2_S6
Entering into /completepath/m_i_int_avpv_1
iavpvint1_S5
Entering into /completepath/m_i_int_avpv_2
iavpvint2_S5
Entering into /completepath/m_p_cast_avpv_1
pavpvcast1_S8
Entering into /completepathd/m_p_int_avpv_1
pavpvint1_S7
Entering into /completepath/m_p_int_avpv_2
pavpvint2_S7
pavpvint2_S7
pavpvint2_S7
pavpvint2_S7
Any recommendations of how to correctly exit the inner loop?
It looks like /completepath/ contains some entries that are not directories. When the loop over /completepath/* sees something that's not a directory, it doesn't enter it, thanks to the [ -d $f ] check.
But it still continues to run the next for y in ... loop.
At that point the script is still in the previous directory it has seen.
One way to solve that is to skip the rest of the loop when $f is not a directory:
if [ -d $f ]; then
cd "$f" && echo Entering into $f
else
continue
fi
There's an even better way. By writing /completepath/*/ only directory entries will be matched, so you can simplify your loop to this:
for f in /completepath/*/
do
cd "$f" && echo "Entering into $f" || { echo "Error: could not enter into $f"; continue; }
for y in $(ls *.fastq.gz | sed 's/_L00[1234]_R1_001.fastq.gz//g' | sort -u)
do
echo ${y}
done
done

recover files deleted with rm command

Please if I run the command
# CREATE TRASH FOLDER
if ! [ -d "$HOME/deleted" ] ; then
mkdir $HOME/deleted
fi
TRASH=$HOME/deleted
mv $# $TRASH
To move file or directory to the trash created. what is the possible command i can run to recover same file to the original directory
If you create a deleted directory like this, you will probably get some unexpected behavior. For example:
rm_script test.txt
cd ../other_directory
rm_script test.txt
will create a single file test.txt with the content of the one in other_dir. Even more fun when you start rm_scripting/moving directories.
Knowing this, and referring once more to trash-cli (see comment Aserre), you might:
if ! [ -d "$HOME/deleted" ] ; then
echo "What a pity! No deleted directory."
echo "Your file(s) is/are lost forever!"
exit 361
fi
TRASH=$HOME/deleted
for file in $# ; do
if [ -f "$TRASH/$file" ] ; then
cp "$TRASH/$file" .
else
echo "Hmm,.. I cannot find $file."
fi
done
This also may have some unwanted results, like removing the file from one directory and un-deleting it in another.
I just do the reverse of the same command line
function undo () {
echo -n "Do you want to recover $*? "
read ANSWER
if [ "$ANSWER" = "y" ]; then
mv $TRASH/$# $PWD
echo "$# successfully recovered"
else
echo "Oops, request denied"
fi
}

move command performs move in source directory too

I have written a shell script to move files from source directory to destination directory.
/home/tmp/ to /home/from/
The move happens correctly but it displays message
mv: /home/tmp/testfile_retry_17072017.TIF
/home/tmp/testfile_retry_17072017.TIF are identical.
and if source directory is empty it displays
mv: cannot rename /home/tmp/* to /home/from/*
for file in /home/tmp/*
if [ -f "$file" ]
then
do
DIRPATH=$(dirname "${file}")
FILENAME=$(basename "${file}")
# echo "Dirpath = ${DIRPATH} Filename = ${FILENAME}"
mv "${DIRPATH}/"${FILENAME} /home/from
echo ${FILENAME} " moved to from directory"
done
else
echo "Directory is empty"
fi
You should use find instead of /home/tmp/* as shown.
for file in $(find /home/tmp/ -type f)
do
if [ -f "$file" ]
then
DIRPATH=$(dirname "${file}")
FILENAME=$(basename "${file}")
# echo "Dirpath = ${DIRPATH} Filename = ${FILENAME}"
mv "${DIRPATH}/"${FILENAME} /home/from
echo ${FILENAME} " moved to from directory"
else
echo "Directory is empty"
fi
done
You have things a bit out of order with:
for file in /home/tmp/*
if [ -f "$file" ]
then
do
Of course "$file" will exist -- you are looping for file in /home/tmp/*. It looks like you intended
for file in /home/tmp/*
do
FILENAME=$(basename "${file}")
if [ ! -f "/home/from/$FILENAME" ] ## if it doesn't already exist in dest
then
Note: POSIX shell include parameter expansions that allow you to avoid calling dirname and basename. Instead you can simply use "${file##*/}" for the filename (which just says remove everything from the left up to (and including) the last /). That is the only expansion you need (as you already know the destination directory name). This allows you to check [ -f "$dest/${f##*/}" ] to determine if a file with the same name you are moving already exists in /home/from
You could use that to your advantage with:
src=/home/tmp ## source dir
dst=/home/from ## destination dir
for f in "$src"/* ## for each file in src
do
[ "$f" = "$src/*" ] && break ## src is empty
if [ -f "$dst/${f##*/}" ] ## test if it already exists in dst
then
printf "file '%s' exists in '%s' - forcing mv.\n" "${f##*/}" "$dst"
mv -f "$f" "$dst" ## use -f to overwrite existing
else
mv "$f" "$dst" ## regular move otherwise
fi
done
There is a great resource for checking your shell code called ShellCheck.net. Just type your code into the webpage (or paste it) and it will analyze your logic and variable use and let you know where problem are identified.
Look things over and let me know if you have further questions.

performing multiple backups at the same time with a bash script

Create a script to backup a file or directory tree by making a zip of the file(s) and copying it $HOME/Backups. The zipfile name should include what it is backing up, and the date the file was created. The script should take a random number of arguments specifying what to backup. If it is not given at least one item to include in the backup, it should complain. Ive got most of it to work but im having issues with multiple files ie file1 file2 to backup at the same time
#!/bin/bash
clear
echo
echo "Use this script to backup files to your home/backups directory"
echo
ls -la
echo
echo "================================================================"
echo
echo -n "Input file(s)/dir to backup: " ; read filez
while [ "$filez" == "" ] ; do
echo -n "You didnt input a filename, try again: " ; read filez
done
while [ ! -e "$filez" ] ; do
echo -n "No such file/dir, try again: " ; read filez
while [ "$filez" == "" ] ; do
echo -n "You didnt input a filename, try again: " ; read filez
done
done
echo
echo "================================================================"
echo
echo -n "Input name of backup file you wish to create(date automatically included): " ; read filezname
while [ "$filezname" == "" ] ; do
echo -n "You didnt input a filename, try again: " ; read filezname
done
zip -r $HOME/backups/$filezname"_$(date +%F)" $filez
Personally, I'm a fan of bash scripts taking arguments right from the command line:
script arg1 arg2 arg3 ...
Bash takes arguments using the special array $#, and you can do arguments processing with shift. Something like this:
#!/bin/bash
filezname=$1
shift
filez=""
for file in "$#"
do
filez="$filez $file"
done
if [[ $filez == "" ]]
then
echo "Give me argz! Nom nom nom!"
exit 0
fi
# Do stuff
What this does is it takes the first argument, takes it as the name of the zip file, and then slurps the rest of the filenames you want to zip up into a big long space-separated string that you can play with.

Resources