I wrote a bash script to read a bunch of CSV files from a folder and create a backup file in a separate backups directory.
#!/bin/sh
files=$(ls /../test/*.csv 2> /dev/null | wc -l)
if [ **"$files" != "0"** ]
then
# What to backup.
backup_files="/../test/test2"
# Where to backup to.
dest="../test/"
# Create archive filename.
day=$(date +%A)
name="some-file.csv"
archive_file="$name-day.tgz"
# Print start status message.
echo "Backing up $backup_files to $dest/$archive_file"
echo
date
# Backup the files using tar.
tar czf $dest/$archive_file $backup_files
# Print end status message.
echo
echo "Backup finished"
date
else
echo nothing there
break
fi
The script should go through each CSV file (file(1).csv, file(2).csv, etc) and write to my backup file one by one, but the problem is that it only writes from the last file (file(3).csv).
The reason is you are not expanding the $day variable. Use this:
archive_file="$name-$day.tgz"
Related
I am new to Unix scripting, I am trying to create Unix script since one week but I couldn't. Please help me in this.
I have a number of different files more than 100 (all the filenames are different) which the filename contains the date string(ex: 20171101)in the directory. I want compare these filename dates with my input date (today - 10days =20171114),with the files in the directories only using filename string date if it is less than with my input date then I have to delete the file. could anyone please help on this. Thanks
My script:
ten_days_ago=$(date -d "10 days ago" +%Y%m%d)
cd "$destination_dir" ;
ls *.* | awk -F '-' '{print $2}'
ls *.* | awk -F '-' '{print $2}' > removal.txt
while read filedate
do
if [ "$filedate" -lt "$ten_days_ago" ] ; then
cd "$destination_dir" ;
rm *-"$filedate"*
echo "deletion done"
fi
done <removal.txt
this script is working fine. but I need to send a email as well - if the deletion has been done then -one pass email else fail email.
but here within while loop if I am writing the emails then that will iterate
You're probably trying to pipe to mail from the middle of your loop. (Your question should really show this code, otherwise we can't say what's wrong.) A common technique is to redirect the loop's output to a file, and then send that. (Using a temporary file is slightly ugly, but avoids sending an empty message when there is no output from the loop.)
Just loop over the files and decide which to remove.
#!/bin/bash
t=$(mktemp -t tendays.XXXXXXXX) || exit
# Remove temp file if interrupted, or when done
trap 'rm -f "$t"' EXIT HUP INT TERM
ten_days_ago=$(date -d "10 days ago" +%Y%m%d)
for file in *-[1-9]*[1-9]-*; do
date=${file#*-} # strip prefix up through first dash
date=${date%-*} # strip from last dash from the previous result
if [ "$date" -lt "$ten_days_ago" ]; then
rm -v "$file"
fi
done >"$t" 2>&1
test -s "$t" || exit # Quit if empty
mail -s "Removed files" recipient#example.net <"$t"
I removed the (repeated!) cd so this can be run in any directory -- just switch to the directory you want before running the script. This also makes it easier to test in a directory with a set of temporary files.
Collecting the script's standard error also means the mail message will contain any error messages if rm fails for some reason or you have other exceptions.
By the by you should basically never use ls in scripts.
I have this snippet:
#!/bin/bash
parent=/parent
newfolder=/newfolder
mkdir "$newfolder"
for folder in "$parent"/*; do
if [[ -d $folder ]]; then
foldername="${folder##*/}"
for file in "$parent"/"$foldername"/*; do
filename="${file##*/}"
newfilename="$foldername"_"$filename"
cp "$file" "$newfolder"/"$newfilename"
done
fi
done
I do need to turn it around in a way that the copied files would be named after the folder they are being moved to (e.g. moving to the /root/Case22 files would be renamed to case22_1.jpg, case22_2.docx, case22_3.JPG etc). The files would be copied from USB and both destination and source directries would be entered by the user. I have written everything else and it works apart from actual renaming and thought I could adapt this snippet.
thanks
p.s. the snippet is written by Jahid and found on stackoverflow
you can try something like this;
#!/bin/bash
parent=/root
a=1
for file in $parent/Case22*; do
filename="${file%.*}"
extension="${file##*.}"
newfilename=$(printf "$filename"_"$a"."$extension")
mv -- "$file" "$newfilename"
let a=a+1
done
Thanks for the help. I have found the solution and thought I might post it here in case someone else will be looking at this.
As the title suggests I needed a Linux shell script to copy and rename multiple files keeping original directory tree (the file source and archive locations would be specified by the user of the script). Here is the code that I came up with after few days research of different sources (it includes a trap so only one instance of script would be running at a time):
lockdir=/var/tmp/mylock #directory for lock file creation
pidfile=/var/tmp/mylock/pid #directory to get the process ID number
if ( mkdir ${lockdir} ) 2> /dev/null; then #main argument to create lock file
echo $$ > $pidfile #if successful script will proceed, otherwise it will skip to the else part of the statement at the end of the script
trap 'rm -rf "$lockdir"; exit $?' INT TERM EXIT #trap to capture reasons of script termination and removal of the lock file so it could be launched again
#start of the main script body, part of successful "if" statement
# read entry_for_testing_only #request for user entry to keep script running and try to run another script instance
findir="$2/$(basename "$1")" #variable that defines final directory to which files from USB will be copied
if [ ! -d "$1" ]; then #testing if first directory entry is a valid directory’’
echo "$1" "is not a directory"
echo ""
exit
else
if [ ! -d "$2" ]; then #testing if second entry is a valid directory
echo "archive directory non existant"
exit
else
if [ -d "$findir" ] && [ "$(ls -A "$findir")" ]; then #testing if second entry directory contains the same name folders and if the folders are empty - to avoid file overwriting
echo "such folder already there and it's not empty"
exit
else
if [ ! -d "$findir" ] ; then #last archive directory argument to create final archive directory
mkdir "$findir"
else true
fi
fi
fi
fi
rsync -a "$1"/ "$findir" #command to copy all files from the source to the archive retaining the directory tree
moved_files="$(find "$findir" -type f)" #variable that finds all files that have been copied to the archive directory
for file in $moved_files; do #start of the loop that renames copied files
counter="$((counter+1))" #incrementation variable
source_name="$(basename "$1")" #variable that captures the name of the source directory
new_name="$source_name"_"$counter" #variable that puts start of the file name and incrementation element together
if echo "$file" | grep "\." #argument that captures the extension of the file
then
extension="$(echo "$file" | cut -f2 -d. )"
else
extension=
fi
full_name="$new_name"."$extension" #variable that defines the final new name of the file
dir="$(dirname "${file}")" #variable that captures the directorry address of currently looped file
mv "$file" "$dir/$full_name" #move command to rename currently looped file with the final new name
done
#end of the main script body, unsuccessful "if" statement continues here
else
echo "Another instance of this script is already running. PID: $(cat $pidfile)"
fi
Anyone able to help me out? I have a shell script I am working on but for the loop below the command after "echo "first file is $firstbd" is not being executed.. the $PROBIN/proutil ?? Not sure why this is...
Basically I have a list of files in a directory (*.list), I grab them and read the first line and pass it as a parameter to the cmdlet then move the .list and the content of the .list to another directory (the .list has a list of files with full path).
for i in $(ls $STAGEDIR/*.list); do
echo "Working with $i"
# grab first .bd file
firstbd=`head -1 $i`
echo "First file is $firstbd"
$PROBIN/proutil $DBENV/$DBNAME -C load $firstbd tenant $TENANT -dumplist $STAGEDIR/$i.list >> $WRKDIR/$i.load.log
#move the list and its content to finished folder
binlist=`cat $i`
for movethis in $binlist; do
echo "Moving file $movethis to $STAGEDIR/finished"
mv $movethis $STAGEDIR/finished/
done
echo "Finished working with list $i"
echo "Moving it to $STAGEDIR/finished"
mv $i $STAGEDIR/finished/
done
The error I was getting is..
./tableload.sh: line 107: /usr4/dlc/bin/proutil /usr4/testdbs/xxxx2 -C load /usr4/dumpdir/xxxxx.bd tenant xxxxx -dumplist /usr4/dumpdir/PUB.xxxxx.list >> /usr4/dumpdir/PUB.xxxx.list.load.log: A file or directory in the path name does not exist... however if I run "/usr4/dlc/bin/proutil"
The fix was to remove ">> $WRKDIR/$i.load.log".. the binary utility wouldn't run when trying to output results to file.. strange..
A couple of really bad practices here
parse the output of ls
not quoting variables
iterating the lines of a file with cat and for
As shelter comments, you don't check that you've created all the directories in the path for your log file.
A rewrite:
for i in "$STAGEDIR"/*.list; do
echo "Working with $i"
# grab first .bd file
firstbd=$(head -1 "$i")
echo "First file is $firstbd"
# ensure the output directory exists
logfile="$WRKDIR/$i.load.log"
mkdir -p "$(dirname "$logfile")"
"$PROBIN"/proutil "$DBENV/$DBNAME" -C load "$firstbd" tenant "$TENANT" -dumplist "$STAGEDIR/$i.list" >> "$logfile"
# move the list and its content to finished folder
while IFS= read -r movethis; do
echo "Moving file $movethis to $STAGEDIR/finished"
mv "$movethis" "$STAGEDIR"/finished/
done < "$i"
echo "Finished working with list $i"
echo "Moving it to $STAGEDIR/finished"
mv "$i" "$STAGEDIR"/finished/
done
I'm trying to gzip a file using a script but it will not work and continues to throw errors. Can someone please giveme some guidance on what is wrong with this script?
DEFAULTDIRECTORY=”/Backup”
if [ -d "$DEFAULTDIRECTORY" ]; then
mkdir -p /backup
fi # Makes directory if the directory does not exist
# Set the timestamp for the backup
TIMESTAMP=`date +%Y%m%d.%H%M`
# let the user choose what they want to backup
echo -n "Select the file or directory you want to backup"
read Chosendata
# read the backup file name file
echo -n "Select the file name"
read FNAME
# start the backup.
echo -e "Starting backup"
# compress the directory and files, direct the tar.gz file to your destination directory
tar -vczf ${FNAME}-${TIMESTAMP}.tar.gz ${Chosendata} > ${DEFAULTDIRECTORY}
# end the backup.
echo -e "Backup complete"
Redirecting output from the tar command to DEFAULTDIRECTORY isn't doing what the comment specifies.
I think what you want to do is save the file in the DEFAULTDIRECTORY.
Change the line
tar -vczf ${FNAME}-${TIMESTAMP}.tar.gz ${Chosendata} > ${DEFAULTDIRECTORY}
to
tar -vczf $DEFAULTDIRECTORY/${FNAME}-${TIMESTAMP}.tar.gz ${Chosendata}
you need to negate this test if [ -d "$DEFAULTDIRECTORY" ]; then -> if [ ! -d "$DEFAULTDIRECTORY" ]; then. You should not use redirect with tar command. instead you should prefix your tar.gz file with it:
tar -vczf $DEFAULTDIRECTORY/${FNAME}-${TIMESTAMP}.tar.gz ${Chosendata}
Try this:
#!/bin/bash
DEFAULTDIRECTORY="/Backup"
# Makes directory if the directory does not exist
# Here you had an inverted statement: use "if [ ! -d ... ] then" or "[ -d ... ] ||"
[ -d "${DEFAULTDIRECTORY}" ] || mkdir -p "${DEFAULTDIRECTORY}"
# Set the timestamp for the backup
# For subshell command you can use `...` or $(...)
TIMESTAMP=$(date +%Y%m%d.%H%M)
# Let the user choose what they want to backup
# You can use the -p (prompt) option instead of using echo
read -p "Select the file or directory you want to backup: " CHOSENDATA
# Read the backup file name
read -p "Select the file name: " FILENAME
# Start the backup.
echo "Starting backup"
# Compress the directory, direct the tar.gz file to your destination directory
# tar 'create' ' zip' 'verbose' 'force' <The output filename> <The data you want to backup>
tar -czvf ${DEFAULTDIRECTORY}/${FILENAME}-${TIMESTAMP}.tar.gz ${CHOSENDATA}
# End the backup.
echo "Backup complete"
For class we have to work on a remote server that the school hosts. So far I have made a lot of files on the server and I would like to back them up in case I want to transfer them to my laptop or in case I accidentally delete a directory or make a silly error. I found a tutorial and a script to back up the file and I decided to modify it so that it would determine what directory it's in (which will be the main user's) and the cd to the Documents. It also creates the directory Backups if it doesn't exist. I am still pretty new to this sort of scripting and any additional advice or post links would be greatly appreciated.
Code:
#!/bin/bash
#######################################################
## Simple backup script..
## Created by Matthew Brunt: (openblue555#gmail.com)
## Licensed under GNU GPL v3 or later, at your option.
## http://www.gnu.org/licenses/gpl.html
##
## Further edited by Michael Garrison to backup the
## directory it is located in and print the contents.
#######################################################
mkdir -p Backup
#Defines our output file
OUTPUT= $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz
#Defines our directory to backup
BUDIR=$( cd Desktop && pwd )
#Display message about starting the backup
echo "Starting backup of directory $BUDIR to file $OUTPUT"
#Start the backup
tar -cZf $OUTPUT $BUDIR
#Checking the status of the last process:
if [ $? == 0 ]; then
#Display confirmation message
echo "The file:"
echo $OUTPUT
echo "was created as a backup for:"
echo $BUDIR
echo ""
echo "Items that were backed up include:"
for i in $BUDIR; do
echo $i
done
echo ""
else
#Display error message message
echo "There was a problem creating:"
echo $OUTPUT
echo "as a backup for:"
echo $BUDIR
fi
I know that the original script works and it worked until I changed the $OUTPUT variable. I currently get the following result:
./backup.sh
./backup.sh: line 15: /Users/mgarrison93/Backup/backup_20121004.tar.gz: No such file
or directory
Starting backup of directory /Users/mgarrison93/Desktop to file
tar: no files or directories specified
There was a problem creating:
as a backup for:
/Users/mgarrison93/Desktop
I can see that it is not accepting the file name, but I don't know how to correct this.
I just tried changing $OUTPUT to /Backups/file-name.tar.gz which I originally had and it works fine. The problem seems to be $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz. Just not sure what is wrong.
Consider these two entirely different pieces of bash syntax: first, you have the syntax for setting a variable to a value permanently (in the current script),
<variable>=<value>
and then there is the syntax for running a command with a variable temporarily set to a value ,
<variable>=<value> <command> <argument> ...
The difference between these two is the space. After the =, once bash runs into an unquoted space, it takes that to mean that the <value> has ended, and anything after it is interpreted as the <command>.
In this line of your script,
OUTPUT= $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz
you have a space after OUTPUT=. bash interprets that to mean that OUTPUT is to be (temporarily) set to the empty string, and the rest of the line, i.e. the result of $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz, is a command and arguments to be run while OUTPUT is equal to the empty string.
The solution is to remove the space. That way bash will know that you're trying to assign the rest of the line as a value to the variable.