For class we have to work on a remote server that the school hosts. So far I have made a lot of files on the server and I would like to back them up in case I want to transfer them to my laptop or in case I accidentally delete a directory or make a silly error. I found a tutorial and a script to back up the file and I decided to modify it so that it would determine what directory it's in (which will be the main user's) and the cd to the Documents. It also creates the directory Backups if it doesn't exist. I am still pretty new to this sort of scripting and any additional advice or post links would be greatly appreciated.
Code:
#!/bin/bash
#######################################################
## Simple backup script..
## Created by Matthew Brunt: (openblue555#gmail.com)
## Licensed under GNU GPL v3 or later, at your option.
## http://www.gnu.org/licenses/gpl.html
##
## Further edited by Michael Garrison to backup the
## directory it is located in and print the contents.
#######################################################
mkdir -p Backup
#Defines our output file
OUTPUT= $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz
#Defines our directory to backup
BUDIR=$( cd Desktop && pwd )
#Display message about starting the backup
echo "Starting backup of directory $BUDIR to file $OUTPUT"
#Start the backup
tar -cZf $OUTPUT $BUDIR
#Checking the status of the last process:
if [ $? == 0 ]; then
#Display confirmation message
echo "The file:"
echo $OUTPUT
echo "was created as a backup for:"
echo $BUDIR
echo ""
echo "Items that were backed up include:"
for i in $BUDIR; do
echo $i
done
echo ""
else
#Display error message message
echo "There was a problem creating:"
echo $OUTPUT
echo "as a backup for:"
echo $BUDIR
fi
I know that the original script works and it worked until I changed the $OUTPUT variable. I currently get the following result:
./backup.sh
./backup.sh: line 15: /Users/mgarrison93/Backup/backup_20121004.tar.gz: No such file
or directory
Starting backup of directory /Users/mgarrison93/Desktop to file
tar: no files or directories specified
There was a problem creating:
as a backup for:
/Users/mgarrison93/Desktop
I can see that it is not accepting the file name, but I don't know how to correct this.
I just tried changing $OUTPUT to /Backups/file-name.tar.gz which I originally had and it works fine. The problem seems to be $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz. Just not sure what is wrong.
Consider these two entirely different pieces of bash syntax: first, you have the syntax for setting a variable to a value permanently (in the current script),
<variable>=<value>
and then there is the syntax for running a command with a variable temporarily set to a value ,
<variable>=<value> <command> <argument> ...
The difference between these two is the space. After the =, once bash runs into an unquoted space, it takes that to mean that the <value> has ended, and anything after it is interpreted as the <command>.
In this line of your script,
OUTPUT= $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz
you have a space after OUTPUT=. bash interprets that to mean that OUTPUT is to be (temporarily) set to the empty string, and the rest of the line, i.e. the result of $( cd Backup && pwd )/backup_$(date +%Y%m%d).tar.gz, is a command and arguments to be run while OUTPUT is equal to the empty string.
The solution is to remove the space. That way bash will know that you're trying to assign the rest of the line as a value to the variable.
Related
I'm writing a bash script to send files from a linux server to a remote Windows FTP server.
I would like to check using FTP if the folder where the file will be stored exists before attempting to create it.
Please note that I cannot use SSH nor SCP and I cannot install new scripts on the linux server. Also, for performance issues, I would prefer if checking and creating the folders is done using only one FTP connection.
Here's the function to send the file:
sendFile() {
ftp -n $FTP_HOST <<! >> ${LOCAL_LOG}
quote USER ${FTP_USER}
quote PASS ${FTP_PASS}
binary
$(ftp_mkdir_loop "$FTP_PATH")
put ${FILE_PATH} ${FTP_PATH}/${FILENAME}
bye
!
}
And here's what ftp_mkdir_loop looks like:
ftp_mkdir_loop() {
local r
local a
r="$#"
while [[ "$r" != "$a" ]]; do
a=${r%%/*}
echo "mkdir $a"
echo "cd $a"
r=${r#*/}
done
}
The ftp_mkdir_loop function helps in creating all the folders in $FTP_PATH (Since I cannot do mkdir -p $FTP_PATH through FTP).
Overall my script works but is not "clean"; this is what I'm getting in my log file after the execution of the script (yes, $FTP_PATH is composed of 5 existing directories):
(directory-name) Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
To solve this, do as follows:
To ensure that you only use one FTP connection, you create the input (FTP commands) as an output of a shell script
E.g.
$ cat a.sh
cd /home/test1
mkdir /home/test1/test2
$ ./a.sh | ftp $Your_login_and_server > /your/log 2>&1
To allow the FTP to test if a directory exists, you use the fact that "DIR" command has an option to write to file
# ...continuing a.sh
# In a loop, $CURRENT_DIR is the next subdirectory to check-or-create
echo "DIR $CURRENT_DIR $local_output_file"
sleep 5 # to leave time for the file to be created
if (! -s $local_output_file)
then
echo "mkdir $CURRENT_DIR"
endif
Please note that "-s" test is not necessarily correct - I don't have acccess to ftp now and don't know what the exact output of running DIR on non-existing directory will be - cold be empty file, could be a specific error. If error, you can grep the error text in $local_output_file
Now, wrap the step #2 into a loop over your individual subdirectories in a.sh
#!/bin/bash
FTP_HOST=prep.ai.mit.edu
FTP_USER=anonymous
FTP_PASS=foobar#example.com
DIRECTORY=/foo # /foo does not exist, /pub exists
LOCAL_LOG=/tmp/foo.log
ERROR="Failed to change directory"
ftp -n $FTP_HOST << EOF | tee -a ${LOCAL_LOG} | grep -q "${ERROR}"
quote USER ${FTP_USER}
quote pass ${FTP_PASS}
cd ${DIRECTORY}
EOF
if [[ "${PIPESTATUS[2]}" -eq 1 ]]; then
echo ${DIRECTORY} exists
else
echo ${DIRECTORY} does not exist
fi
Output:
/foo does not exist
If you want to suppress only the messages in ${LOCAL_LOG}:
ftp -n $FTP_HOST <<! | grep -v "Cannot create a file" >> ${LOCAL_LOG}
I'm not a pro in shell scripting, thats why I ask here :).
Let's say I got a folder. I need a script that monitors that folder for new files (no prefix name of files is given). When a new file gets copied into that folder, another script should start. Has the second script processed the file successfully the file should be deleted.
I hope you can give me some ideas on how to achieve such script :)
Thank you very much in advance.
Thomas
Try this:
watcher.sh:
#!/bin/bash
if [ -z $1 ];
then
echo "You need to specify a dir as argument."
echo "Usage:"
echo "$0 <dir>"
exit 1
fi
while true;
do
for a in $(ls -1 $1/* 2>/dev/null);
do
otherscript $a && rm $a #calls otherscript with the file a as argument and removes it if otherscript returned something non-zero
done
sleep 2s
done
Don't forget to make it executable
chmod +x ./watcher.sh
call it with:
./watcher.sh <dirname>
try inotify(http://man7.org/linux/man-pages/man7/inotify.7.html)
or you may need to install inotify-tools (http://www.ibm.com/developerworks/linux/library/l-ubuntu-inotify/) to use it by shell.
I have a cron job which runs every 30 minutes to generate log files with time-stamp like this:
test20130215100531.log,
test20130215102031.log
I would like to create one folder daily with date time-stamp and push log files in to respective date folder when generated.
I need to achieve this on AIX server with bash.
Maybe you are looking for a script like this:
#!/bin/bash
shopt -s nullglob # This line is so that it does not complain when no logfiles are found
for filename in test*.log; do # Files considered are the ones starting with test and ending in .log
foldername=$(echo "$filename" | awk '{print (substr($0, 5, 8));}'); # The foldername is characters 5 to 13 from the filename (if they exist)
mkdir -p "$foldername" # -p so that we don't get "folder exists" warning
mv "$filename" "$foldername"
echo "$filename $foldername" ;
done
I only tested with your sample, so do a proper testing before using in a directory that contains important stuff.
Edit in response to comments:
Change your original script to this:
foldername=$(date +%Y%m%d)
mkdir -p /home/app/logs/"$foldername"
sh sample.sh > /home/app/logs/"$foldername"/test$(date +%Y%m%d%H%M%S).log
Or if the directory is created somewhere else, just do this:
sh sample.sh > /home/app/logs/$(date +%Y%m%d)/test$(date +%Y%m%d%H%M%S).log
You should use logrotate! It can do this for you already, and you can just write to the same log file.
Check their man pages for info:
http://linuxcommand.org/man_pages/logrotate8.html
I have the following script sample:
#!/bin/bash
# Aborts the script on "simple command failure" (does not cover pipes)
set -e
# Makes sure we do not run the script outside the correct directory (i.e. the backup directory)
projects_directory='~/projects'
backup_drectory="${projects_directory}/backup/"
echo "Backup directory: ${backup_drectory}"
if [ ! -d "$projects_directory" ]; then
mkdir "$projects_directory"
echo "${projects_directory} created successfully"
fi
Which fails miserably with the following output:
Backup directory: ~/projects/backup/
mkdir: cannot create directory `~/projects': No such file or directory
I do not understand why. If I enter the mkdir ~/projects command manually in a Terminal, the directory gets created. Any suggestion is most welcome.
Remove the single quotes:
projects_directory=~/projects
The quoting prevents the shell from expanding the ~ character.
I am attempting to write a bash script that changes directory and then runs an existing script in the new working directory.
This is what I have so far:
#!/bin/bash
cd /path/to/a/folder
./scriptname
scriptname is an executable file that exists in /path/to/a/folder - and (needless to say), I do have permission to run that script.
However, when I run this mind numbingly simple script (above), I get the response:
scriptname: No such file or directory
What am I missing?! the commands work as expected when entered at the CLI, so I am at a loss to explain the error message. How do I fix this?
Looking at your script makes me think that the script you want to launch a script which is locate in the initial directory. Since you change you directory before executing it won't work.
I suggest the following modified script:
#!/bin/bash
SCRIPT_DIR=$PWD
cd /path/to/a/folder
$SCRIPT_DIR/scriptname
cd /path/to/a/folder
pwd
ls
./scriptname
which'll show you what it thinks it's doing.
I usually have something like this in my useful script directory:
#!/bin/bash
# Provide usage information if not arguments were supplied
if [[ "$#" -le 0 ]]; then
echo "Usage: $0 <executable> [<argument>...]" >&2
exit 1
fi
# Get the executable by removing the last slash and anything before it
X="${1##*/}"
# Get the directory by removing the executable name
D="${1%$X}"
# Check if the directory exists
if [[ -d "$D" ]]; then
# If it does, cd into it
cd "$D"
else
if [[ "$D" ]]; then
# Complain if a directory was specified, but does not exist
echo "Directory '$D' does not exist" >&2
exit 1
fi
fi
# Check if the executable is, well, executable
if [[ -x "$X" ]]; then
# Run the executable in its directory with the supplied arguments
exec ./"$X" "${#:2}"
else
# Complain if the executable is not a valid
echo "Executable '$X' does not exist in '$D'" >&2
exit 1
fi
Usage:
$ cdexec
Usage: /home/archon/bin/cdexec <executable> [<argument>...]
$ cdexec /bin/ls ls
ls
$ cdexec /bin/xxx/ls ls
Directory '/bin/xxx/' does not exist
$ cdexec /ls ls
Executable 'ls' does not exist in '/'
One source of such error messages under those conditions is a broken symlink.
However, you say the script works when run from the command line. I would also check to see whether the directory is a symlink that's doing something other than what you expect.
Does it work if you call it in your script with the full path instead of using cd?
#!/bin/bash
/path/to/a/folder/scriptname
What about when called that way from the command line?