I need to synchronize files in my computer with server to which I have only FTP access and I would prefer a single line to add to crontab which checks every day that new or updated files are transferred (overwritten).
The question has been asked years ago but no simple answer was forthcoming and I want to know if there are better solutions today than ncftput, wput etc. which do not allow to
ncftpput -R -z -u "USER" -p "PASS" webxx.at /dir/ /source/
is rumored to work, but the -z switch seems "of label" use. My experiments seem to indicate that times are not reliable checked.
Just put this code into a file FtpSync.sh and add this file into your crontab.
You can adjust the parameters in the file. I tried to create "speaking" parameters so that they explain themselves.
A call of this script will either download or upload files (depending on the parameter CopyServerdataToLocal). If you want to do both, simply start the script twice with different parameters (or create two script files).
FtpSync.sh
#!/bin/bash
# Michael Hutter, 20.11.2021
# This Script can be used to synchronize a remote FTP directory and a local directory
HOST='ftp.mywebspace.de'
USER='web1234567f1'
PASS='YourSecretPwd'
SERVERFOLDER='/FolderOnWebspace'
PCFOLDER='/home/michael/ftpsync/MyLocalFolder'
CopyMoreThanOneFileSimultaneously="--parallel=10"
CopyServerdataToLocal=1 # 0=Upload, 1=Download
IgnoreSubdirectories=1
ContinuePartiallyTransmittedFiles=0
DontOverwriteNewerExistingFiles=0 # If this is used ContinuePartiallyTransmittedFiles will not work!
DeleteAdditionalFilesInDestinationDir=0 # Deletes files in DestDir which are not present in SourceDir
RemoveSourceFilesAfterTransfer=0 # Moves the files instead of copying them
ExcludeParams='--exclude-glob .* --exclude-glob .*/' # Exclude (hidden) files and direcories -> starting with a dot
OnlyShowChangesButDontChangeFiles=0 # DryRun mode
OutputAsMuchInfoAsPossible=1 # Verbose mode
################################################################################
if [ $CopyServerdataToLocal -eq 1 ]; then
if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
echo "Modus=Download"
fi
DIRECTORIES="$SERVERFOLDER $PCFOLDER"
else
if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
echo "Modus=Upload"
fi
REVERSE='--reverse'
DIRECTORIES="$PCFOLDER $SERVERFOLDER"
fi
if [ $IgnoreSubdirectories -eq 1 ]; then
IGNORESUBDIRS='--no-recursion'
fi
if [ $ContinuePartiallyTransmittedFiles -eq 1 ]; then
CONTINUEFILES='--continue'
fi
if [ $DontOverwriteNewerExistingFiles -eq 1 ]; then
ONLYNEWER='--only-newer'
fi
if [ $DeleteAdditionalFilesInDestinationDir -eq 1 ]; then
DELETE='--delete'
fi
if [ $RemoveSourceFilesAfterTransfer -eq 1 ]; then
REMOVE='--Remove-source-files'
fi
if [ $OnlyShowChangesButDontChangeFiles -eq 1 ]; then
DRYRUN='--dry-run'
fi
if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
VERBOSE='--verbose'
fi
lftp -f "
open $HOST
user $USER $PASS
lcd $PCFOLDER
mirror $DRYRUN $REVERSE $IGNORESUBDIRS $DELETE $REMOVE $CONTINUEFILES $ONLYNEWER $CopyMoreThanOneFileSimultaneously --use-cache $ExcludeParams $VERBOSE $DIRECTORIES
bye
"
Related
I'm currently writing a bash script which should install all the software that I need. The process looks promising so far: First, I write a "software-list.txt" file which contains all dependencies for multiple distros. Afterwards bash split these values into an array and reads the corresponding value out of it. Finally the script should combine the distro package manager name (e.g. dnf, if I'm using Fedora Linux) with the operator ("install") with the arguments (which are the software packages).One last info: All variable names which don't appear in the source code, we're defined beforehand
The script looks like this:
One last info: All variable names which don't appear in the source code, were defined beforehand
case "$DISTRO_NAME" in
"Fedora")
PROGRAMM="dnf"
CSV_INDEX=0;;
"Debian")
PROGRAMM="apt-get"
CSV_INDEX=1;;
esac
# Read all required packages
while IFS= read -r line
do
IFS=','
LINE=($line)
if [ $CURR_LINE_INDEX -gt 1 ] && [ $CURR_LINE_INDEX -lt $LINE_COUNT ]
then
ARGUMENTS+="${LINE[$CSV_INDEX]} "
elif [ $CURR_LINE_INDEX -eq $LINE_COUNT ]
then
ARGUMENTS+="${LINE[$CSV_INDEX]}"
fi
CURR_LINE_INDEX=$((CURR_LINE_INDEX+1))
done < "software-list.txt"
# Run installation script
$PROGRAMM $OPERATOR $ARGUMENTS
However, whenever I run the script, the command is correct. But the output is always the same "couldn't find any match for packagex packagey"
I followed Jetchisel's advice and did a shellcheck. How is it now?
#!/bin/bash
case "$DISTRO_NAME" in
"Fedora")
PROGRAMM="dnf"
CSV_INDEX=0;;
"Debian")
PROGRAMM="apt-get"
CSV_INDEX=1;;
esac
# Read all required packages
while IFS= read -r -a line
do
IFS=','
LINE=("${line[#]}")
if [ "$CURR_LINE_INDEX" -gt 1 ] && [ "$CURR_LINE_INDEX" -lt "$LINE_COUNT" ]
then
ARGUMENTS+="${LINE[$CSV_INDEX]} "
elif [ "$CURR_LINE_INDEX" -eq "$LINE_COUNT" ]
then
ARGUMENTS+="${LINE[$CSV_INDEX]}"
fi
CURR_LINE_INDEX=$((CURR_LINE_INDEX+1))
done < "software-list.txt"
# Run installation script
"$PROGRAMM $OPERATOR $ARGUMENTS"
apps="http:git.abc.com";
cluster-ui="http:git.xyz.com";
customer-ui="http:git.xxx.com";
SERVICE=$1;
My requirement is if I pass service name as a 'apps' then I need to clone the $apps url.
Here
if [ $Service -eq apps ]
not think a good approach as my repo url might get increased so more and more loop will come
Any suggestions?
The $ sign assigns the input argument, so we're getting first input if it matches the below variable, so do what you want inside if condition.
#!/bin/bash
apps="http:git.abc.com";
clusterui="http:git.xyz.com";
customerui="http:git.xxx.com";
#SERVICE=$1;
#Store global
repo=''
# if empty parameter is passed
if [ $# -lt 1 ] ; then
echo "Parameters Need"
exit 1
fi;
# for search the correct parameter
if [ $1 = "apps" ]; then
repo=$apps
elif [ $1 = "cluster-ui" ] ; then
repo=$clusterui
elif [ $1 = "customer-ui" ] ; then
repo=$customerui
else
echo "Not found"
fi;
echo $repo
Note just repeat elif [ ] ;then for more entries or think!
how to access run this file like this sh ./file.sh apps just replace apps with yours. make sure you have permission to execute the file if you don't have, give it to permission like below
chmod 766 file
now run the shell script sh ./file.sh clusterui
'Case statement' would suit here more than if ladder
I'm having issues with a script I'm writing in bash with regards to backing up or restoring. What I'm trying to do is check for parameters and then if none are presented, loop until a name is provided or they quit. I can check for parameters and loop to quit but the problem I am having is getting the user input and then using that for the backup file name. Here's my script so far, can someone advise on how to loop for filename/q and how to get said filename input to work with the rest of the script?
#!/bin/bash
# Purpose - Backup world directory
# Author - Angus
# Version - 1.0
FILENAME=$1.tar.gz
SRCDIR=World
DESDIR=backupfolder
if [ $# -eq 0 ]
then
echo "No filename detected. To use this utility a filename is
required. Usage:tarscript filename"
else
echo "The filename to be used will be $filename"
fi
while [ $# -eq 0 ]
do
echo "Please provide a filename or press q to exit."
read response
if [ $response == 'q' ]
then
exit
else [ $response == '$FILENAME' ]
echo -n 'Would you like to backup or restore? (B/R)'
read response
if [ $response == 'B' ]
then
tar cvpzf $DESDIR/$FILENAME $SRCDIR
echo 'Backup completed'
exit
fi
fi
done
I finally managed to get it working in the end. I realised what my mistakes were thanks to Jens and changed things enough that it now responds to input and supplied parameters. Of course the code is nearly twice as big now with all my changes but hey ho.
I have been making script that will back up and restore a directory. I want to make it better but I need some help.
At the moment the I have the file being saved as just backup.tgz I did have the date added onto the end but when I ran the restore function the I could only have it look for the backup.tgz and not the backup with the date extension. Is there any way to have it look for the most recent backup? Or even look for the backup given by user input?
I have also tried to add a progress bar and make incremental back ups but had no luck there either if someone could help?
Tar Code
#!/bin/bash
ROOT="/Users/Rory/Documents"
ROOT_EXCLUDE="--exclude=/dev --exclude=/proc --exclude=/sys --exclude=/temp --exclude=/run --exlucde=/mnt --exlcude=/media --exlude=$
DESTIN="/Users/Rory/BackUps"
BACKUP="backup.tgz"
CREATE="/dev /proc /sys /temp /run /mnt /media "
if [ "$USER" != "root" ]; then
echo "You are not the root user"
echo "To use backup please use: sudo backup"
exit
fi
clear
echo "************************************************"
echo "********* Backup Menu **************************"
echo "************************************************"
OPTIONS="BACKUP RESTORE DESTINATION EXIT"
LIST="1)BACKUP 2)RESTORE 3)DESTINATION 4)EXIT"
select opt in $OPTIONS; do
if [ "$opt" = "EXIT" ]; then
echo "GOODBYE!"
sleep 3
clear
exit
elif [ "$opt" = "BACKUP" ]; then
echo "BACKING UP FILES..."
sleep 2
tar cvpfz $DESTIN/backup.tgz $ROOT $ROOT_EXCLUDE
echo "BACKUP COMPLETE"
sleep 2
clear
echo $LIST
elif [ "$opt" = "RESTORE" ]; then
echo "RESTOTING FILES..."
sleep 2
tar xvpfz $DESTIN/$BACKUP -C /Users/Rory/BackUps
sleep 2
if [[ -e "/proc" ]]; then
echo "$CREATE already exists! "
else
mkdir $CREATE
echo "$CREATE are created! "
fi
echo "RESTORE COMPLETE..."
clear
exit
elif [ "$opt" = "DESTINATION" ]; then
echo "CURRENT DESTINATION: $DESTIN/backup.tgz "
echo "TO CHANGE ENTER THE NEW DESTINATION..."
echo "TO LEAVE IT AS IS JUST PRESS ENTER..."
read NEW_DESTIN
#IF GREATER THEN 0 ASSIGN NEW DESTINATION
if [ ${#NEW_DESTIN} -gt 0 ]; then
DESTIN = "$NEW_DESTIN"
fi
clear
else
clear
echo "BAD INPUT!"
echo "ENTER 1 , 2, 3 or 4.."
echo $LIST
fi
done
Well, in the code snippet you posted, you are only looking for backup.tgz.
If you wanted to pick a specific one, you could modify your script to accept an argument and pick one based on a string you input. Or...if you wanted to do based on "how old", you could sort the backup files by date and allow the user to pick 0th, 1st, 2nd, etc.
One thing you may want to check out is rsync. Rsync can only copy files that have changed.
Plus, you can also enable a progress bar with rsync = )
rsync -avP /source/path/ /dest/path/
Check out the man page for more details man rsync
To enable progress bar on restore, you can untar the file and use rsync in reverse, and then you have progress updates = )
If you want to make this a custom numeric progress bar you'll probably need to do something more complicated than simply taking output from rsync.
I am trying to write the script which creates a newfolder(1,2,...) and collecs the log in sdcard for each rebooting i .e each time when device gets reboot one folder will get create in sdcard.
I have a problem with this script that is if devices gets reboot for 2000 times then i have not enough space for collecting the logs in folder so what i am planning is folder count has to be 5 only i mean if it count goes more then 5 then top most folder should get delete.
#!/system/bin/sh
sleep 2
#create crash log dir on sdcard
if [ ! -d /sdcard/crash_logs ]; then
mkdir /sdcard/crash_logs
fi
if [ ! -f /sdcard/crash_logs/log_num.txt ]; then
echo "1" > /sdcard/crash_logs/log_num.txt
fi
num=$(cat /sdcard/crash_logs/log_num.txt)
if [ -z $num ]; then
num=1
fi
echo $((num+1)) > /sdcard/crash_logs/log_num.txt
if [ ! -d /sdcard/crash_logs/$num ]; then
mkdir /sdcard/crash_logs/$num
fi
You can just iterate over 1...5 numbers.
#get current number
num=...
#next number can be from 1 to 5:
num=$(( ($num+1)%5 +1 ))
#remove previous logs if any
rm -f /sdcard/crash_logs/$num/*
#copy log...