I need sending a few files to FTP server but the following script runs only one time, even if there are more entries in the document that have string "example". In /abc.txt I have paths to files which I want sending to FTP server.
#!/bin/sh
if grep -q example "/abc.txt" ;
then
var=$( cat /abc.txt )
HOST='X.X.X.X'
USER='USER'
PASSWD='PASSWORD'
cd $var
FILE='./*.txt'
ftp -nv $HOST > /abc.log.txt <<ENDSCRIPT
quote USER $USER
quote PASS $PASSWD
passive
put $FILE
bye
quit
ENDSCRIPT
echo $FILE
sed -i '1d' "/abc.txt"
else
echo "error"
fi
I recommend using for, and I think using -q option for grep here is unnecessary:
for var in $(grep example ./abc.txt); do
your_code_here
done
Here var will each time contain the name of the file.
Related
My aim is to create a shell script such that it logins and filter the list of files available and select a file to get. Here I need to run commands like in bash.
My sample code is:
sshpass -p password sftp user#10.10.10.10 <<EOF
cd /home/
var=$(ls -rt)
echo $var
echo "select a folder"
read folder
cd $folder
filen=&(ls -rt)
echo $filen
echo "select a file"
read name
get $name
bye
EOF
The above approach will not work. Remember that the 'here document' (<<EOF ... EOF) is evaluate as input to the sftp session. Prompts will be displayed, and user input will be requested BEFORE any output (ls in this case) will be available from sftp.
Consider using lftp, which has more flexible construct. In particular, it will let you use variables, create command dynamically, etc.
lftp sftp://user#host <<EOF
cd /home
ls
echo "Select Folder"
shell 'read folder ; echo "cd $folder" >> temp-cmd'
source temp-cmd
ls
echo "Select Folder"
shell 'read file ; echo "get $file" >> temp-cmd'
source temp-cmd
EOF
In theory, you can create similar constructs with pipes and sftp (may be a co-process ?), but this is much harder.
Of course, the other alternative is to create different sftp sessions for listing, but this will be expensive/inefficient.
After some research and experimentation, found a way to create batch/interactive sessions with sftp. Posting as separate answer, as I still believe the easier way to go is with lftp (see other answer). Might be used on system without lftp
The initial exec create FD#3 - pointing to the original stdout - probably user terminal. Anything send to stdout will be executed by the sftp in the pipeline.
The pipe is required to allow both process to run concurrently. Using here doc will result in sequential execution. The sleep statement are required to allow SFTP to complete data retrieval from remote host.
exec 3>&1
(
echo "cd /home/"
echo "ls"
sleep 3 # Allow time for sftp
echo "select a folder" >&3
read folder
echo "cd $folder"
echo "ls"
sleep 3 # Allow time for sftp
echo "select a file" >&3
read name
echo "get $name"
echo "bye"
) | sshpass -p password sftp user#10.10.10.10
I would suggest you to create a file with pattern of the files you want downloaded and then you can get files downloaded in one single line:
sftp_connection_string <<< $"ls -lrt"|grep -v '^sftp'|grep -f pattern_file|awk '{print $9}'|sed -e 's/^/get -P /g'|sftp_connection_string
if there are multiple definite folders to be looked into, then:
**Script version**
for fldr in folder1 folder2 folder3;do
sftp_connection_string <<< $"ls -lrt ${fldr}/"|grep -v '^sftp'|grep -f pattern_file|awk '{print $9}'|sed -e "s/^/get -P ${fldr}/g"|sftp_connection_string
done
One-liner
for fldr in folder1 folder2 folder3;do sftp_connection_string <<< $"ls -lrt ${fldr}/"|grep -v '^sftp'|grep -f pattern_file|awk '{print $9}'|sed -e "s/^/get -P ${fldr}\//g"|sftp_connection_string;done
let me know if it works.
I have a script I am trying to work out to scan my LAN and send me notification if there is a new MAC address that does not appear in my master list. I believe my variables may be messed up. This is what I have:
#!/bin/bash
LIST=$HOME/maclist.log
MASTERFILE=$HOME/master
FILEDIFF="$(diff $LIST $MASTERFILE)"
# backup the maclist first
if [ -f $LIST ]; then
cp $LIST maclist_`date +%Y%m%H%M`.log.bk
else
touch $LIST
fi
# this will scan the network and extract the IP and MAC address
nmap -n -sP 192.168.122.0/24 | awk '/^Nmap scan/{IP=$5};/^MAC/{print IP,$3};{next}' > $LIST
# this will use a diff command to compare the maclist created above and master list of known good devices on the LAN
if [ $FILEDIFF ] 2> /dev/null; then
echo
echo "---- All is well on `date` ----" >> macscan.log
echo
else
# echo -e "\nWARNING!!" | `mutt -e 'my_hdr From:user#email.com' -s "WARNIG!! NEW DEVICE ON THE LAN" -i maclist.log user#email.com`
echo "emailing you"
fi
When I execute this when the maclist.log does not exist I get this response:
diff: /root/maclist.log: No such file or directory
If I execute it again with the maclist.log file existing the file gets renamed from the cp line without any issue.
The line
FILEDIFF="$(diff $LIST $MASTERFILE)"
executes the diff when it is run (not when you use $FILELIST later). At that time the list file hasn't been created.
The easiest fix is just to put the diff command in full where $FILELIST is currently used.
I need to do an FTP of the files available in the file files_to_download.
I have put an FTP script in between but it throws and error saying
" Syntax error: end of file unexpected (expecting "done")". Do I need to do an FTP login for downloading the file every time. I want to download all the files in a single FTP login?
if [ $update -eq 1 ]
then
#echo "File needs to be updated"
while read file_data
do
#echo $file_data
file_name=`echo $file_data | cut -d':' -f1` #truncate the file path
echo $file_name
#ftp -inv <<!
#open ${SERVER}
#user ${USERNAME} ${PASSWORD}
#binary
#cd $REMOTEDIR
#get server_version
#lcd $LOCALDIR
#close
#quit
#!
done < files_to_download
fi
You can use an outline script like this:
{
cat << EOF
open ${SERVER}
user ${USERNAME} ${PASSWORD}
binary
cd ${REMOTEDIR}
get server_version
EOF
sed -e 's/:.*//' -e 's/^/get /' files_to_download
cat <<EOF
lcd ${LOCALDIR}
close
quit
EOF
} | ftp -inv
The first cat sets up the connection. The sed edits the list of file names into get statements. The final cat puts out the remaining commands. The surrounding { and } send all the output of the commands within to the ftp command. The chances are that simply omitting all the second cat would work fine; the FTP command would read EOF on its input after the final file transfer and then exit of its own accord.
The get server_version can be deleted if server_version was meant to be a file name rather than a request for the version of the FTP server. The lcd probably isn't necessary either.
I've used the ${VAR} notation consistently; the original code used that an $VAR. Consistency is good.
You should not indent !, that is, you should place it at the beginning of the line, without any whitespaces before it. You indented it, so it's not parsed as the end of the heredoc.
To download all files in a single login, you can print the FTP commands in a subshell. Or, you can also generate read the filenames beforehand and store it into a variable.
I'm writing a bash script to send files from a linux server to a remote Windows FTP server.
I would like to check using FTP if the folder where the file will be stored exists before attempting to create it.
Please note that I cannot use SSH nor SCP and I cannot install new scripts on the linux server. Also, for performance issues, I would prefer if checking and creating the folders is done using only one FTP connection.
Here's the function to send the file:
sendFile() {
ftp -n $FTP_HOST <<! >> ${LOCAL_LOG}
quote USER ${FTP_USER}
quote PASS ${FTP_PASS}
binary
$(ftp_mkdir_loop "$FTP_PATH")
put ${FILE_PATH} ${FTP_PATH}/${FILENAME}
bye
!
}
And here's what ftp_mkdir_loop looks like:
ftp_mkdir_loop() {
local r
local a
r="$#"
while [[ "$r" != "$a" ]]; do
a=${r%%/*}
echo "mkdir $a"
echo "cd $a"
r=${r#*/}
done
}
The ftp_mkdir_loop function helps in creating all the folders in $FTP_PATH (Since I cannot do mkdir -p $FTP_PATH through FTP).
Overall my script works but is not "clean"; this is what I'm getting in my log file after the execution of the script (yes, $FTP_PATH is composed of 5 existing directories):
(directory-name) Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
Cannot create a file when that file already exists.
To solve this, do as follows:
To ensure that you only use one FTP connection, you create the input (FTP commands) as an output of a shell script
E.g.
$ cat a.sh
cd /home/test1
mkdir /home/test1/test2
$ ./a.sh | ftp $Your_login_and_server > /your/log 2>&1
To allow the FTP to test if a directory exists, you use the fact that "DIR" command has an option to write to file
# ...continuing a.sh
# In a loop, $CURRENT_DIR is the next subdirectory to check-or-create
echo "DIR $CURRENT_DIR $local_output_file"
sleep 5 # to leave time for the file to be created
if (! -s $local_output_file)
then
echo "mkdir $CURRENT_DIR"
endif
Please note that "-s" test is not necessarily correct - I don't have acccess to ftp now and don't know what the exact output of running DIR on non-existing directory will be - cold be empty file, could be a specific error. If error, you can grep the error text in $local_output_file
Now, wrap the step #2 into a loop over your individual subdirectories in a.sh
#!/bin/bash
FTP_HOST=prep.ai.mit.edu
FTP_USER=anonymous
FTP_PASS=foobar#example.com
DIRECTORY=/foo # /foo does not exist, /pub exists
LOCAL_LOG=/tmp/foo.log
ERROR="Failed to change directory"
ftp -n $FTP_HOST << EOF | tee -a ${LOCAL_LOG} | grep -q "${ERROR}"
quote USER ${FTP_USER}
quote pass ${FTP_PASS}
cd ${DIRECTORY}
EOF
if [[ "${PIPESTATUS[2]}" -eq 1 ]]; then
echo ${DIRECTORY} exists
else
echo ${DIRECTORY} does not exist
fi
Output:
/foo does not exist
If you want to suppress only the messages in ${LOCAL_LOG}:
ftp -n $FTP_HOST <<! | grep -v "Cannot create a file" >> ${LOCAL_LOG}
#!/bin/bash
# See if registry is set to expire updates
filename=hostnames
> test.log
PARAMETER=Updates
FILE=/etc/.properties
CODE=sudo if [ ! -f $FILE] && grep $PARAMETER $FILE; then echo "File found, parameter not found."
#CODE=grep $PARAMETER $FILE || sudo tee -a /etc/.properties <<< $PARAMETER
while read -r -a line
do
hostname=${line//\"}
echo $hostname":" >> test.log
#ssh -n -t -t $hostname "$CODE" >> test.log
echo $CODE;
done < "$filename"
exit
I want to set "Updates 30" in /etc/.properties on about 50 servers if:
The file exists (not all servers have the software installed)
The parameter "Updates" is not already set in the file (e.g. in case of multiple runs)
I am a little puzzled so far how, because I am not sure if this can be done in 1 line of bash code. The rest of the script works fine.
Ok, here's what i think would be a solution for you. Like explained in this article http://www.unix.com/shell-programming-scripting/181221-bash-script-execute-command-remote-servers-using-ssh.html
invoke the script which contains the commands that you want to be executed at the remote server
Code script 1:
while read -r -a line
do
ssh ${line} "bash -s" < script2
done < "$filename"
To replace a line in a text file, you can use sed (http://www.cyberciti.biz/faq/unix-linux-replace-string-words-in-many-files/)
Code script 2:
PARAMETER=Updates
FILE=/etc/.properties
NEWPARAMETER=Updates ###(What you want to write there)
if [ ! -f $FILE] && grep $PARAMETER $FILE; then exit
sed -i 's/$PARAMETER/$NEWPARAMETER/g' $FILE
So, I'm not certain this covers all your use case, I hope this helps you out if there is anything feel free to ask!