Im working on a bash script to check an ftp site to see if some files exist on it. It works by looping through an array of files using curl to check if the file exists on the ftp server. If it doesnt then curl should output an error which im looking for in my if loop and echoing a result based on it. Code below:
#!/bin/bash
# script variables
ftpaddress=ftpaddress
ftplocation=folderlocation
ftpusername=ftpusername
ftppassword=ftppassword
err="curl: (19) Given file does not exist"
# Initialise array of files
files=( "file1.xml" "file2.xml" "zy.xml" )
for f in "${files[#]}"; do
result=(curl ftp://$ftpaddress$ftplocation$f --user $ftpusername:$ftppassword --head)
if [[ "$result" == "$err" ]]; then
echo $f " Does not exist!!!!"
else
echo $f "Exists"
fi
done
Something about the if loop is failing, ive tried lots of variations on it but have been unable to find one that works. At the moment the result is never matching the error. When I run the curl just from the command line it outputs the error i have set to $err if the file doesnt exist but when running the script the else branch is being selected everytime saying the file does exist. I've even tried setting the error to be*"19"*and it still doesnt match. I've spent a lot of time looking it up and testing it but have had no luck so would appreciate any help that can be given.
Thanks
( ) will get you the return value $( ) will get you stdout
result=$(curl ftp://$ftpaddress$ftplocation$f --user $ftpusername:$ftppassword --head 2>&1)
Related
I've been searching the web but couldn't find the answer for my situation. I believe it's a first?
Anyway, here's what I'm trying to do;
#!/bin/bash
if source <(curl -s -f http://example.com/$1.txt); then
echo "Found the file"
else
echo "Couldn't find it"
fi
This is supposed to source the bash from http://example.com/$1.txt when I run the script like this; ./myscript.sh fileName while hiding any success of error outputs because I don't want them to show up.
However, while it works fine for the files that exist, it still says "Found the file" even if the file isn't there and sources the bash from an empty file because of the -f flag. If I remove the -f flag then it works and says "Couldn't find it" but it also returns an HTTP error since the file isn't there and as I said, I want to hide the errors too.
How can I work this around?
The result code from source is simply the last command in the sourced file. If the file is empty (as it will be if curl fails) that's a success.
What you can do is guard against an error from curl separately.
if source <(curl -s -f "http://example.com/$1.txt" || echo "exit $?"); then
I currently have a directory that is supposed to have 500 files. Each file is of the name form List.1.rds, ... List.500.rds. The way I can see which ones are missing is by the following code in bash:
for((i=1; i<=500; i++)); do name="List.${i}.rds"; [[ ! -e "$name" ]] && echo "missing $name"; done
If a file is missing, it returns the missing file name. However, I would like to go one step further and stop the entire script should any file be missing. Is there a way to do this? thanks.
It can be as simple as setting a flag when a file is missing:
miss=0
for ((i=1;i<=500;i++)); do
file=List.$i.rds
if [[ ! -e $file ]]; then
echo "Missing $file"
miss=1
fi
done
# exit if "miss" flag is 1
((miss)) && exit 1
I'm currently writing code for a script to move files to a "dustbin".
As it stands - the code works fine but I want it to produce a message when a file has been moved correctly and a message when a specific file has failed to move/doesn't exist.
My code will accept multiple filenames for input, e.g.
# del test1.txt *.html testing.doc
# Successfully moved to Dustbin
However if only one of these does not exist it still produces an error message.
How do I do this but still allow it to accept arguments as seen in the above example?
My code is as follows:
#!/bin/sh
mv -u "$#" /root/Dustbin 2>/dev/null
# END OF SCRIPT
Sorry for what is probably an obvious question! I'm completely new to shellscript !
Many thanks
You would have to iterate over the arguments and try to move each one:
for path in "$#"; do
if mv -u "$path" /root/Dustbin 2>/dev/null; then
echo "Success"
else
printf 'Failed to move %s\n' "$path"
fi
done
As a shorthand for iterating over the arguments you can omit in "$#" like
for path; do
if mv -u "$path" /root/Dustbin 2>/dev/null; then
echo "Success"
else
printf 'Failed to move %s\n' "$path"
fi
done
I need to check if a file exists in directory using bash. I have tried below method but it needs complete path as input.
if [ -e /*/my_file.txt ] ;
then
echo "file found"
else
echo "not found"
fi
Is there any way that I can check if the file exists at any depth dynamically.
NOTE: I don't want to use "find" as it takes lot of time to execute.
If you are using bash 4, you can write patterns that recursively descend a hierarchy:
shopt -s globstar
for f in /**/myfile.txt; do
if [[ -e $f ]]; then
found=1
echo "File found"
break
fi
done
if [[ $found -ne 1 ]]; then
echo "File not found"
fi
Using find:
found=$( find / -name myfile.txt )
if [[ -n $found ]]; then
echo "File found"
else
echo "File not found"
fi
If really speed is your concern, file globbing like ls * */* */*/* is not helping you that much. And it has its limit with this error: Argument list too long. find is a useful tool for finding stuff. It is very flexible. But like using file globbing, it has to scan the directory tree with every invocation. For occasional searches, like for maintenance, this is totally acceptable. But if this is part processing pipeline, the speed is not acceptable. You need an optimised database for that.
The simplistic way
Most every UNIX I know is shipped with locate.
If it is preinstall you can search like this:
$ locate -b '\my_file.txt'
The backslash in front of my_file.txt is intended. It switches off wildcard search. Adding -i gives case insensitive search.
If the command is not available it should be installable from your OS repository. For Debian/Ubuntu: apt install locate. For first init run /etc/cron.daily/locate as root or with sudo.
The database is updated on a daily basis. For some applications, this interval is probably too long. By moving the cronjob from daily to like every 3 hours, you get more recent results.
The realtime way ...
This is a bit out of the scope of this answer. But you would need some kind of deamon, that would watch kernel inotify events for directory changes. These in turn would be reflected in a database, that can be queried through some API. Like Spotlight from MacOS or Tracker from Gnome.
find is the proper solution.
however you can use bash expansion feature
if ls */* | grep -q my_file.txt
then echo file found
else echo file not found
fi
note
that above solution will not find my_file.txt if a top level.
if my_file.txt is part of a directory name you might get a wrong result.
if there are many (thousands) directories and many files / expansion might get paste bash limit (arg list too long)
you can ls * */* */*/* | grep with limit state above.
Could someone help me understand the following piece of code which is deciding on the start and end dates to pick data out of a db.
# Get the current time as the stop time.
#
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
#
# Read the lasttime file to get the start time
#
if test -f $1/optlasttime
then
starttime=`cat $1/optlasttime`
# if the length of the chain is zero
# (lasttime is empty) It is updated properly
# (and I wait for the following hour)
if test -z "$starttime"
then
echo "Empty file lasttime"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 5
fi
else
# If lasttime does not exist I create, it with the present date
# and I wait for the following hour
echo "File lasttime does not exist"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 6
fi
Thanks
The script checks to see if there's a non-empty file named optlasttime in the directory specified as an argument ($1). If so, the script exits successfully (status 0). If the file doesn't exist or is empty, the current hour formatted as 2010-01-07 14:00 is written to the file, another file named .optpamo.pid is deleted from the argument directory and the script exits unsuccessfully (status 5 or 6).
This script is obviously a utility being called by some outer process, to which you need to refer for full understanding.
1.) Sets stop time to current time
2.) Checks if file $1/optlasttime exists (where $1 is passed into the script)
a.) if $1/optlasttime exists it checks the contents of the file (which it is assumed that if it does have contents it is a timestamp)
b.) if $1/optlasttime does not exist it populates the $1/optlasttime file with the stoptime.
I copied and pasted a small snippet of this into a file I called test.ksh
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
Then I ran it at the commandline, like so:
zhasper#berens:~$ ksh -x ./temp.ksh
+ date '+%Y-%m-%d %H:00'
+ stoptime='2010-01-08 18:00'
+ test 0 -ne 0
The -x flag to ksh makes it print out each commandline, in full, as it executes. Comparing what you see here with the snippet of shell script above should tell you something about how ksh is interpreting the file.
If you run this over the whole file you should get a good feel for what it's doing.
To learn more, you can read man ksh, or search for ksh scripting tutorial online.
Together, these three things should help you learn a lot more than us simply telling you what the script does.