I'd like to write a shell script which checks if a certain file, archived_sensor_data.json, exists, and if so, deletes it. Following http://www.cyberciti.biz/tips/find-out-if-file-exists-with-conditional-expressions.html, I've tried the following:
[-e archived_sensor_data.json] && rm archived_sensor_data.json
However, this throws an error
[-e: command not found
when I try to run the resulting test_controller script using the ./test_controller command. What is wrong with the code?
You're missing a required space between the bracket and -e:
#!/bin/bash
if [ -e x.txt ]
then
echo "ok"
else
echo "nok"
fi
Here is an alternative method using ls:
(ls x.txt && echo yes) || echo no
If you want to hide any output from ls so you only see yes or no, redirect stdout and stderr to /dev/null:
(ls x.txt >> /dev/null 2>&1 && echo yes) || echo no
The backdrop to my solution recommendation is the story of a friend who, well into the second week of
his first job, wiped half a build-server clean. So the basic task is to figure out if a file exists,
and if so, let's delete it. But there are a few treacherous rapids on this river:
Everything is a file.
Scripts have real power only if they solve general tasks
To be general, we use variables
We often use -f force in scripts to avoid manual intervention
And also love -r recursive to make sure we create, copy and destroy in a timely fashion.
Consider the following scenario:
We have the file we want to delete: filesexists.json
This filename is stored in a variable
<host>:~/Documents/thisfolderexists filevariable="filesexists.json"
We also hava a path variable to make things really flexible
<host>:~/Documents/thisfolderexists pathtofile=".."
<host>:~/Documents/thisfolderexists ls $pathtofile
filesexists.json history20170728 SE-Data-API.pem thisfolderexists
So let's see if -e does what it is supposed to. Does the files exist?
<host>:~/Documents/thisfolderexists [ -e $pathtofile/$filevariable ]; echo $?
0
It does. Magic.
However, what would happen, if the file variable got accidentally be evaluated to nuffin'
<host>:~/Documents/thisfolderexists filevariable=""
<host>:~/Documents/thisfolderexists [ -e $pathtofile/$filevariable ]; echo $?
0
What? It is supposed to return with an error... And this is the beginning of the story how that entire
folder got deleted by accident
An alternative could be to test specifically for what we understand to be a 'file'
<host>:~/Documents/thisfolderexists filevariable="filesexists.json"
<host>:~/Documents/thisfolderexists test -f $pathtofile/$filevariable; echo $?
0
So the file exists...
<host>:~/Documents/thisfolderexists filevariable=""
<host>:~/Documents/thisfolderexists test -f $pathtofile/$filevariable; echo $?
1
So this is not a file and maybe, we do not want to delete that entire directory
man test has the following to say:
-b FILE
FILE exists and is block special
-c FILE
FILE exists and is character special
-d FILE
FILE exists and is a directory
-e FILE
FILE exists
-f FILE
FILE exists and is a regular file
...
-h FILE
FILE exists and is a symbolic link (same as -L)
Internally, the rm command must test for file existence anyway,
so why add another test? Just issue
rm filename
and it will be gone after that, whether it was there or not.
Use rm -f is you don't want any messages about non-existent files.
If you need to take some action if the file does NOT exist, then you must test for that yourself. Based on your example code, this is not the case in this instance.
If you're using a NFS, "test" is a better solution, because you can add a timeout to it, in case your NFS is down:
time timeout 3 test -f
/nfs/my_nfs_is_currently_down
real 0m3.004s <<== timeout is taken into account
user 0m0.001s
sys 0m0.004s
echo $?
124 <= 124 means the timeout has been reached
A "[ -e my_file ]" construct will freeze until the NFS is functional again:
if [ -e /nfs/my_nfs_is_currently_down ]; then echo "ok" else echo "ko" ; fi
<no answer from the system, my session is "frozen">
You could also uses stat :
stat /
File: /
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: fd01h/64769d Inode: 2 Links: 26
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2009-01-01 02:00:00.000000000 +0200
Modify: 2009-01-01 02:00:00.000000000 +0200
Change: 2009-01-01 02:00:00.000000000 +0200
Birth: -
On a path that doesn't exist, you will get:
stat /aaa
stat: cannot stat '/aaa': No such file or directory
Related
In shell, I want to check if a file exists or not then create if it doesn't exist or delete if it exists. For this I need a one liner and am trying to do something like:
ls | awk '\filename\' <if exist delete else create>
I need the ls as my problem has some command that outputs a list of strings that need to be pipelined to awk then possibly touch/mkdir.
#!/bin/bash
if [ -z "$1" ] || [ ! -f "$1" ] # $1 is input filename and -f check if $1 is a regular file
then
rm "$1" #delete the file
else
touch "$1" #create the file
fi
save the file as filecreator.sh
change the permission to allow execution with sudo chmod a+rx
while running the script use ./filecreator.sh yourfile.extension
You can see the file in your directory.
Using oc projects and oc new-project instad of ls and touch as indicated in a comment.
oc projects |
while read -r proj; do
if [ -d "$proj" ]; then
rm -rf "$proj"
else
oc new-project "$proj"
fi
done
I don't think there is a useful way to write this as a one-liner. If you like, you can replace the newlines with semicolons, except after then and else.
You really should put your actual requirements in the question itself. ls is a superbly useless example because it cannot list a file which doesn't already exist, and you should not use ls in scripts at all.
rm yourfile 2>/dev/null || touch yourfile
If the file existed before, rm will succeed and erase the file, and the touch won't be executed. You end up with no file afterwards.
If the file did not exist before, rm will fail (but the error message is not visible, since it is directed to the bitbucket), and due to the non-zero exit code of rm, the touch will be executed. You end up with an empty file afterwards.
Caveat: If the file exists, but you don't have permissions to remove it, you won't notice this error, due to the redirection of stderr. Hence, for debugging and later diagnosis, it might be better to redirect stderr to some file instead.
I am working on some stuff where I am storing data in a file.
But each time I run the script it gets appended to the previous file.
I want help on how I can remove the file if it already exists.
Don't bother checking if the file exists, just try to remove it.
rm -f /p/a/t/h
# or
rm /p/a/t/h 2> /dev/null
Note that the second command will fail (return a non-zero exit status) if the file did not exist, but the first will succeed owing to the -f (short for --force) option. Depending on the situation, this may be an important detail.
But more likely, if you are appending to the file it is because your script is using >> to redirect something into the file. Just replace >> with >. It's hard to say since you've provided no code.
Note that you can do something like test -f /p/a/t/h && rm /p/a/t/h, but doing so is completely pointless. It is quite possible that the test will return true but the /p/a/t/h will fail to exist before you try to remove it, or worse the test will fail and the /p/a/t/h will be created before you execute the next command which expects it to not exist. Attempting this is a classic race condition. Don't do it.
Another one line command I used is:
[ -e file ] && rm file
You can use this:
#!/bin/bash
file="file_you_want_to_delete"
if [ -f "$file" ] ; then
rm "$file"
fi
If you want to ignore the step to check if file exists or not, then you can use a fairly easy command, which will delete the file if exists and does not throw an error if it is non-existing.
rm -f xyz.csv
A one liner shell script to remove a file if it already exist (based on Jindra Helcl's answer):
[ -f file ] && rm file
or with a variable:
#!/bin/bash
file="/path/to/file.ext"
[ -f $file ] && rm $file
Something like this would work
#!/bin/sh
if [ -fe FILE ]
then
rm FILE
fi
-f checks if it's a regular file
-e checks if the file exist
Introduction to if for more information
EDIT : -e used with -f is redundant, fo using -f alone should work too
if [ $( ls <file> ) ]; then rm <file>; fi
Also, if you redirect your output with > instead of >> it will overwrite the previous file
So in my case I wanted to remove a FIFO file before I create it again, so this worked for me:
#!/bin/bash
file="/tmp/test"
rm -rf $file | true
mkfifo $file
| true will continue the script even if file is not found.
I currently have this bash script (which is located in my home directory, i.e., /home/username/ and I am running it as root as it's necessary for the icon copying lines):
cd /home/username/Pictures/Icon*
declare -a A={Arch,Debian,Fedora,Mageia,Manjaro,OpenSUSE}
declare -a B={Adwaita,Faenza,gnome,Humanity}
for i in $A; do
for j in $B; do
if test -e /usr/share/icons/$j/scalable ; else
mkdir /usr/share/icons/$j/scalable/
fi
if test -e /usr/share/icons/$j/scalable/$i.svg ; else
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
fi
done
done
What I want this script to do is to copy icons from my Pictures/Icons and logos directory to the scalable theme (specified in $B) subdirectories in /usr/share/icons. Before it does this, however, I'd like it to create a scalable directory in these theme subdirectories if it does not already exist. The problem is that the else part of the conditionals is not being read properly, as I keep receiving this error:
./copyicon.sh: line 8: syntax error near unexpected token `else'
./copyicon.sh: line 8: ` if test -e /usr/share/icons/$j/scalable ; else'
If you're wondering why the test -e ... in the conditional it's based on a textbook on bash scripting I've been following.
Checking file and/or directory existence
To check whether a file exists in bash, you use the -f operator. For directories, use -d. Example usage:
$ mkdir dir
$ [ -d dir ] && echo exists!
exists!
$ rmdir dir
$ [ -d dir ] && echo exists!
$ touch file
$ [ -f file ] || echo "doesn't exist..."
$ rm file
$ [ -f file ] || echo "doesn't exist..."
doesn't exist...
For more information simply execute man test.
A note on -e, this test operator checks whether a file exists. While this may seem like a good choice, it's better to use -f which will return false if the file isn't a regular file. /dev/null for example is a file but nor a regular file. Having the check return true is undesired in this case.
A note on variables
Be sure to quote variables too, once you have a space or any other special character contained in a variable it can have undesired side effects. So when you test for existence of files and directories, wrap the file/dir in double quotes. Something like [ -f "/path/to/some/${dir}/" ] will work while the following would fail if there is a space in dir: [ -f /path/to/some/${dir}/ ].
Fixing the syntax error
You are experiencing a syntax error in the control statements. A bash if clause is structured as following:
if ...; then
...
fi
Or optional with an else clause:
if ...; then
...
else
...
fi
You cannot omit the then clause. If you wish to only use the else clause you should negate the condition. Resulting in following code:
if [ ! -f "/usr/share/icons/$j/scalable" ]; then
mkdir "/usr/share/icons/$j/scalable/"
fi
Here we add an exclamation point (!) to flip the expression's evaluation. If the expression evaluates to true, the same expression preceded by ! will return false and the other way around.
You can't skip the then part of the if statement, easiest solution would be to just negate the test
if [[ ! -e /usr/share/icons/${j}/scalable ]] ; then
mkdir /usr/share/icons/${j}/scalable/
fi
if [[ ! -e /usr/share/icons/${j}/scalable/${i}.svg ]] ; then
cp -a ${i}*.svg /usr/share/icons/${j}/scalable/${i}.svg
fi
I left it with -e (exists), but you might consider using -d for directories or -f for files and some error handling to catch stuff (e.g. /usr/share/icons/$j/scalable/ exists, but is a file and not a directory for whatever reason.)
I also noticed that in your original code you are potentially trying to copy multiple files into one:
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
I left it that way in my example in case you are sure that it is always only one file and are intentionally renaming it. If not I'd suggest only specifying a target directory.
When i use this:
*/5 6-18 * * 1-6 [ "$(ls -A /DIR_WHERE_FILES_ARE_OR_NOT/)" ] &&
rsync -au /DIR_WHERE_FILES_ARE_OR_NOT/ /DIR_WHERE_FILES_SHOLD_GO; \
mv /DIR_WHERE_FILES_ARE_OR_NOT/* /SAVE_DIR/ ||
mail -s "DIR IS EMPTY" myemail#klkldkl.de <<< "message"
i get two mails:
mv: cannot stat `/DIR_WHERE_FILES_ARE_OR_NOT/*': No such file or
directory
and
"DIR IS EMPTY"
Why?
You get
mv: cannot stat `/DIR_WHERE_FILES_ARE_OR_NOT/*': No such file or directory
for exactly the reason stated: that directory is empty, hence it does not contain a file named * (asterisk). It's just the way glob expansion works in the shell: if the glob doesn't match anything it is passed literally to the command. Since mv attemps to rename a non-existing file, it complains as shown.
This would all be much more readable, if instead of a sequence of && and || operators in a crontab you would place the whole logic in a script with equivalent if/else/fi constructs and just call the script from cron.
You get two mails because you explicitly send the first with mail -s. The second is from cron because the output on stderr and stdout is not empty.
Your commands are equivalent to
if [ "$(ls ...)" ]; then
rsync
fi
if ! mv; then
mail
fi
Note that there is no else.
Just like user Jens already mentioned, and also from my experience, unless you are using a very simple and usually single command, you should stick to script files. So, in your case, I would go with a script file. I'll give you an example.
#!/bin/bash
dir_where_files_are_or_not=/filespath
dir_where_files_should_go=/another/filespath
save_dir=/savefiles/path
# ok, lets start by checking if dir contains files
if [ "$(ls -A $dir_where_files_are_or_not)" ]; then
# dir contains files, so lets rsync and mv them
rsync -au $dir_where_files_are_or_not/ $dir_where_files_should_go
mv $dir_where_files_are_or_not/* $save_dir
else
# dir is empty, lets send email
mail -s "DIR IS EMPTY" myemail#klkldkl.de <<< "message"
fi
Now, I just put this code in a file. Give it a name, for example "chkfiles" and save it in a directory (I use /usr/local/sbin for all of my scripts).
Next, in a shell, run the command chmod +x /usr/local/sbin/chkfiles to make the file executable. Then add the script to your crontab.
I would suggest the following line inside crontab:
*/5 6-18 * * 1-6 /bin/bash /usr/local/sbin/chkfiles
I used /bin/bash to call the right interpreter for this script. It should now work as expected.
Important Notes:
Before running the script, you need to change the dir_where_files_are_or_not, dir_where_files_should_go and save_dir vars to your needs.
Do NOT include trailing slashes in the dirs, otherwise the rsync and mv might not do what you really want
Regards
You get two mails because when mv fails, cron captures what is written to standard error and mails it to the owner, then runs the mail command. You can suppress the error message from mv to avoid the mail from cron.
mv /DIR_WHERE_FILES_ARE_OR_NOT/* /SAVE_DIR/ 2> /dev/null || mail -s "DIR IS EMPTY" myemail#klkldkl.de <<< "message"
I'm getting started with bash-scripting and I am currently writing a script to backup a directory if the directory was modified since the last time it was backed-up . I am using the diff command to check for this, but I think I might not be using it right.
EDIT
The reason I am getting differences is because the files newFile and oldFile are being created in the directory which has the script and that is the same directory which I am trying to backup. How do I use diff such that it returns differences other than those 2 files ?
ls -lR $1 > oldFile
while [ 0 ];
do
ls -lR $1 > newFile
if [ ! diff newFile oldFile ] ; then
echo "they differ"
else
echo "they don't differ"
fi
done
First, the if statement doesn't work at all. use:
if [ diff oldFile newFile ] ; then
echo "they differ"
fi
Note the [ ] brackets! (note that they are not really 'brackets'. [ is a shorty for the test command and ] is the last argument to it. Crazy, but cool!
You can use
diff -rN OLD_DIR NEW_DIR
to get a recursive diff. -N also tracks new files
Also you might have a look at inotifywait and friends. Inotify is a mechanism in the Linux kernel that allows to register hooks when an inode (file or directory,...) has changed. Using inotifywait (and fiends) you are enabled to register a shell script that will be triggered at every file operation that takes place on a file / directory of interest. In that script you can doing backup directly after changes (if you want)
A small search would have revealed this: Bash: using a the result of a diff in a if statement.
DIFF=$(diff newfile oldfile)
if [ "$DIFF" != "" ]; then
...
You could also use md5deep to hash the files in the directory and then hash the output of that command for a final checksum:
hash=$(md5deep -rs folder | md5sum)
Brackets around diff in your code are not mandatory, the idea being that the if statement tests by itself the return code of the command that follows. So no need to call test another time. You can also write:
if diff -q newFile oldFile &> /dev/null ; then...
(the redirection to /dev/null being here because diff returns a - useless - output )