I need a script to keep polling "receive_dir" directory till "stopfile" get written in the directory.
This has to run despite empty directory.
So far i have this but fails if receive_dir is empty with no files with "unary operator expected". Help !!
#!/usr/bin/ksh
until [ $i = stopfile ]
do
for i in `ls receive_dir`; do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i;
done
done
This will do what you ask for (loop until the stop file exist). I added a "sleep 1" to lower resource usage. It's also good practice to use "#!/usr/bin/env ksh" as shebang.
#!/usr/bin/env ksh
until [ -e receive_dir/stopfile ]
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
sleep 1
done
If you have empty dir, the
until [ $i = stopfile ]
is evaluated as
until [ = stopfile ]
what is ofcourse syntax error.
One comment: Never parse output from ls.
#!/bin/bash
do_something() {
echo $(date +%m-%d-%Y-%H:%M:%S) "$1"
}
dir="."
until [[ -f "$dir/stopfile" ]]
do
find "$dir" -print0 | while IFS= read -r -d '' filename
do
do_something "$filename"
done
done
or (much slower)
do_something() {
echo $(date +%m-%d-%Y-%H:%M:%S) "$1"
}
export -f do_something
dir="."
until [[ -f "$dir/stopfile" ]]
do
find "$dir" -exec bash -c 'do_something "{}"' \;
done
You're evaluating nothing, and the 'test' isn't able to evaluate it.
~> [ $empty_var = stopfile ]
-bash: [: =: unary operator expected
First, don't parse ls:
http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29
EDIT: Part of the issue is your loop is actually doing the test, try something like this (assuming receive_dir is relative):
#user000001 is right; my original find example would suffer the same issue, so this:
for i in receive_dir/*
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i
[ $i = stopfile ] && break
done
EDIT: Adding in another example based on your comment:
How about this...
FOUND="N"
while [ "${FOUND}" = "N" ]
do
for i in receive_dir/*
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i
[ "$i" = stopfile ] && FOUND="Y"
done
sleep 60
done
Another option is to use inotifywait to monitor the status of the directory. For example, this script will run the loop until the file "stopfile" is touched.
until inotifywait "receive_dir" | grep "stopfile"
do
echo "running"
done
echo "done"
The advantage is that these is no busy loop, and that you don't have to repeatedly call the (potentially expensive) find command
Related
I have a homework using for loop but I'm not quite understand the task that I have to do in there. I wrote a script but I feel like it's not a correct script. Please help!
Here is the question:
Write a shell script to list out the contents of any directory, and indicate for each file (including invisible ones) whether the file is a directory, a plain file, and whether it is public and/or executable to this process
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
Poorly written specifications are the bane of education. "Public" sounds like the wrong word here. I'll assume it means "readable".
You check if there's an argument, but you don't exit the program if there is not. I'd also confirm it's a directory, and readable.
The manual will do you a lot of good. Expect to do a lot of reading till you learn this stuff, and then reference it a lot to be sure.
Read this section carefully, create some tests for yourself to prove they work and that you understand them, and your job will be more than half done.
Don't use [. Generally it's just better to always use [[ instead, unless you are using (( or case or some other construct.
I don't see that a for loop was specified, but it ought to be fine. Just be aware that you might have to specify $1/* and $1/.* separately.
Put all your tests in one loop, though. For each file, test for whether it's a directory - if it is, report it. Test if it's a plain file - if it is, report it.
I do NOT like doing homework for someone, but it looks like you could use an example that simplifies this. I recommend you not use this as written - break it out and make it clearer, but this is a template for the general logic.
#! /bin/env bash
(( $# )) && [[ -d "$1" ]] && [[ -r "$1" ]] || {
echo "use: $0 <dir>" >&2
exit 1
}
for e in "$1"/.* "$1"/*
do echo "$e:"
[[ -d "$e" ]] && echo " is a directory"
[[ -f "$e" ]] && echo " is a plain file"
[[ -r "$e" ]] && echo " is readable"
[[ -x "$e" ]] && echo " is executable"
done
If you read the links I provided you should be able to break this apart and understand it.
Generally, your script is long and a bit convoluted. Simpler is easier to understand and maintain. For example, be very careful about block indentation to understand scope.
$: for i in 1 2 3
> do echo $i
> done
1
2
3
$: echo $i
3
Compare this to -
for i in $*
do if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
You are testing each entry to see if it is a file, and if it is, reporting "THIS IS A LIST OF FILE and DIRECTORY in $i" every time...
but then only testing the last one to see if it's a directory, because the [ -d $i ] is after the done.
...did you run this somewhere to try it, and look at the results?
I have a placeholder file and would like to override it with the output of a command only if the output is not zero length. I guess I could do FOO="$(command)" then [-z $FOO]. Is there a better way?
There are different ways, but I don't know about "better". You could block on a read and only set up the redirection once some data come through:
cmd | { read j && { echo "$j"; cat; } > placeholder; }
(Note, if your command generates output but no newlines, this will ignore the data.)
If you don't need the output of the command, you can run: [ -z "$(command)" ] directly. For example, the following prints "empty":
#!/bin/sh
if [ -z "$(echo -n)" ]; then
echo "empty"
fi
Your example with -z will work, but you can also check if a variable is non-empty just with
[ "$var" ]
So, a simple solution could look like this:
#!/bin/sh
output="$( command )"
[ "$output" ] && echo "$output" > your_file.txt
If you are going to do this type of thing a lot, better make a function:
write_if_non_zero(){
local msg=$1
local file=$2
if [[ ! -z "$msg" ]]; then
echo "$msg" > "$file"
fi
}
Then
write_if_non_zero "$FOO" "$FILE"
I am new to bash scripting and I have to create this script that takes 3 directories as arguments and copies in the third one all the files in the first one that are NOT in the second one.
I did it like this:
#!/bin/bash
if [ -d $1 && -d $2 && -d $3 ]; then
for FILE in [ ls $1 ]; do
if ! [ find $2 -name $FILE ]; then
cp $FILE $3
done
else echo "Error: one or more directories are not present"
fi
The error I get when I try to execute it is: "line 7: syntax error near unexpected token `done' "
I don't really know how to make it work!
Also even if I'm using #!/bin/bash I still have to explicitly call bash when trying to execute, otherwise it says that executing is not permitted, anybody knows why?
Thanks in advance :)
Couple of suggestions :
No harm double quoting variables
cp "$FILE" "$3" # prevents wordsplitting, helps you filenames with spaces
for statement fails for the fundamental reason -bad syntax- it should've been:
for FILE in ls "$1";
But then, never parse ls output. Check [ this ].
for FILE in ls "$1"; #drastic
Instead of the for-loop in step2 use a find-while-read combination:
find "$1" -type f -print0 | while read -rd'' filename #-type f for files
do
#something with $filename
done
Use lowercase variable names for your script as uppercase variables are reserved for the system. Check [this].
Use tools like [ shellcheck ] to improve script quality.
Edit
Since you have mentioned the input directories contain only files, my alternative approach would be
[[ -d "$1" && -d "$2" && -d "$3" ]] && for filename in "$1"/*
do
[ ! -e "$2/${filename##*/}" ] && cp "$filename" "$3"
done
If you are baffled by ${filename##*/} check [ shell parameter expansion ].
Sidenote: In linux, although discouraged it not uncommon to have non-standard filenames like file name.
Courtesy: #chepner & #mklement0 for their comments that greatly improved this answer :)
Your script:
if ...; then
for ...; do
if ...; then
...
done
else
...
fi
Fixed structure:
if ...; then
for ...; do
if ...; then
...
fi # <-- missing
done
else
...
fi
If you want the script executable, then make it so:
$ chmod +x script.sh
Notice that you also have other problems in you script. It is better written as
dir1="$1"
dir2="$2"
dir3="$3"
for f in "$dir1"/*; do
if [ ! -f "$dir2/$(basename "$f")" ]; then
cp "$f" "$dir3"
fi
done
this is not totally correct:
for FILE in $(ls $1); do
< whatever you do here >
done
There is a big problem with that loop if in that folder there is a filename like this: 'I am a filename with spaces.txt'.
Instead of that loop try this:
for FILE in "$1"/*; do
echo "$FILE"
done
Also you have to close every if statement with fi.
Another thing, if you are using BASH ( #!/usr/bin/env bash ), it is highly recommended to use double brackets in your test conditions:
if [[ test ]]; then
...
fi
For example:
$ a='foo bar'
$ if [[ $a == 'foo bar' ]]; then
> echo "it's ok"
> fi
it's ok
However, this:
$ if [ $a == 'foo bar' ]; then
> echo "it's ok";
> fi
bash: [: too many arguments
You've forgot fi after the innermost if.
Additionally, neither square brackets nor find do work this way. This one does what your script (as it is now) is intended to on my PC:
#!/bin/bash
if [[ -d "$1" && -d "$2" && -d "$3" ]] ; then
ls -1 "$1" | while read FILE ; do
ls "$2/$FILE" >/dev/null 2>&1 || cp "$1/$FILE" "$3"
done
else echo "Error: one or more directories are not present"
fi
Note that after a single run, when $2 and $3 refer to different directories, those files are still not present in $2, so next time you run the script they will be copied once more despite they already are present in $3.
Sorry for asking this question again. I have already received answer but with using find but unfortunately I need to write it without using any predefined commands.
I am trying to write a script that will loop recursively through the subdirectories in the current directory. It should check the file count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like:
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirectories don't include subdirectories in turn.
I am confused about this because it doesn't work as I expect.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
How can I modify my script to produce the desired output?
This bash script produces the output that you want:
#!/bin/bash
bigfile="$PWD/BigList"
shortfile="$PWD/ShortList"
shopt -s nullglob
loop_folder_recurse() {
(
[[ -n "$1" ]] && cd "$1"
for i in */; do
[[ -d "$i" ]] && loop_folder_recurse "$i"
count=0
files=''
for j in *; do
if [[ -f "$j" ]]; then
files+="$j"$'\n'
((++count))
fi
done
if ((count > 10)); then
outfile="$bigfile"
else
outfile="$shortfile"
fi
echo "$i" >> "$outfile"
echo "$files" >> "$outfile"
done
)
}
loop_folder_recurse
Explanation
shopt -s nullglob is used so that when a directory is empty, the loop will not run. The body of the function is within ( ) so that it runs within a subshell. This is for convenience, as it means that the function returns to the previous directory when the subshell exits.
Hopefully the rest of the script is fairly self-explanatory but if not, please let me know and I will be happy to provide additional explanation.
I'm using the find command in my bash script like so
for x in `find ${1} .....`;
do
...
done
However, how do I handle the case where the input to my script is a file/directory that does not exist? (ie I want to print a message out when that happens)
I've tried to use -d and -f, but the case I am having trouble with is when ${1} is "." or ".."
When the input is something that doesn't exist it does not enter my for loop.
Thanks!
Bash gives you this out of the box:
if [ ! -f ${1} ];
then
echo "File/Directory does not exist!"
else
# execute your find...
fi
Bash scripting is a bit weird. Practice before implementation. But this site seems to break it down well.
If the file exists, this works:
if [ -e "${1}" ]
then
echo "${1} file exists."
fi
If the file does not exist, this works. Note the '!' to denote 'not':
if [ ! -e "${1}" ]
then
echo "${1} file doesn't exist."
fi
assign the find to a variable and test against the variable.
files=`find ${1} .....`
if [[ "$files" != “file or directory does not exist” ]]; then
...
fi
You can try something like this:
y=`find . -name "${1}"`
if [ "$y" != "" ]; then
for x in $y; do
echo "found $x"
done
else
echo "No files/directories found!"
fi