I have three variables:
taille : number of iterations
racine : directory from where to copy
rep : directory where to copy<>br
So the code is supposed to start copying recursively from racine to rep, number of files copied is restricted by taille. I cant seem to make the cp command to work, and i dont know how to make the recursivity to work either. My code is as follows
if [ -z "$1" ]
then
taille=0
else
taille=$1
fi
if [ -z "$2" ]
then
racine=`pwd`
else
racine=$2
fi
if [ -z "$3" ]
then
rep="test2"
else
rep=$3
fi
count=0
for i in `ls $racine`;
do
if [ $count -lt $((taille+1)) ]
then
echo $i
`cp $i test2`
fi
count=$((count+1))
done
Can somebody help me?
The following quick and dirty test script seems to do what you want (more or less).
I'm trying to convey a general approach, not provide a complete solution. What's below will misbehave (look at the mkdir -p line) given a relative path (e.g. ../../) for the source directory, so you may have to give that problem a little thought. I hope it conveys the idea clearly enough though.
#!/bin/bash
racine="$1"
rep="$2"
declare -i taille=3
declare -i count=0
while read -r -d $'\0'; do
if [ $count -ge $taille ]; then
break
fi
if [ -d "$REPLY" ]; then
mkdir -p "$rep/$REPLY";
else
cp "$REPLY" "$rep"
fi
count=$((count+1))
done < <(find $racine -print0)
A couple of things:
You don't need the back ticks around the copy command, so just cp $i test2.
You increment a variable like this: count=`expr $count + 1`.
EDIT: The count=$((count+1)) syntax works if you are using Bash.
Related
I'm having the hardest time finding out how to do a fairly simple task succesfully:
Archive files and folders to a different location (eg cp -rp /source/file /destination/file)
Remove them once coppied (rm -rf /source/file)
But only if they are 30+ days or older, which I know requires find -mtime 30. But I do not know how to make the following code work with find:
#!/bin/bash
#functions
help()
{
echo "archiveer <doelmap>
archiveer /home/goedvoorbeeld/testmap
Dit commando zorgt ervoor dat gebackupte users van meer dan 30 dagen oud worden gearchiveerd. Alleen de meest recente data wordt opgeslagen.
De doelmap parameter is verplicht.";
exit
}
doemaar()
{
echo "Doe maar wordt uitgevoerd"
for f in *; do
if [[ "${f}" -nt "${doelmap}${f}" ]]
then
echo "file $f is nieuwer of niet aanwezig, gebacked up"
cp -rp "$f" "${doelmap}${f}"
rm -rf "$f"
else
echo "file $f is ouder"
fi
done
}
#check for parameters
if [[ $1 = "help" ]]
then
help
elif [[ $1 = "" ]]
then
echo "Missende parameter!"
help
fi
#set variables
doelmap=$1
#script gaat runnen
echo "${doelmap}" && sleep 1
if [ ! -d "${doelmap}" ]
then
mkdir -p "${doelmap}"
echo "${doelmap} is gemaakt"
else
echo "doelmap bestaat al"
fi
doemaar
echo "Script end"
Note that * can sometimes cause problems if filenames have spaces or special characters in them. That is why I always redirect any lists to a file, from which I work from. Also, the results list from * might exceed shell limits, causing different issues.
Also, "-rp" implies recursive, so directory contents. But earlier you stated that you were only looking at files. So the assumption is that there are no sub-directories under the directory where you are trying to find old files.
I would process the file archiving, then I would check the parent directories to purge directories that are left empty after the archiving. This is a safety measure in case of power loss.
You could consider adapting the following into your script:
cd "${LIVE_DIR}"
TOPIC=`basename "${LIVE_DIR}" `
find . -type f -mtime 30+ -print >joblist.files
mkdir "${ARCHIVE_ROOT}/${TOPIC}"
if [ $? -ne 0 ] ; then echo "unable to create dir" ; exit 1 ; fi
while read file
do
ARC_FILE="${ARCHIVE_ROOT}/${TOPIC}/${file}"
cp -pv "${file}" "${ARC_FILE}"
if [ $? -eq 0 ]
then rm -f "${file}"
if [ $? -ne 0 ]
then echo "PURGE|rm -f \"${file}\"" >&2
fi
else rm -fv "${ARC_FILE}"
echo "ARCHIVE|cp -pv \"${file}\" \"${ARC_FILE}\"" >&2
fi
echo "LAST|${file}" >joblist.last
done <joblist.files 2>joblist.failed
joblist.failed contains any cleanup that needs to be performed, commands that need to be repeated to complete the initial job.
joblist.last contains the last file processed if you need to kill the job and keep track of where you left off.
I have a homework using for loop but I'm not quite understand the task that I have to do in there. I wrote a script but I feel like it's not a correct script. Please help!
Here is the question:
Write a shell script to list out the contents of any directory, and indicate for each file (including invisible ones) whether the file is a directory, a plain file, and whether it is public and/or executable to this process
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
Poorly written specifications are the bane of education. "Public" sounds like the wrong word here. I'll assume it means "readable".
You check if there's an argument, but you don't exit the program if there is not. I'd also confirm it's a directory, and readable.
The manual will do you a lot of good. Expect to do a lot of reading till you learn this stuff, and then reference it a lot to be sure.
Read this section carefully, create some tests for yourself to prove they work and that you understand them, and your job will be more than half done.
Don't use [. Generally it's just better to always use [[ instead, unless you are using (( or case or some other construct.
I don't see that a for loop was specified, but it ought to be fine. Just be aware that you might have to specify $1/* and $1/.* separately.
Put all your tests in one loop, though. For each file, test for whether it's a directory - if it is, report it. Test if it's a plain file - if it is, report it.
I do NOT like doing homework for someone, but it looks like you could use an example that simplifies this. I recommend you not use this as written - break it out and make it clearer, but this is a template for the general logic.
#! /bin/env bash
(( $# )) && [[ -d "$1" ]] && [[ -r "$1" ]] || {
echo "use: $0 <dir>" >&2
exit 1
}
for e in "$1"/.* "$1"/*
do echo "$e:"
[[ -d "$e" ]] && echo " is a directory"
[[ -f "$e" ]] && echo " is a plain file"
[[ -r "$e" ]] && echo " is readable"
[[ -x "$e" ]] && echo " is executable"
done
If you read the links I provided you should be able to break this apart and understand it.
Generally, your script is long and a bit convoluted. Simpler is easier to understand and maintain. For example, be very careful about block indentation to understand scope.
$: for i in 1 2 3
> do echo $i
> done
1
2
3
$: echo $i
3
Compare this to -
for i in $*
do if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
You are testing each entry to see if it is a file, and if it is, reporting "THIS IS A LIST OF FILE and DIRECTORY in $i" every time...
but then only testing the last one to see if it's a directory, because the [ -d $i ] is after the done.
...did you run this somewhere to try it, and look at the results?
I am new to bash scripting and I have to create this script that takes 3 directories as arguments and copies in the third one all the files in the first one that are NOT in the second one.
I did it like this:
#!/bin/bash
if [ -d $1 && -d $2 && -d $3 ]; then
for FILE in [ ls $1 ]; do
if ! [ find $2 -name $FILE ]; then
cp $FILE $3
done
else echo "Error: one or more directories are not present"
fi
The error I get when I try to execute it is: "line 7: syntax error near unexpected token `done' "
I don't really know how to make it work!
Also even if I'm using #!/bin/bash I still have to explicitly call bash when trying to execute, otherwise it says that executing is not permitted, anybody knows why?
Thanks in advance :)
Couple of suggestions :
No harm double quoting variables
cp "$FILE" "$3" # prevents wordsplitting, helps you filenames with spaces
for statement fails for the fundamental reason -bad syntax- it should've been:
for FILE in ls "$1";
But then, never parse ls output. Check [ this ].
for FILE in ls "$1"; #drastic
Instead of the for-loop in step2 use a find-while-read combination:
find "$1" -type f -print0 | while read -rd'' filename #-type f for files
do
#something with $filename
done
Use lowercase variable names for your script as uppercase variables are reserved for the system. Check [this].
Use tools like [ shellcheck ] to improve script quality.
Edit
Since you have mentioned the input directories contain only files, my alternative approach would be
[[ -d "$1" && -d "$2" && -d "$3" ]] && for filename in "$1"/*
do
[ ! -e "$2/${filename##*/}" ] && cp "$filename" "$3"
done
If you are baffled by ${filename##*/} check [ shell parameter expansion ].
Sidenote: In linux, although discouraged it not uncommon to have non-standard filenames like file name.
Courtesy: #chepner & #mklement0 for their comments that greatly improved this answer :)
Your script:
if ...; then
for ...; do
if ...; then
...
done
else
...
fi
Fixed structure:
if ...; then
for ...; do
if ...; then
...
fi # <-- missing
done
else
...
fi
If you want the script executable, then make it so:
$ chmod +x script.sh
Notice that you also have other problems in you script. It is better written as
dir1="$1"
dir2="$2"
dir3="$3"
for f in "$dir1"/*; do
if [ ! -f "$dir2/$(basename "$f")" ]; then
cp "$f" "$dir3"
fi
done
this is not totally correct:
for FILE in $(ls $1); do
< whatever you do here >
done
There is a big problem with that loop if in that folder there is a filename like this: 'I am a filename with spaces.txt'.
Instead of that loop try this:
for FILE in "$1"/*; do
echo "$FILE"
done
Also you have to close every if statement with fi.
Another thing, if you are using BASH ( #!/usr/bin/env bash ), it is highly recommended to use double brackets in your test conditions:
if [[ test ]]; then
...
fi
For example:
$ a='foo bar'
$ if [[ $a == 'foo bar' ]]; then
> echo "it's ok"
> fi
it's ok
However, this:
$ if [ $a == 'foo bar' ]; then
> echo "it's ok";
> fi
bash: [: too many arguments
You've forgot fi after the innermost if.
Additionally, neither square brackets nor find do work this way. This one does what your script (as it is now) is intended to on my PC:
#!/bin/bash
if [[ -d "$1" && -d "$2" && -d "$3" ]] ; then
ls -1 "$1" | while read FILE ; do
ls "$2/$FILE" >/dev/null 2>&1 || cp "$1/$FILE" "$3"
done
else echo "Error: one or more directories are not present"
fi
Note that after a single run, when $2 and $3 refer to different directories, those files are still not present in $2, so next time you run the script they will be copied once more despite they already are present in $3.
Sorry for asking this question again. I have already received answer but with using find but unfortunately I need to write it without using any predefined commands.
I am trying to write a script that will loop recursively through the subdirectories in the current directory. It should check the file count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like:
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirectories don't include subdirectories in turn.
I am confused about this because it doesn't work as I expect.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
How can I modify my script to produce the desired output?
This bash script produces the output that you want:
#!/bin/bash
bigfile="$PWD/BigList"
shortfile="$PWD/ShortList"
shopt -s nullglob
loop_folder_recurse() {
(
[[ -n "$1" ]] && cd "$1"
for i in */; do
[[ -d "$i" ]] && loop_folder_recurse "$i"
count=0
files=''
for j in *; do
if [[ -f "$j" ]]; then
files+="$j"$'\n'
((++count))
fi
done
if ((count > 10)); then
outfile="$bigfile"
else
outfile="$shortfile"
fi
echo "$i" >> "$outfile"
echo "$files" >> "$outfile"
done
)
}
loop_folder_recurse
Explanation
shopt -s nullglob is used so that when a directory is empty, the loop will not run. The body of the function is within ( ) so that it runs within a subshell. This is for convenience, as it means that the function returns to the previous directory when the subshell exits.
Hopefully the rest of the script is fairly self-explanatory but if not, please let me know and I will be happy to provide additional explanation.
I need a script to keep polling "receive_dir" directory till "stopfile" get written in the directory.
This has to run despite empty directory.
So far i have this but fails if receive_dir is empty with no files with "unary operator expected". Help !!
#!/usr/bin/ksh
until [ $i = stopfile ]
do
for i in `ls receive_dir`; do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i;
done
done
This will do what you ask for (loop until the stop file exist). I added a "sleep 1" to lower resource usage. It's also good practice to use "#!/usr/bin/env ksh" as shebang.
#!/usr/bin/env ksh
until [ -e receive_dir/stopfile ]
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
sleep 1
done
If you have empty dir, the
until [ $i = stopfile ]
is evaluated as
until [ = stopfile ]
what is ofcourse syntax error.
One comment: Never parse output from ls.
#!/bin/bash
do_something() {
echo $(date +%m-%d-%Y-%H:%M:%S) "$1"
}
dir="."
until [[ -f "$dir/stopfile" ]]
do
find "$dir" -print0 | while IFS= read -r -d '' filename
do
do_something "$filename"
done
done
or (much slower)
do_something() {
echo $(date +%m-%d-%Y-%H:%M:%S) "$1"
}
export -f do_something
dir="."
until [[ -f "$dir/stopfile" ]]
do
find "$dir" -exec bash -c 'do_something "{}"' \;
done
You're evaluating nothing, and the 'test' isn't able to evaluate it.
~> [ $empty_var = stopfile ]
-bash: [: =: unary operator expected
First, don't parse ls:
http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29
EDIT: Part of the issue is your loop is actually doing the test, try something like this (assuming receive_dir is relative):
#user000001 is right; my original find example would suffer the same issue, so this:
for i in receive_dir/*
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i
[ $i = stopfile ] && break
done
EDIT: Adding in another example based on your comment:
How about this...
FOUND="N"
while [ "${FOUND}" = "N" ]
do
for i in receive_dir/*
do
time=$(date +%m-%d-%Y-%H:%M:%S)
echo $time
echo $i
[ "$i" = stopfile ] && FOUND="Y"
done
sleep 60
done
Another option is to use inotifywait to monitor the status of the directory. For example, this script will run the loop until the file "stopfile" is touched.
until inotifywait "receive_dir" | grep "stopfile"
do
echo "running"
done
echo "done"
The advantage is that these is no busy loop, and that you don't have to repeatedly call the (potentially expensive) find command