in my shell scripts I need to rotate log directories.
I am looking for a more compact, scaleable and elegant way than this, but have currently no idea how to solve this i.e. in a while loop and to calculate with variables.
function f_rotate_logdirs()
{
if [ -d $LOGDIR_OLD14 ]; then
# be extra cautious, no rm -rf operation ...
rm -rf $LOGDIR_OLD14
fi
if [ -d $LOGDIR_OLD13 ]; then
mv $LOGDIR_OLD13 $LOGDIR_OLD14
fi
[...]
if [ -d $LOGDIR_OLD1 ]; then
mv $LOGDIR_OLD1 $LOGDIR_OLD2
fi
if [ -d $LOGDIR ]; then
mv $LOGDIR $LOGDIR_OLD1
fi
mkdir -p $LOGDIR
echo $DATE > $LOGDIR/0.DATE
}
Do you have an idea for a more compact code which easily scales up to n_days ?
Any help on this would be much appreachiated.
Many thanks for this upfront.
The following works:
rm -r -f LOGDIR15
seq 1 14 | tac | xargs -t -n1 sh -c 'if [ -d "$1$2" ]; then mv -n "$1$2" "$1$(($2+1))"; fi' -- LOGDIR
if [ -d "LOGDIR" ]; then mv -n "LOGDIR" "LOGDIR1"; fi
mkdir LOGDIR
We need to handle the first and last separately. Making a function out of it, would be:
backup() {
rm -r -f "$1$2"
seq 1 "$(($2 - 1))" | tac |
xargs -t -n1 sh -c 'if [ -d "$1$2" ]; then mv -n "$1$2" "$1$(($2+1 ))"; fi' -- "$1"
if [ -d "$1" ]; then mv -n "$1" "$1"1; fi
mkdir "$1"
}
with usage:
backup LOGDIR 15
would move the directory named LOGDIR to LOGDIR1 and LOGDIR2 and .. LOGDIR15.
It seems unknown to me, why you use variables $LOGDIR_OLD15 and not just directory names themselves.
Thanks for your input, the commands seq and tac were new to me.
Now I found some time to code it newly and I took some of your nice ideas.
My aim was to get a scaleable solution which is easy to understand and maintain.
I decided to name the current logdir "$dir.0" because some of my scripts gather config diffs of devices that are being taking on daily basis. This makes coding a little bit easier for getting the diffs between i.e. "5 and 3 days ago" or between "2 days ago and today".
#! /bin/sh
# Customizeable settings
LOGDIR_BACKUPS=14 # max number of backups
# Directories
LOGDIR_NAME=logs
LOGDIR=$LOGDIR_NAME.0
# Defines
DATE=`date +'%Y%m%d-%H%M'`
function f_rotate_logdirs() {
local dir=$1 # name of logdir folder
local max=$2 # max #
local min=0 # $dir.0 = current logdir
for i in `seq $min $max | tac`
do
case $i in
$max) if [ -d $dir.$i ]; then
rm -f $dir.$i/*
rmdir $dir.$i
fi
;;
$min) if [ -d $dir.$i ]; then
mv $dir.$i $dir.$((i+1))
fi
mkdir $dir.$i
echo $DATE > $dir.$i/0.DATE
;;
*) if [ -d $dir.$i ]; then
mv $dir.$i $dir.$((i+1))
fi
;;
esac
done
}
f_rotate_logdirs $LOGDIR_NAME $LOGDIR_BACKUPS
Related
I searched and couldn't find anything, maybe I can't understand the problem properly.
I have a bash function who read files in current dir and sub dir's, I'm trying to arrange the text and analyze the data but somehow I'm losing lines if I'm using pipeline.
the code:
function recursiveFindReq {
for file in *.request; do
if [[ -f "$file" ]]; then
echo handling "$file"
echo ---------------with pipe-----------------------
cat "$file" | while read -a line; do
if (( ${#line} > 1 )); then
echo ${line[*]}
fi
done
echo ----------------without pipe----------------------
cat "$file"
echo
echo num of lines: `cat "$file" | wc -l`
echo --------------------------------------
fi
done
for dir in ./*; do
if [[ -d "$dir" ]]; then
echo cd to $dir
cd "$dir"
recursiveFindReq "$1"
cd ..
fi
done
}
the output is:
losing lines even when they meet requirements
I marked with 2 red arrows the place I'm losing info
actual_size="du -h $some_folder";
y='eval $actual_size`
max_size=2MB
if [[ y -lt $max_size ]]; then
echo `du -h $some_folder`
cp "$file" $some_folder` #$file points to some file
Whats wrong with above lines of code? I also tried
if [[ $(stat -c %s $some_folder) -lt $max_size ]]; then
any help would be appreciated, Thnaks!
In order to use POSIX du in a script it helps to have a wrapper function which only prints the output of its second to last field. This can be accomplished with awk, or here is how I do it using eval to expand the second to last argument to a function:
#!/bin/sh
duprint()
{
if [ "$#" -gt 1 ]
then printf "%s" $( eval printf '$'$(( $# - 1 )))
fi
}
duwrap()
{
duprint $(du -s $# 2>/dev/null)
}
duwrap "$#"
This is useful for instance if you want the total size of several directories in anticipation of copying them over to a new filesystem:
./duwrapper /bin /etc /lib /boot
Finally, one can use their du wrapper function to compare directory sizes:
if [ "$(duwrap /lib)" -gt "$(duwrap /bin)" ]
then echo "/lib is greater"
fi
I'm trying to write a shell script that will recursively count all the files and sub-directories in a directory and also all the hidden ones. My script can count them however it can't detect hidden files and directories that are in a sub-directory. How can i change it so that it is able to do this? Also i cannot use find, du or ls -R
#!/bin/bash
cd $1
dir=0
hiddendir=0
hiddenfiles=0
x=0
items=( $(ls -A) )
amount=( $(ls -1A | wc -l) )
counter() {
if [ -d "$i" ]; then
let dir+=1
if [[ "$i" == .* ]]; then
let hiddendir+=1
let dir-=1
fi
search "$i"
elif [ -f "$i" ]; then
let files+=1
if [[ "$i" == .* ]]; then
let files-=1
let hiddenfiles+=1
fi
fi
}
search() {
for i in $1/*; do
counter "$i"
done
}
while [ $x -lt $amount ]; do
i=${items[$x]}
counter "$i"
let x+=1
done
#!/bin/bash -e
shopt -s globstar dotglob # now ** lists all entries recursively
cd "$1"
dir=0 files=0 hiddendir=0 hiddenfiles=0
counter() {
if [ -f "$1" ]; then local typ=files
elif [ -d "$1" ]; then local typ=dir
else continue
fi
[[ "$(basename "$1")" == .* ]] && local hid=hidden || local hid=""
((++$hid$typ))
}
for i in **; do
counter "$i"
done
echo $dir $files $hiddendir $hiddenfiles
Consider using this:
find . | wc -l
Sorry for asking this question again. I have already received answer but with using find but unfortunately I need to write it without using any predefined commands.
I am trying to write a script that will loop recursively through the subdirectories in the current directory. It should check the file count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like:
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirectories don't include subdirectories in turn.
I am confused about this because it doesn't work as I expect.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
How can I modify my script to produce the desired output?
This bash script produces the output that you want:
#!/bin/bash
bigfile="$PWD/BigList"
shortfile="$PWD/ShortList"
shopt -s nullglob
loop_folder_recurse() {
(
[[ -n "$1" ]] && cd "$1"
for i in */; do
[[ -d "$i" ]] && loop_folder_recurse "$i"
count=0
files=''
for j in *; do
if [[ -f "$j" ]]; then
files+="$j"$'\n'
((++count))
fi
done
if ((count > 10)); then
outfile="$bigfile"
else
outfile="$shortfile"
fi
echo "$i" >> "$outfile"
echo "$files" >> "$outfile"
done
)
}
loop_folder_recurse
Explanation
shopt -s nullglob is used so that when a directory is empty, the loop will not run. The body of the function is within ( ) so that it runs within a subshell. This is for convenience, as it means that the function returns to the previous directory when the subshell exits.
Hopefully the rest of the script is fairly self-explanatory but if not, please let me know and I will be happy to provide additional explanation.
I have this script that copies image files from source directory to destination directory. There are some image files in the source directory that have the same name but different file size. This script also compares the two files with the same name using a stat command. Now, I want to add a string suffix e.g. IMG0897.DUP.JPG before the file extension to the files with the same file name that are going to be copied over to the destination folder. At the moment, my script adds the file size of the file to the file name.
I need help on how to add a string of text of my own rather than the size of the file.
Here's my script:
#!/bin/sh
SEARCH="IMG_*.JPG"
SOURCE= $1
DEST=$2
test $# -ne 2 && echo Usage : phar image_path archive_path
if [ ! -e $1 ]
then echo Source folder does not exist
fi
if [ ! -e $2 ]
then mkdir $2/
fi
# Execute the script.
if [ "${SEARCH%% *}" = "$SEARCH" ]; then
command="find \"$1\" -name \"$SEARCH\""
else
command="find \"$1\" -name \"${SEARCH%% *}\""$(for i in ${SEARCH#* }; do echo -n " -o -name \"$i\""; done)
fi
# Run the main loop.
eval "$command" | while read file; do
bn=$(basename "$file")
bc=$(stat -c%s "$file")
if [ -f "${2}/$bn" ] && [ "$bc" -ne $(stat -c%s "${2}/$bn") ]; then
bn="$bn.$bc"
fi
if [ -f "${2}/$bn" ]; then
echo "File ${2}/$bn already exists."
else
echo "Copying $file to $2/$bn"
cp -a "$file" "$2/$bn"
fi
done
exit 0
else
echo "Error : Can't find $1 or $2"
exit 1
fi
I modified your scripte slightly.
#!/bin/sh
SEARCH="IMG_*.JPG"
SOURCE=$1
DEST=$2
SUFFIX=DUP
test $# -ne 2 && echo Usage : phar image_path archive_path
if [ ! -e $1 ]
then echo Source folder does not exist
fi
if [ ! -e $2 ]
then mkdir $2/
fi
# Execute the script.
if [ "${SEARCH%% *}" = "$SEARCH" ]; then
command="find \"$1\" -name \"$SEARCH\""
else
command="find \"$1\" -name \"${SEARCH%% *}\""$(for i in ${SEARCH#* }; do echo -n " -o -name \"$i\""; done)
fi
# Run the main loop.
eval "$command" | while read file; do
bn=$(basename "$file")
bc=$(stat -c%s "$file")
if [ -f "${2}/$bn" ] && [ "$bc" -ne $(stat -c%s "${2}/$bn") ]; then
bc=$(echo ${bn}|cut -d. -f2)
bn=$(echo ${bn}|cut -d. -f1)
bn=$bn.$SUFFIX.$bc**
fi
if [ -f "${2}/$bn" ]; then
echo "File ${2}/$bn already exists."
else
echo "Copying $file to $2/$bn"
cp -a "$file" "$2/$bn"
fi
done
exit 0
else
echo "Error : Can't find $1 or $2"
exit 1
fi
My execution result is:
root#precise32:/vagrant# sh JPG_moves.sh /root/dir1/ /root/destination/
Copying /root/dir1/IMG_0897.JPG to /root/destination//IMG_0897.JPG
root#precise32:/vagrant# sh JPG_moves.sh /root/dir2/ /root/destination/
Copying /root/dir2/IMG_0897.JPG to /root/destination//IMG_0897.DUP.JPG