how to create incremental files with version in shell - shell

how to create files dynamically when i run shell script.
Intially in the /tmp folder i need to check if file like(CFT_AH-120_v1.txt) exist in tmp folder else create CFT_AH-120_v1.txt. Next time when i run shell script it should create CFT_AH-120_v2.txt and in each run it should increment the version number of the file.
In tmp folder i should have files like
CFT_AH-120_v1.txt
CFT_AH-120_v2.txt
CFT_AH-120_v3.txt
i will get CFT_AH-120 from variable dynamically.
#!/bin/bash
export filename
temp=$(find CFT_AH-120-V* | sort -V | tail -1)
if [ -e $temp ]
then echo "ok"
echo $temp
fname="${temp%.*}"
echo $fname
temp1="${temp%[[:digit:]]*}$((${temp##*[[:alpha:]]} + 1))"
echo $temp1
touch $temp1 ".txt"
else
touch CFT_AH-120-V1.txt
echo "nok"
fi

I am not sure the exact requirement for you. As per my understanding this is a simple way of approach.
#!/bin/bash
file_temp=$(find . -name "CFT_AH-120-V*" -type f | sort -V | tail -1)
echo $temp
if [ -z "$temp" ]; then
echo "File not found!"
else
num_temp=$(echo $temp | cut -d '-' -f3- | sed 's/V//')
num_value_incr=$(expr $new_temp + 1)
touch "CFT_AH-120-V$num_value_incr"
echo "New file created!"
fi
Note: This code search for the largest value of the "V"....number.... (eg.***V120) and increment based on that value and also the intermediate number wont be increment. If you need the missing numbers to be created, then the logic for this code needs to be changed.
Hope this might help you!

Related

How to rename the filename of that folder name and copy all files into one folder

I am new to unix . I got a requirement like this .
I have xml folder in the server . In that folder , everyday , i will get different employee details for each employee in one folder .
/server/user/home/xml/e1100123/Employeedetails.xml
/server/user/home/xml/e1100123/Employeesalary.xml
/server/user/home/xml/e1100123/Employeeleaves.xml
/server/user/home/xml/e1100123/Employeestatus.xml
/server/user/home/xml/e1100155/Employeedetails.xml
/server/user/home/xml/e1100155/Employeesalary.xml
/server/user/home/xml/e1100155/Employeeleaves.xml
/server/user/home/xml/e1100155/Employeestatus.xml
I have to group all employees in one folder with filename_employeenumber as shown below .
/server/user/home/xml/allemployees/Employeedetails-e1100123.xml
/server/user/home/xml/allemployees/Employeesalary-e1100123.xml
/server/user/home/xml/allemployees/Employeeleaves-e1100123.xml
/server/user/home/xml/allemployees/Employeestatus-e1100123.xml
/server/user/home/xml/allemployees/Employeedetails-e1100155.xml
/server/user/home/xml/allemployees/Employeesalary-e1100155.xml
/server/user/home/xml/allemployees/Employeeleaves-e1100155.xml
/server/user/home/xml/allemployees/Employeestatus-e1100155.xml
How to write a code in unix shell script ?
Thank you ......
Sai
Bare bone of your desired script could be like so:
script.sh
#!/bin/bash
base_dir=/server/user/home/xml
all_dir=${base_dir}/allemployes
e_dirs=$(ls -d ${base_dir}/e*)
e_files=(Employeedetails.xml Employeesalary.xml Employeestatus.xml Employeeleaves.xml)
if [ ! -d "${base_dir}" ]; then
echo "ERROR: ${base_dir} doesn't exists!"
fi
if [ ! -d "${all_dir}" ]; then
echo "INFO: Creating all employees directory at ${all_dir}"
mkdir "${all_dir}"
fi
for e in ${e_dirs}; do
pushd "${e}" > /dev/null
for f in ${e_files[#]}; do
if [ ! -f ${f} ]; then
echo "ERROR: Missing ${f} file for employee $(basename ${e})"
else
new_file=${all_dir}/${f%.*}-$(basename ${e}).xml
cp "${e}/${f}" "${new_file}"
fi
done
popd > /dev/null
done
Notes:
Script assumes that in xml/ folder are only employees folder and every folder starts with lower character e, hence we do e_dirs=$(ls -d ${base_dir}/e*)
Script works only with files Employeedetails.xml Employeesalary.xml Employeestatus.xml Employeeleaves.xml file, ignoring others, in each employee folder.
Script creates allemployes/ folder if it does not exists.
Script makes copies instead of moving (not destructive), so you can use mv instead of cp
I haven't tested the script! So please do not run it on live data :). Feel free to modify it.
Use this script :
#!/bin/bash
sourceDir=$1
destDir=$2
dirname=$(basename $1)
if [ ! -d $2 ] ; then
mkdir $2
fi
for I in $sourceDir/*
do
fname=$(basename $I)
prefix=$(echo $fname | awk -F "." 'NF{NF-=1};1')
suffix=$(echo $fname | awk -F "." '{print $NF}')
target="$destDir/$prefix-$dirname.$suffix"
cp $I $target
# I use cp(Copy) command , you can replace this to mv(Move) command .
echo $target
done
Usage :
./script.sh [SOURCEDIR] [DESTDIR]
Example :
./script.sh /server/user/home/xml/e1100123/ /server/user/home/xml/allemployees/

Nested for loop to enter and exit multiple directories Bash script

As an example, I have 7 directories each containing 4 files. The 4 files follow the following naming convention name_S#_L001_R1_001.fastq.gz. The sed command is to partially keep the unique file name.
I have a nested for loop in order to enter a directory and perform a command, exit the directory and proceed to the next directory. Everything seems to be working beautifully, however the code gets stuck on the last directory looping 4 times.
for f in /completepath/*
do
[ -d $f ] && cd "$f" && echo Entering into $f
for y in `ls *.fastq.gz | sed 's/_L00[1234]_R1_001.fastq.gz//g' | sort -u`
do
echo ${y}
done
done
Example output-
Entering into /completepath/m_i_cast_avpv_1
iavpvcast1_S6
Entering into /completepath/m_i_cast_avpv_2
iavpvcast2_S6
Entering into /completepath/m_i_int_avpv_1
iavpvint1_S5
Entering into /completepath/m_i_int_avpv_2
iavpvint2_S5
Entering into /completepath/m_p_cast_avpv_1
pavpvcast1_S8
Entering into /completepathd/m_p_int_avpv_1
pavpvint1_S7
Entering into /completepath/m_p_int_avpv_2
pavpvint2_S7
pavpvint2_S7
pavpvint2_S7
pavpvint2_S7
Any recommendations of how to correctly exit the inner loop?
It looks like /completepath/ contains some entries that are not directories. When the loop over /completepath/* sees something that's not a directory, it doesn't enter it, thanks to the [ -d $f ] check.
But it still continues to run the next for y in ... loop.
At that point the script is still in the previous directory it has seen.
One way to solve that is to skip the rest of the loop when $f is not a directory:
if [ -d $f ]; then
cd "$f" && echo Entering into $f
else
continue
fi
There's an even better way. By writing /completepath/*/ only directory entries will be matched, so you can simplify your loop to this:
for f in /completepath/*/
do
cd "$f" && echo "Entering into $f" || { echo "Error: could not enter into $f"; continue; }
for y in $(ls *.fastq.gz | sed 's/_L00[1234]_R1_001.fastq.gz//g' | sort -u)
do
echo ${y}
done
done

Bash Script - Will not completely execute

I am writing a script that will take in 3 outputs and then search all files within a predefined path. However, my grep command seems to be breaking the script with error code 123. I have been staring at it for a while and cannot really seem the error so I was hoping someone could point out my error. Here is the code:
#! /bin/bash -e
#Check if path exists
if [ -z $ARCHIVE ]; then
echo "ARCHIVE NOT SET, PLEASE SET TO PROCEED."
echo "EXITING...."
exit 1
elif [ $# -ne 3 ]; then
echo "Illegal number of arguments"
echo "Please enter the date in yyyy mm dd"
echo "EXITING..."
exit 1
fi
filename=output.txt
#Simple signal handler
signal_handler()
{
echo ""
echo "Process killed or interrupted"
echo "Cleaning up files..."
rm -f out
echo "Finsihed"
exit 1
}
trap 'signal_handler' KILL
trap 'signal_handler' TERM
trap 'signal_handler' INT
echo "line 32"
echo $1 $2 $3
#Search for the TimeStamp field and replace the / and : characters
find $ARCHIVE | xargs grep -l "TimeStamp: $2/$3/$1"
echo "line 35"
fileSize=`wc -c out.txt | cut -f 1 -d ' '`
echo $fileSize
if [ $fileSize -ge 1 ]; then
echo "no"
xargs -n1 basename < $filename
else
echo "NO FILES EXIST"
fi
I added the echo's to know where it was breaking. My program prints out line 32 and the args but never line 35. When I check the exit code I get 123.
Thanks!
Notes:
ARCHIVE is set to a test directory, i.e. /home/'uname'/testDir
$1 $2 $3 == yyyy mm dd (ie a date)
In testDir there are N number of directories. Inside these directories there are data files that have contain data as well as a time tag. The time tag is of the following format: TimeStamp: 02/02/2004 at 20:38:01
The scripts goal is to find all files that have the date tag you are searching for.
Here's a simpler test case that demonstrates your problem:
#!/bin/bash -e
echo "This prints"
true | xargs false
echo "This does not"
The snippet exits with code 123.
The problem is that xargs exits with code 123 if any command fails. When xargs exits with non-zero status, -e causes the script to exit.
The quickest fix is to use || true to effectively ignore xargs' status:
#!/bin/bash -e
echo "This prints"
true | xargs false || true
echo "This now prints too"
The better fix is to not rely on -e, since this option is misleading and unpredictable.
xargs makes the error code 123 when grep returns a nonzero code even just once. Since you're using -e (#!/bin/bash -e), bash would exit the script when one of its commands return a nonzero exit code. Not using -e would allow your code to continue. Just disabling it on that part can be a solution too:
set +e ## Disable
find "$ARCHIVE" | xargs grep -l "TimeStamp: $2/$1/$3" ## If one of the files doesn't match the pattern, `grep` would return a nonzero code.
set -e ## Enable again.
Consider placing your variables around quotes to prevent word splitting as well like "$ARCHIVE".
-d '\n' may also be required if one of your files' filename contain spaces.
find "$ARCHIVE" | xargs -d '\n' grep -l "TimeStamp: $2/$1/$3"

script to check last file modified

SHELL SCRIPT TO GET MAIL IF FILE GET MODIFIED
I am writing script to get mail if file has been modified
recip="mungsesagar#gmail.com"
file="/root/sagar/ldapadd.sh"
#stat $file
last_modified=$(stat --printf=%y $file | cut -d. -f1)
#echo $last_modified
mail -s "File ldapadd.sh has changed" $recip
Now I get mail when I run this script but I want to compare two variables so that I can get mail only if file modified or content changed.
How can I store output in variable to compare
Thanks in advance
Sagar
I'd do it this way:
recip="you#example.com"
file="/root/sagar/ldapadd.sh"
ref="/var/tmp/mytimestamp.dummy"
if [ "$file" -nt "$ref" ]; then
mail -s "File ldapadd.sh has changed" $recip
fi
touch -r "$file" "$ref" # update our dummy file to match
The idea is to store the last seen timestamp of the file of interest by copying it to another file (using touch). Then we always know what the last time was, and can compare it against the current timestamp on the file and email as needed.
If I understand your question correct, the logic can be changed by storing the output of "ls -ltr filename" in a temp1 file and comparing the same with the ls -ltr output
I would use find to see the last modifyed file
#!/bin/bash
file=timestamp.txt
if [ ! -f timestamp.txt ];
then
stat -f %Sm -t %Y%m%d%H%M%S $file > timestamp.txt
else
timestamp=$(stat -f %Sm -t %Y%m%d%H%M%S $file)
filetime=$(cat filetime.txt)
if [ "$filetime" = "$timestamp" ];
then
#Do nothing
else
echo "$file has been modified" >> /tmp/email.txt
mail -s "File has changed" "email#domain.com" < /tmp/email.txt
fi
fi

help with shell script for finding and moving files based on condition

Looking for some help with my bash script. I am trying to write this shell script to do the following:
find files in a dir named:
server1-date.done
server2-date.done
server3-date.done
...
server10-date.done
print to a listA
find files in a dir (*.gz) and print to a listB
if listA has a count of 10 (basically found 10 .done files), then
proceed with the moving the files in listB to its new directory
after moving the files from listB, then remove the old directory which is similarly named (server1-date, server2-date, ...) and the .done files.
So far, I have this in the works. I can't get the condition for the if section working. I don't think I coded that correctly. Any code suggestions, improvements, etc would be appreciated. Thanks.
#Directories
GZDIR=/mydumps/mytest
FINALDIR=/mydumps/mytest/final
FLGDIR=/backup/mytest/flags
export GZDIR FINALDIR FLGDIR
#lists
FLGLIST="/mydumps/mytest/lists/userflgs.lst"
GZLIST="/mydumps/mytest/lists/gzfiles.lst"
export FLGLIST GZLIST
#Find files
find $FLGDIR -name \*.done -print > $FLGLIST
find $GZDIR -name \*.gz -print > $GZLIST
#Get need all (10) flags found before we do the move
FLG_VAL =`cat $FLGLIST | wc -l`
export $FLG_VAL
if [ "$FLG_VAL" = "10" ]; then
for FILE in $GZLIST
do
echo "mv $GZLIST $FINALDIR" 2>&1
for FLAG in $FLGLIST
do
echo "rmdir -f $FLAG" 2>&1
done
done
else
echo "Cannot move file" 2>&1
exit 0
fi
I do not know if this will work, but it will fix all the obvious problems:
#!/bin/sh
#Directories
GZDIR=/mydumps/mytest
FINALDIR=/mydumps/mytest/final
FLGDIR=/backup/mytest/flags
export GZDIR FINALDIR FLGDIR
#lists
FLGLIST="/mydumps/mytest/lists/userflgs.lst"
GZLIST="/mydumps/mytest/lists/gzfiles.lst"
#Find files
find "$FLGDIR" -name '*.done' -print > "$FLGLIST"
find "$GZDIR" -name '*.gz' -print > "$GZLIST"
#Get need all (10) flags found before we do the move
FLG_VAL=$(wc -l <"$FLGLIST") # Always prefer $( ... ) to backticks.
if [ "$FLG_VAL" -ge 10 ]; then
for FILE in $(cat "$GZLIST")
do
echo "mv $FILE $FINALDIR" 2>&1
done
for FLAG in $(cat "$FLGLIST")
do
echo "rmdir -f $FLAG" 2>&1
done
else
echo "Cannot move file" 2>&1
exit 0
fi
First of all, I really recommend that you as the default approach always test for exceptions, and do not include the "normal" case inside a test unless that is necessary.
...
FLG_VAL=`wc -l < $FLGLIST` # no need for cat, and no space before '='
export $FLG_VAL
if [ "$FLG_VAL" != "10" ]; then
echo "Cannot move file" 2>&1
exit 0
fi
for FILE in $GZLIST
do
echo "mv $GZLIST $FINALDIR" 2>&1
for FLAG in $FLGLIST
do
echo "rmdir -f $FLAG" 2>&1
done
done
See how much easier the code is to read now that the error check is extracted and stands by itself?
FLG_VAL =`cat $FLGLIST | wc -l`
Should be:
FLG_VAL=`cat $FLGLIST | wc -l`

Resources