I wrote the following script:
for filename in `find . -name '*'.cpp | grep $IN_REGEX | grep -v $OUT_REGEX`
do
echo "Output file is $OUTPUT_FILE"
count=`git log --pretty=format: --name-only $filename | grep -v ^$ | wc -l`
echo "$count $filename" >> $OUTPUT_FILE
done
But nothing gets written into the output file.
Please note:
I have set the values for OUTPUT_FILE, IN_REGEX and OUT_REGEX.
The code inside the loop is being executed. I checked this with an sh -x invokation.
When I remove the >> $OUTPUT_FILE I get the output.
I tried a touch $OUTPUT_FILE inside the script and that is working fine.
Can someone please point out what is my mistake here?
This line of code
for filename in `find . -name '*'.cpp
is a
(break on space in file names)
You should instead do :
while IFS= read -r file; do
echo "Output file is $OUTPUT_FILE"
count=$(git log --pretty=format: --name-only "$file" | grep -v '^$' | wc -l)
echo "$count $file" >> "$OUTPUT_FILE"
done < <(find . -name '*.cpp' | grep "$IN_REGEX" | grep -v "$OUT_REGEX")
For this to work, ensure that $OUTPUT_FILE have a path in it.
Related
what I am trying to achieve is, to delete same filenames(filename+modfiedtimestamp)exisitng in Src_Dir1 and Src_Dir2
So first i have tried to deploy all the filenames to tempa(Src_Dir1) and tempb(Src_Dir2) respectively.
Below is the screenshot of the source directory.
Files inside archive be like this and few files outside too..
So, initially I am want to deal with the files inside Archive(SRC_Dir1) and later outside Archive(SRC_Dir2) what I am trying to do is to use a while loop to read each and every filename and string concat with the modified timestamp(mtime) and input to tempc(like for example it should be like AirTimeActs_2018-12-03.csv+2019-01-24 14:41:53.000000000 -0500 = AirTimeActs_2018-12-03.csv_2019-01-24 14:41:53.000000000 -0500 this is how it should be generating into tempc file for each and every filename inside Archive(SRC_Dir1). This is where I am stuck under string concat variable section on how to proceed. Please help me with the code, hope I am comprehensible.
IMPORTANT
(Really appreciate it, if you help me out with the extension of the code which i haven't mentioned here and yet to achieve which is - >
Have to implement the same code(which I am trying to do for tempa, I'd like to do it for tempb too and name it as tempd) and then do a file data compare between tempc and tempd) if there is any kind of same data filename, then delete the file existing in Src_Dir2, if there is no same data filename, then do nothing.)
#!/bin/bash
Src_Dir1=path/Airtime_Activation/Archive
Src_Dir2=path/Airtime_Activation/
find "$Src_Dir1" -maxdepth 1 -name "*.xlsx" -o -name "*.csv" | sed "s/.*\///" > -print>path/Airtime_Activation/temp_a
find "$Src_Dir2" -maxdepth 1 -name "*.xlsx" -o -name "*.csv" | sed "s/.*\///" > -print>path/Airtime_Activation/temp_b
echo 'phase1'
cat path/Airtime_Activation/temp_a | while read file;
do
echo 'phase1.5'
echo "$file"
echo 'phase2'
mtime=$(stat -c '%y' $file)
Full_name=${file}_${mtime}
echo "$Full_name" >> path/Airtime_Activation/temp_c
echo 'phase3'
done
#!/bin/bash
Src_Dir1=path/Airtime_Activation/Archive
Src_Dir2=path/Airtime_Activation/
find "$Src_Dir1" -maxdepth 1 -name "*.xlsx" -o -name "*.csv" | sed "s/.*\///" > -print>path/Airtime_Activation/temp_a
find "$Src_Dir2" -maxdepth 1 -name "*.xlsx" -o -name "*.csv" | sed "s/.*\///" > -print>path/Airtime_Activation/temp_b
echo 'phase1'
cat path/Airtime_Activation/temp_a | while read file;
do
echo 'phase1.5'
echo "$file"
echo 'phase2'
mtime=$(stat -c '%y' $file)
Full_name=${file}_${mtime}
echo "$Full_name" >> path/Airtime_Activation/temp_c
echo 'phase3'
done
cat /path/Airtime_Activation/temp_b | while read file
#while IFS="" read -r -d $'\0' file;
do
#echo "$file"
echo 'phase2'
mtime=$(stat -c '%y' $Src_Dir2/$file)
Full_name=${file}_${mtime}
echo "$Full_name" >> path/temp_d
echo 'phase3'
done
#file compare and delete old files from outisde archive
grep -Ff temp_d temp_c > path/Airtime_Activation/temp_e
cat path/Airtime_Activation/temp_e | while read file
#while IFS="" read -r -d $'\0' file;
do
#echo "$file"
echo 'phase2'
echo "${file%_*}"
rm $Src_Dir2/${file%_*}
echo 'phase3'
done
I run the below script in a reference env and get the correct output in the output file, but when I run the same in production the output file is empty.
I tried debugging the using set -x, and understand that the for loop is not getting executed in the prod env.
please suggest what might be the issue.
#!/bin/ksh
DIR=/some/log/dir/error
DATE=$1
OUTPUT=/some/logs/dir/scripts/output/output.csv
. /calling/env/setupscript.ksh
for file in $(find $DIR 2>/dev/null| grep $ASOF | grep -i something1 | grep -vi someotherthing2 | grep -iv someotherthing3 | grep -vi someotherthing4 | grep -vi someotherthing4 )
do
echo "Checking file $file..."
if [[ -z "$DONE" ]] then
head -1 $file | read line
echo "File name;Filter name;Type;Scenario;$line" > $OUTPUT.tmp
DONE=1
fi
fullfile=$file
file=$(basename $file)
file=${file#SPR_RPT_}
file=${file#SPR_BY__}
echo $file | awk -F_ '{ print $(NF-2)" "$(NF-1) }' | read type scen
filter=${file%%_$type\_*}
grep ERROR $fullfile | while read line
do
echo "$(basename $fullfile);$filter;$type;$scen;$line" >> ${OUTPUT}.tmp
done
done
i'm trying to write a code which will print all files taking more than min_size (lets say 10G) in a directory. the problem is output off the below code is all files irrespective of the min_size. i will be getting other details like mtime , owner as well later in the code but this part itself doesnt work fine, whats wrong here ?
#!/bin/sh
if (( $# <3 )); then
echo "$0 dirname min_size count"
exit 1
else
dirname="$1";
min_size="$2";
count="$3";
#shift 3
fi
tmpfile=$(mktemp /lawdump/pulkit/files.XXXXXX)
exec 3> "$tmpfile"
find "${dirname}" -type f -print0 2>&1 | grep -v "Permission denied" | xargs -0 -I {} echo "{}" > "$tmpfile"
for i in `cat tmpfile`
do
x="`du -ah $i | awk '{print $1}' | grep G | sort -nr -k 1`"
size=$(echo $x | sed 's/[A-Za-z]*//g')
if [ size > $min_size ];then
echo $size
fi
done
Note : i know this can be done through find or du but i need to write a shell script to have an email sent out regularly with all the details.
Can you tell me how can you count the files with the extension ".txt" you delete from a folder? Shell script in Unix
Thank you for your answer :)
I tried to delete them this way :
deleted=0
while read line
do
if test -d "$line"
then
for i in "$line"/*
do
if test -f "$i"
then
deleted=`ls -l $line |grep "*.o" | wc -l`
echo "From: " $line " I deleted : " $deleted
find . -type f -name "*.o" -exec rm -f {} \;
else
echo "Not file " $i
fi
done
else
echo "NOT a directory!"
fi
done
Try doing this :
LANG=C rm -v *.txt | grep -c "^removed "
An answer - though not necessarily the right one:
files=*.txt
ls -1 "$files" | wc -l
rm "$files"
Ruth
How do I list newest directory and add as variable to bash script to process files recursively
ls -t1 | head -n1
Works perfectly to list the latest directory, but I want to add that directory name to my script so I can process the files within using the following script:
#!/bin/bash
ls | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr ' ' '_' `
done
ls | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr '\*.JPEG' '\*.jpg' `
done
mogrify -resize 750 *.jpg
wait
jpegoptim *.jpg –max=70 --strip-all
exit
I also want to process the files recursively, there might be at most one level of sub directories.
Basically keep the bash script at the root of the directory and process all latest directories and sub directories files.
OK I modified the script to this:
#!/bin/bash
DIR=ls -t1 | head -n1
ls $DIR | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr ' ' '_' `
done
ls $DIR | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr '\*.JPEG' '\*.jpg' `
done
mogrify -resize 750 $DIR/*.jpg
wait
jpegoptim $DIR/*.jpg –max=70 --strip-all
exit
But it does not seem to recognise the $DIR variable.
This bash script will rename and convert the jpg files in the newest directory in the current directory and the files in the first level of directories under that directory.
#!/bin/bash
FIRST_DIR=`ls -t1F | grep / | head -n1`
DIR="./${FIRST_DIR}"
ls -t1F $DIR | while read -r FILE
do
if [ "$FILE" ]
then
if [[ $FILE = */ ]]
then
echo "here ${DIR}${FILE}."
DEEP_DIR="${DIR}${FILE}"
ls -t1 $DEEP_DIR | while read -r FILE2
do
if [ "$FILE2" ]
then
if [[ $FILE2 != */ ]]
then
RENAME=`echo ${FILE2//\*/} | tr ' ' '_' `
mv -v "${DEEP_DIR}${FILE2//\*/}" "${DEEP_DIR}${RENAME}"
FILE2=$RENAME
RENAME2=`echo ${FILE2//\*/} | tr '\*.JPEG' '\*.jpg' `
mv -v "${DEEP_DIR}${FILE2//\*/}" "${DEEP_DIR}${RENAME2}"
FILE2=$RENAME2
fi
fi
done
mogrify -resize 750 "$DEEP_DIR*.jpg"
wait
jpegoptim "$DEEP_DIR*.jpg" –max=70 --strip-all
fi
if [[ $FILE != */ ]]
then
RENAME=`echo ${FILE//\*/} | tr ' ' '_' `
mv -v "${DIR}${FILE//\*/}" "${DIR}${RENAME}"
FILE=$RENAME
RENAME2=`echo ${FILE//\*/} | tr '\*.JPEG' '\*.jpg' `
mv -v "${DIR}${FILE//\*/}" "${DIR}${RENAME2}"
FILE=$RENAME2
fi
fi
done
if [ "$FIRST_DIR" ]
then
mogrify -resize 750 "$DIR*.jpg"
wait
jpegoptim "$DIR*.jpg" –max=70 --strip-all
fi
Here are a couple of good links about bash programming:
http://tldp.org/LDP/abs/html/comparison-ops.html
http://tldp.org/LDP/abs/html/string-manipulation.html