I searched and couldn't find anything, maybe I can't understand the problem properly.
I have a bash function who read files in current dir and sub dir's, I'm trying to arrange the text and analyze the data but somehow I'm losing lines if I'm using pipeline.
the code:
function recursiveFindReq {
for file in *.request; do
if [[ -f "$file" ]]; then
echo handling "$file"
echo ---------------with pipe-----------------------
cat "$file" | while read -a line; do
if (( ${#line} > 1 )); then
echo ${line[*]}
fi
done
echo ----------------without pipe----------------------
cat "$file"
echo
echo num of lines: `cat "$file" | wc -l`
echo --------------------------------------
fi
done
for dir in ./*; do
if [[ -d "$dir" ]]; then
echo cd to $dir
cd "$dir"
recursiveFindReq "$1"
cd ..
fi
done
}
the output is:
losing lines even when they meet requirements
I marked with 2 red arrows the place I'm losing info
I would like to compare two binary files (very small, 100Kb each) and replace the oldest with the last modified one.
I have created a simple script, but I would need your help to make it running properly:
#!/bin/sh
# select the two files
FILE1="/dir1/file1.binary"
FILE2="/dir2/file2.binary"
# create the hash of the two files
HASH1="$(md5sum $FILE1 | cut -c 1-32)"
HASH2="$(md5sum $FILE2 | cut -c 1-32)"
# compare the two hashes
if [ "$HASH1" == "$HASH2" ];
# if the two hashes are the same, exit
then
echo "the two files are identical"
exit 0
# otherwise compare which of them has been last modified
fi
DATE1="(stat -c %Y $FILE1)"
DATE2="(stat -c %Y $FILE2)"
# if FILE1 is newer than FILE2, replace FILE2 with FILE1
if [ "${DATE1}" -gt "${DATE2}" ];
then
cp $FILE1 $FILE2
echo "${FILE2} was replaced by ${FILE1}"
# if FILE2 is newer than FILE1, replace FILE1 with FILE2
fi
cp $FILE2 $FILE1
echo "${FILE1} was replaced by ${FILE2}"
exit 0
The file seems working (at least if the two files are identical), but if one file has been modified, I receive the following error:
line 24: [: {(stat -c %Y test1)}: integer expression expected
What is wrong?
By the way, is there a better way to solve this problem?
Thanks
Thank you so much everybody for your help. Here is how the script looks like now. There is also notification on QTS for QNAP, but it can be taken out if running elsewhere or not needed.
#!/bin/sh
# select the two files
FILE1="/dir1/file1"
FILE2="/dir2/file2"
# use or create a log file with timestamp of the output
LOG="/dir1/ScriptLog.txt"
TIMESTAMP=$(date +"%Y-%m-%d %Hh:%M")
if [ ! -e $LOG ]; then
touch $LOG
echo "$TIMESTAMP - INFO: '$LOG' does not exists but has been created." >&2
# else
# echo "$TIMESTAMP - INFO: '$LOG' exists and it will be used if any change to '$FILE1'
# or to '$FILE2' is needed." >&2
fi
# You can also pass the two file names as arguments for the script
if [[ $# == 2 ]]; then
FILE1=$1
FILE2=$2
fi
# check if the two files exist and are regular
if [ -f "$FILE1" -a -f "$FILE2" ]; then
# meanwhile compare FILE1 against FILE2
# if files are identical, stop there
if cmp "$FILE1" "$FILE2" 2>/dev/null>/dev/null; then
echo "$TIMESTAMP - INFO: '$FILE1' and '$FILE2' are identical." >&2 | >> $LOG
# if FILE1 is newer than FILE2, copy FILE1 over FILE2
elif [ "$FILE1" -nt "$FILE2" ]; then
if cp -p "$FILE1" "$FILE2"; then
echo "$TIMESTAMP - INFO: '$FILE1' replaced '$FILE2'." >&2 | >> $LOG
# if copy is successful, notify it into QTS
/sbin/notice_log_tool -a "$TIMESTAMP - INFO: '$FILE1' replaced '$FILE2'." --severity=5 >&2
else
echo "$TIMESTAMP - ERROR: FAILED to replace '$FILE2' with '$FILE1'." >&2 | >> $LOG
exit 1
fi
# if FILE1 is older than FILE2, copy FILE2 over FILE1
elif [ "$FILE1" -ot "$FILE2" ]; then
if cp -p "$FILE2" "$FILE1"; then
echo "$TIMESTAMP - INFO: '$FILE2' replaced '$FILE1'." >&2 | >> $LOG
# if copy is successful, notify it into QTS
/sbin/notice_log_tool -a "$TIMESTAMP - INFO: '$FILE2' replaced '$FILE1'." --severity=5 >&2
else
echo "$TIMESTAMP - ERROR: FAILED to replace '$FILE2' with '$FILE1'." >&2 | >> $LOG
exit 1
fi
# if two files are not identical but with same modification date
else
echo "$TIMESTAMP - ERROR: We should never reach this point. Something is wrong in the script." >&2 | >> $LOG
exit 1
fi
# if one file does not exist or is not valid, exit
else
echo "$TIMESTAMP - ERROR: One of the files does not exist, has been moved or renamed." >&2 | >> $LOG
# if error, notify it into QTS
/sbin/notice_log_tool -a "$TIMESTAMP - ERROR: One of the files does not exist, has been moved or renamed." --severity=5 >&2
exit 1
fi
I'm also going to suggest refactoring this, both to simplify the code, and to save your CPU cycles.
#!/bin/sh
# If both files exist....
if [ -f "$1" -a -f "$2" ]; then
# If they have the same content...
if cmp "$1" "$2" >/dev/null 2>/dev/null; then
echo "INFO: These two files are identical." >&2
# If one is newer than the other...
elif [ "$1" -nt "$2" ]; then
if cp -p "$1" "$2"; then
echo "INFO: Replaced file '$2' with '$1'." >&2
else
echo "ERROR: FAILED to replace file." >&2
exit 1
fi
# If the other is newer than the one...
elif [ "$1" -ot "$2" ]; then
if cp -p "$2" "$1"; then
echo "INFO: Replaced file '$1' with '$2'." >&2
else
echo "ERROR: FAILED to replace file." >&2
exit 1
fi
else
echo "ERROR: we should never reach this point. Something is wrong." >&2
exit 1
fi
else
echo "ERROR: One of these files does not exist." >&2
exit 1
fi
A few things that you may find useful.
This avoids calculating an md5 on each of the files. While comparing sums may be fine for small files like yours, it gets mighty expensive as your files grow. And it's completely unnecessary, because you have the cmp command available. Better to get in the habit of writing code that will work with less modification when you recycle it for the next project.
An if statement runs a command, usually [ or [[, but it can be any command. Here, we're running cmp and cp within an if, so that we can easily check the results.
This doesn't use stat anymore. While it's possible that you may never look beyond Linux, it's always a good idea to keep portability in mind, and if you can make your script portable, that's great.
This is not a bash script. Neither was your script -- if you call your script with /bin/sh, then you're in POSIX compatibility mode, which already makes this more portable than you thought. :-)
Indenting helps. You might want to adopt it for your own scripts, so that you can have a better visual idea of what commands are associated with the various conditions that are being tested.
What about something a bit simpler like the following?
#!/bin/sh
# select the two files from cli
# $1 = current file
# $2 = new file
FILE1=$1
FILE2=$2
# otherwise compare which of them has been last modified
DATE1=`(stat -c %Y $FILE1)`
DATE2=`(stat -c %Y $FILE2)`
if [ $DATE2 -gt $DATE1 ]; then
echo "cp -f $FILE2 $FILE1"
# cp -f $FILE2 $FILE1
fi
Almost there. Cleaning up your code and tweaking it a bit here is what I got
#!/bin/bash
# select the two files (default option)
FILE1="/dir1/file1.binary"
FILE2="/dir1/file2.binary"
# You can also pass the two file names as arguments for the script
if [ $# -eq 2 ]; then
FILE1=$1
FILE2=$2
fi
# create the hash of the two files
HASH1="$(md5sum $FILE1 | sed -n -e 's/^.*= //p')"
HASH2="$(md5sum $FILE2 | sed -n -e 's/^.*= //p')"
# get the dates of last modification
DATE1="$(stat -f '%m%t%Sm' $FILE1 | cut -c 1-10)"
DATE2="$(stat -f '%m%t%Sm' $FILE2 | cut -c 1-10)"
# Uncomment to see the values
#echo $FILE1 ' = hash: ' $HASH1 ' date: ' $DATE1
#echo $FILE2 ' = hash: ' $HASH2 ' date: ' $DATE2
# compare the two hashes
if [ $HASH1 == $HASH2 ]; then
# if the two hashes are the same, exit
echo "the two files are identical"
exit 0
fi
# compare the dates
if [ $DATE1 -gt $DATE2 ]; then
# if FILE1 is newer than FILE2, replace FILE2 with FILE1
cp $FILE1 $FILE2
echo "${FILE2} was replaced by ${FILE1}"
elif [ $DATE1 -lt $DATE2 ]; then
# else if FILE2 is newer than FILE1, replace FILE1 with FILE2
cp $FILE2 $FILE1
echo "${FILE1} was replaced by ${FILE2}"
else
# else the files are identical
echo "the two files are identical"
fi
Your way of getting the date was wrong, at least on my machine. So I rewrote it.
Your hash string was wrong. You were effectively cropping the string to the first 32 characters. By using sed you can actually get rid of the first part of the command and simply store the result of the md5sum.
You also misused the conditional statements as HuStmpHrrr pointed out.
The rest is cosmetics.
Sorry for asking this question again. I have already received answer but with using find but unfortunately I need to write it without using any predefined commands.
I am trying to write a script that will loop recursively through the subdirectories in the current directory. It should check the file count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like:
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirectories don't include subdirectories in turn.
I am confused about this because it doesn't work as I expect.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
How can I modify my script to produce the desired output?
This bash script produces the output that you want:
#!/bin/bash
bigfile="$PWD/BigList"
shortfile="$PWD/ShortList"
shopt -s nullglob
loop_folder_recurse() {
(
[[ -n "$1" ]] && cd "$1"
for i in */; do
[[ -d "$i" ]] && loop_folder_recurse "$i"
count=0
files=''
for j in *; do
if [[ -f "$j" ]]; then
files+="$j"$'\n'
((++count))
fi
done
if ((count > 10)); then
outfile="$bigfile"
else
outfile="$shortfile"
fi
echo "$i" >> "$outfile"
echo "$files" >> "$outfile"
done
)
}
loop_folder_recurse
Explanation
shopt -s nullglob is used so that when a directory is empty, the loop will not run. The body of the function is within ( ) so that it runs within a subshell. This is for convenience, as it means that the function returns to the previous directory when the subshell exits.
Hopefully the rest of the script is fairly self-explanatory but if not, please let me know and I will be happy to provide additional explanation.
I want to write a shell script to check a file on Linux and notify me if the file has been modified or changed.
For example: if /etc/sysconfig/network-scripts/ifcfg-eth0 has been changed, the script should notify me.
This is my code and it doesn't work, please help me:
#!/bin/bash
while true
do
#ATIME=`stat -c %Z /:path/to/the/file.txt`
#lmodiff= `stat -c %y /etc/sysconfig/network-scripts/ifcfg-eth0 | sed 's/^\([0-9\-]*\).*/\1/'`
lmd="2014-09-15"
cmd=`stat -c %y /etc/sysconfig/network-scripts/ifcfg-eth0 | sed 's/^\([0-9\-]*\).*/\1/'`
if [[ "$lmd" != "$cmd" ]]
then
echo "RUN COMMNAD"
#LTIME=$ATIME
else
echo "equal"
fi
done
Can you help me to improve my code?
I might do something like this:
#!/bin/bash
SLEEP=5
FILE=/path/to/file
SUM=$( md5sum $FILE )
while : ; do
sleep $SLEEP
NEWSUM=$( md5sum $FILE )
if [ "$SUM" != "$NEWSUM" ]; then
echo "RUN COMMAND"
SUM=$NEWSUM
else
echo "equal"
fi
done
I have shell Script written which does a job of comparing two files and and gives me result in a HTML format for defects. But i want to improve it so that i can only get modified files defects instead of legacy defects also. I am using this script to get Coverity report.
while read line; do
n=$((++n))
if echo $line | grep '^[[:space:]]*>' &>/dev/null; then
if [ $(($n % 2)) -eq 1 ]; then
# TODO somehow get proper defect number from html
# echo "Defect num: $(($n/2 + 1))"
def_num=$((++def_num))
fi
echo $line | sed -n -e 's/>[[:space:]]*\(.*\)/\1/p'
if [ $(($n % 2)) -eq 0 ]; then
echo "-------------------------------"
fi
done < <(diff -y -W 200 ./cov-results-base/result.filt ./cov-results-changed/result.filt)
echo "==============================="
echo
echo "Number of defects in old code: $(tac cov-results-base/summary.xml |
sed -n '/num/{s|<num>\(.*\)</num>|\1|p; q;}')"
echo "Number of defects in new code: $(tac cov-results-changed/summary.xml |
sed -n '/num/{s|<num>\(.*\)</num>|\1|p; q;}')"
This enables you to get the last modification time of a file and the compare with the current time.
now=`date +%s`
modified=`stat -c "%Y" $file`
if [ $(($now-$modified)) -gt 0 ]; then
echo "not modified";
else
echo "modified";
fi
I hope that this is what you wanted.