Move a file which is constantly in use - bash

I need to move the whole content of a file (test.log) which is constantly in use. Moving the file could result in an error for the application which is writing to the file.
My approach is to redirect the output to test.log_bck, copy the original file (test.log) to test.log_cp, clear it and then check if there was any data added in the same time the test.log file was cleared. If any data was missing from the _cp file then merge it with the _bck file without redundant data. It is a lot of effort for such an easy task and my question is: is there another, easier / more efficient way to do it.
#/usr/bin/bash
#redirect the output to /tmp/logging/test.log_bck
bck(){
`tail -f /tmp/logging/test.log &> /tmp/logging/test.log_bck`
}
#run it in background
bck &
#copy the original file to test.log_cp
`cp /tmp/logging/test.log /tmp/logging/test.log_cp` | echo "Copped"
#clear the original file
echo "" > /tmp/logging/test.log | echo "Cleared"
#get the PID of the redirection process and check if there are other running and kill them
bck_pid=`ps -ef | grep "tail -f /tmp/logging/test.log" | grep -v grep | awk '{print$2}' | head -1`
while [ "$bck_pid" != "" ]
do
echo $bck_pid
kill $bck_pid | echo "Killed"
bck_pid=`ps -ef | grep "tail -f /tmp/logging/test.log" | grep -v grep | awk '{print$2}' | head -1`
done
date=$(date '+%Y_%m_%d_%H_%M')
cat /tmp/logging/test.log_cp /tmp/logging/test.log_bck > /tmp/logging/test.log_$date
cat -n /tmp/logging/test.log_$date | sort -uk2 | sort -nk1 | cut -f2-

Related

ps working on command line but not in script

I have a simple script that checks the process id of some process using ps. When I run it directly on command line, it works fine but does not when I run it in a script. What am I doing wrong?
This works fine:
ps auwx | grep elasticsearch | grep -v grep | grep user | awk '{print $2}' | tail -1
In script, it does not:
#!/bin/bash
#Setting ES Heap to 50GB
ES_HEAP_SIZE="50g"
#Finding dump file to be deleted
FILE_ID=$(ps auwx | grep elasticsearch | grep -v grep | grep user | awk '{print $2}' | tail -1)
FILE_NAME="java_pid$FILE_ID.hprof"
echo "Elasticsearch pid: $FILE_ID"
echo "Dump file name if it exists: $FILE_NAME. Checking now."
if [ -s $FILE_NAME ]
then
rm $FILE_NAME
kill -9 $FILE_ID
#Starting elasticsearch daemon
/data/elasticsearch-1.4.4/bin/elasticsearch -d
else
echo "All good. Dump file $FILE_NAME does not exist."
fi
Personal pet peeve: why do you have six executions in your pipeline when two will handle everything you need?
ps auwx | awk '/elasticsearch/ && /user/ { x=$2 } END{ print x; }'
As an aside, you wanted PID? Because it looks like you're reading PPID.
Hope that helps.

grep search with filename as parameter

I'm working on a shell script.
OUT=$1
here, the OUT variable is my filename.
I'm using grep search as follows:
l=`grep "$pattern " -A 15 $OUT | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
The issue is that the filename parameter I must pass is test.log.However, I have the folder structure :
test.log
test.log.001
test.log.002
I would ideally like to pass the filename as test.log and would like it to search it in all log files.I know the usual way to do is by using test.log.* in command line, but I'm facing difficulty replicating the same in shell script.
My efforts:
var-$'.*'
l=`grep "$pattern " -A 15 $OUT$var | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
However, I did not get the desired result.
Hopefully this will get you closer:
#!/bin/bash
for f in "${1}*"; do
grep "$pattern" -A15 "$f"
done | grep -w $i | awk 'END{print $8}'

Bash filename expansion identifies items in file tree, called command not

..poky/build$ for SUBPATH in $(bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \") ; do ls ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH 2>&1 ; done | grep -e "No such file or directory" | wc -l
2855
..poky/build$ for SUBPATH in $(bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \") ; do ls ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH 2>&1 ; done | grep -v -e "No such file or directory" | wc -l
15
Here one of all those No such file or directory
ls: cannot access ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package/usr/lib/libicalss.so.1.0.0: No such file or directory
where
..poky/build$ bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \"
/usr/bin/* /usr/sbin/* /usr/lib/alsa-lib/* /usr/lib/lib*.so.* /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d /usr/lib/udev/rules.d /usr/share/alsa-lib /usr/lib/alsa-lib/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers /usr/lib/alsa-lib/smixer/*.so
and
poky/build$ echo $SHELL
/bin/bash
Apparently Bash file name expansion finds 2855 items in identified sub-paths the called ls command can't identify.
Actually in every iteration instead of ls ... I need to do find with search root point set to ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH and -nameargument set few times (logical OR) to some patterns.
Where is my mistake?
Is this that file name expansion takes place in the for-loop instead of on invoking ls command (as programmer wishes it)?
#
Following alternative found under my limited expertise level and time resources
cd ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package && echo "/usr/bin/* /usr/sbin/* /usr/lib/alsa-lib/* /usr/lib/lib*.so.* /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d /usr/lib/udev/rules.d /usr/share/alsa-lib /usr/lib/alsa-lib/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers" | sed -r 's/(^\/)/.\//g' | sed -r 's/( \/)/ .\//g' | ls -Ralh $(awk '{print $0}') ; cd -
Glue for pasting long string with sub-paths to this command pipe from building block presented earlier out of scope as for this Q.
If possible please review. Thanks.
Does this qualify to be this Q's answer?

Bash script any reason why it wont write the file

I am helping debug some code that exec the following script is there any reason why its not writing a file to the server? - if that what it does. All the $ data and permissions are ok:
Script:
#!/bin/bash
RANGE=$1
ALLOCATION=`echo $RANGE | cut -f1,2,3 -d'.'`
/sbin/ip rule add from $1 lookup $2
echo $ALLOCATION
rm /path/too/file/location/$ALLOCATION
for i in `seq 3 254`
do
echo $ALLOCATION.$i >> /path/too/file/location/$ALLOCATION
done
ETH=`/sbin/ifconfig | grep eth0 | tail -n1 | cut -f2 -d':' | cut -f1 -d' '`

bash scripting for mysqldump

i use the following code to download all mysql database in a different file and not in one file (like --all-databases) and put them in the /backup/mysql folder
#!/bin/bash
mysqldump=`which mysqldump`
echo $mysqldump
mkdir /backup/mysql/$(date '+%d-%b-%Y')
echo "creating folder for current date done"
for line in "$(mysqlshow |cut -f1 -d"-" | cut -c 3- | cut -f1 -d" ")"
do
$mysqldump $line > /backup/mysql/$(date '+%d-%b-%Y')/"$line"
echo "$line\n"
done
I used the cut pipes to remove dashes and empty space before and at the end of the database name and it gave me what I want.
The problem is at line 13 according to bash but with no more details. Any ideas what I'm doing wrong?
mysqlshow output format
+---------------------+
| Databases |
+---------------------+
| information_schema |
| gitlabhq_production |
| mysql |
| performance_schema |
| phpmyadmin |
| test |
+---------------------
Your script doesn't manage the first 3 lines nor the last one, so you $line variable is invalid.
Solution
mysql | tail -n +4 | head -n -1 | tr -d '| '
tail -n +4: skip first four lines (may need adjustement);
head -n -1: ignore last line ;
tr -d '| ': remove pipe and space.
Advices
quotes your variables ;
use $() instead of backtick ;
don't use for i in $(ls *.mp3).
read How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
Better solution
Instead of a for loop you should use a while with a Process Substitution:
while read -r db; do
echo "[$db]";
done < <(mysqlshow -u root -p | tail -n +3 | head -n -1 | tr -d ' |' )

Resources