using tail to follow a log and execute a command instantly? Only seems to work by starting a new line - bash

I am trying to figure out a command which will enable me to read a log file in real time and execute a command when the string matches? I am using logkeys and trying to make it when I type a word it immediately triggers a command. This script works, but only when I press enter (start a newline) does it execute, and it seems anything I have found online also requires the press of the enter key to work. Is there a way to get around this somehow?
#/bin/bash
echo Waiting...
string='test'
tail /path/to/logfile -n0 -f | while read line; do
if [[ $line =~ $string ]]; then
echo "hello"
fi
done

I've played with buffering settings to no avail, so my conclusion is that read waits for a newline before it finishes. If you instead did read -n1, read would read exactly one character, which isn't quite what we want either, because then $line would always be just that one char.
Unfortunately, grep appears to have the same behavior (even with buffering options changed), even with grep -o:
$ tail logfile -f -n0 | grep -o test &
[1] 25524
$ echo -n test >> logfile
$ echo -n test >> logfile
$ echo test >> logfile
test
test
test
I think the general solution would be to roll our own "ring buffer grep" search tool that reads character per character into a ring buffer.
Here's my perl version of that, hope it helps. (Save as: ringgrep.pl)
#!/usr/bin/perl -w
use strict;
if (!$ARGV[0]) {
print "Usage: $0 needle\n";
exit 1;
}
my $needle = $ARGV[0];
my $buffer_len = length($needle);
my #buffer = (0) x $buffer_len;
my $i = 0;
my $input;
while(sysread(STDIN, $input, 1)) {
$buffer[$i] = $input;
my $string = join("", #buffer);
$string = (($i+1)%$buffer_len == 0 ? "" : substr($string, $i-$buffer_len+1)) . substr($string, 0, $i+1);
# print "string is: $string\n";
if ($string =~ /$needle/) {
print "got test!\n";
#buffer = (0) x $buffer_len;
$i = 0
} else {
$i = (++$i) % $buffer_len
}
}
Usage:
$ chmod +x ringgrep.pl
$ tail logfile -n0 -f | ./ringgrep.pl "this is a test" &
[1] 25756
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "this is a test" >> logfile
got test!
$ echo -n "this is a test" >> logfile
got test!
$ echo -n "this is a test" >> logfile
got test!
$ (echo -n t; echo -n h; echo -n i; echo -n s; echo -n ' '; echo -n i; echo -n s; echo -n ' '; echo -n a; echo -n ' '; echo -n t; echo -n e; echo -n s; echo -n t) >> logfile
got test!

Related

bash optional command in variable

i have a code:
L12(){
echo -e "/tftpboot/log/archive/L12/*/*$sn*L12*.log /tftpboot/log/diag/*$sn*L12*.log"
command="| grep -v hdd"
}
getlog(){
echo $(ls -ltr $(${1}) 2>/dev/null `${command}` | tail -1)
}
however $command does not seem to be inserting | grep -v hdd correctly
i need $command to be either empty or | grep
is there a simple solution to my issue or should i go for different approach
edit:
there may be another problem in there
i am loading a few "modules"
EVAL.sh
ev(){
case "${1}" in
*FAIL*) paint $red "FAIL";;
*PASS*) paint $green "PASS";;
*)echo;;
esac
result=${1}
}
rackinfo.sh (the "main script")
#! /bin/bash
#set -x
n=0
for src in $(ls modules/)
do
source modules/$src && ((n++))
## debugging
# source src/$src || ((n++)) || echo "there may be an issue in $src"
done
## debugging
# x=($n - $(ls | grep src | wc -l))
# echo -e "$x plugin(s) failed to laod correctly"
# echo -e "loaded $n modules"
########################################################################
command=cat
tests=("L12" "AL" "BI" "L12-3")
while read sn
do
paint $blue "$sn\t"
for test in ${tests[#]}
do
log="$(ev "$(getlog ${test})")"
if [[ -z ${log} ]]
then
paint $cyan "${test} "; paint $red "!LOG "
else
paint $cyan "${test} ";echo -ne "$log "
fi
done
echo
done <$1
the results i get are still containing "hdd" for L12()
Set command to cat as a default.
Also, it's best to use an array for commands with arguments, in case any of the arguments is multiple words.
There's rarely a reason to write echo $(command). That's essentially the same as just writing command.
#default command does nothing
command=(cat)
L12(){
echo -e "/tftpboot/log/archive/L12/*/*$sn*L12*.log /tftpboot/log/diag/*$sn*L12*.log"
command=(grep -v hdd)
}
getlog(){
ls -ltr $(${1}) 2>/dev/null | "${command[#]}" | tail -1)
}

reading from serial using shellscript

I have a serial port device that I would like to test using Linux command line.
And if I run the following command from terminal, it gives output
cat < /dev/ttyS0 &
This command opens the serial port and relays what it reads from it to its stdout.So, I tried it from shell script file but it is not working
fName="test.txt";
awk '
BEGIN { RS = "" ; FS = "\n" }
{
address = '/dev/ttyS0';
system("cat < " address );
}
END {
}' "$fName";
But it is not working and giving output.How can I listen to communication between a process and a serial port? Thanks
Using awk timeouts
I've successfully read something under dash, be using GAWK_READ_TIMEOUT environment variable:
out=`GAWK_READ_TIMEOUT=3000 awk '{print}' </dev/ttyS0 & sleep 1 ; echo foo >/dev/ttyS0`
On my terminal, this output:
echo "$out"
foo
Password:
or
echo "$out"
Login incorrect
testhost login:
Using bash timeouts
You could use FD under bash as:
exec 5>/dev/ttyS0
exec 6</dev/ttyS0
while read -t .1 -u 6 line;do
echo $line
done
or, to read unfinished lines:
while IFS= read -d '' -t .1 -u 6 -rn 1 char;do
echo -n "$char"
done
echo
So you could:
echo 'root' >&5
while IFS= read -d '' -t .1 -u 6 -rn 1 char;do
echo -n "$char"
done
echo 'password is 1234' >&5
while IFS= read -d '' -t .1 -u 6 -rn 1 char;do
echo -n "$char"
done
... Once done, you could close FD by running:
exec 6<&-
exec 5>&-
Sample bash poor terminal script
I've logged and test some commands with:
#!/bin/bash
exec 5>/dev/ttyS0
exec 6</dev/ttyS0
readbuf() {
while IFS= read -d '' -t .1 -u 6 -rn 1 char;do
echo -n "$char"
done
};
while [ "$cmd" != "tquit" ] ;do
readbuf
read cmd
echo >&5 "$cmd"
done

Redirection based on variables assigned within while loop setup

I want to access while loop variable out side the loop
while read line
do
...
...
...
done < $file > /home/Logs/Sample_$line_$(date +"%Y%m%d_%H%M%S").log
In the above example whatever the log file is getting generated that doesn't have the value for the line variable. i.e. $line is not working here.
Please let me know how this can be written to make it work.
#!/bin/sh
exec 1> /home/Logs/`basename $0 | cut -d"." -f1 | sed 's/\.sh//g'`_$(date +"%Y%m%d_%H%M%S").log 2>&1
echo "Execution Started : `date` \n"
SQL_DIR=/home/Sql
INFILE=in_file
TEMPFILE=temp_file
RETURN_CODE=0
ls -ltr $SQL_DIR|grep ".sql"|awk -F" " '{print $9}'|awk -F"." '{print $1}' > $INFILE
sed '/^$/d' $INFILE > $TEMPFILE; mv $TEMPFILE $INFILE
while read line
do
{
START_TIME=`date +%s`
printf "\n SQL File Executed Is : $line.sql"
SQL_FILE_NM=$line.sql
SQL_FILE=$SQL_DIR/$SQL_FILE_NM
nzsql -db netezza_db -Atqf $SQL_FILE > /dev/null
RETURN_CODE=$?
if [ $RETURN_CODE -eq 0 ]; then
echo "Time taken to execute sqlfile $SQL_FILE=$TT_HRS:$TT_MINS:$TT_REM_SECS HH:MM:SS" > $TEMPFILE
printf "\n Success: Sql completed successfully at `date` \n"
cat $TEMPFILE|mailx -s "Time taken to execute sqlfile $SQL_FILE=$TT_HRS:$TT_MINS:$TT_REM_SECS HH:MM:SS" 'koushik.chandra#a.com'
else
printf "\n Error: Failed in sql execution at `date`"
exit $RETURN_CODE
fi
END_TIME=`date +%s`
TT_SECS=$(( END_TIME - START_TIME))
TT_HRS=$(( TT_SECS / 3600 ))
TT_REM_MS=$(( TT_SECS % 3600 ))
TT_MINS=$(( TT_REM_MS / 60 ))
TT_REM_SECS=$(( TT_REM_MS % 60 ))
printf "\n"
printf "Total time taken to execute the sql $line="$TT_HRS:$TT_MINS:$TT_REM_SECS HH:MM:SS
printf "\n"
} > /home/Logs/sql_query_time_$line_$(date +"%Y%m%d_%H%M%S").log
done < $INFILE
rm -f $INFILE $TEMPFILE
exit $RETURN_CODE
You actually need redirection inside the while loop:
while read -r line; do
{ cmd1; cmd2; cmd3; } > "/home/Logs/Sample_${line}_$(date +"%Y%m%d_%H%M%S").log"
done < "$file"
When you have > outfile after done then output is redirected to one file only.

Bash - sometimes creates only empty output

I am trying to create a bash dictionary script that accepts first argument and creates file named after that, then script accepts next arguments (which are files inside same folder) and outputs their content into file (first argument). It also sorts, deletes symbols etc., but main problem is, that sometimes ouptut file is empty (I am passing one non empty file and one non existing file), after deleting and running script few more times it is sometimes empty sometimes not.
#!/bin/bash
numberoffileargs=$(( $# - 1 ))
exitstat=0
counterexit=0
acceptingstdin=0;
> "$1";
#check if we have given input files given
if [ "$#" -gt 1 ]; then
#for cycle going through input files
for i in "${#:2}"
do
#check whether input file is readable
if [ -r "${i}" ]; then
cat "${i}" >> "$1"
#else redirect to standard output
else
exitstat=2
counterexit=$((counterexit + 1))
echo "file does not exist" 1>&2
fi
done
else
echo "stdin code to be done"
acceptingstdin=1
#stdin input to output file
#stdin=$(cat)
fi
#one word for each line, alphabetical sort, alphabet only, remove duplicates
#all lowercase
#sort -u >> "$1"
if [ "$counterexit" -eq "$numberoffileargs" ] && [ "$acceptingstdin" -eq 0 ]; then
exitstat=3
fi
cat "$1" | sed -r 's/[^a-zA-Z\-]+/ /g' | tr A-Z a-z | tr ' ' '\n' | sort -u | sed '/^$/d' > "$1"
echo "$numberoffileargs"
echo "$counterexit"
echo "$exitstat"
exit $exitstat
Here is your script with some syntax improvement. Your trouble came from the fact that the dictionary was both on input and output on your pipeline; I added a temp file to fix it.
#!/bin/bash
(($# >= 1)) || { echo "Usage: $0 dictionary file ..." >&2 ; exit 1;}
dict="$1"
shift
echo "Creating $dict ..."
>| "$dict" || { echo "Failed." >&2 ; exit 1;}
numberoffileargs=$#
exitstat=0
counterexit=0
acceptingstdin=0
if (($# > 0)); then
for i ; do
#check whether input file is readable
if [ -r "${i}" ]; then
cat "${i}" >> "$dict"
else
exitstat=2
let counterexit++
echo "file does not exist" >&2
fi
done
else
echo "stdin code to be done"
acceptingstdin=1
fi
if ((counterexit == numberoffileargs && acceptingstdin == 0)); then
exitstat=3
fi
sed -r 's/[^a-zA-Z\-]+/ /g' < "$dict" | tr '[:upper:]' '[:lower:]' | tr ' ' '\n' |
sort -u | sed '/^$/d' >| tmp$$
mv -f tmp$$ "$dict"
echo "$numberoffileargs"
echo "$counterexit"
echo "$exitstat"
exit $exitstat
The pipeline might be improved.

count words in a file without using wc

Working in a shell script here, trying to count the number of words/characters/lines in a file without using the wc command. I can get the file broken into lines and count those easy enough, but I'm struggling here to get the words and the characters.
#define word_count function
count_stuff(){
c=0
w=0
l=0
local f="$1"
while read Line
do
l=`expr $line + 1`
# now that I have a line I want to break it into words and characters???
done < "$f"
echo "Number characters: $chars"
echo "Number words: $words"
echo "Number lines: $line"
}
As for characters, try this (adjust echo "test" to where you get your output from):
expr `echo "test" | sed "s/./ + 1/g;s/^/0/"`
As for lines, try this:
expr `echo -e "test\ntest\ntest" | sed "s/^.*$/./" | tr -d "\n" | sed "s/./ + 1/g;s/^/0/"`
===
As for your code, you want something like this to count words (if you want to go at it completely raw):
while read line ; do
set $line ;
while true ; do
[ -z $1 ] && break
l=`expr $l + 1`
shift ;
done ;
done
You can do this with the following Bash shell script:
count=0
for var in `cat $1`
do
count=`echo $count+1 | bc`
done
echo $count

Resources