I want to grep the log from only START to the end in shell
I am using this script but getting only up to TIME as there is a space?
cat cdrdump_test.log |
while read line
do
sed -n 's/.*CDRQ]:\([^ ]*\).*/\1/p;d' >>new_cdrdump_test.log
done
echo "done"
LOG::
2017-02-13 08:16:14,808 [20170213081614665-88] INFO nxl.vxml.manager.VxmlSessionManager ( 807) - [TRX-20170213081614665-88][CALLFLOW] [CDRQ]:START|START|>>ACTION=TM2CM.ALERTING,SYSTEMIDENTIFIER=192.168.100.10,SESSIONID=20170213081614665-88,MSISDN=773930863,CHANNEL=24,DNIS=851851255777759523,TYPE=IBD,CID=-1,TRUNK=2,VXML=null,BOARD=1,DOMAIN=GSM,PROMO=DEFAULT,DIAL_TIME=null,OPTIONS={},TIME=13-02-2017 08:16:14.0666|STARTTIME=1486984574666|<>ACTION=TM2CM.CONNECTED,SYSTEMIDENTIFIER=192.168.100.10,SESSIONID=20170213081614665-88,TIME=13-02-2017 08:16:14.0760|<>ACTION=CM2CM.LOG,SYSTEMIDENTIFIER=192.168.100.10,SESSIONID=20170213081614665-88,MESSAGE=CRBT_STATUS:NoRecord,TIME=13-02-2017 08:16:14.0806|<
echo "started "
cat cdrdump_test.log| while read line
do
sed -n 's/.CDRQ]:(.)./\1/p;d' >>DeleteEND.log
done
echo "start1"
cat DeleteEND.log| while read line
do
sed s/'\w$'// >>Final_test.log
done
echo "end"
Related
trying to tail and grep certain logs with $installingFil and adding Command: word before output in $DNLOGfile, but file is empty /private/tmp/DownLog.log
What will be the correct approach to echo "Command:" before every line.
I am not able to add/echo "Command:" word before every line which is output in /private/tmp/DownLog.log
like:-
Command: bla bla
Command: downloding finish
Scrip -
#!/bin/bash
downFil=$(tail -f /var/log/system.log | grep 'Downloading files of')
dLog="/private/tmp/DownLog.log"
if [ -f /var/log/system.log ]; then
echo "Command:" "$downFil" >> "$dLog" # dLog is blank
else
echo "Command: Please wait.." >> $dLog # this works fine
fi
If I understand you correctly, you should try:
tail -f /var/log/system.log | grep 'Downloading files of' | sed 's/^/Command: /' >> /private/tmp/DownLog.log
i'm using git bash for windows (git version 2.18.0.windows.1) to write a bash script like this one:
file=$1
if [[ ! -s "$file" ]] ; then #empty
echo -e "${RED}Invalid argument. Pass the file with all GCIDs as INPUT!!!${NOCOLOR}"
else
number=$(cat $file | wc -l )
number=$(($number+1))
echo -e "** ${number} GCID detected **"
echo ""
while read -r gcidRead
do
gcid=${gcidRead}
echo -e "select distinct operation from audit_trail.audit_trail where gcid='$gcid';" >> query.txt
value=$(psql "host=XXXX port=62013 dbname=prodemeagcdm user=XXXX password=XXXX" <<-EOF
select distinct operation from audit_trail.audit_trail where gcid='$gcid';
\q
EOF
)
echo -e "${value}" >> output.txt
if grep -q delete_bupa output.txt ; then
echo -e "${gcid}" >> gcidDeleted.txt
fi
done < $file
fi
I created just to debug the query.txt file in which the output is:
';lect distinct operation from audit_trail.audit_trail where gcid='XXX
instead of
select distinct operation from audit_trail.audit_trail where gcid='XXX'
In short, every string after $gcid parameter will be written at the beginning of the entire string.
If I use a unix terminal the echo output is ok.
Why in git bash the "echo" command has the wrong output mentioned?
Thanks in advance
I think you see the output on terminal of a string which contains a CR (13 in dec or 0D in hex) : the last '; cahracters are written from the beginning of the line, thus overwriting the first two characters of the string.
The string actually consists of select dist... gcid='XXX\r';, and is just printed awkwardly (to a human) on the terminal.
There are many ways to drop CR from the input, here are two of them :
# remove all CR chars from the input :
cat $file | tr -d '\r' | while read -r gcdiRead; do
...
done
# remove all CR chars only at end of lines (e.g : when followed by LF) :
cat $file | sed -e 's/\r$//' | while read -r gcdiRead; do
...
done
I have an executable that accepts queries from stdin and responds to them, reading until EOF. Additionally I have an input file and a special command, let's call those EXEC, FILE and CMD respectively.
What I need to do is:
Pass FILE to EXEC as input.
Disregard all the output corresponding to commands read from FILE (/dev/null/).
Pass CMD as the last command.
Fetch output for the last command and save it in a variable.
EXEC's output can be multiline for each query.
I know how to pass FILE + CMD into the EXEC:
echo ${CMD} | cat ${FILE} - | ${EXEC}
but I have no idea how to fetch only output resulting from CMD.
Is there a magical one-liner that does this?
After looking around I've found the following partial solution:
mkfifo mypipe
(tail -f mypipe) | ${EXEC} &
cat ${FILE} | while read line; do
echo ${line} > mypipe
done
echo ${CMD} > mypipe
This allows me to redirect my input, but now the output gets printed to screen. I want to ignore all the output produced by EXEC in the while loop and get only what it prints for the last line.
I tried what first came into my mind, which is:
(tail -f mypipe) | ${EXEC} > somefile &
But it didn't work, the file was empty.
This is race-prone -- I'd suggest putting in a delay after the kill, or using an explicit sigil to determine when it's been received. That said:
#!/usr/bin/env bash
# route FD 4 to your output routine
exec 4> >(
output=; trap 'output=1' USR1
while IFS= read -r line; do
[[ $output ]] && printf '%s\n' "$line"
done
); out_pid=$!
# Capture the PID for the process substitution above; note that this requires a very
# new version of bash (4.4?)
[[ $out_pid ]] || { echo "ERROR: Your bash version is too old" >&2; exit 1; }
# Run your program in another process substitution, and close the parent's handle on FD 4
exec 3> >("$EXEC" >&4) 4>&-
# cat your file to FD 3...
cat "$file" >&3
# UGLY HACK: Wait to let your program finish flushing output from those commands
sleep 0.1
# notify the subshell writing output to disk that the ignored input is done...
kill -USR1 "$out_pid"
# UGLY HACK: Wait to let the subprocess actually receive the signal and set output=1
sleep 0.1
# ...and then write the command for which you actually want content logged.
echo "command" >&3
In validating this answer, I'm doing the following:
EXEC=stub_function
stub_function() {
local count line
count=0
while IFS= read -r line; do
(( ++count ))
printf '%s: %s\n' "$count" "$line"
done
}
cat >file <<EOF
do-not-log-my-output-1
do-not-log-my-output-2
do-not-log-my-output-3
EOF
file=file
export -f stub_function
export file EXEC
Output is only:
4: command
You could pipe it into a sed:
var=$(YOUR COMMAND | sed '$!d')
This will put only the last line into the variable
I think, that your proram EXEC does something special (open connection or remember state). When that is not the case, you can use
${EXEC} < ${FILE} > /dev/null
myvar=$(echo ${CMD} | ${EXEC})
Or with normal commands:
# Do not use (printf "==%s==\n" 1 2 3 ; printf "oo%soo\n" 4 5 6) | cat
printf "==%s==\n" 1 2 3 | cat > /dev/null
myvar=$(printf "oo%soo\n" 4 5 6 | cat)
When you need to give all input to one process, perhaps you can think of a marker that you can filter on:
(printf "==%s==\n" 1 2 3 ; printf "%s\n" "marker"; printf "oo%soo\n" 4 5 6) | cat | sed '1,/marker/ d'
You should examine your EXEC what could be used. When it is running SQL, you might use something like
(cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d'
and write this in a var with
myvar=$( (cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d' )
I am reading a text file with the following code and I want to echo the output on the same line on screen. Preceding to that, I want to do some trimming with sed but at the end I failed to echo the output on the same line.
while read line; do {
var="$(echo $line | sed 's/<[^>]*>//g')";
echo -n "$var"
} done < file.txt
So if I echo -n "$line" it prints the output on the same line but when `sed' comes in it failed to do so. What is it that I am doing wrong ?
cat "file.log"| grep -q '2013-11-10'
while read line
do
echo file_content_time=`echo $line | sed -e 's/\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0- 9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]\).*/\1/'`
if [ $? -eq 0 ]
then
echo comparison_start_date=`date -d "$file_content_time" +%Y%m%d`
fi
done < 'file.log'
/* Here I am trying find out the line containing '2013-11-10' and from that line onwards date has to display .*/
To output everything from a line containing a pattern up to the end-of-file all you need is
awk '/2013-11-10/,/pattern-not-in-file/' file.log
awk '/pattern/{p=1}p' your_file
initial_time=$(grep -o -m1 "2013-11-10 [0-9][0-9]:[0-9][0-9]:[0-9][0-9]" file.log)