I am trying to get the logs from journalctl after a specified time with the use of cursor option.
Below is code in script file.
value=$( journalctl -n 0 --show-cursor | tail -1 | cut -f2- -d: | sed 's/ /"/;s/$/"/')
echo "$value"
sleep 20
echo "journalctl --after-cursor=$value"
journalctl --after-cursor=$value
The ouput of this script file is
"s=3057f92d5b3e4afdb5eb91c22e880074;i=1f646;b=0bc7e6e9f16f4847b2d50e0a0dd31023;m=a10d4c4d1;
t=5bba8ac2477ae;x=1cc1645fed6ffc79"
journalctl --after-cursor="s=3057f92d5b3e4afdb5eb91c22e880074;i=1f646;
b=0bc7e6e9f16f4847b2d50e0a0dd31023;m=a10d4c4d1;t=5bba8ac2477ae;x=1cc1645fed6ffc79"
Failed to seek to cursor: Invalid argument
As we can see above the journalctl --after-cursor results in "Failed to seek cursor error".
However if the same is executed in command line terminal, the --after-cursor gives the output.
Is there something needed to be done before calling journalctl with after-cursor option in shell script?
Bash is very finicky about interpolation. You need to trim the double-quotes around $value and then quote it later on:
value=$( journalctl -n 0 --show-cursor | tail -1 | cut -f2- -d: | sed 's/ /"/;s/$/"/') | tr -d '"')
echo "$value"
sleep 20
echo "journalctl --after-cursor=$value"
journalctl --after-cursor="$value"
I've confirmed that this works by running the above and I've also used this same method on another script which uses cursors.
Related
I was working on a script that would keep monitoring login to my server or laptop via ssh.
this was the code that I was working with.
slackmessenger() {
curl -X POST -H 'Content-type: application/json' --data '{"text":"'"$1"'"}' myapilinkwashere
## removed it the api link due to slack restriction
}
while true
do
tail /var/log/auth.log | grep sshd | head -n 1 | while read LREAD
do
echo ${LREAD}
var=$(tail -f /var/log/auth.log | grep sshd | head -n 1)
slackmessenger "$var"
done
done
The issue I'm facing is that it keeps sending the old logs due to the while loop. can there be a condition that the loop only sends the new entries/updated enter as opposed to sending the old one over and over again. could not think of a condition that would skip the old entries and only shows old one.
Instead of using head -n 1 to extract a line at a time, iterate over the filtered output of tail -f /var/log/auth.log | grep sshd and process each line once as it comes through.
#!/usr/bin/env bash
# ^^^^- this needs to be a bash script, not a sh script!
case $BASH_VERSION in '') echo "Needs bash, not sh" >&2; exit 1;; esac
while IFS= read -r line; do
printf '%s\n' "$line"
slackmessenger "$line"
done < <(tail -f /var/log/auth.log | grep --line-buffered sshd)
See BashFAQ #9 describing why --line-buffered is necessary.
You could also write this as:
#!/usr/bin/env bash
case $BASH_VERSION in '') echo "Needs bash, not sh" >&2; exit 1;; esac
tail -f /var/log/auth.log |
grep --line-buffered sshd |
tee >(xargs -d $'\n' -n 1 slackmessenger)
I'm working on a server and to show detailed GPU information I use these commands:
nvidia-smi
ps -up `nvidia-smi |tail -n +16 | head -n -1 | sed 's/\s\s*/ /g' | cut -d' ' -f3`
However as you can see, nvidia-smi is called twice. How can I make the output of nvidia-smi go to output and pipe to another command at the same time?
Use tee:
ps -up `nvidia-smi |tee /dev/stderr |tail -n +16 | head -n -1 | sed 's/\s\s*/ /g' | cut -d' ' -f3`
Since stdout is piped, you can't make a copy to it, so I picked stderr to show output.
If /dev/stderr is not available, use /proc/self/fd/2.
I'm currently monitoring a log file and my ultimate goal is to write a script that uses tail -n0 -f and execute a certain command once grep finds a correspondence. My current code:
tail -n 0 -f $logfile | grep -q $pattern && echo $warning > $anotherlogfile
This works but only once, since grep -q stops when it finds a match. The script must keep searching and running the command, so I can update a status log and run another script to automatically fix the problem. Can you give me a hint?
Thanks
use a while loop
tail -n 0 -f "$logfile" | while read LINE; do
echo "$LINE" | grep -q "$pattern" && echo "$warning" > "$anotherlogfile"
done
awk will let us continue to process lines and take actions when a pattern is found. Something like:
tail -n0 -f "$logfile" | awk -v pattern="$pattern" '$0 ~ pattern {print "WARN" >> "anotherLogFile"}'
If you need to pass in the warning message and path to anotherLogFile you can use more -v flags to awk. Also, you could have awk take the action you want instead. It can run commands via the system() function where you pass the shell command to run
Given the following 2 lines in a bash script,
LOCATION=$(curl -i -H "Windmill-Name: $APPLICATION_NAME" -H "Windmill-Identifier: $CFBundleIdentifier" -F "ipa=#$IPA" -F "plist=#$PLIST" $WINDMILL_BASE_URL/windmill/rest/windmill/$USER | grep ^Location | awk '{print $2}')
echo "[windmill] Use $LOCATION for accessing '$APPLICATION_NAME'"
In some cases, the echo string appears like below.
for accessing 'MultiPartIOSDemo'[a truncated $LOCATION]
The behaviour is not consistent but when it reproduces, the malformed output is consistent (i.e. a truncated $LOCATION at some range).
It looks like echo outputs the string to the buffer but the piping of curl isn't yet done and ends up writing its output on top.
Can't quite tell.
Update
Tried all of your suggestions, but the same problem occurs.
Have now dropped grep and the script looks like so
LOCATION=$(curl -i -H "Windmill-Name: $APPLICATION_NAME" -H "Windmill-Identifier: $CFBundleIdentifier" -F "ipa=#$IPA" -F "plist=#$PLIST" "$WINDMILL_BASE_URL/windmill/rest/windmill/$USER" | awk -W '/^Location/ {print $2}')
echo "[windmill] Use $LOCATION for accessing '$APPLICATION_NAME'"
Here are some more details.
The script that includes the above lines is wrapped in a
(
(
# bash script
) 2>&1 | tee $HOME/.windmill/$PROJECT_NAME.log
) 2>&1 | tee $HOME/.windmill/windmill.log
Hence the output of the echo is on both logs.
Just noticed that the above behaviour occurs while tailing, e.g.
tail -fn 20 ~/.windmill/windmill.log
However if I do a
more ~/.windmill/windmill.log
I can see that the echo message appears correctly. Notice the newline character "^M". Wondering if it has anything to do with the way tail parses the log.
[windmill] Use [correct $LOCATION] ^M for accessing 'MultiPartIOSDemo'
Clarified Question
Guess after all the above, there are really 2 questions.
Under what circumstances does the ^M appear in the log?
Why is tail parsing the log wrong, i.e. parsing ^M in such a way that it first outputs the "for accessing 'MultiPartIOSDemo'" then the "Use $LOCATION" on top.
I would (1) change
$WINDMILL_BASE_URL/windmill/rest/windmill/$USER
to
"$WINDMILL_BASE_URL/windmill/rest/windmill/$USER"
(2) use tee to debug
and (3) use awk to match "^location" instead of grep
LOCATION=$(curl -i -H "Windmill-Name: $APPLICATION_NAME" -H "Windmill-Identifier: $CFBundleIdentifier" -F "ipa=#$IPA" -F "plist=#$PLIST" "$WINDMILL_BASE_URL/windmill/rest/windmill/$USER" | tee /tmp/debug.$$ | awk ' /^Location/ {print $2}')
echo "[windmill] Use $LOCATION for accessing '$APPLICATION_NAME'"
Perhaps curl is returing an error. You can check the results and only parse it if no error occured
# Create a trace file
tfile=/tmp/trace.$$
# uncomment this line if you want to delete the trace file at the end of the script
#trap "/bin/rm $tfile" 0 1 15
curl -i -H "Windmill-Name: $APPLICATION_NAME" -H "Windmill-Identifier: $CFBundleIdentifier" -F "ipa=#$IPA" -F "plist=#$PLIST" "$WINDMILL_BASE_URL/windmill/rest/windmill/$USER" >$tfile
status=$?
if [ $status -eg 0 ]
then
echo curl was successful
LOCATION=$(awk '/^Location/ {print $2}' <$tfile)
echo "[windmill] Use $LOCATION for accessing '$APPLICATION_NAME'"
else
echo curl exited with status $status
fi
Here is a minimal test case which fails
#!/bin/tcsh
#here is some code in tcsh I did not write which spawns many processes.
#let us pretend that it spawns 100 instances of stupid_test which the user kills
#manually after an indeterminate period
/bin/bash <<EOF
#!/bin/bash
while true
do
if [[ `ps -e | grep stupid_test | wc -l` -gt 0 ]]
then
echo 'test program is still running'
echo `ps -e | grep stupid_test | wc -l`
sleep 10
else
break
fi
done
EOF
echo 'test program finished'
The stupid_test program is consists of
#!/bin/bash
while true; do sleep 10; done
The intended behavior is to run until stupid_test is killed (in this case manually by the user), and then terminate within the next ten seconds. The observed behavior is that the script does not terminate, and evaluates ps -e | grep stupid_test | wc -l == 1 even after the program has been killed (and it no longer shows up under ps)
If the bash script is run directly, rather than in a here document, the intended behavior is recovered.
I feel like I am doing something very stupidly wrong, I am not the most experienced shell hacker at all. Why is it doing this?
Usually when you try to grep the name of a process, you get an extra matching line for grep itself, for example:
$ ps xa | grep something
57386 s002 S+ 0:00.01 grep something
So even when there is no matching process, you will get one matching line. You can fix that by adding a grep -v grep in the pipeline:
ps -e | grep stupid_test | grep -v grep | wc -l
As tripleee suggested, an even better fix is writing the grep like this:
ps -e | grep [s]tupid_test
The meaning of the pattern is exactly the same, but this way it won't match grep itself anymore, because the string "grep [s]tupid_test" doesn't match the regular expression /[s]tupid_test/.
Btw I would rewrite your script like this, cleaner:
/bin/bash <<EOF
while :; do
s=$(ps -e | grep [s]tupid_test)
test "$s" || break
echo test program is still running
echo "$s"
sleep 10
done
EOF
Or a more lazy but perhaps sufficient variant (hinted by bryn):
/bin/bash <<EOF
while ps -e | grep [s]tupid_test
do
echo test program is still running
sleep 10
done
EOF