I'd like Ansible to tail a log file and wait for idle time - say for example XX seconds where the logs are idle for that time.
If the logs are not idle within that XX seconds, continue to wait until we have XX seconds of idle time.
If the idle time elapses then Ansible will restart the server.
Idle time can be calculated by checking the last 2 log entries and the time difference between them.
How can I go about achieving this with Ansible?
There's no need to wait for two log entries XX seconds apart rather than just waiting until XX seconds after the last log entry since if you've gone longer than XX seconds the next write will match your condition.
And on that basis, just do this (for example purposes XX = 5)
- find: paths="/tmp" patterns="logfile" age="5s" age_stamp="mtime"
register: modified
until: modified.matched == 1
retries: 20
delay: 5
Just make sure the paths and patterns specifications match only your log file.
If you have a command to print the time difference in seconds, you can use:
- shell: /opt/myscripts/log_difference.sh
register: result
until: result.stdout | int > 60
retries: 5
delay: 10
This will execute log_difference.sh until number of retries is exceeded or number in stdout is greater than 60.
Related
When viewing submitted jobs managed by Slurm, I would like to have the time limit column (specified by %l) to show only hours, instead of the usual days-hours:minutes:seconds format. This is the command I am currently using:
squeue --format="%.6i %.5P %.25j %.8u %.8T %.10M %.5l %.15b %.5C %.6D %R" --sort=+i --me
and this is the example output:
276350 qgpu jobname username RUNNING 1:14:14 1-00:00:00 gres:gpu:v100:1 18 1 s31n02
So, in this case, I would like the elapsed time to remain as is (1:14:14), but the time limit to change from 1-00:00:00 to 24. Is there a way to do it?
This is the way Slurm displays the dates. Elapsed time will eventually be displayed the same way (days-hours:minutes:seconds) after 23:59:59.
You can use a wrapper script to convert into a different format. Or if you know the time limit is no more than a day, just set the time limit to 23:59:00 by using --time=1439.
salloc -N1 --time=1439 bash
Using your squeue command:
166 mypartition interactive jyvet RUNNING 7:36 23:59:00 N/A 1 1 mynode
I'd like to write a script for Windows 10 to add 1 to a count in a .txt each time a print job completes. Ideally a separate count for each day, so I can see how many print jobs were completed in a day.
Any help in understanding how to go about this is appreciated!
The print service already logs every time it prints - you just need to enable the appropriate event log channel and consume the resulting log events:
# Enable the Microsoft-Windows-PrintService/Operational log channel
wevtutil.exe set-log Microsoft-Windows-PrintService/Operational /enabled:true
Now that the log channel is enabled, the print service will log an event with event ID 307 everytime it executes a local print job. Since the log events all have timestamps, getting a count per day is as simple as using the Group-Object cmdlet:
# Fetch the print job events from the event log
$printJobEvents = Get-WinEvent -FilterHashtable #{ LogName='Microsoft-Windows-PrintService/Operational'; EventId=307 }
# Group by date logged, to get a count-per-day
$printJobEvents |Group-Object { '{0:yyyy-MM-dd}' -f $_.TimeCreated.Date } -NoElement |Sort-Object Name
One technique that might be useful is to query stats for the spooler service like this:
Get-CimInstance 'Win32_PerfFormattedData_Spooler_PrintQueue' |
Format-Table -Property Name,Jobs,TotalJobsPrinted,TotalPagesPrinted -AutoSize
This gives output like this:
Name Jobs TotalJobsPrinted TotalPagesPrinted
---- ---- ---------------- -----------------
Printer1 0 50 212
Printer2 3 13 118
Printer3 1 33 306
_Total 4 96 636
The stats are reset each time the Print Spooler service restarts, so you'll need to take that into account in your final script, which might make this a trickier option than Mathias' event log solution.
Currently, I have a sequence job in DataStage.
Here is the flow:
StartLoop Activity --> UserVariables Activity --> Job Activity --> Execute Command --> Endloop Activity
The job will run every 30 minutes (8 AM - 8 PM) to get real data. The first loop iteration will load data from 8 PM the previous day to 8 AM the current day, and the others will load data that happens in the last 30 minutes.
The UserVariables Activity is to pass variables (SQL statement) to filter data getting in the Job Activity. The first iteration the UserVariables pass variable A (SQL statement 1) to the Job Activity, from the second iteration, it will pass variable B (SQL statement 2) to the Job Activity.
The Execute Command I currently set the 'Sleep 1800' command for the job to sleep 30 minutes to end the iteration of the loop. But I realized now that it is affected by the running time of each iteration. So with my knowing-nothing about shell script, I have searched for solutions and have this file to sleep until a specific time when minute like 30 or 00 (delay 0-1 minute but it's fine).
The shell script is below, I ran it fine on my system but no success on making it as part of the job.
#!/bin/bash
minute=$(date +%M)
num_1=30
num_2=60
if [ $minute -le 30 ];
then
wait=$((($num_1 - $minute)*$num_2))
sleep $wait
fi
if [ $minute -gt 30 ];
then
wait=$((($num_2 - $minute)*$num_2))
sleep $wait
fi
I am now facing 2 problems right now that I need your help with.
The job runs the first iteration fine with the variable A below:
select * from my_table where created_date between trunc(sysdate-1) + 20/24 and trunc(sysdate) + 8/24;
But from the second iteration it failed with the Job Activity with the variable B below:
select * from my_table where created_date between trunc(sysdate-1/48, 'hh') + 30*trunc(to_number(to_char(sysdate-1/48,'MI'))/30)/1440 and trunc(sysdate, 'hh') + 30*trunc(to_number(to_char(sysdate,'MI'))/30)/1440;
In the parallel job, the log said:
INPUT,0: The following SQL statement failed: select * from my_table where created_date between trunc(sysdate-1/48, hh) + 30*trunc(to_number(to_char(sysdate-1/48,MI))/30)/1440 and trunc(sysdate, hh) + 30*trunc(to_number(to_char(sysdate,MI))/30)/1440.
I realized that maybe it failed to run the parallel job because it removed the single quote in hh and MI.
Is it because when passing variables from UserVariables Activity to Job Activity the variable will remove all the quotes? And how can I fix this?
2. How can I make the shell script above as part of the job like Execute Command or some other stage. I have searched for solutions and I think it's about the ExecSH Before/ After Routine Activity. But after reading from IBM pages, I still don't know where to start with it.
Sorry for adding 2 questions at 1 post that makes it so long but it's very relative to each other so it will take lots of time to answer if I separate it into 2 posts and you guys need more information about it.
Thank you!
Try escaping the single quote characters (precede each with a backslash).
Execute the shell script from an Execute Command activity ahead of the Job activity.
I noticed that the TailFile processor consumes CPU on the AIX operating system.
Can I do anything to reduce the consumption?
Processors:
- id: xxxxxxxxxxxxxxxxxxxxxxxxxxx
name: TailFile
class: org.apache.nifi.processors.standard.TailFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
- success
Properties:
File Location: Local
File to Tail: *.log
Initial Start Position: Beginning of File
Rolling Filename Pattern:
tail-base-directory: /WorkingDir85/log/
tail-mode: Multiple files
tailfile-lookup-frequency: 10 minutes
tailfile-maximum-age: 24 hours
tailfile-recursive-lookup: 'false'
The scheduling period is 0 sec which basically means run as fast as possible. Setting to something like '10 ms' or even '1 ms' should lighten the CPU usage.
Basically, I have a series of commands I want to run every other sunday. I set a cron task to run the script every sunday, then this script only allows the script to run on even weeks, thus it only runs every other sunday. My question is, will this script still work going from year to year.
if [ $(($(date +'%U') % 2)) -eq 0 ]
then
some command
fi
You have what's known as the XY problem here.
You have a problem with this part of your shell script, and you want to solve the problem by fixing the script. In reality, fixing the root cause of the problem is easier.
Simply alter your cron job to run every other Sunday:
#----+-----+-----+-----+-----+-------------------------------------------------
#min |hour |day |month|day |command
# | |of mn| |of wk|
#----+-----+-----+-----+-----+-------------------------------------------------
03 04 * * 7 expr `date +%W` % 2 >/dev/null || fortnightly.sh
See How to instruct cron to execute a job every second week? for more info.
If you don't want to specify this with cron syntax, you can use the %s format instead of %U. This will give you the number of seconds since 1st Jan 1970 UTC. You can divide this to get a week number:
$(($(date +'%s') / 604800))
Then you can do your modulo test on that.
Note the number 604800 = 7 * 86400 = 7 * 24 * 60 * 60 ie the number of seconds in one week.
If you're running this every day, you'll want to know that it's actually a Sunday. So in this case, you would divide by 86400 to get a day number. Then, armed with the knowledge that day zero was a Thursday, you can check that the result (modulo 14) is either 3 or 10, depending on which Sunday you started at.