Systemctl bash variable - bash

I'm trying to get this variable work, but I'm getting always an error about systemd-escape. Although I escape the special character, the variable is not working:
status="systemctl status syslog-ng | grep Active: | sed 's/.*: //' | sed 's/since.*//g'"
The result I'm getting is here:
Invalid unit name "|" was escaped as "\x7c" (maybe you should use systemd-escape?)
Invalid unit name "|" was escaped as "\x7c" (maybe you should use systemd-escape?)
Invalid unit name "'s/.*:" was escaped as "\x27s-.*:" (maybe you should use systemd-escape?)
Invalid unit name "|" was escaped as "\x7c" (maybe you should use systemd-escape?)
Invalid unit name "'s/since.*//g'" was escaped as "\x27s-since.*--g\x27" (maybe you should use systemd-escape?)
Unit \x7c.service could not be found.
Unit grep.service could not be found.
Unit Active:.service could not be found.
Unit \x7c.service could not be found.
Unit sed.service could not be found.
Unit \x27.mount could not be found.
Unit \x7c.service could not be found.
Unit sed.service could not be found.
● syslog-ng.service - System Logger Daemon
Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-10-11 14:01:23 CEST; 26min ago
Docs: man:syslog-ng(8)
Main PID: 3020944 (syslog-ng)
Tasks: 3 (limit: 101081)
Memory: 8.9M
CGroup: /system.slice/syslog-ng.service
└─3020944 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid
Oct 11 14:01:23 syslog-ng systemd[1]: Starting System Logger Daemon...
Oct 11 14:01:23 syslog-ng syslog-ng[3020944]: [2021-10-11T14:01:23.798405] Plugin module not found in 'module-path'; module-path='/usr/lib64/syslog-ng'>
Oct 11 14:01:23 syslog-ng syslog-ng[3020944]: [2021-10-11T14:01:23.801271] Plugin module not found in 'module-path'; module-path='/usr/lib64/syslog-ng'>
Oct 11 14:01:23 syslog-ng syslog-ng[3020944]: [2021-10-11T14:01:23.801828] Plugin module not found in 'module-path'; module-path='/usr/lib64/syslog-ng'>
Oct 11 14:01:23 syslog-ng syslog-ng[3020944]: [2021-10-11T14:01:23.808340] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!;
Oct 11 14:01:23 syslog-ng systemd[1]: Started System Logger Daemon.
I just want to have the status of my syslog-ng as a variable

We suggest to follow systemd service unit definition documentation.
As stated in the Command lines section.
This syntax is inspired by shell syntax, but only the meta-characters and expansions described in the following paragraphs are understood, and the expansion of variables is different. Specifically, redirection using "<", "<<", ">", and ">>", pipes using "|", running programs in the background using "&", and other elements of shell syntax are not supported.
We cannot assume bash nor sh syntax/rules writing systemd service unit.
Environment variables are better defined statically.
In case you need to store values in dynamic variables. We suggest create a script service-unit-xyx.bash that executes all the required assignments and eventually run the target service.
Using a single script service-unit-xyx.bash you can easily log and debug every line.
Eventually execute service-unit-xyx.bash script with full path in ExecStart= command.
Using execution script service-unit-xyx.bash also simplify the service unit development, from systemd service unit file to bash/sh script development.
It is also possible to use a ExecStartPre=, ExecStartPost= commands to control service runtime environments.
Read and follow sample service units from the internet.
All commands/scripts should specify absolute path reference.

Related

Running awesome-client from a script executing as root

Running Awesome on Debian (11) testing
awesome v4.3 (Too long)
• Compiled against Lua 5.3.3 (running with Lua 5.3)
• D-Bus support: ✔
• execinfo support: ✔
• xcb-randr version: 1.6
• LGI version: 0.9.2
I'm trying to signal to Awesome when systemd triggers suspend. After fiddling with D-Bus directly for awhile and getting nowhere, I wrote a couple of functions that somewhat duplicate the functionality of signals.
I tested it by running the following command in a shell, inside of my Awesome session:
$ awesome-client 'require("lib.syskit").signal("awesome-client", "Hello world!")'
This runs just fine. A notification posts to the desktop "Hello world!" as expected. I added the path to my lib.syskit code to the $LUA_PATH in my ~/.xsessionrc. Given the error described below, I doubt this is an issue.
Now for the more difficult part. I put the following in a script located at /lib/systemd/system-sleep/pre-suspend.sh
#!/bin/bash
if [ "${1}" == "pre" ]; then
ERR=$(export DISPLAY=":0"; sudo -u naddan awesome-client 'require("lib.syskit").signal("awesome-client", "pre-suspend")' 2>&1)
echo "suspending at `date`, ${ERR}" > /tmp/systemd_suspend_test
elif [ "${1}" == "post" ]; then
ERR=$(export DISPLAY=":0"; sudo -u naddan awesome-client 'require("lib.syskit").signal("awesome-client", "post-suspend")' 2>&1)
echo "resuming at `date`, ${ERR}" >> /tmp/systemd_suspend_test
fi
Here's the output written to /tmp/systemd_suspend_test
suspending at Thu 22 Jul 2021 10:58:01 PM MDT, Failed to open connection to "session" message bus: /usr/bin/dbus-launch terminated abnormally without any error message
E: dbus-send failed.
resuming at Thu 22 Jul 2021 10:58:05 PM MDT, Failed to open connection to "session" message bus: /usr/bin/dbus-launch terminated abnormally without any error message
E: dbus-send failed.
Given that I'm already telling it the $DISPLAY that Awesome is running under (this is a laptop), and that I'm running awesome-client as my user, not root, what else am I missing that's keeping this from working?
Is there a better way that I could achieve telling Awesome when the system suspends?
awesome-client is a shell script. It is a thin wrapper around dbus-send. Thus, since you write "After fiddling with D-Bus directly for awhile and getting nowhere", I guess the same reasoning applies.
Given that I'm already telling it the $DISPLAY that Awesome is running under (this is a laptop), and that I'm running awesome-client as my user, not root, what else am I missing that's keeping this from working?
You are missing the address of the dbus session bus. For me, it is:
$ env | grep DBUS
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
Is there a better way that I could achieve telling Awesome when the system suspends?
Instead of sending a message directly to awesome via some script, you could use the existing mechanism for this: Dbus signals. These are broadcasts that interested parties can listen to.
Google suggests that there is already a PrepareForSleep signal:
https://serverfault.com/questions/573379/system-suspend-dbus-upower-signals-are-not-seen
Based on this, Google then gave me the following AwesomeWM lua code that listens for logind's PrepareForSleep signal (written by yours truely - thanks Google for finding that!):
https://github.com/awesomeWM/awesome/issues/344#issuecomment-328354719
local lgi = require("lgi")
local Gio = lgi.require("Gio")
local function listen_to_signals()
local bus = lgi.Gio.bus_get_sync(Gio.BusType.SYSTEM)
local sender = "org.freedesktop.login1"
local interface = "org.freedesktop.login1.Manager"
local object = "/org/freedesktop/login1"
local member = "PrepareForSleep"
bus:signal_subscribe(sender, interface, member, object, nil, Gio.DBusSignalFlags.NONE,
function(bus, sender, object, interface, signal, params)
-- "signals are sent right before (with the argument True) and
-- after (with the argument False) the system goes down for
-- reboot/poweroff, resp. suspend/hibernate."
if not params[1] then
-- This code is run before suspend. You can replace the following with something else.
require("gears.timer").start_new(2, function()
mytextclock:force_update()
end)
end
end)
end
listen_to_signals()

custom pattern to filter strings when using telegraf inputs.logparser.grok

I'm trying filter for particular words in a log file using regex, the goal is that any log line that matches the regex in custom_pattern will go into influxdb, log lines that do not match willbe ignored. When I tested the regex it works, even in golang playground (https://play.golang.org/p/_apzOVwwgl2). But when I use it in the telegraf conf file as it is below, it doesn't work, there's no input into influxdb. Is there something I'm missing that should added to the configuration?
I've tested the regex on http://grokdebug.herokuapp.com/ and https://play.golang.org/p/_apzOVwwgl2 it works but not in the custom_patterns under [inputs.logparser.grok].
Here is my grok config
[[inputs.logparser]]
files = ["/var/log/test1"]
from_beginning = true
[inputs.logparser.grok]
patterns = ["%{FAIL_LOG}"]
custom_patterns = '''FAIL_LOG ^.*?\b(multipathd?)\b.*?\b(failed|failing|(remaining active paths))\b.*?$'''
The pattern is supposed to match first 2 log lines like below and ignore the third line.
Oct 29 03:29:03 dc-as-5p multipath: checker failed interface 8:0 in map 150gb
Oct 29 03:29:03 dc-as-5p multipathd: checker failing interface 8:0 in map 150gb
Oct 29 03:26:03 dc-as-5p link: checker down remaining active paths interface 8:0 in map 150gb
What am I doing wrong?
I summarised how I got custom log parsing in Telegraf/GROK to work in the following post: Custom log parsing with Telegraf/Tail Plugin/GROK. Maybe it helps you or others debug similar problems.
Maybe interessting for others reading this in 2020, that Telegraf's logparser is now replaced by the Tail plugin. Example in my post above.
PS: My approach for your problem would be to not use regex at all, but to define three different patterns for each of the lines. This of course will only work if you have a low number of possible log errors/lines.
If you run telegraf with the --debug flag, you will see that it is having an issue parsing the logs.
$ telegraf --debug --config ./telegraf.conf
...
2019-11-17T05:01:07Z D! Grok no match found for: "Oct 29 03:29:03 dc-as-5p multipath: checker failed interface 8:0 in map 150gb"
2019-11-17T05:01:07Z D! Grok no match found for: "Oct 29 03:29:03 dc-as-5p multipathd: checker failing interface 8:0 in map 150gb value=3"
2019-11-17T05:01:07Z D! Grok no match found for: "Oct 29 03:26:03 dc-as-5p link: checker down remaining active paths interface 8:0 in map 150gb"
This error message is misleading because, as your testing has shown, your regex pattern is correct. The real issue is that you have not included a value to be logged in your regex.
A version of your regex to store the error message and timestamp might be:
custom_patterns = '''FAIL_LOG %{SYSLOGTIMESTAMP:timestamp}.*(multipath).?: %{GREEDYDATA:message:string}'''
The value pattern can be found between ${}. Additional premade patterns can be found here. This will eliminate the first two errors above. The results of these can be seen using the --test flag.
$telegraf --test --config ./telegraf.conf
...
> logparser,host=pop-os,path=./test1 message="checker failed interface 8:0 in map 150gb",timestamp="Oct 29 03:29:03 " 1573968174161853621
For some reason the --test flag did not always output the results. I would have to run the command multiple times before getting the above output.

User defined (or emmited) username when using the logger(1) linux bash tool command

I am trying to log some custom logs. The problem is that if I use the logger command, the username running the command is also logged. I would like to ommit that info so I can manually fill anything I want. I have read the manual but could not find anything like that. I also tried implementing it in a script (java) but not quit succeed.
Example. Now I am seeing this:
Mar 2 10:31:28 $HOSTNAME $USERNAME: Hello world!
What I would like to see is this:
Mar 2 10:31:28 suhosin[666]: ALERT - canary mismatch on efree() - heap overflow detected (attacker '000.000.000.000', file 'xyz')
Use the -t option to set the tag.
$ logger -t 'nobody' 'hello'
Produces log:
Feb 28 10:25:37 myhostname nobody: hello
Relevant man page section (bold added for emphasis):
-t, --tag tag
Mark every line to be logged with the specified tag. The default tag is the name of the user logged in on the terminal (or a user name based on effective user ID).

libvirt image based provisioning using logical volumes

Are there known issues with image based provisioning using logical volumes in libvirt? I am getting this error while trying to do the same
Unable to save
Failed to create a compute kvm2 (Libvirt) instance test3.xxx.local: Call
to virNetworkCreateXML failed:
internal error: Child process (/usr/sbin/lvcreate --name
test3.xxx.local-disk1 -L 1K --type snapshot --virtualsize 10485760K -s
/vm-images-pool/images-vol/template_minimal) unexpected exit status 3: 2017-
01-05 00:42:08.133+0000: 12330: debug : virFileClose:102 : Closed fd 29
2017-01-05 00:42:08.133+0000: 12330: debug : virFileClose:102 : Closed fd 31
2017-01-05 00:42:08.133+0000: 12330: debug : virFileClose:102 : Closed fd 27
Volume group name expected (no slash) Run `lvcreate --help' for more
information
This link from Red Hat flags it as a known issue:
https://access.redhat.com/solutions/1995053
That doc has a date of October 20 2015. Not sure if anythig changed after that to support LV.
I tried to satisfy the requirement in that doc by creating a pool based on dir like this:
Setup:
Storage pool vm-images-pool-dir of type dir
Storage pool vm-images-pool of type logical
template_minimal is the image template.
[root#kvm2 libvirt]# virsh vol-list vm-images-pool-dir
Name Path
----------------------------------------------------------------------------
template_minimal /vm-images-pool/images-vol/template_minimal
vm-images-pool storage pool is of type VG with one volume:
images-vol vm-images-pool -wi-ao---- 249.00g
images-vol is mounted under /vm-images-pool/images-vol/
Any insight is appreciated.
Thanks,
TG
=======================================
more details.
Daniel, Thanks. I am a bit confused. I couldn't put the actual commands earlier since I had cleaned them up. I recreated the setup. Here are the commands I used:
virsh pool-define-as vm-images-pool logical --source-dev /dev/mapper/mpathd
virsh pool-build vm-images-pool
virsh pool-start vm-images-pool
virsh vol-create-as vm-images-pool images-vol --capacity 249G
virsh pool-define-as vm-images-pool-dir dir - - - - /vm-images-pool/images- vol/
virsh pool-build vm-images-pool-dir
virsh pool-start vm-images-pool-dir
[root#kvm2 ~]# virsh vol-list vm-images-pool-dir
Name Path
---------------------------------------------------------------------------- --
lost+found /vm-images-pool/images-vol/lost+found
template_minimal /vm-images-pool/images-vol/template_minimal
=======================================
/vm-images-pool/images-vol/template_minimal is the path used for template image
==================================
more tests:
mounted the logical volume at a mount point to match the directory based storage pool:
[root#kvm2 ~]# df -h /vm-images-pool-dir/images-vol
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vm--images--pool-images--vol 245G 1.2G 232G 1% /vm-images- pool-dir/images-vol
[root#kvm2 ~]# virsh vol-list vm-images-pool-dir
Name Path
------------------------------------------------------------------------------
lost+found /vm-images-pool-dir/images-vol/lost+found
template_minimal /vm-images-pool-dir/images-vol/template_minimal
[root#kvm2 ~]#
used /vm-images-pool-dir/images-vol/template_minimal as the template path
same result
Unable to save
Failed to create a compute kvm2 (Libvirt) instance test3.xxx.local: Call
to virNetworkCreateXML failed: internal error: Child process
(/usr/sbin/lvcreate --name test3.xxx.local-disk1 -L 1K --type
snapshot --virtualsize 10485760K - s /vm-images-pool-dir/images-
vol/template_minimal) unexpected exit status 3: 2017-01-05
16:45:10.694+0000: 40712: debug : virFileClose:102 : Closed fd 27 2017-
01-05 16:45:10.694+0000: 40712: debug : virFileClose:102 : Closed fd 29
2017-01-05 16:45:10.694+0000: 40712: debug : virFileClose:102 : Closed fd 24
Volume group name expected (no slash) Run `lvcreate --help' for more
information.
the source of the image is "/vm-images-pool-dir/images-vol/template_minimal" and the guest's target back end is a LV of 10G on another storage pool called "virtual-machines"
Not understanding what the 'lvcreate' commmand is trying to do, shouldnt it at least use "virtual-machines" as the target VG. The tool I am using is Satellite 6.2. I am thinking its something silly that I am overlooking. Not sure where :)
Thanks
TG
Based on the paths in that command, it seems you wanted to create a new file based volume in the /vm-images-pool/images-vol/, ie your "vm-images-pool-dir" pool. The fact that you are seeing an error from "lvcreate" though, suggests that you mistakenly specified "vm-images-pool" to libvirt as the pool to use, causing it to try to create a logical volume instead. You don't show the actual command / API you are running, but check that you've given the right pool name to it.
I know the question has long been asked, but I just hit the same problem and found the answer. I couldn't find the exact virsh command you are using leading to this error, but here I used the following XML file with virsh vol-create libvirtVG logical.xml
<volume >
<name>vol02</name>
<capacity unit='KiB'>2097152</capacity>
<allocation unit='KiB'>0</allocation>
<backingStore>
<path>/dev/libvirtVG/sles15sp1</path>
</backingStore>
</volume>
To be able to get rid of the error I had to set the allocation to the value of the capacity. You can also see that virt-manager is automatically doing it for you:
https://github.com/virt-manager/virt-manager/blob/master/virtinst/storage.py#L646
The equivalent using the virsh vol-create-as command would be:
virsh vol-create-as libvirtVG vol02 2048MiB --allocation 2048MiB \
--backing-vol /dev/libvirtVG/sles15sp1

How do I read / understand ansible logs on target host (written by syslog)

When you execute ansible on some host, it will write to syslog on that host, something like this:
Dec 1 15:00:22 run-tools python: ansible-<stdin> Invoked with partial=False links=None copy_links=None perms=None owner=False rsync_path=None dest_port=22 _local_rsync_path=rsync group=False existing_only=False archive=True _substitute_controller=False verify_host=False dirs=False private_key=None dest= compress=True rsync_timeout=0 rsync_opts=None set_remote_user=True recursive=None src=/etc/ansible/repo/external/golive/ checksum=False times=None mode=push ssh_args=None delete=False
Dec 1 15:00:22 run-tools python: ansible-<stdin> Invoked with partial=False links=None copy_links=None perms=None owner=False rsync_path=None dest_port=22 _local_rsync_path=rsync group=False existing_only=False archive=True _substitute_controller=False verify_host=False dirs=False private_key=None dest= compress=True rsync_timeout=0 rsync_opts=None set_remote_user=True recursive=None src=/etc/ansible/repo/external/golive/ checksum=False times=None mode=push ssh_args=None delete=False
Dec 1 15:00:22 run-tools python: ansible-<stdin> Invoked with partial=False links=None copy_links=None perms=None owner=False rsync_path=None dest_port=22 _local_rsync_path=rsync group=False existing_only=False archive=True _substitute_controller=False verify_host=False dirs=False private_key=None dest= compress=True rsync_timeout=0 rsync_opts=None set_remote_user=True recursive=None src=/etc/ansible/repo/external/golive/ checksum=False times=None mode=push ssh_args=None delete=False
Dec 1 15:00:56 run-tools python: ansible-<stdin> Invoked with filter=* fact_path=/etc/ansible/facts.d
Dec 1 15:09:56 run-tools python: ansible-<stdin> Invoked with checksum_algorithm=sha1 mime=False get_checksum=True path=/usr/local/bin/check_open_files_generic.sh checksum_algo=sha1 follow=False get_md5=False
Dec 1 15:09:56 run-tools python: ansible-<stdin> Invoked with directory_mode=None force=False remote_src=None path=/usr/local/bin/check_open_files_generic.sh owner=root follow=False group=root state=None content=NOT_LOGGING_PARAMETER serole=None diff_peek=None setype=None dest=/usr/local/bin/check_open_files_generic.sh selevel=None original_basename=check_open_files_generic.sh regexp=None validate=None src=check_open_files_generic.sh seuser=None recurse=False delimiter=None mode=0755 backup=None
Dec 1 15:20:03 run-tools python: ansible-<stdin> Invoked with warn=True executable=None _uses_shell=False _raw_params=visudo -c removes=None creates=None chdir=None
Is there any documentation or explanation of these logs that would help me understand how to read them? Specifically I would like to be able to see what exactly ansible did, which files it touched etc. Is it possible to find it there? Or reconfigure ansible so that it writes this kind of information in there?
Is it possible to configure these logs at all? How?
I am not aware of documentation that explains the contents of syslog messages specifically. However, you can look at some of the logging code in AnsibleModule.log() to see what's going on. Basically, it's reporting module names and the parameters they were called with.
For configuring logs, there are some good suggestions in response to this related question. The summary is that you can get more information - including your request about what ansible did - by specifying a log path and running with the verbose -v flag. For more fine-grained control, you can attack the problem from two different angles:
From the playbook side, you can use the debug module or tailor your handling of changed/failed results to suit your needs. Both of those changes can add useful context to your log output.
Outside of playbooks, you can use Ansible callback plugins to control logging. Here is an example of a callback plugin which intercepts logs and outputs something more human readable.

Resources