SElinux blocking Google-Chrome (headless) - google-chrome-headless

Good evening all,
Last month I commissioned a new OracleLinux server, and installed the latest Chrome version on it.
The project currently, is a small website which needs to generate a PDF report.
First, I create a HTML file with the entire content (it's only a header image, a table, and a footer - nothing too fancy).
Then, PHP issues this to generate the PDF file, after which it's downloaded by the user.
$output = shell_exec('/usr/bin/xvfb-run /usr/bin/google-chrome --no-sandbox --headless --disable-gpu --no-margins --print-to-pdf-no-header --print-to-pdf=/location/to/pdffile.pdf /location/to/htmlfile.html ');
This works great when I perform the action on the commandline (copy & paste the command).
However, when the PHP script executes the action, the following errors are generated and shown in the browser;
[0110/224510.775219:ERROR:file_path_watcher_linux.cc(71)] Failed to read /proc/sys/fs/inotify/max_user_watches
[0110/224510.779754:ERROR:udev_watcher.cc(51)] Failed to initialize a udev monitor. # # Fatal error in , line 0 # ignored # # # #FailureMessage Object: 0x7fff1b3fa650#0 0x55d633e63289 [0110/224510.816882:WARNING:crash_handler_host_linux.cc(341)] Could not translate tid, attempt = 1 retry ...
[0110/224510.830491:ERROR:file_path_watcher_linux.cc(71)] Failed to read /proc/sys/fs/inotify/max_user_watches
[0110/224510.918277:WARNING:crash_handler_host_linux.cc(341)] Could not translate tid, attempt = 2 retry ...
[0110/224511.018809:WARNING:crash_handler_host_linux.cc(341)] Could not translate tid, attempt = 3 retry ...
[0110/224511.119405:WARNING:crash_handler_host_linux.cc(365)] Could not translate tid - assuming crashing thread is thread group leader; syscall_supported=1 [0110/224511.121596:ERROR:crash_handler_host_linux.cc(444)] Failed to write crash dump for pid 106039 Cannot upload crash dump: failed to open --2021-01-10 22:45:11-- https://clients2.google.com/cr/report Resolving clients2.google.com (clients2.google.com)...
[0110/224511.507094:ERROR:headless_shell.cc(391)] Abnormal renderer termination. 172.217.17.142 Connecting to clients2.google.com (clients2.google.com)|172.217.17.142|:443... failed: Permission denied. Giving up. Unexpected crash report id length Failed to get crash dump id. Report Id:
Trying to find anything useless on the internet on those errors, failed.
So I looked further in /var/log/messages
It appears that Chrome needs all kinds of permissions to print a HTML into a PDF.
SElinux is prohibiting Chrome certain actions.
Now I'm wondering ... What is the solution?
Can I simply execute the commands suggested by SElinux in the errors below, or is it better to do something else? If so, what?
After spending the entire weekend on the internet, I found no real information regarding this.
[root#testserver1 bin]# > /var/log/messages
[root#testserver1 bin]# cat /var/log/messages
Jan 10 22:45:10 testserver1 dbus-daemon[1112]: [system] Activating service name='org.fedoraproject.Setroubleshootd' requested by ':1.150' (uid=0 pid=26234 comm="/usr/sbin/sedispatch " label="system_u:system_r:auditd_t:s0") (using servicehelper)
Jan 10 22:45:11 testserver1 kernel: traps: chrome[106039] trap invalid opcode ip:55d636e755d2 sp:7fff1b3fa640 error:0 in chrome[55d6312bf000+7b80000]
Jan 10 22:45:11 testserver1 systemd[1]: Started Process Core Dump (PID 106054/UID 0).
Jan 10 22:45:11 testserver1 dbus-daemon[1112]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Jan 10 22:45:11 testserver1 systemd-coredump[106057]: Process 106039 (chrome) of user 48 dumped core.#012#012Stack trace of thread 106039:#012#0 0x000055d636e755d2 n/a (chrome)#012#1 0x000055d636e7270e n/a (chrome)#012#2 0x000055d6331e2ce1 n/a (chrome)#012#3 0x000055d6331efcef n/a (chrome)#012#4 0x000055d63311e086 n/a (chrome)#012#5 0x000055d63311ec19 n/a (chrome)#012#6 0x000055d6334eb437 n/a (chrome)#012#7 0x000055d63301a159 n/a (chrome)#012#8 0x000055d636e7785a n/a (chrome)#012#9 0x000055d636d6df47 n/a (chrome)#012#10 0x000055d636d6e6d3 n/a (chrome)#012#11 0x000055d636e9f4d6 n/a (chrome)#012#12 0x000055d6385bf1c2 n/a (chrome)#012#13 0x000055d6385af82e n/a (chrome)#012#14 0x000055d6385b1d0f n/a (chrome)#012#15 0x000055d638d3add0 n/a (chrome)#012#16 0x000055d633d7d3e2 n/a (chrome)#012#17 0x000055d633d7b659 n/a (chrome)#012#18 0x000055d633dcae1c n/a (chrome)#012#19 0x000055d633dcac51 n/a (chrome)#012#20 0x000055d633dc94d5 n/a (chrome)#012#21 0x000055d63160676d ChromeMain (chrome)#012#22 0x00007ff0a7daf7a3 __libc_start_main (libc.so.6)#012#23 0x000055d6312bfa6a _start (chrome)#012#012Stack trace of thread 106042:#012#0 0x00007ff0a7e89257 epoll_wait (libc.so.6)#012#1 0x000055d633f9efeb n/a (chrome)#012#2 0x000055d633f9c9cb n/a (chrome)#012#3 0x000055d633e83a4b n/a (chrome)#012#4 0x000055d633e3572a n/a (chrome)#012#5 0x000055d633e0b811 n/a (chrome)#012#6 0x000055d633e4792d n/a (chrome)#012#7 0x000055d633e4e809 n/a (chrome)#012#8 0x000055d633e73b94 n/a (chrome)#012#9 0x00007ff0ada4a16a start_thread (libpthread.so.0)#012#10 0x00007ff0a7e88f23 __clone (libc.so.6)#012#012Stack trace of thread 106043:#012#0 0x00007ff0ada50708 pthread_cond_timedwait##GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d633e6fbd1 n/a (chrome)#012#2 0x000055d633e70243 n/a (chrome)#012#3 0x000055d633e489bf n/a (chrome)#012#4 0x000055d633e485ad n/a (chrome)#012#5 0x000055d633e73b94 n/a (chrome)#012#6 0x00007ff0ada4a16a start_thread (libpthread.so.0)#012#7 0x00007ff0a7e88f23 __clone (libc.so.6)#012#012Stack trace of thread 106044:#012#0 0x00007ff0a7e89257 epoll_wait (libc.so.6)#012#1 0x000055d633f9efeb n/a (chrome)#012#2 0x000055d633f9c9cb n/a (chrome)#012#3 0x000055d633e83a4b n/a (chrome)#012#4 0x000055d633e3572a n/a (chrome)#012#5 0x000055d633e0b811 n/a (chrome)#012#6 0x000055d633e4e809 n/a (chrome)#012#7 0x000055d633e73b94 n/a (chrome)#012#8 0x00007ff0ada4a16a start_thread (libpthread.so.0)#012#9 0x00007ff0a7e88f23 __clone (libc.so.6)#012#012Stack trace of thread 106045:#012#0 0x00007ff0ada5031c pthread_cond_wait##GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d633e6fa3d n/a (chrome)#012#2 0x000055d633e70396 n/a (chrome)#012#3 0x000055d633e7000e n/a (chrome)#012#4 0x000055d633de6487 n/a (chrome)#012#5 0x000055d633e3572a n/a (chrome)#012#6 0x000055d633e0b811 n/a (chrome)#012#7 0x000055d633e4e809 n/a (chrome)#012#8 0x000055d633e73b94 n/a (chrome)#012#9 0x00007ff0ada4a16a start_thread (libpthread.so.0)#012#10 0x00007ff0a7e88f23 __clone (libc.so.6)#012#012Stack trace of thread 106051:#012#0 0x00007ff0ada50708 pthread_cond_timedwait##GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d633e6fbd1 n/a (chrome)#012#2 0x000055d633e70243 n/a (chrome)#012#3 0x000055d633e4875c n/a (chrome)#012#4 0x000055d633e485ad n/a (chrome)#012#5 0x000055d633e73b94 n/a (chrome)#012#6 0x00007ff0ada4a16a start_thread (libpthread.so.0)#012#7 0x00007ff0a7e88f23 __clone (libc.so.6)
Jan 10 22:45:11 testserver1 systemd[1]: systemd-coredump#31-106054-0.service: Succeeded.
Jan 10 22:45:11 testserver1 dbus-daemon[1112]: [system] Activating service name='org.fedoraproject.SetroubleshootPrivileged' requested by ':1.3219' (uid=995 pid=106033 comm="/usr/libexec/platform-python -Es /usr/sbin/setroub" label="system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023") (using servicehelper)
Jan 10 22:45:12 testserver1 dbus-daemon[1112]: [system] Successfully activated service 'org.fedoraproject.SetroubleshootPrivileged'
Jan 10 22:45:13 testserver1 setroubleshoot[106033]: SELinux is preventing /usr/bin/bash from write access on the directory /dev/fd/63. For complete SELinux messages run: sealert -l 472c7d4e-7e04-47a1-9dae-17cd3c696468
Jan 10 22:45:13 testserver1 setroubleshoot[106033]: SELinux is preventing /usr/bin/bash from write access on the directory /dev/fd/63.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that bash should be allowed write access on the 63 directory by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'google-chrome' --raw | audit2allow -M my-googlechrome#012# semodule -X 300 -i my-googlechrome.pp#012
Jan 10 22:45:13 testserver1 setroubleshoot[106033]: failed to retrieve rpm info for /proc/sys/fs/inotify/max_user_watches
Jan 10 22:45:14 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from read access on the file /proc/sys/fs/inotify/max_user_watches. For complete SELinux messages run: sealert -l 1c33cba5-b4df-430d-9bdb-dec10b11639f
Jan 10 22:45:14 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from read access on the file /proc/sys/fs/inotify/max_user_watches.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed read access on the max_user_watches file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'ThreadPoolForeg' --raw | audit2allow -M my-ThreadPoolForeg#012# semodule -X 300 -i my-ThreadPoolForeg.pp#012
Jan 10 22:45:15 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the execmem access on a process. For complete SELinux messages run: sealert -l 2c8783bf-dec7-454d-997e-6a2c1dfbb240
Jan 10 22:45:15 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the execmem access on a process.#012#012***** Plugin allow_execmem (53.1 confidence) suggests *********************#012#012If you know why chrome needs to map a memory region that is both executable and writable and understand that this is a potential security problem.#012Then you can allow the mapping by switching one of the following booleans: httpd_execmem#012Do#012follow the advice of the catchall_boolean plugin, otherwise contact your security administrator and report this issue#012#012***** Plugin catchall_boolean (42.6 confidence) suggests ******************#012#012If you want to allow httpd to execmem#012Then you must tell SELinux about this by enabling the 'httpd_execmem' boolean.#012#012Do#012setsebool -P httpd_execmem 1#012#012***** Plugin catchall (5.76 confidence) suggests **************************#012#012If you believe that chrome should be allowed execmem access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:16 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:16 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:16 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:16 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:17 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:17 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:18 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:18 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:19 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:19 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
Jan 10 22:45:19 testserver1 setroubleshoot[106033]: failed to retrieve rpm info for /proc/sys/fs/inotify/max_user_watches
Jan 10 22:45:20 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from read access on the file /proc/sys/fs/inotify/max_user_watches. For complete SELinux messages run: sealert -l 1c33cba5-b4df-430d-9bdb-dec10b11639f
Jan 10 22:45:20 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from read access on the file /proc/sys/fs/inotify/max_user_watches.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed read access on the max_user_watches file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'ThreadPoolForeg' --raw | audit2allow -M my-ThreadPoolForeg#012# semodule -X 300 -i my-ThreadPoolForeg.pp#012
Jan 10 22:45:21 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process. For complete SELinux messages run: sealert -l 87b4832d-2497-4a8f-a5b6-7c0124a90cf8
Jan 10 22:45:21 testserver1 setroubleshoot[106033]: SELinux is preventing /opt/google/chrome/chrome from using the ptrace access on a process.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that chrome should be allowed ptrace access on processes labeled httpd_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'chrome' --raw | audit2allow -M my-chrome#012# semodule -X 300 -i my-chrome.pp#012
[root#testserver1 bin]#
Any help is appreciated.

I feel like an idiot... I should have broaden my search. Instead of looking just for my specific error, look for generic errors and solutions. So I spent a weekend for nothing.
The solution will work for anyone I guess, I found this URL here:
https://www.serverlab.ca/tutorials/linux/administration-linux/troubleshooting-selinux-centos-red-hat/
It explains how to troubleshoot and fix these errors.
Install the setroubleshoot packages using Yum.
sudo yum install setroubleshoot setools
Because I wanted to start with a fresh logfile, I rotated the audit.log;
sudo service auditd rotate
Then... Do whatever you did to generate the errors.
After that, start the just installed troubleshoot package;
sudo sealert -a /var/log/audit/audit.log
It provides a report with all errors found, and ways to fix them.
The report will sort the errors and fixes by confidence level.
My approach was, fix the errors with a 100% confedence level.
Then rotate the logs again, try again what I wanted to do, see if it works or not, and if it doesn't, generate a new report.
Etc.

Related

Dronekit-sitl fails to bind on default port 5760

I have dronekit-sitl installed in a python3 virtual environment on my Windows 10 machine and have used it before by running dronekit-sitl copter with no issues. However, as of today I am running across what seems to be a permission issue when trying to execute the ArduCopter sitl.
$ dronekit-sitl copter
os: win, apm: copter, release: stable
SITL already Downloaded and Extracted.
Ready to boot.
Execute: C:\Users\kyrlon\.dronekit\sitl\copter-3.3\apm.exe --home=-35.363261,149.165230,584,353 --model=quad -I 0
SITL-0> Started model quad at -35.363261,149.165230,584,353 at speed 1.0
SITL-0.stderr> bind port 5760 for 0
Starting sketch 'ArduCopter'
bind failed on port 5760 - Operation not permitted
Starting SITL input
Not sure what might have triggered a new operation permission issue, and I tried to start over with a fresh Python environment, but even after a complete PC shutdown, I am still having the error as shown above.
It turns out that having docker on my system was the culprit and excluding the port I was attempting to use as mentioned in this SO post that led me to this github issue. Running the command in an elevated terminal:
netsh interface ipv4 show excludedportrange protocol=tcp
Provided me the results of the following excluded ports:
Protocol tcp Port Exclusion Ranges
Start Port End Port
---------- --------
1496 1595
1658 1757
1758 1857
1858 1957
1958 2057
2058 2157
2180 2279
2280 2379
2380 2479
2480 2579
2702 2801
2802 2901
2902 3001
3002 3101
3102 3201
3202 3301
3390 3489
3490 3589
3590 3689
3693 3792
3793 3892
3893 3992
3993 4092
4093 4192
4193 4292
4293 4392
4393 4492
4493 4592
4593 4692
4768 4867
4868 4967
5041 5140
5141 5240
5241 5340
5357 5357
5358 5457
5458 5557
5558 5657
5700 5700
5701 5800
8005 8005
8884 8884
15202 15301
15302 15401
15402 15501
15502 15601
15602 15701
15702 15801
15802 15901
15902 16001
16002 16101
16102 16201
16202 16301
16302 16401
16402 16501
16502 16601
16602 16701
16702 16801
16802 16901
16993 17092
17093 17192
50000 50059 *
* - Administered port exclusions.
Turns out that docker or possibly Hyper-V excluded the range that included 5760:
5701 5800
And as mentioned from the github issue, I probably resolved this issue before after a set number of restarts that incremented the port ranges, or possibly got lucky in the past starting dronekit-sitl before docker ran on my system.
Either way, to resolve this issue of Operation not permitted, running the command as admin:
net stop winnat
net start winnat
solved the issue with dronekit-sitl without having to specify a different port besides the default 5760.

Start systemd service after DRM card0 device

I have kiosk display running an electron app.
This app needs to be started on boot.
[Unit]
Description=Display Application
After=network.target getty#tty1.service
Conflicts=getty#tty1.service
[Service]
Type=simple
ExecStart=/usr/bin/xinit /usr/bin/electron main.js -- :0 -nocursor -nolisten tcp
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
But for some reason this service ist started to early and xinit complains with no screens found. After the first systemd restart the app is running.
I searchd systemd device which sounded promising. But using dev-dri-card0 as Requires and After results in Missing Dependency in startup.
# ls /dev/dri
by-path card0 renderD128
# dmesg | grep drm
[ 3.609070] systemd[1]: Starting Load Kernel Module drm...
[ 3.672741] systemd[1]: modprobe#drm.service: Succeeded.
[ 3.673192] systemd[1]: Finished Load Kernel Module drm.
[ 4.327074] fb0: switching to inteldrmfb from EFI VGA
[ 4.332654] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/bxt_dmc_ver1_07.bin (v1.7)
[ 4.356201] i915 0000:00:02.0: [drm] Panel advertises DPCD backlight support, but VBT disagrees. If your backlight controls don't work try booting with i915.enable_dpcd_backlight=1. If your machine needs this, please file a _new_ bug report on drm/i915, see https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs for details.
[ 4.367399] [drm] Initialized i915 1.6.0 20200917 for 0000:00:02.0 on minor 0
[ 4.376167] fbcon: i915drmfb (fb0) is primary device
[ 4.402186] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
# systemctl list-units -a -t device | grep card0
dev-dri-card0.device loaded inactive dead /dev/dri/card0
I have no glue whats wrong here. Found some posts saying that drm module is loaded to late (after systemd) ... added drm to initramfs ... no luck.
Ok found a solution
[Unit]
Description=Display Application
After=network.target systemd-user-sessions.service
Conflicts=getty#tty1.service
[Service]
TTYPath=/dev/tty1
PAMName=systemd-user
Environment=NODE_ENV=production
ExecStart=/usr/bin/xinit /usr/bin/electron main.js -- :0 -quiet -nocursor -nolisten tcp
Restart=on-failure
RestartSec=5
[Install]
Alias=display-manager.service
WantedBy=multi-user.target
This way the systemd-logind brings up a user session and all the drm stuff is ready.

yum and dnf install/update takes 5 minutes to start on RHEL8

If I run yum or dnf as root on most recent updated RHEL8 it works smoothly. As soon as I try running it with sudo users (added to wheel group) it takes up to 5 minutes.
Doing a yum clean did not help eiher.
I usually run updates using Ansible from a remote host but Ansible disconnects after several minutes of trying to run the yum module.
I set SELinux to disabled, just in case and checked proxy settings in dnf.conf - It all looks fine. Any input is very much appreciated.
This is the dnf.log
2020-02-20T15:33:13Z INFO Updating Subscription Management repositories.
2020-02-20T15:37:36Z DEBUG DNF version: 4.2.7
2020-02-20T15:37:36Z DDEBUG Command: yum update
2020-02-20T15:37:36Z DDEBUG Installroot: /
2020-02-20T15:37:36Z DDEBUG Releasever: 8
2020-02-20T15:37:36Z DEBUG cachedir: /var/cache/dnf
2020-02-20T15:37:36Z DDEBUG Base command: update
2020-02-20T15:37:36Z DDEBUG Extra commands: ['update']
2020-02-20T15:37:37Z DEBUG repo: using cache for: epel-modular
2020-02-20T15:37:37Z DEBUG epel-modular: using metadata from Sat Feb 15 03:19:39 2020.
2020-02-20T15:37:37Z DEBUG repo: using cache for: epel
2020-02-20T15:37:37Z DEBUG epel: using metadata from Thu Feb 20 06:38:22 2020.
2020-02-20T15:37:38Z DEBUG reviving: 'rhel-8-for-x86_64-baseos-rpms' can be revived - repomd matches.
2020-02-20T15:37:38Z DEBUG rhel-8-for-x86_64-baseos-rpms: using metadata from Thu Feb 13 10:30:57 2020.
2020-02-20T15:37:38Z DEBUG reviving: 'rhel-8-for-x86_64-appstream-rpms' can be revived - repomd matches.
2020-02-20T15:37:39Z DEBUG rhel-8-for-x86_64-appstream-rpms: using metadata from Thu Feb 20 11:06:31 2020.
2020-02-20T15:37:39Z DDEBUG timer: sack setup: 3550 ms
2020-02-20T15:37:39Z DEBUG Completion plugin: Generating completion cache...
2020-02-20T15:37:40Z DEBUG --> Starting dependency resolution
2020-02-20T15:37:40Z DEBUG --> Finished dependency resolution
2020-02-20T15:37:40Z DDEBUG timer: depsolve: 277 ms
2020-02-20T15:37:40Z INFO Dependencies resolved.
2020-02-20T15:37:40Z INFO Nothing to do.
2020-02-20T15:37:40Z INFO Complete!
2020-02-20T15:37:40Z DDEBUG Cleaning up.
I resolved the issue. The proxy was not correctly set in /etc/rhsm/rhsm.conf
I don't understand why running as root would not cause the same problem.

Couchbase-Server Community Edition 5.0.1 Installation issue on MacOS High Sierra version 10.13.2

I had previous working copy of the Couchbase Server community Edition 4*, this stopped opening the Admin Console page from the menu options post upgrade to the macOs high sierra.
Action performed
I have wiped deleted and re-install with the latest version of CB Server 5.0.1.
Clicked the Couchbase Server.app to begin with it has pinned the couchbase icon in the menu bar.
Selected "Open Admin Console"
Browser complained 'Safari Can't Connect to 127.0.0.1:8091'
Checked if there is any process running? And found there are.
Rizwans-MacBook-Pro:logs rizwan$ ps -ef | grep couch
501 6411 6396 0 9:08AM ?? 0:01.48 /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/erlang/erts-5.10.4.0.0.1/bin/beam.smp -A 16 -- -root /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/erlang -progname erl -- -home /Users/rizwan -- -kernel inet_dist_listen_min 21100 inet_dist_listen_max 21299 -sasl sasl_error_logger false -hidden -name babysitter_of_ns_1#127.0.0.1 -setcookie nocookie -run ns_babysitter_bootstrap -- -couch_ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.d/capi.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.d/geocouch.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/local.ini /Users/rizwan/Library/Preferences/couchbase-server.ini /Users/rizwan/Library/Application Support/Couchbase/etc/couch-platform.ini /Users/rizwan/Library/Application Support/Couchbase/etc/couch-custom.ini -ns_babysitter cookiefile "/Users/rizwan/Library/Application Support/Couchbase/var/lib/couchbase/couchbase-server.cookie" -ns_server config_path "/Users/rizwan/Library/Application Support/Couchbase/etc/couchbase/static_config" -ns_server pidfile "/Users/rizwan/Library/Application Support/Couchbase/couchbase-server.pid" -ns_server cookiefile "/Users/rizwan/Library/Application Support/Couchbase/var/lib/couchbase/couchbase-server.cookie-ns-server" -ns_server dont_suppress_stderr_logger true -ns_server loglevel_stderr info
501 6704 6411 0 9:12AM ?? 0:00.80 /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/erlang/erts-5.10.4.0.0.1/bin/beam.smp -A 16 -sbt u -P 327680 -K true -swt low -MMmcs 30 -e102400 -- -root /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/erlang -progname erl -- -home /Users/rizwan -- -smp enable -setcookie nocookie -kernel inet_dist_listen_min 21100 inet_dist_listen_max 21299 error_logger false -sasl sasl_error_logger false -user user_io -run child_erlang child_start ns_bootstrap -- -couch_ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.d/capi.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/default.d/geocouch.ini /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/etc/couchdb/local.ini /Users/rizwan/Library/Preferences/couchbase-server.ini /Users/rizwan/Library/Application Support/Couchbase/etc/couch-platform.ini /Users/rizwan/Library/Application Support/Couchbase/etc/couch-custom.ini
Further to it try the couchbase server starting from the command line via the script available in download. It failed with the below log
Rizwans-MacBook-Pro:Resources rizwan$ ./start-server.sh
Starting epmd ...
Starting Couchbase Server ...
{error_logger,{{2018,1,7},{20,33,26}},"Protocol: ~tp: the name babysitter_of_ns_1#127.0.0.1 seems to be in use by another Erlang node",["inet_tcp"]}
{error_logger,{{2018,1,7},{20,33,26}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.20.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,320}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.57>,<0.17.0>]},{dictionary,[{longnames,true}]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,1105}],[]]}
{error_logger,{{2018,1,7},{20,33,26}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[['babysitter_of_ns_1#127.0.0.1',longnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2018,1,7},{20,33,26}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2018,1,7},{20,33,26}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}
{error_logger,{{2018,1,7},{20,33,26}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Ambari dashboard retrieving no statistics

I have a fresh install of Hortonworks Data Platform 2.2 installed on a small cluster (4 machines) but when I login to the Ambari GUI, the majority of dashboard stats boxes (HDFS disk usage, Network usage, Memory usage etc) are not populated with any statistics, instead they show the message:
No data There was no data available. Possible reasons include inaccessible Ganglia service
Clicking on the HDFS service link gives the following summary:
NameNode Started
SNameNode Started
DataNodes 4/4 DataNodes Live
NameNode Uptime Not Running
NameNode Heap n/a / n/a (0.0% used)
DataNodes Status 4 live / 0 dead / 0 decommissioning
Disk Usage (DFS Used) n/a / n/a (0%)
Disk Usage (Non DFS Used) n/a / n/a (0%)
Disk Usage (Remaining) n/a / n/a (0%)
Blocks (total) n/a
Block Errors n/a corrupt / n/a missing / n/a under replicated
Total Files + Directories n/a
Upgrade Status Upgrade not finalized
Safe Mode Status n/a
The Alerts and Health Checks box to the right of the screen is not displaying any information but if I click on the settings icon this opens the Nagios frontend and again, everything looks healthy here!
The install went smoothly (CentOS 6.5) and everything looks good as far as all services are concerned (all started with green tick next to service name). There are some stats displayed on the dashboard: 4/4 datanodes are live, 1/1 Nodemanages live & 1/1 Supervisors are live. I can write files to HDFS so its looks like it's a Ganglia issue?
The Ganglia daemon seems to be working ok:
ps -ef | grep gmond
nobody 1720 1 0 12:54 ? 00:00:44 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPHistoryServer/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPHistoryServer/gmond.pid
nobody 1753 1 0 12:54 ? 00:00:44 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPFlumeServer/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPFlumeServer/gmond.pid
nobody 1790 1 0 12:54 ? 00:00:48 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPHBaseMaster/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPHBaseMaster/gmond.pid
nobody 1821 1 1 12:54 ? 00:00:57 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPKafka/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPKafka/gmond.pid
nobody 1850 1 0 12:54 ? 00:00:44 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPSupervisor/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPSupervisor/gmond.pid
nobody 1879 1 0 12:54 ? 00:00:45 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPSlaves/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPSlaves/gmond.pid
nobody 1909 1 0 12:54 ? 00:00:48 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPResourceManager/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPResourceManager/gmond.pid
nobody 1938 1 0 12:54 ? 00:00:50 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPNameNode/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPNameNode/gmond.pid
nobody 1967 1 0 12:54 ? 00:00:47 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPNodeManager/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPNodeManager/gmond.pid
nobody 1996 1 0 12:54 ? 00:00:44 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPNimbus/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPNimbus/gmond.pid
nobody 2028 1 1 12:54 ? 00:00:58 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPDataNode/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPDataNode/gmond.pid
nobody 2057 1 0 12:54 ? 00:00:51 /usr/sbin/gmond --conf=/etc/ganglia/hdp/HDPHBaseRegionServer/gmond.core.conf --pid-file=/var/run/ganglia/hdp/HDPHBaseRegionServer/gmond.pid
I have checked the Ganglia service on each node, the processes are running as expected
ps -ef | grep gmetad
nobody 2807 1 2 12:55 ? 00:01:59 /usr/sbin/gmetad --conf=/etc/ganglia/hdp/gmetad.conf --pid-file=/var/run/ganglia/hdp/gmetad.pid
I have tried restarting Ganglia services with no luck, restarted all services but still the same. Does anyone have any ideas how I get the dashboard to work properly? Thank you.
It turns out to be a proxy issue, to access the internet I had to add my proxy details to the file /var/lib/ambari-server/ambari-env.sh
export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Dhttp.proxyHost=theproxy -Dhttp.proxyPort=80 -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false'
When ganglia was trying to access each node in the cluster the request was going via the proxy and never resolving, to overcome the issue I added my nodes to the exclude list (add the flag -Dhttp.nonProxyHosts) like so:
export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Dhttp.proxyHost=theproxy -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts="localhost|node1.dms|node2.dms|node3.dms|etc" -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false'
After adding the exclude list the stats were retrieved as expected!

Resources