I am Trying to install DC/OS but facing issues in On-Premises
Below is the Error Log
Connection to 172.XX.XXX.XX closed.
Starting DC/OS Install Process
Running preflight checks
Checking if DC/OS is already installed: PASS (Not installed)
PASS Is SELinux disabled?
Checking if docker is installed and in PATH: PASS
Checking docker version requirement (>= 1.6): PASS (1.11.1)
Checking if curl is installed and in PATH: PASS
Checking if bash is installed and in PATH: PASS
Checking if ping is installed and in PATH: PASS
Checking if tar is installed and in PATH: PASS
Checking if xz is installed and in PATH: PASS
Checking if unzip is installed and in PATH: PASS
Checking if ipset is installed and in PATH: PASS
Checking if systemd-notify is installed and in PATH: PASS
Checking if systemd is installed and in PATH: PASS
Checking systemd version requirement (>= 200): PASS (219)
Checking if group 'nogroup' exists: PASS
Checking if port 80 (required by mesos-ui) is in use: PASS
Checking if port 53 (required by mesos-dns) is in use: PASS
Checking if port 15055 (required by dcos-history) is in use: PASS
Checking if port 5050 (required by mesos-master) is in use: PASS
Checking if port 2181 (required by zookeeper) is in use: PASS
Checking if port 8080 (required by marathon) is in use: PASS
Checking if port 3888 (required by zookeeper) is in use: PASS
Checking if port 8181 (required by exhibitor) is in use: PASS
Checking if port 8123 (required by mesos-dns) is in use: PASS
Checking Docker is configured with a production storage driver: WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
PASS (overlay)
Creating directories under /etc/mesosphere
Creating role file for master
Configuring DC/OS
Setting and starting DC/OS
Job for dcos-setup.service failed because the control process exited with error code. See "systemctl status dcos-setup.service" and "journalctl -xe" for details.
172.XX.XXX.XX:XX
Connection to 172.XX.XXX.XXX closed.
Starting DC/OS Install Process
Running preflight checks
Checking if DC/OS is already installed: PASS (Not installed)
PASS Is SELinux disabled?
Checking if docker is installed and in PATH: PASS
Checking docker version requirement (>= 1.6): PASS (1.11.1)
Checking if curl is installed and in PATH: PASS
Checking if bash is installed and in PATH: PASS
Checking if ping is installed and in PATH: PASS
Checking if tar is installed and in PATH: PASS
Checking if xz is installed and in PATH: PASS
Checking if unzip is installed and in PATH: PASS
Checking if ipset is installed and in PATH: PASS
Checking if systemd-notify is installed and in PATH: PASS
Checking if systemd is installed and in PATH: PASS
Checking systemd version requirement (>= 200): PASS (219)
Checking if group 'nogroup' exists: PASS
Checking if port 80 (required by mesos-ui) is in use: PASS
Checking if port 53 (required by mesos-dns) is in use: PASS
Checking if port 15055 (required by dcos-history) is in use: PASS
Checking if port 5050 (required by mesos-master) is in use: PASS
Checking if port 2181 (required by zookeeper) is in use: PASS
Checking if port 8080 (required by marathon) is in use: PASS
Checking if port 3888 (required by zookeeper) is in use: PASS
Checking if port 8181 (required by exhibitor) is in use: PASS
Checking if port 8123 (required by mesos-dns) is in use: PASS
Checking Docker is configured with a production storage driver: WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
PASS (overlay)
Creating directories under /etc/mesosphere
Creating role file for slave
Configuring DC/OS
Setting and starting DC/OS
Job for dcos-setup.service failed because the control process exited with error code. See "systemctl status dcos-setup.service" and "journalctl -xe" for details.
172.XX.XX.XXX:22
As suggested in Error Log we looked into "systemctl status dcos-setup.service" and "journalctl -xe" but not able to determine the actual cause
[root#macXX ~]# systemctl status dcos-setup.service
â dcos-setup.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
[root#macXX ~]# journalctl -xe
May 27 09:40:01 mac58 kernel: SELinux: initialized (dev overlay, type overlay), uses xattr
May 27 09:40:01 mac58 kernel: device veth0db88ce entered promiscuous mode
May 27 09:40:01 mac58 kernel: IPv6: ADDRCONF(NETDEV_UP): veth0db88ce: link is not ready
May 27 09:40:01 mac58 NetworkManager[776]: <warn> (veth3d6605b): failed to find device 58 'veth3d6605b' with udev
May 27 09:40:01 mac58 NetworkManager[776]: <info> (veth3d6605b): new Veth device (carrier: OFF, driver: 'veth', ifindex: 58)
May 27 09:40:01 mac58 NetworkManager[776]: <warn> (veth0db88ce): failed to find device 59 'veth0db88ce' with udev
May 27 09:40:01 mac58 NetworkManager[776]: <info> (veth0db88ce): new Veth device (carrier: OFF, driver: 'veth', ifindex: 59)
May 27 09:40:01 mac58 NetworkManager[776]: <info> (docker0): bridge port veth0db88ce was attached
May 27 09:40:01 mac58 NetworkManager[776]: <info> (veth0db88ce): enslaved to docker0
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev proc, type proc), uses genfs_contexts
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev devpts, type devpts), uses transition SIDs
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
May 27 09:40:02 mac58 kernel: SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
May 27 09:40:02 mac58 avahi-daemon[739]: Withdrawing workstation service for veth3d6605b.
May 27 09:40:02 mac58 NetworkManager[776]: <warn> (veth3d6605b): failed to disable userspace IPv6LL address handling
May 27 09:40:02 mac58 NetworkManager[776]: <info> (veth0db88ce): link connected
May 27 09:40:02 mac58 NetworkManager[776]: <info> (docker0): link connected
May 27 09:40:02 mac58 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth0db88ce: link becomes ready
May 27 09:40:02 mac58 kernel: docker0: port 1(veth0db88ce) entered forwarding state
May 27 09:40:02 mac58 kernel: docker0: port 1(veth0db88ce) entered forwarding state
May 27 09:40:03 mac58 avahi-daemon[739]: Registering new address record for fe80::4410:e6ff:fedd:be2a on veth0db88ce.*.
May 27 09:40:17 mac58 kernel: docker0: port 1(veth0db88ce) entered forwarding state
May 27 09:47:10 mac58 kernel: traps: python3[17232] general protection ip:7f2be234423 sp:7f2beceb0394 error:0 in libc-2.22.s
May 27 09:47:11 mac58 kernel: docker0: port 1(veth0db88ce) entered disabled state
May 27 09:47:11 mac58 NetworkManager[776]: <info> (veth0db88ce): link disconnected
May 27 09:47:11 mac58 NetworkManager[776]: <warn> (veth3d6605b): failed to find device 58 'veth3d6605b' with udev
May 27 09:47:11 mac58 NetworkManager[776]: <info> (veth3d6605b): new Veth device (carrier: OFF, driver: 'veth', ifindex: 58)
May 27 09:47:11 mac58 NetworkManager[776]: <info> (docker0): link disconnected (deferring action for 4 seconds)
May 27 09:47:11 mac58 abrt-server[17352]: Executable '/opt/mesosphere/packages/python--e3169ded66609d3cb4055a3f9f8f0b1113a557
May 27 09:47:11 mac58 abrt-server[17352]: 'post-create' on '/var/spool/abrt/ccpp-2016-05-27-09:47:10-17214' exited with 1
May 27 09:47:11 mac58 abrt-server[17352]: Deleting problem directory '/var/spool/abrt/ccpp-2016-05-27-09:47:10-17214'
May 27 09:47:11 mac58 avahi-daemon[739]: Withdrawing address record for fe80::4410:e6ff:fedd:be2a on veth0db88ce.
May 27 09:47:11 mac58 kernel: docker0: port 1(veth0db88ce) entered disabled state
May 27 09:47:11 mac58 avahi-daemon[739]: Withdrawing workstation service for veth3d6605b.
May 27 09:47:11 mac58 avahi-daemon[739]: Withdrawing workstation service for veth0db88ce.
May 27 09:47:11 mac58 NetworkManager[776]: <warn> (veth3d6605b): failed to disable userspace IPv6LL address handling
May 27 09:47:11 mac58 NetworkManager[776]: <info> (docker0): bridge port veth0db88ce was detached
May 27 09:47:11 mac58 NetworkManager[776]: <info> (veth0db88ce): released from master docker0
May 27 09:47:11 mac58 NetworkManager[776]: <warn> (veth0db88ce): failed to disable userspace IPv6LL address handling
May 27 09:47:11 mac58 kernel: device veth0db88ce left promiscuous mode
May 27 09:47:11 mac58 kernel: docker0: port 1(veth0db88ce) entered disabled state
May 27 09:47:11 mac58 docker[10803]: time="2016-05-27T09:47:11.828750505+05:30" level=error msg="Handler for POST /v1.23/cont
May 27 09:47:16 mac58 NetworkManager[776]: <info> (docker0): link disconnected (calling deferred action)
Your Help is Highly Appreciated!!!...Thank you
Related
I want to connect my OpenVPN server (Ubuntu 16.4) in my office to my Mikrotik at home as client.
I already have OpenVPN server set based on this tutorial (link). If I try to connect it connects with OpenVPN client Windows app (no errors), and asks for username and password, with Client.ovpn added in Program Files/OpenVPN/config.
Here is my server.conf in OpenVPN server:
port 51333
proto tcp
dev tun5
ca ca.crt
cert server.crt
key server.key
dh dh2048.pem
server 10.8.101.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "route 192.168.20.10 255.255.255.255" #This is my server that I want to connect in Office
keepalive 10 120
tls-auth /etc/openvpn/ta.key
key-direction 0
cipher AES-256-CBC
auth SHA1
max-clients 10
user nobody
group nogroup
persist-key
persist-tun
username-as-common-name
plugin /usr/lib/openvpn/openvpn-plugin-auth-pam.so /etc/pam.d/openvpn
status /etc/openvpn/mikrotik.log
verb 5
mute-replay-warnings
client-config-dir ccd
management localhost 7505
Here is my configuration of base.conf in OpenVPN server (Ubuntu) for creating client.ovpn files:
client
dev tun
proto tcp
remote mydomain.com 51333 #in my DNS I redirect this domain to my public static domain in office there in firewall I am portforwarding this port to my server in office
resolv-retry infinite
nobind
user nobody
group nogroup
auth-user-pass
#ca ca.crt
#cert client.crt
#key client.key
remote-cert-tls server
tls-auth ta.key
cipher AES-256-CBC
auth SHA1
# More reliable detection when a system loses its connection.
ping 15
ping-restart 45
ping-timer-rem
persist-tun
persist-key
verb 5
key-direction 1
Now in Mikrotik (OS is: v6.42.12 in hAP lite (simps) I have:
Import my Client.ovpn in Files
Import Certificate Client.ovpn and set for T (name: ca.crt_0) and for KT (name: client.crt_0).
Create new PPP Profile: ppp profile add name=OVPN-client change-tcp-mss=yes only-one=yes use-encryption=required use-mpls=no
Create new interface: interface ovpn-client add connect-to=mydomain.com port 51333 add-default-route=no auth=sha1 certificate=client.crt_0 disabled=no user=vpnuser password=vpnpass name=myvpn profile=OVPN-client
But with this configuration, I cannot establish a connection. I cannot get "R - status" on OVPN-client, I only get this error:
* ovpn-out1: connecting
* ovpn-out1: terminating - peer disconnected
* ovpn-out1: disconnected
If I check logs in server I get this:
openVPN1 ovpn-server[2050]: MULTI: multi_create_instance called
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Re-using SSL/TLS context
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Control Channel MTU parms
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Data Channel MTU parms
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Local Options String: 'V4,dev-type tun,link-mtu 1559,tun-mtu 1500,proto TCPv4_SERVER,keydir 0,cipher AES-256-CBC,auth SHA1,keysize 256,tls-auth,key-method 2,tls-server'
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Expected Remote Options String: 'V4,dev-type tun,link-mtu 1559,tun-mtu 1500,proto TCPv4_CLIENT,keydir 1,cipher AES-256-CBC,auth SHA1,keysize 256,tls-auth,key-method 2,tls-client'
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Local Options hash (VER=V4): '7ac8f09f'
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: Expected Remote Options hash (VER=V4): '53276059'
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: TCP connection established with [AF_INET]XX.XXX.XXX.XX:60345
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: TCPv4_SERVER link local: [undef]
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: TCPv4_SERVER link remote: [AF_INET]XX.XXX.XXX.XX:60345
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: XX.XXX.XXX.XX:60345 TLS: Initial packet from [AF_INET]XX.XXX.XXX.XX:60345
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: XX.XXX.XXX.XX:60345 TLS Error: cannot locate HMAC in incoming packet from [AF_INET]XX.XXX.XXX.XX:60345
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: XX.XXX.XXX.XX:60345 Fatal TLS error (check_tls_errors_co), restarting
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: XX.XXX.XXX.XX:60345 SIGUSR1[soft,tls-error] received, client-instance restarting
Oct 26 09:33:03 openVPN1 ovpn-server[2050]: TCP/UDP: Closing socket
MikroTik does not support UDP in OpenVPN only TCP.
ROS 7 will support it but it's still in development phase.
As of 2022 - you would need to use ROS version 7 or later, where UPP support for OpenVPN is implemented. Still, Mikrotik's implementation of OpenVPN is limited, eg. no support for TLS auth with a static key.
I am installing tor in my ubuntu 18.04 as per link.After completing all the steps, i am getting this error
$ sudo service tor status
● tor.service - Anonymizing overlay network for TCP (multi-instance-master)
Loaded: loaded (/lib/systemd/system/tor.service; enabled; vendor preset: enabled)
Active: active (exited) since Fri 2018-07-06 11:47:19 IST; 13min ago
Main PID: 10894 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4554)
CGroup: /system.slice/tor.service
Jul 06 11:47:19 aks-Vostro-1550 systemd[1]: Starting Anonymizing overlay network for TCP (multi-instance-master)...
Jul 06 11:47:19 aks-Vostro-1550 systemd[1]: Started Anonymizing overlay network for TCP (multi-instance-master).
My /lib/systemd/system/tor.service file is:
# This service is actually a systemd target,
# but we are using a service since targets cannot be reloaded.
[Unit]
Description=Anonymizing overlay network for TCP (multi-instance-master)
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecReload=/bin/true
[Install]
WantedBy=multi-user.target
I will be thankful for your help and support.
I have solved my problem in Ubuntu 18.04 using the suggestion given by link
I am using a Raspberry Pi. To reduce I/O on my SD-Card I symlink all important log files to an external USB-mounted Harddrive.
Example:
ln -s /media/usb-device/logs/auth.log /var/log/auth.log
The logging works fine. But fail2ban seems not to like that. When I enable my ssh-monitoring in my /etc/fail2ban/jail.local file,
# [sshd]
enabled = true
bantime = 3600
fail2ban crash during executing this command systemctl restart fail2ban.service
I have tried to hardcode the path:
# logpath = %(sshd_log)s
logpath = /media/usb-devive/logs/auth.log
But fail2ban throws the same error:
fail2ban.service - Fail2Ban Service
Loaded: loaded (/lib/systemd/system/fail2ban.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2018-04-28 20:42:33 CEST; 45s ago
Docs: man:fail2ban(1)
Process: 3014 ExecStop=/usr/bin/fail2ban-client stop (code=exited, status=0/SUCCESS)
Process: 3045 ExecStart=/usr/bin/fail2ban-client -x start (code=exited, status=255)
Main PID: 658 (code=killed, signal=TERM)
Apr 28 20:42:33 raspberrypi systemd[1]: fail2ban.service: Service hold-off time over, scheduling restart.
Apr 28 20:42:33 raspberrypi systemd[1]: Stopped Fail2Ban Service.
Apr 28 20:42:33 raspberrypi systemd[1]: fail2ban.service: Start request repeated too quickly.
Apr 28 20:42:33 raspberrypi systemd[1]: Failed to start Fail2Ban Service.
Apr 28 20:42:33 raspberrypi systemd[1]: fail2ban.service: Unit entered failed state.
Apr 28 20:42:33 raspberrypi systemd[1]: fail2ban.service: Failed with result 'exit-code'.
Any ideas?
"devive" in the logpath is spelt incorrectly
I'm a newbie in Docker/Docker Swarm and I want to create a custom mosquitto service on Swarm. I created a custom mosquitto image
lcsf/mosquitto3 from ubuntu:latest, then I added some tools (ping, ipconfig). I can run a single container with docker run and /bin/bash, but I can't create a Swarm service with that image. The service isn't created successfully. There are some outputs below.
Dockerfile:
FROM ubuntu:latest
RUN apt-get -y update
RUN apt-get install -y mosquitto mosquitto-clients
EXPOSE 80 443 1883 8083 8883
Docker service create output:
overall progress: 0 out of 1 tasks
1/1: preparing [========> ]
verify: Detected task failure
This output is shown in a loop, then I stop it using ctrl+c, the service is created, but doesn't run, with 0/1 replicas.
Docker service ps mqtt (my custom name) output, there are 3 nodes.
ID NAME IMAGE NODE
DESIRED STATE CURRENT STATE ERROR PORTS
llqr0gysz4bj mqtt.1 lcsf/mosquitto3:latest Docker02 Ready Ready 2 seconds ago
kcwfqovyn2mp \_ mqtt.1 lcsf/mosquitto3:latest Docker03 Shutdown Complete 2 seconds ago
ruisy599nbt4 \_ mqtt.1 lcsf/mosquitto3:latest Docker03 Shutdown Complete 7 seconds ago
xg1lib5x8vt9 \_ mqtt.1 lcsf/mosquitto3:latest Docker02 Shutdown Complete 13 seconds ago
fgm9wu25t0lj \_ mqtt.1 lcsf/mosquitto3:latest Docker03 Shutdown Complete 18 seconds ago
That's it, I hope someone can help me. Thanks in advance and I'm sorry 'bout my English and Stack Overflow skills.
UPDATE #1
Output from journalctl -f -n10 command after tying to create the service:
Sep 25 09:01:03 Docker01 dockerd[1230]: time="2017-09-25T09:01:03.692391553-04:00" level=info msg="Node join event for Docker02-a9b6d39043d3/192.168.222.51"
Sep 25 09:01:15 Docker01 systemd-udevd[31966]: Could not generate persistent MAC address for veth8e5ebcb: No such file or directory
Sep 25 09:01:15 Docker01 systemd-udevd[31967]: Could not generate persistent MAC address for vethaf2978b: No such file or directory
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered blocking state
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered disabled state
Sep 25 09:01:15 Docker01 kernel: device vethaf2978b entered promiscuous mode
Sep 25 09:01:15 Docker01 kernel: IPv6: ADDRCONF(NETDEV_UP): vethaf2978b: link is not ready
Sep 25 09:01:15 Docker01 kernel: eth0: renamed from veth8e5ebcb
Sep 25 09:01:15 Docker01 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethaf2978b: link becomes ready
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered blocking state
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered forwarding state
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered disabled state
Sep 25 09:01:15 Docker01 kernel: veth8e5ebcb: renamed from eth0
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered disabled state
Sep 25 09:01:15 Docker01 kernel: device vethaf2978b left promiscuous mode
Sep 25 09:01:15 Docker01 kernel: docker0: port 1(vethaf2978b) entered disabled state
Sep 25 09:01:33 Docker01 dockerd[1230]: time="2017-09-25T09:01:33.693508463-04:00" level=info msg="Node join event for Docker03-f71a448c54c7/192.168.222.52"
Sep 25 09:01:46 Docker01 dockerd[1230]: time="2017-09-25T09:01:46.541311475-04:00" level=info msg="Node join event for Docker02-a9b6d39043d3/192.168.222.51"
Sep 25 09:01:57 Docker01 dockerd[1230]: sync duration of 3.001217113s, expected less than 1s
Sep 25 09:02:03 Docker01 dockerd[1230]: time="2017-09-25T09:02:03.694876667-04:00" level=info msg="Node join event for Docker03-f71a448c54c7/192.168.222.52"
Sep 25 09:02:33 Docker01 dockerd[1230]: time="2017-09-25T09:02:33.695993259-04:00" level=info msg="Node join event for Docker03-f71a448c54c7/192.168.222.52"
UPDATE #2
This is the output from docker service ps --no-trunc mqtt command
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
bour693j8jbbrt799fz0nkpwr mqtt.1 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker03 Ready Ready 4 seconds ago
wro6254cs94gkijs8s4v9cvim \_ mqtt.1 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker03 Shutdown Complete 4 seconds ago
7vgx2mehaxki2p680fesn5jww \_ mqtt.1 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker03 Shutdown Complete 10 seconds ago
52hv6da6mj72s64po3hze4ham \_ mqtt.1 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker03 Shutdown Complete 15 seconds ago
e3s383vtg0idw8ryxwh2y3gmu \_ mqtt.1 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker03 Shutdown Complete 21 seconds ago
90i30f3riwka8xs187xi7uxt2 mqtt.2 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker02 Ready Ready less than a second ago
p2lzd04tinjdjkwkr26umlh9a \_ mqtt.2 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker02 Shutdown Complete less than a second ago
q8awoj8uu7gad6hvonhl4t9f1 \_ mqtt.2 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker02 Shutdown Complete 6 seconds ago
1fuqt0et7vw1vntd8p62jiiut \_ mqtt.2 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker02 Shutdown Complete 11 seconds ago
k3vlusok792zw0v3yddxqlmg3 \_ mqtt.2 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker02 Shutdown Complete 17 seconds ago
i4tywshqv4pxsyz5tz0z0evkz mqtt.3 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker01 Ready Ready less than a second ago
44ee4iqqpkeome4lokx9ykmbo \_ mqtt.3 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker01 Shutdown Complete less than a second ago
kdx273e9fkpqkafztif1dz35q \_ mqtt.3 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker01 Shutdown Complete 6 seconds ago
l2oewfnwbkia94r6rifbcfi4h \_ mqtt.3 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker01 Shutdown Complete 11 seconds ago
dyekgkd0swsualssw4dtvk681 \_ mqtt.3 lcsf/mosquitto3:latest#sha256:beca44e5f916d08730dd19d9d10dd2dcbd3502866f69316806a63bc094a179a9 Docker01 Shutdown Complete 17 seconds ago
Your issue is your dockerfile. You are running bash command on a swarm service. You need to run a command which doesn't exist
FROM ubuntu:latest
RUN apt-get -y update
RUN apt-get install -y mosquitto mosquitto-clients
EXPOSE 80 443 1883 8083 8883
CMD ["tail", "-f", "/dev/null"]
This is a infinite tail command, which will make sure your container doesn't exit. Run a command in image that is not looking for user input when deploying to swarm.
I have kubernetes running on 4 centos 7 boxes, master and minions. I also have flannel and skydns installed. flannel overlay ip is 172.17.0.0/16 and my service cluster ip is 10.254.0.0/16. I'm running spinnaker pods on the k8 cluster. what I see is that the spinnaker services are unable to find each other. Each pod gets an ip from the 172.17 slice and I can ping the pods from any of the nodes using that ip. However the services themselves uses the cluser ip and are unable to talk to each other. Since Kube-proxy is the one that should be forwarding this traffic, I looked at the iptable rules and I see this:
[root#MultiNode4 ~$]iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-ISOLATION all -- anywhere anywhere
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere /* kubernetes service portals */
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain KUBE-SERVICES (1 references)
target prot opt source destination
REJECT tcp -- anywhere 10.254.206.105 /* spinnaker/spkr-clouddriver: has no endpoints */ tcp dpt:afs3-prserver reject-with icmp-port-unreachable
REJECT tcp -- anywhere 10.254.162.75 /* spinnaker/spkr-orca: has no endpoints */ tcp dpt:us-srv reject-with icmp-port-unreachable
REJECT tcp -- anywhere 10.254.62.109 /* spinnaker/spkr-rush: has no endpoints */ tcp dpt:8085 reject-with icmp-port-unreachable
REJECT tcp -- anywhere 10.254.68.125 /* spinnaker/spkr-echo: has no endpoints */ tcp dpt:8089 reject-with icmp-port-unreachable
REJECT tcp -- anywhere 10.254.123.127 /* spinnaker/spkr-front50: has no endpoints */ tcp dpt:webcache reject-with icmp-port-unreachable
REJECT tcp -- anywhere 10.254.36.197 /* spinnaker/spkr-gate: has no endpoints */ tcp dpt:8084 reject-with icmp-port-unreachable
Seems like kube-proxy is unable to forward. I have no errors on kube-proxy startup:
[root#MultiNode4 ~$]systemctl status kube-proxy -l
kube-proxy.service - Kubernetes Kube-Proxy Server
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-07 02:54:54 EDT; 1h 10min ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 7866 (kube-proxy)
Memory: 3.6M
CGroup: /system.slice/kube-proxy.service
└─7866 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://centos-master:8080
Jul 07 02:54:54 clm-aus-015349.bmc.com systemd[1]: Started Kubernetes Kube-Proxy Server.
Jul 07 02:54:54 clm-aus-015349.bmc.com systemd[1]: Starting Kubernetes Kube-Proxy Server...
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: E0707 02:54:54.754845 7866 server.go:340] Can't get Node "multiNode4", assuming iptables proxy: nodes "MultiNode4" not found
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.756460 7866 server.go:200] Using iptables Proxier.
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.756527 7866 proxier.go:208] missing br-netfilter module or unset br-nf-call-iptables; proxy may not work as intended
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.756551 7866 server.go:213] Tearing down userspace rules.
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.770100 7866 conntrack.go:36] Setting nf_conntrack_max to 262144
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.770145 7866 conntrack.go:41] Setting conntrack hashsize to 65536
Jul 07 02:54:54 clm-aus-015349.bmc.com kube-proxy[7866]: I0707 02:54:54.771445 7866 conntrack.go:46] Setting nf_conntrack_tcp_timeout_established to 86400
What am I missing?
I have the same problem, and it turns out that the selector is wrong.
After fixing that, everything works just fine.
The REJECT is inserted when a particular service has 0 endpoints. The selector in your Service.spec must be wrong or you don't have any pods running.