I am attempting to start gliderlabs/registrator and have it connect to consul on the COREOS_PRIVATE_IPV4 ip address.
[Unit]
Description=registrator
After=consul-server#%i.service
Requires=consul-server#%i.service
[Service]
EnvironmentFile=/etc/environment
ExecStartPre=-/usr/bin/docker kill registrator
ExecStartPre=-/usr/bin/docker rm registrator
ExecStartPre=/usr/bin/docker pull gliderlabs/registrator
ExecStart=/usr/bin/docker run --volume=/var/run/docker.sock:/var/run/docker.sock --net=host --hostname ${HOSTNAME} --name=registrator gliderlabs/registrator:latest consul://${COREOS_PRIVATE_IPV4}:8500
ExecStop=/usr/bin/docker stop registrator
[X-Fleet]
#Global=true
I am running into an error on starting the service that complains about the EnvironmentFile directive.
Dec 13 16:23:41 core-01 systemd[1]: [/run/fleet/units/registrator.service:5] Unknown lvalue 'EnvironmentFile' in section 'Unit'
I am currently running coreos 835.9.0. Does anyone have any thoughts on why this might be failing?
The unit provided does not match up with what the error is returning. The error is essentially saying that the EnvironmentFile=option is in the [Unit] section of your systemd-unit, and that the option isn't valid within that section.
Either what you actually have in the unit is different from what you put here, or it's possible fleet messed it up when it parsed and rendered out the unit.
If you look at the file in /run/fleet/units/registrator.service you should be able to verify where the EnvironmentFile option is. Make sure it's in the [Service] section and not the [Unit] section.
You can also run fleetctl cat registrator.service and fleet will output the unit file definition. It's possible you submitted the unit, made changes and then didn't destroy the unit before re-submitting it.
Related
I'm using Ubuntu 18.04.4 LTS which uses systemd, but the squid package packaged with this version of Ubuntu is configured to start via init.d. It starts and runs via systemctl start squid.service if I start it manually after the system has booted.
However, I'm using a ZFS mount point ("/media") to store the cache data, and during the boot process squid is starting before this mount point is active. Consequently I'm getting the error "Failed to verify one of the swap directories". Full output of systemctl status squid is here
I'd like to tell systemd to wait until after media.mount has completed in the most minimally invasive way possible (e.g. without modifying the /etc/init.d/squid file that is maintained by the package). To that end I created the /etc/systemd/system/squid.service.d/override.conf file like so:
% cat /etc/systemd/system/squid.service.d/override.conf
[Unit]
Wants=network.target network-online.target nss-lookup.target media.mount
After=network.target network-online.target nss-lookup.target media.mount
[Install]
WantedBy=multi-user.target
But squid is still starting too early.
Is what I want to do possible? Or do I have to bite the bullet and define a native /etc/systemd/system/squid.service file and remove the /etc/init.d/squid init script?
My golang web app cannot start when I use systemd, but it works fine when manually start. What are problems with my systemd configuration?
goweb.service
$ cat goweb.service
[Unit]
Description=Backend service
After=network.target
[Service]
User=myapp
Group=myapp
Restart=on-failure
ExecStart=/u01/backend
[Install]
WantedBy=multi-user.target
backend is a binary file compiled with command: env GOOS=linux GOARCH=amd64 go build -v bitbucket.org/myapp/backend
systemd service status
$ sudo service goweb status
Redirecting to /bin/systemctl status goweb.service
● goweb.service - Backend service
Loaded: loaded (/usr/lib/systemd/system/goweb.service; disabled; vendor preset: disabled)
Active: inactive (dead)
May 18 10:55:56 instance-1 systemd[1]: Started Backend service.
May 18 10:55:56 instance-1 systemd[1]: Starting Backend service...
P/S: It looks like my web app started, but then stoped immediately.
I try config Type=forking, then service status show as below. Could someone explains why the log Started Backend service. and Starting Backend service... order is reversed.
$ sudo service goweb status
Redirecting to /bin/systemctl status goweb.service
● goweb.service - Backend service
Loaded: loaded (/usr/lib/systemd/system/goweb.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Wed 2016-05-18 11:06:02 UTC; 2s ago
Process: 25847 ExecStart=/u01/backend (code=exited, status=0/SUCCESS)
May 18 11:06:02 instance-1 systemd[1]: Starting Backend service...
May 18 11:06:02 instance-1 systemd[1]: Started Backend service.
Result when I run web app manually (from terminal):
$ /u01/backend
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST /upload --> main.uploadFileHandler (3 handlers)
[GIN-debug] Environment variable PORT="9005"
[GIN-debug] Listening and serving HTTP on :9005
Update:
I try supervisord and it works fine (same binary file)
With systemd, I changed service config to Restart=always, RestartSec=15. And systemd keeps restarting my web app.
Does anyone know why supervisord work fine but systemd not? I think that systemd would work fine with a basic task like that!
I know this is old, but saw that there weren't any sufficient answers, so thought I would post. My app was also being run manually, but when using systemd it wouldn't work. I realized it had to do with the filepaths that the Go program would state, such as:
tpl = template.Must(template.ParseGlob("templates/*"))
When running the app manually, the templates folder was there relative to my main.go file, but when systemd would run it, the filepath was different (not sure why or how to solve this yet), but for now I just hardcoded an absolute filepath, such as:
tpl = template.Must(template.ParseGlob("./home/ubuntu/templates/*"))
Now systemd works.
Hope this helps. If anyone can elaborate on how to come up with a better solution, would totally help me as well!
I'll take a wild stab and guess your app listens on port 80 and/or 443, so your best bet is to use setcap on it to give it permission.
example: sudo setcap 'cap_net_bind_service=+ep' /u01/backend, this will need to be done every time you compile your app.
From #RijulSudhir comment above, insert:
WorkingDirectory=/home/ubuntu/
below:
[service]
of systemd service file.
Replace your app temporarily with a minimal version that just dumps the environment out. Are the environment for running manually and under systemd sufficiently similar?
In this specific instance, I guess it's because you don't specify
ExecStart=/path/to/app
As a result, there's nothing to start.
Is there some way to trigger an event (e.g. running a script to push some logs to S3) when an EC2 instance is stopped/terminated?
I have looked into triggering the script using a service in /usr/lib/systemd/system but I haven't had any luck with that yet. I have heard that networking capabilities on the instance can be shutdown before a service is triggered and if true, that could be why the script is not executing correctly.
So the answer is not really AWS specific, but it is working for me now (tested on EC2 instance stopping and terminating).
I've created a system.d service file:
/usr/lib/systemd/system/my_shutdown.service
[Unit]
Description=my_shutdown Service
Before=shutdown.target reboot.target halt.target
Requires=network-online.target network.target
[Service]
KillMode=none
ExecStart=/bin/true
ExecStop=/path/to/my_script.sh
RemainAfterExit=yes
Type=oneshot
[Install]
WantedBy=multi-user.target
Added this service to multi-user.target:
systemctl enable my_shutdown.service
Alternatively you can manually create the symlink:
ln -s /usr/lib/systemd/system/my_shutdown.service /etc/systemd/system/multi-user.target.wants/my_shutdown.service
Started the service and tested by stopping/terminating the instance.
systemctl start my_shutdown.service
My understanding:
Description: a description of our service.
Before: we want our service to stop before these targets are started.
Requires: our service requires that network capabilities are available. These targets must not be stopped before our service starts/stops.
KillMode: none; do not kill our process.
ExecStart: /bin/true; a command that does nothing but returns a success. Run when are service is started.
ExecStop: the script to run. Run when are service is being stopped.
RemainAfterExit: consider our service active even when all its processes exited.
Type: oneshot; it is expected that the process has to exit before systemd starts follow-up units.
WantedBy: the target we want to add our service to.
References:
https://www.freedesktop.org/software/systemd/man/systemd.service.html
https://www.freedesktop.org/software/systemd/man/systemd.kill.html#
https://www.freedesktop.org/software/systemd/man/systemd.special.html
https://www.freedesktop.org/software/systemd/man/systemd.target.html
You can trigger events, such as pushing logs to S3 on specific events, with CloudWatch... Learn more here: https://aws.amazon.com/cloudwatch/
I have installed Mesosphere DCOS in AWS using the provided template. Now I would like to restart all the nodes but adding the --insecure-registry parameter to all the slave nodes (and master as well) so that they communicate with my docker registry. So I was reading the best way to do this is on the cloud-config script for the AWS template.
So in the AWS EC2 Launch Configurations I copied the configuration of the master node, then adjusted the User Data then updated the auto scaling groups and restarted the master. (awesome answer how to do this at the end of How do I use insecure docker registries with Amazon EC2 Container Service (ECS)?)
The lines were added to the end of the units section in cloud-config as suggested by CoreOS docs:
https://coreos.com/os/docs/latest/cloud-config.html
units
....Many lines here
- name: docker.service
drop-ins: |-
- name: 50-insecure-registry.conf
content: |
[Service]
Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
But then, the master wouldn't restart. So I had to revert my change.
So many questions:
a. Why is there no docker.service block in this template cloud-config? How and when docker starts?
b. Do I need to edit the flannel_docker_opts.env file? Again there is no mention to such file in this cloud-config. But there is mention in this page:
https://coreos.com/flannel/docs/latest/flannel-config.html
Of particular interest at the end of that page:
ExecStartPost in flanneld.service converts information in /run/flannel/subnet.env into Docker daemon command line args (such as --bip and --mtu), storing them in /run/flannel_docker_opts.env
...
docker.service sources in /run/flannel_docker_opts.env which contains env variables with command line options and starts the Docker with them.
And in fact I can see the mentioned files like early-docker.service, but again no mention of flannel in the cloud-config.
But indeed I found the service files mentioned in the page above:
/usr/lib64/udev/rules.d/80-docker.rules
/usr/lib64/systemd/system/early-docker.service
/usr/lib64/systemd/system/early-docker.socket
/usr/lib64/systemd/system/docker.service
/usr/lib64/systemd/system/docker.socket
/usr/lib64/systemd/system/sockets.target.wants/docker.socket
/usr/lib64/systemd/system/early-docker.target
And indeed the /run/flannel_docker_opts.env file is mentioned in the docker.service file, but does not exist in the /run folder:
vi /usr/lib64/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=docker.socket early-docker.target network.target
Requires=docker.socket early-docker.target
[Service]
Environment=TMPDIR=/var/tmp
EnvironmentFile=-/run/flannel_docker_opts.env <<<<<<<<<< HERE!!!!!
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
ExecStart=/usr/lib/coreos/dockerd --daemon --host=fd:// $DOCKER_OPTS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
[Install]
c. Where does this docker.service file comes from! There is no mention to it in the cloud-config. Is it part of the CoreOS architecture?
d. If this docker.service is integral part of CoreOS, why are all the files in this /usr/lib64 path? CoreOS docs mentions other path locations for all the files.
Any suggestion would be appreciated. I'm going blind now. I will try and create to this non existent flannel_docker_conf.env file. But I'm not sure if what I'm doing is the correct way.
Thanks!
I am trying to setup tomcat 7 on Digital Ocean CoreOS machine but facing some problem, not sure how to solve them. I am following below tutorial provided by the Digital Ocean to setup Apache.
https://www.digitalocean.com/community/tutorials/how-to-create-and-run-a-service-on-a-coreos-cluster
I created docker container and run it using following command.
docker run -i -t ubuntu:14.04 /bin/bash
I was successfully able to install tomcat 7 by using below commands. (I followed this tutorial to setup tomcat 7 within the docker container: https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-7-on-ubuntu-14-04-via-apt-get)
sudo apt-get update
sudo apt-get install tomcat7
Then I can created service unit file named as tomcat#.service
[Unit]
Description=Tomcat 7 web server service
After=etcd.service After=docker.service
Requires=tomcat-discovery#%i.service
[Service]
TimeoutStartSec=0 KillMode=none
EnvironmentFile=/etc/environment
ExecStartPre=-/usr/bin/docker kill tomcat%i
ExecStartPre=-/usr/bin/docker rm tomcat%i
ExecStartPre=/usr/bin/docker pull attacomsian/tomcat
ExecStart=/usr/bin/docker run –name tomcat%i -p ${COREOS_PUBLIC_IPV4}:%i:8080 attacomsian/tomcat `service tomcat7 start` -D FOREGROUND
ExecStop=/usr/bin/docker stop tomcat%i
[X-Fleet]
X-Conflicts=tomcat#*.service
Then I created tomcat-discovery#.service to register service states with Etcd as below
[Unit]
Description=Announce Tomcat#%i service
BindsTo=tomcat#%i.service
[Service]
EnvironmentFile=/etc/environment
ExecStart=/bin/sh -c “while true; do etcdctl set /announce/services/tomcat%i ${COREOS_PUBLIC_IPV4}:%i –ttl 60; sleep 45; done”
ExecStop=/usr/bin/etcdctl rm /announce/services/tomcat%i
[X-Fleet]
X-ConditionMachineOf=tomcat#%i.service
I submitted and loaded files to Fleet as below
fleetctl submit tomcat#.service tomcat-discovery#.service
fleetctl load tomcat#8080.service
fleetctl load tomcat-discovery#8080.service
Everything worked fine so far. I did not see any error. But when I tried to run the service as below
fleetctl start tomcat#8080.service
But it did not started. I can see it is appearing as dead.
I am new to CoreOS and still learning. I am managing servers at Digital Ocean and I quite about it quite well. I googled about this issue but did not found any help. I personally think following line is actually causing the trouble.
ExecStart=/usr/bin/docker run –name tomcat%i -p ${COREOS_PUBLIC_IPV4}:%i:8080 attacomsian/tomcat `service tomcat7 start` -D FOREGROUND
I would really appreciate any kind help to get this up.
Many Thanks
Attacomsian
I was going so suggest you take a look at what others have done and then discovered you have posted a similar question on the Docker Hub registry.
Did you take a look at the Docker file used by the tutum/tomcat image?
https://github.com/tutumcloud/tutum-docker-tomcat/blob/master/7.0/Dockerfile
https://github.com/tutumcloud/tutum-docker-tomcat/blob/master/7.0/run.sh
It runs a script called "run.sh" that runs tomcat in the foreground.
The thing that is tricky to understand is that Docker is not a virtual machine and therefore does not have any services running. You must run the docker processes explicitly or setup a process manager like runit or supervisord.
Hope this helps.