How to reach a linked service in docker-compose? - bash

According to https://docs.docker.com/compose/compose-file/#links, if I specify the name of another service under links in docker-compose, I should be able to reach that service at a hostname identical to the service name.
To test this, I tried the following docker-compose.yml:
version: '3'
services:
tor:
build: ./tor
use_tor:
build: ./use_tor
links:
- tor
where the tor and use_tor directories contain Dockerfiles:
.
├── docker-compose.yml
├── tor
│   └── Dockerfile
└── use_tor
└── Dockerfile
which are, for tor:
FROM alpine:latest
EXPOSE 9050
RUN apk --update add tor
CMD ["tor"]
and for use_tor:
FROM alpine:latest
CMD ["nc", "-z", "tor", "9050"]
However, if I do docker-compose build followed by docker-compose up, I see from the logs that the use_tor service exits with status code 1:
Starting scrapercompose_tor_1
Recreating scrapercompose_use_tor_1
Attaching to scrapercompose_tor_1, scrapercompose_use_tor_1
tor_1 | May 02 15:36:34.123 [notice] Tor v0.2.8.12 running on Linux with Libevent 2.0.22-stable, OpenSSL LibreSSL 2.4.4 and Zlib 1.2.8.
tor_1 | May 02 15:36:34.123 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
tor_1 | May 02 15:36:34.123 [notice] Configuration file "/etc/tor/torrc" not present, using reasonable defaults.
tor_1 | May 02 15:36:34.129 [notice] Opening Socks listener on 127.0.0.1:9050
tor_1 | May 02 15:36:34.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
tor_1 | May 02 15:36:34.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
tor_1 | May 02 15:36:34.000 [warn] You are running Tor as root. You don't need to, and you probably shouldn't.
tor_1 | May 02 15:36:34.000 [notice] We were built to run on a 64-bit CPU, with OpenSSL 1.0.1 or later, but with a version of OpenSSL that apparently lacks accelerated support for the NIST P-224 and P-256 groups. Building openssl with such support (using the enable-ec_nistp_64_gcc_128 option when configuring it) would make ECDH much faster.
tor_1 | May 02 15:36:34.000 [notice] Bootstrapped 0%: Starting
scrapercompose_use_tor_1 exited with code 1
tor_1 | May 02 15:36:35.000 [notice] Bootstrapped 80%: Connecting to the Tor network
tor_1 | May 02 15:36:36.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
tor_1 | May 02 15:36:36.000 [notice] Bootstrapped 90%: Establishing a Tor circuit
tor_1 | May 02 15:36:36.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
tor_1 | May 02 15:36:36.000 [notice] Bootstrapped 100%: Done
Apparently the command nc -z tor 9050 doesn't return the expected status code 0 on the use_tor container. However, it would seem to me that this should work. For example, if I modify the tor service to map port 9050 on the container to the host as follows,
services:
tor:
build: ./tor
ports:
- "9050:9050"
Then in my ordinary terminal, I do see that nc -z localhost 9050 yields an exit code of 0:
kurt#kurt-ThinkPad:~$ nc -z localhost 9050
kurt#kurt-ThinkPad:~$ echo $?
0
In short, I would expect the hostname tor to behave like localhost on my the host after the port mapping, but this appears not to be the case. Why is this not working?

This question made me gawk at it for once. Although I cloned this example but was not able to get the solution. According to docker docs
The EXPOSE instruction informs Docker that the container listens on
the specified network ports at runtime. EXPOSE does not make the ports
of the container accessible to the host. To do that, you must use
either the -p flag to publish a range of ports or the -P flag to
publish all of the exposed ports. You can expose one port number and
publish it externally under another number.
So I think that may be because the tor service is running on 127.0.0.1 instead of 0.0.0.0 (for difference between them you can look here)
tor_1 | May 02 15:36:34.129 [notice] Opening Socks listener on
127.0.0.1:9050
It is accessible through terminal is because of the ports argument in docker-compose.yml which does the same as -p argument.
All in all if the tor service listens on 0.0.0.0 then it should work as expected.

Related

dnsmasq can't bind listen-address

I tried to setup hostapd and dnsmasq to braodcast a WiFi from a Raspberry Pi 3. I only want to broadcast the WiFi so devices can connect to a http-server running on the Raspberry, no ethernet bridge is required.
I installed hostapd and dnsmasq and configured them as followed:
dhcpcd.conf
# A sample configuration for dhcpcd.
# See dhcpcd.conf(5) for details.
# Allow users of this group to interact with dhcpcd via the control socket.
#controlgroup wheel
# Inform the DHCP server of our hostname for DDNS.
hostname
# Use the hardware address of the interface for the Client ID.
clientid
# or
# Use the same DUID + IAID as set in DHCPv6 for DHCPv4 ClientID as per RFC4361.
# Some non-RFC compliant DHCP servers do not reply with this set.
# In this case, comment out duid and enable clientid above.
#duid
# Persist interface configuration when dhcpcd exits.
persistent
# Rapid commit support.
# Safe to enable by default because it requires the equivalent option set
# on the server to actually work.
option rapid_commit
# A list of options to request from the DHCP server.
option domain_name_servers, domain_name, domain_search, host_name
option classless_static_routes
# Respect the network MTU. This is applied to DHCP routes.
option interface_mtu
# Most distributions have NTP support.
#option ntp_servers
# A ServerID is required by RFC2131.
require dhcp_server_identifier
# Generate SLAAC address using the Hardware Address of the interface
#slaac hwaddr
# OR generate Stable Private IPv6 Addresses based from the DUID
slaac private
#denyinterfaces wlan0
# Example static IP configuration:
#interface eth0
#static ip_address=192.168.0.5/24
#static ip6_address=fd51:42f8:caae:d92e::ff/64
#static routers=192.168.0.5
#static domain_name_servers=192.168.0.5
interface wlan0
allow-hotplug wlan0
#iface wlan0 inet static
static ip_address=192.168.0.5/24
nohook wpa_supplicant
#netmask 255.255.255.0
#network 192.168.0.0
#broadcast 192.168.0.255
# It is possible to fall back to a static IP if DHCP fails:
# define static profile
#profile static_eth0
#static ip_address=192.168.1.23/24
#static routers=192.168.1.1
#static domain_name_servers=192.168.1.1
# fallback to static profile on eth0
#interface eth0
#fallback static_eth0
As you can see I already tried different options using denyinterfaces and other things I found in different tutorials, but none did work.
hostapd.conf
interface=wlan0
driver=nl80211
ssid=****
hw_mode=g
channel=6
ieee80211n=1
wmm_enabled=1
ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40]
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_key_mgmt=WPA-PSK
wpa_passphrase=****
rsn_pairwise=CCMP
dnsmasq.conf
interface=wlan0
listen-address=192.168.0.5
bind-interfaces
server=8.8.8.8
domain-needed
bogus-priv
dhcp-range=192.168.0.25,192.168.0.150,255.255.255.0,240h
Now I got two problems:
hostapd does not run on startup although I did define the daemon_conf and it is working when I run hostapd /path/to/config
My main problem, dnsmasq is not running. When I try to start the service, it crashes with an error cannot bind listen-address.
service dnsmasq status
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset:
Active: failed (Result: exit-code) since Tue 2022-03-29 12:58:34 CEST; 17min
Process: 483 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCC
Process: 491 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=2
Mär 29 12:58:33 raspberrypitop systemd[1]: Starting dnsmasq - A lightweight DHCP
Mär 29 12:58:33 raspberrypitop dnsmasq[483]: dnsmasq: Syntaxprüfung OK.
Mär 29 12:58:34 raspberrypitop dnsmasq[491]: dnsmasq: Konnte Empfangs-Socket für
Mär 29 12:58:34 raspberrypitop dnsmasq[491]: Konnte Empfangs-Socket für 192.168.
Mär 29 12:58:34 raspberrypitop dnsmasq[491]: Start fehlgeschlagen
Mär 29 12:58:34 raspberrypitop systemd[1]: dnsmasq.service: Control process exit
Mär 29 12:58:34 raspberrypitop systemd[1]: dnsmasq.service: Failed with result '
Mär 29 12:58:34 raspberrypitop systemd[1]: Failed to start dnsmasq - A lightweig
lines 1-14/14 (END)
I guess I messed up the configuration somehow, but since this is something new for me and there are many different tutorials for several different kinds of OSs and OS versions, it's very hard to understand, what is going wrong.
Ok, figured it out myself.
In my case hostapd not starting automatically was actually causing the second issue, since it prevented the wlan0 interface from coming up.
I had to sudo systemctl unmask hostapd and reboot. dnsmasq would still not start since it tried to start before hostapd was finished setting everything up, even if told to wait for hostapd.service. So i edited the systemd/dnsmasq.service config and added
[service]
restart=always
retry=2
So it tries to restart every 2sec until hostapd has done it's job and so it all is working.

Why can't write certificate.crt with acme?

root#vultr:~# systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-07-28 02:16:44 UTC; 23min ago
Docs: man:nginx(8)
Main PID: 12999 (nginx)
Tasks: 2 (limit: 1148)
Memory: 8.2M
CGroup: /system.slice/nginx.service
├─12999 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
└─13000 nginx: worker process
Jul 28 02:16:44 vultr.guest systemd[1]: Starting A high performance web server and a reverse proxy server...
Jul 28 02:16:44 vultr.guest systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argument
Jul 28 02:16:44 vultr.guest systemd[1]: Started A high performance web server and a reverse proxy server.
The nginx is in good status.
I want to create and write certificate.crt with acme:
sudo su -l -s /bin/bash acme
curl https://get.acme.sh | sh
export CF_Key="xxxx"
export CF_Email="yyyy#yahoo.com"
CF_Key is my global api key in cloudflare,CF_Email is the register email to login cloudflare.
acme#vultr:~$ acme.sh --issue --dns dns_cf -d domain.com --debug 2
The output content is so long that i can't post here,so i upload into the termbin.com ,we share the link below:
https://termbin.com/taxl
Please open the webpage,you can get the whole output info,and check which result in error,there are two main issues:
1.My nginx server is in good status,acme.sh can't detect it.
2.How can set the config file?
[Wed Jul 28 03:04:38 UTC 2021] config file is empty, can not read CA_EAB_KEY_ID
[Wed Jul 28 03:04:38 UTC 2021] config file is empty, can not read CA_EAB_HMAC_KEY
[Wed Jul 28 03:04:38 UTC 2021] config file is empty, can not read CA_EMAIL
To write key into specified directory:
acme.sh --install-cert -d domain.com
--key-file /usr/local/etc/certfiles/private.key
--fullchain-file /usr/local/etc/certfiles/certificate.crt
It encounter problem:
[Tue Jul 27 01:12:15 UTC 2021] Installing key to:/usr/local/etc/certfiles/private.key
cat: /home/acme/.acme.sh/domain.com/domain.com.key: No such file or directory
To check files in /usr/local/etc/certfiles/
ls /usr/local/etc/certfiles/
private.key
No certificate.crt in /usr/local/etc/certfiles/.
How to fix then?
From acme.sh v3.0.0, acme.sh is using Zerossl as default ca, you must
register the account first(one-time) before you can issue new certs.
Here is how ZeroSSL compares with LetsEncrypt.
With ZeroSSL as CA
You must register at ZeroSSL before issuing a certificate. To register run the below command (assuming yyyy#yahoo.com is email with which you want to register)
acme.sh --register-account -m yyyy#yahoo.com
Now you can issue a new certificate (assuming you have set CF_Key & CF_Email or CF_Token & CF_Account_ID)
acme.sh --issue --dns dns_cf -d domain.com
Without ZeroSSL as CA
If you don't want to use ZeroSSL and say want to use LetsEncrypt instead, then you can provide the server option to issue a certificate
acme.sh --issue --dns dns_cf -d domain.com --server letsencrypt
Here are more options for the CA server.

How to exit a postgres docker container from a bash script

I have a docker-compose.yml configured to create and add data to a database. My compose file then loads in a bash script which will run a few short tests on the data and exit 1 upon a test failing. This is working as intended, however when I want my container to exit upon all tests passing but the docker-entrypoint.sh script seems to ignore my exit 0 command. I've attempted to write a different entrypoint script which will call on the docker-entrypoint.sh in hopes that I can then exit the container, but I'm not having any luck. Is there any easy way to implement this?
Output example:
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-04-06 19:29:58.511 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-04-06 19:29:58.511 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-04-06 19:29:58.511 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-04-06 19:29:58.517 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-04-06 19:29:58.534 UTC [102] LOG: database system was shut down at 2020-04-06 19:29:58 UTC
postgres_1 | 2020-04-06 19:29:58.540 UTC [1] LOG: database system is ready to accept connections
it's normal; entrypoint will ignore any command , to solve this you can use CDM parameter after entrypoint declaration in Dockerfile.
For example, the following snippet in Dockerfile
ENTRYPOINT ["/bin/echo", "Hello"]
CMD ["world"]
when container runs as
docker run -it <image>
will produce output:
Hello world
but when container runs as
docker run -it <image> John
will result in:
Hello John

How to check if Cloud Pub/Sub emulator is up and running?

I have GC functions which I develop and test locally by using Cloud Pub/Sub emulator.
I want to be able to check from within Go code if Cloud Pub/Sub emulator is up and running. If not, I would like to inform a developer that he/she should start emulator before he/she execute code locally.
When the emulator starts I noticed a line
INFO: Server started, listening on 8085
Maybe I can check if port is available or similar.
I guess you have used this command:
gcloud beta emulators pubsub start
And you got the following output:
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub]
[pubsub] INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub]
[pubsub] INFO: Applied Java 7 long hostname workaround.
[pubsub]
[pubsub] INFO: Server started, listening on 8085
If you take a look at the second INFO message you'll notice that the process name will be JAVA. Now you can run this command:
sudo lsof -i -P -n
Getting all the listening ports and applications, the output should be something like this:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
XXXX
XXXX
java XXX XXX XX IPv4 XXX 0t0 TCP 127.0.0.1:8085 (LISTEN)
Alternatively you can modify the previous command to show only what is happening on the desired port:
sudo lsof -i -P -n | grep 8085

How to connect to SSHD inside a Docker container from Windows?

I have a Ruby on Rails environment, and I'm converting it to run in Docker. This is largely because the development machine is a Windows laptop and the server is not. I have the Docker container mainly up and running, and now I want to connect the RubyMine debugger. To accomplish this the recommendation is to setup an SSH server in the container.
https://intellij-support.jetbrains.com/hc/en-us/community/posts/207649545-Use-RubyMine-and-Docker-for-development-run-and-debug-before-deployment-for-testing-
I successfully added SSHD to the container using the dockerfile lines from https://docs.docker.com/engine/examples/running_ssh_service/#build-an-egsshd-image minus the EXPOSE 22 (because it wasn't working with the port mapping in the docker-compose.yml). But the port is not accessible on the local machine
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6652389d248c civilservice_web "bundle exec rails..." 16 minutes ago Up 16 minutes 0.0.0.0:3000->3000/tcp, 0.0.0.0:3022->22/tcp civilservice_web_1
When I try to point PUTTY at localhost and 3022, it says that the server unexpectedly closed the connection.
What am I missing here?
This is my dockerfile
FROM ruby:2.2
RUN apt-get update && apt-get install -y \
build-essential \
libpq-dev \
nodejs \
openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:screencast' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# SSH login fix. Otherwise user is kicked off after login
RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
CMD ["/usr/sbin/sshd", "-D"]
RUN mkdir /MyApp
WORKDIR /MyApp
ADD Gemfile /MyApp/Gemfile
ADD Gemfile.lock /MyApp/Gemfile.lock
RUN bundle install
ADD . /MyApp
and this is my docker-compose.yml
version: '2'
services:
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/CivilService
ports:
- "3000:3000"
- "3022:22"
DOCKER_HOST doesn't appear to be an environment variable
docker version outputs the following
Client:
Version: 17.03.0-ce
API version: 1.26
Go version: go1.7.5
Git commit: 60ccb22
Built: Thu Feb 23 10:40:59 2017
OS/Arch: windows/amd64
Server:
Version: 17.03.0-ce
API version: 1.26 (minimum version 1.12)
Go version: go1.7.5
Git commit: 3a232c8
Built: Tue Feb 28 07:52:04 2017
OS/Arch: linux/amd64
Experimental: true
docker run -it --rm --net container:civilservice_web_1 busybox netstat -lnt outputs
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.11:35455 0.0.0.0:* LISTEN
SSHD is now running along side the Rails app, but the recipe that I was working from for setting up the service is not correct for the flavor of Linux that came with my base image https://docs.docker.com/engine/examples/running_ssh_service/#build-an-egsshd-image
The image I'm using is based on Debian 8. Could someone point me at where the example breaks down?
Your sshd process isn't running. That's visible in the netstat output:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.11:35455 0.0.0.0:* LISTEN
But as user2105103 points out, I should have realized that if I compared your docker-compose.yml with the Dockerfile. You define the sshd command in the image with a Dockerfile line:
CMD ["/usr/sbin/sshd", "-D"]
But then you override your image setting when running the container with the docker-compose command:
command: bundle exec rails s -p 3000 -b '0.0.0.0'
So, the only thing run, as you can see in the netstat, is the rails app listening on 3000. If you need multiple commands to run, then you can docker exec to kick off the second command (not recommended for a second service like this), use a command that launches sshd in the background and rails in the foreground (fairly ugly), or you can consider something like supervisord.
Personally, I'd skip sshd and just use docker exec -it civilservice_web_1 /bin/bash to get a prompt inside the container when you need it.

Resources