i'm new to Docker and facing with an issue for days and i don't know how to manage that. here is the situation;
i have a sudoer user in a vps server that runs such services like apache2 and mysql and ... and also serves couple application that i don't want to hurt them by upgrade or change services like php version or change web server from apache to nginx.
i want to serve a laravel application in docker and i built this docker-compose.prod.yml file in my project root folder that lives in /home/myUser/www/laravelProject (contains laravel application):
version : '3'
networks:
laravel:
volumes :
dbdata:
driver: local
services:
nginx:
image : nginx:stable-alpine
container_name: nginx
ports :
- "8080:80"
volumes :
- ./:/var/www/html ## i think it would make sens if i do like ./:/home/myuser/www/laravelProject ???!!!
- ./nginx/default.prod.conf:/etc/nginx/conf.d/default.conf
depends_on :
- php
- mysql
- redis
- artisan
networks :
- laravel
mysql :
image : mysql:5.7.29
container_name: mysql
restart : unless-stopped
tty : true
ports :
- "4406:3306"
environment :
MYSQL_DATABASE : ${DB_DATABASE}
MYSQL_USER : ${DB_USERNAME}
MYSQL_PASSWORD : ${DB_PASSWORD}
MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
SERVICE_TAGS : dev
SERVICE_NAME : mysql
volumes :
- dbdata:/var/lib/mysql
- ./mysql/my.cnf:/etc/mysql/my.cnf
networks :
- laravel
php :
build :
context : .
dockerfile: php.prod.dockerFile
container_name: php
volumes :
- ./:/var/www/html
ports :
- "9000:9000"
networks :
- laravel
artisan :
build :
context : .
dockerfile: php.dockerfile
container_name: artisan
volumes :
- ./:/var/www/html
depends_on :
- mysql
working_dir : /var/www/html
entrypoint : ['php', '/var/www/html/artisan']
networks :
- laravel
redis :
container_name: redis
image : redis
ports :
- "6379:6379"
volumes :
- ./:/data
entrypoint : redis-server --appendonly yes
restart : always
networks :
- laravel
i'm sorry i think i missed tab's and spaces. however, and this is my php.prod.dockerfile
FROM php:7.4-fpm-alpine
WORKDIR /home/www/html // again it's not supposed to be /home/myUser/www/project ???!!!
RUN docker-php-ext-install pdo pdo_mysql
RUN chown -R www-data:www-data /var/www // i don't really have any sense why i'm doing this !!
i build the containers successfully using command (in my project root):
docker-compose -f docker-compose.prod.yml up -d --build
and the default.prod.nginx file is :
server {
listen 80;
index index.php index.html;
server_name api.myUser.com www.api.myUser.com;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /var/www/html/public;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
}
unfortunately when i try to reach to the endpoint api.myUser.com:8080 i see
This site can’t be reached
http://api.myUser.com:8080/ is unreachable.
ERR_ADDRESS_UNREACHABLE
what i'm missing ?
(sorry for my english)
EDIT: here is the output of docker-compose up --build
Creating network "kooche-mobl_laravel" with the default driver
Building php
Step 1/4 : FROM php:7.4-fpm-alpine
---> f9f075c5a926
Step 2/4 : WORKDIR /var/www/html
---> Using cache
---> 5478beb70e23
Step 3/4 : RUN docker-php-ext-install pdo pdo_mysql
---> Using cache
---> 049d5b4134d8
Step 4/4 : RUN chown -R www-data:www-data /var/www
---> Using cache
---> b697f0fb62a1
Successfully built b697f0fb62a1
Successfully tagged kooche-mobl_php:latest
Building artisan
Step 1/4 : FROM php:7.4-fpm-alpine
---> f9f075c5a926
Step 2/4 : WORKDIR /var/www/html
---> Using cache
---> 5478beb70e23
Step 3/4 : RUN docker-php-ext-install pdo pdo_mysql
---> Using cache
---> 049d5b4134d8
Step 4/4 : RUN chown -R www-data:www-data /var/www
---> Using cache
---> b697f0fb62a1
Successfully built b697f0fb62a1
Successfully tagged kooche-mobl_artisan:latest
Creating redis ... done
Creating mysql ... done
Creating php ... done
Creating artisan ... done
Creating nginx ... done
Attaching to redis, php, mysql, artisan, nginx
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Entrypoint script for MySQL Server 5.7.29-1debian10 started.
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Switching to dedicated user 'mysql'
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Entrypoint script for MySQL Server 5.7.29-1debian10 started.
mysql | 2020-09-04T06:46:11.850419Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use -- explicit_defaults_for_timestamp server option (see documentation for more details).
mysql | 2020-09-04T06:46:11.857132Z 0 [Note] mysqld (mysqld 5.7.29-log) starting as process 1 ...
mysql | 2020-09-04T06:46:11.866317Z 0 [Note] InnoDB: PUNCH HOLE support available
mysql | 2020-09-04T06:46:11.866418Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql | 2020-09-04T06:46:11.866455Z 0 [Note] InnoDB: Uses event mutexes
mysql | 2020-09-04T06:46:11.866506Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mysql | 2020-09-04T06:46:11.866533Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql | 2020-09-04T06:46:11.866578Z 0 [Note] InnoDB: Using Linux native AIO
mysql | 2020-09-04T06:46:11.868019Z 0 [Note] InnoDB: Number of pools: 1
mysql | 2020-09-04T06:46:11.868769Z 0 [Note] InnoDB: Using CPU crc32 instructions
mysql | 2020-09-04T06:46:11.876267Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
mysql | 2020-09-04T06:46:11.918223Z 0 [Note] InnoDB: Completed initialization of buffer pool
mysql | 2020-09-04T06:46:11.936304Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysql | 2020-09-04T06:46:11.955456Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
mysql | 2020-09-04T06:46:12.004799Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql | 2020-09-04T06:46:12.004958Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql | 2020-09-04T06:46:12.055925Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql | 2020-09-04T06:46:12.058036Z 0 [Note] InnoDB: 96 redo
rollback segment(s) found. 96 redo rollback segment(s) are active.
mysql | 2020-09-04T06:46:12.058061Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
mysql | 2020-09-04T06:46:12.059038Z 0 [Note] InnoDB: Waiting for purge to start
mysql | 2020-09-04T06:46:12.109390Z 0 [Note] InnoDB: 5.7.29 started; log sequence number 12758905
mysql | 2020-09-04T06:46:12.110224Z 0 [Note] Plugin 'FEDERATED' is disabled.
mysql | 2020-09-04T06:46:12.115113Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql | 2020-09-04T06:46:12.126409Z 0 [Note] InnoDB: Buffer pool(s) load completed at 200904 6:46:12
mysql | 2020-09-04T06:46:12.137920Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
mysql | 2020-09-04T06:46:12.138013Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
mysql | 2020-09-04T06:46:12.139676Z 0 [Warning] CA certificate ca.pem is self signed.
mysql | 2020-09-04T06:46:12.139893Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
mysql | 2020-09-04T06:46:12.141123Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
mysql | 2020-09-04T06:46:12.141277Z 0 [Note] IPv6 is available.
mysql | 2020-09-04T06:46:12.141345Z 0 [Note] - '::' resolves to '::';
mysql | 2020-09-04T06:46:12.141398Z 0 [Note] Server socket created on IP: '::'.
mysql | 2020-09-04T06:46:12.168549Z 0 [Note] Event Scheduler: Loaded 0 events
mysql | 2020-09-04T06:46:12.169118Z 0 [Note] mysqld: ready for connections.
mysql | Version: '5.7.29-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not
empty, will attempt to perform configuration
nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
php | [04-Sep-2020 06:46:10] NOTICE: fpm is running, pid 1
php | [04-Sep-2020 06:46:10] NOTICE: ready to handle connections
nginx | 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
redis | 1:C 04 Sep 2020 06:46:10.064 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 04 Sep 2020 06:46:10.064 # Redis version=6.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 04 Sep 2020 06:46:10.064 # Configuration loaded
redis | 1:M 04 Sep 2020 06:46:10.068 * Running mode=standalone, port=6379.
redis | 1:M 04 Sep 2020 06:46:10.068 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis | 1:M 04 Sep 2020 06:46:10.068 # Server initialized
redis | 1:M 04 Sep 2020 06:46:10.068 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis | 1:M 04 Sep 2020 06:46:10.070 * Ready to accept connections
nginx | 10-listen-on-ipv6-by-default.sh: error:
/etc/nginx/conf.d/default.conf differs from the packages version
nginx | /docker-entrypoint.sh: Launching /docker-
entrypoint.d/20-envsubst-on-templates.sh
nginx | /docker-entrypoint.sh: Configuration complete; ready
for start up
i've also this line on netstat -tulnp | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 16093/docker-proxy
EDIT 2:
and about the firewall (iptables)
cat /etc/sysconfig/iptables |grep ACCEPT
its out put
:PREROUTING ACCEPT [884:49904]
:INPUT ACCEPT [262:14960]
:OUTPUT ACCEPT [60:4106]
:POSTROUTING ACCEPT [77:5110]
:INPUT ACCEPT [247:22593]
:OUTPUT ACCEPT [199:50843]
-A INPUT -p tcp -m tcp --dport 8090 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 8090 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 8080 -j ACCEPT
-A FORWARD -o br-4a4e80432e3f -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-4a4e80432e3f ! -o br-4a4e80432e3f -j ACCEPT
-A FORWARD -i br-4a4e80432e3f -o br-4a4e80432e3f -j ACCEPT
-A FORWARD -o br-8e73d0dbe1df -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-8e73d0dbe1df ! -o br-8e73d0dbe1df -j ACCEPT
-A FORWARD -i br-8e73d0dbe1df -o br-8e73d0dbe1df -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --gid-owner 209 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --gid-owner 12 -j ACCEPT
-A OUTPUT -d 127.0.0.1/32 -p tcp -m multiport --dports 25,465,587 -m owner --uid-owner 201 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --uid-owner 0 -j ACCEPT
-A DOCKER -d 192.168.32.2/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 3306 -j ACCEPT
-A DOCKER -d 192.168.32.3/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 9000 -j ACCEPT
-A DOCKER -d 192.168.32.4/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 6379 -j ACCEPT
-A DOCKER -d 192.168.32.6/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 80 -j ACCEPT
i think it seems ok, but when i try telnet myPort 8080
says:
telnet: Unable to connect to remote host: No route to host
Related
I've wrote a script to start openvpn (called vpn_up) but then I want it to also run my firewall script (called firewall_up) after starting the vpn. Here is the script that works:
#!/bin/bash
#script called vpn_up
exp_login=mylogin
exp_pass=mypass
config_file=$1
expect -c "
spawn openvpn --config $config_file --script-security 2 --up /etc/openvpn/update-systemd-resolved --down /etc/openvpn/update-systemd-resolved --dhcp-option 'DOMAIN-ROUTE .' --down-pre
expect \"Auth Username:\"
send \"$exp_login\r\";
expect \"for no echo)\"
send \"$exp_pass\r\";
interact
"
After opening the vpn, I want it to run my script firewall_up
#!/bin/bash
#script called firewall_up
# get your IP address
curl -s ifconfig.me > /tmp/ip_address
#Clear any iptables rules you might have at the moment
iptables -F
#Start building the firewall by allowing tun and your localhost
iptables -A INPUT -i tun+ -j ACCEPT
iptables -A OUTPUT -o tun+ -j ACCEPT
#Add the IP address of the VPN to the firewall
IP_LIST=$(tr '\n' ' ' < /tmp/ip_address)
for IP in $IP_LIST; do
iptables -A INPUT -s $IP -j ACCEPT
iptables -A OUTPUT -d $IP -j ACCEPT
done
iptables -A INPUT -p udp --sport 1195 -j ACCEPT
iptables -A INPUT -p udp --dport 1195 -j ACCEPT
iptables -A OUTPUT -p udp --sport 1195 -j ACCEPT
iptables -A OUTPUT -p udp --dport 1195 -j ACCEPT
#iptables -A INPUT -p udp --sport 53 -j ACCEPT
#iptables -A INPUT -p udp --dport 53 -j ACCEPT
#iptables -A OUTPUT -p udp --sport 53 -j ACCEPT
#iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
iptables -A INPUT -p udp -j DROP
iptables -A OUTPUT -p udp -j DROP
# Stop anything not from VPN or localhost
iptables -A INPUT -j DROP
iptables -A OUTPUT -j DROP
#Clean up your tempoary files
rm /tmp/ip_address
Currently, I have to run
sudo vpn_up some_config_file
in one terminal window and then run
sudo firewall_up
in another window and then it all works just fine.
I'd like to have just one script to do everything. I tried adding
--up /usr/sbin/firewall_up
to the spawn openvpn command to my first script i.e.
#!/bin/bash
# script called vpn_up
exp_login=mylogin
exp_pass=mypass
config_file=$1
expect -c "
spawn openvpn --config $config_file --script-security 2 --up /etc/openvpn/update-systemd-resolved --up /usr/sbin/firewall_up --down /etc/openvpn/update-systemd-resolved --dhcp-option 'DOMAIN-ROUTE .' --down-pre
expect \"Auth Username:\"
send \"$exp_login\r\";
expect \"for no echo)\"
send \"$exp_pass\r\";
interact
"
But it ends up running firewall_up before the vpn is actually up. I.e it uses my Initial IP address, not the IP address after the vpn is up and running. Is there any way to just add more code after the expect is finished and openvpn is done?
Any suggestions?
Thanks
As asked, here is the key (I think) output from openvpn:
Fri May 14 11:39:31 2021 /sbin/ip link set dev tun0 up mtu 1500
Fri May 14 11:39:31 2021 /sbin/ip addr add dev tun0 local 10.167.0.50 peer 10.167.0.49
Fri May 14 11:39:31 2021 /usr/sbin/firewall_up tun0 1500 1557 10.167.0.50 10.167.0.49 init
Fri May 14 11:39:33 2021 /sbin/ip route add 185.195.19.203/32 via 192.168.0.1
Fri May 14 11:39:33 2021 /sbin/ip route add 0.0.0.0/1 via 10.167.0.49
Fri May 14 11:39:33 2021 /sbin/ip route add 128.0.0.0/1 via 10.167.0.49
Fri May 14 11:39:33 2021 /sbin/ip route add 10.167.0.1/32 via 10.167.0.49
Fri May 14 11:39:33 2021 Initialization Sequence Completed
The problem appears to be that it runs firewall_up before completing the initialization.
I am unable to use JMX exporter to expose kafka metrics. Can you look at my steps and correct me where needed.?
I am following steps here to enable kafka with JMX exporter.
Following are step by step instruction I followed
#get kafka
wget kafka_2.11-2.0.0
# Download Prometheus JMX exporter:
sudo wget -P /opt/kafka/prometheus/ https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.3.0/jmx_prometheus_javaagent-0.3.0.jar
sudo wget -P /opt/kafka/prometheus/ https://raw.githubusercontent.com/prometheus/jmx_exporter/master/example_configs/kafka-0-8-2.yml
#Edit Prometheus JMX exporter config file; and append following lines
echo “- pattern : kafka.producer<type=producer-metrics, client-id=(.+)><>(.+):\w* name: kafka_producer_$2” >> /opt/kafka/prometheus/kafka-0-8-2.yml
echo “— pattern : kafka.consumer<type=consumer-metrics, client-id=(.+)><>(.+):\w* name: kafka_consumer_$2” >> /opt/kafka/prometheus/kafka-0-8-2.yml
echo “— pattern : kafka.consumer<type=consumer-fetch-manager-metrics, client-id=(.+)><>(.+):\w* name: kafka_consumer_$2” >> /opt/kafka/prometheus/kafka-0-8-2.yml
#start zookeeper in terminal 0
/opt/kafka/bin/zookeeper-server-start.sh config/zookeeper.properties
#start kafka broker in terminal 1
KAFKA_HEAP_OPTS=’”-Xmx1000M -Xms1000M”’
KAFKA_OPTS=”-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.3.0.jar=7071:/opt/kafka/prometheus/kafka-0–8–2.yml”
JMX_PORT=7071
/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
#start kafka consumer in terminal 2
KAFKA_OPTS=”-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.3.0.jar=7072:/opt/kafka/prometheus/kafka-0–8–2.yml”
JMX_PORT=7072
/opt/kafka/bin/kafka-console-consumer.sh — bootstrap-server 0.0.0.0:9092 — topic test — from-beginning
#start kafka producer in terminal 3
KAFKA_OPTS=”-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.3.0.jar=7073:/opt/kafka/prometheus/kafka-0–8–2.yml”
JMX_PORT=7073
/opt/kafka/bin/kafka-console-producer.sh — broker-list 0.0.0.0:9092 — topic test
After above steps zookeeper and kafka is running fine.
I can type in producer terminal a message and it is received on consumer console. However no kafka metrics is visible on Prometheus. To debug this I checked ports 7071/2/3 by
netstat -tlnp | grep 7071
netstat -tlnp | grep 7072
netstat -tlnp | grep 7073
which resulted in blank response; this means no service is using above ports. I feel like JMX exporter is not enabled correctly.
Can you help me with above issues?
From the looks of your question, you put the variables on their own lines, while the blog has them on the same line...
e.g. This is how to start the Kafka server
KAFKA_HEAP_OPTS='"-Xmx1000M -Xms1000M"' KAFKA_OPTS='-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.3.0.jar=7071:/opt/kafka/prometheus/kafka-0–8–2.yml' JMX_PORT=7081 /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
Otherwise, you need to export the variables so the sub-process will pick them up like you did in your previous question, which seemed to be working fine for exposing the metrics
export KAFKA_HEAP_OPTS='"-Xmx1000M -Xms1000M"'
export KAFKA_OPTS='-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.3.0.jar=7071:/opt/kafka/prometheus/kafka-0–8–2.yml'
export JMX_PORT=7081
/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
Note: The blog you linked to doesn't use JMX_PORT, but those ports cannot be the same as the exporter.
I would also suggest at least downloading a version newer than 0.3 - https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/
and using the configs for Kafka 2.0 - https://github.com/prometheus/jmx_exporter/blob/master/example_configs/kafka-2_0_0.yml
Sidenote: netstat -tlnp | grep 707 would show you all them at once
thank you cricket-007 for your help.
I am listing steps i followed here for simplicity
wget -q -O /tmp/kafka.tgz https://archive.apache.org/dist/kafka/2.0.0/kafka_2.11-2.0.0.tgz
tar -xf /tmp/kafka.tgz --directory /opt/kafka --strip 1
rm -f /tmp/kafka.tgz
wget -q -O /tmp/kafka.tgz https://archive.apache.org/dist/kafka/2.0.0/kafka_2.11-2.0.0.tgz
mkdir /opt/kafka
tar -xf /tmp/kafka.tgz --directory /opt/kafka --strip 1
rm -f /tmp/kafka.tgz
sudo wget -P /opt/kafka/prometheus/ https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar
wget https://raw.githubusercontent.com/prometheus/jmx_exporter/master/example_configs/kafka-2_0_0.yml
cd kafka
export KAFKA_OPTS="-javaagent:/opt/kafka/prometheus/jmx_prometheus_javaagent-0.12.0.jar=7071:/opt/kafka/prometheus/kafka-2_0_0.yml"
export KAFKA_HEAP_OPTS="-Xmx1000M -Xms1000M"
mv ../kafka-2_0_0.yml prometheus/
/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
netstat -tlnpu |grep 70
tcp6 0 0 :::7071 :::* LISTEN 209455/java
udp6 0 0 :::40705 :::*
curl -s localhost_or_IP:7071 | grep -i kafka
long list of metrics will be dumped on stdout -
I am running the following setup:
Magento 1.9.3.1
Varnish 3.0.5-2 - package installed from here https://www.magentocommerce.com/magento-connect/turpentine-varnish-cache.html
Ubuntu 14.04
Plesk 17.0.17
I have installed both varnish on the server and the plugin within magento, saved the configuration properly (from magento admin), however I am missing a configuration setting somewhere since the Varnish headers do not show up.
/etc/default/varnish has the following content:
DAEMON_OPTS="-a :443 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,8192m\
-p esi_syntax=0x2\
-p cli_buffer=16384"
I did set it up to listen on 443 because the website has a ssl certificate.
When I try to see who is listening 443 with netstat -ntlp | grep -w 443 I get the following response:
tcp 0 0 91.250.103.173:443 0.0.0.0:* LISTEN 9171/nginx: worker
tcp6 0 0 :::443 :::* LISTEN 6109/varnishd
So in addition to this I tried to modify the port where nginx listens by setting the listening port to 444 like this listen 444 ssl;. It did not work.
When I try to see the headers with curl there is no X-Varnish header.
What am I missing?
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,8192m\
-p esi_syntax=0x2\
-p cli_buffer=16384"
you should change port listen to nginx or apache to 8080
and config default.vcl change port backend 8080
I created a bash script to switch between using Apache and Varnish
But the commands of restarting the httpd service not working RECENTLY
the script was working fine before months
#!/bin/bash
echo "Switching between Apache and Varnish cache"
if grep -Fxq "apache_port=0.0.0.0:80" /var/cpanel/cpanel.config
then
sed -i '/apache_port/c\apache_port=0.0.0.0:8080' /var/cpanel/cpanel.config
else
sed -i '/apache_port/c\apache_port=0.0.0.0:80' /var/cpanel/cpanel.config
fi
/usr/local/cpanel/whostmgr/bin/whostmgr2 –updatetweaksettings &&
/scripts/rebuildhttpdconf &&
service httpd restart &&
service varnish restart &&
echo "Done"
I don't know Why the restarting oof httpd is not working exactly
What about to use iptables port redirection instead?
Basically, you have your varnish and apache running on their own unprivileged ports concurrently and redirect all traffic to port 80 to varnish or apache on kernel level.
Let run varnish on 0.0.0.0:6081 and apache on 0.0.0.0:8080 and play with these 2 command sets (under root or sudo):
To switch traffic to apache (assuming we have already it directed to varnish):
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 && \
iptables -t nat -D PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 6081
# first command adds rule to redirect all traffic on interface eth0 (adjust as needed) from port 80 to port 8080, rule is added to the end of rules list, so already active rule redirecting traffic to 6081 is still in charge with higher priority
# second line deletes rule redirecting traffic from port 80 to port 6081, to new rule can come into effect. moreover, it's executed only if previous command (-A) was finished successfully.
To switch it back to varnish:
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 6081
iptables -t nat -D PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
# inverted rules from above, adds redirection to 6081 and removes redirection to 8080 if addition was successful
Pros:
No varnish restarts, so no cold cache.
No port 80 outage overall if you first add the new redirection rule and then delete still valid older one.
Much safer, if addition of the first rule fails, skip deleting still valid rule and report problem. You are still running as before failed attempt. Just play with scripts as needed.
Cons:
No cache eviction as there is no varnish restart. But I guess this is not the reason you try to switch traffic to apache. And you can evict varnish separately if you need afterall. :)
I have a trouble. Can`t access to my ethernet-connected MFP from Windows 7 clients, but by Ubuntu (and router/server) machine it can get access to it.
MFP = Epson Stylus Color 730
network:
MFP (192.168.0.100) + win7clients (192.168.0.101-200) ---> Ubuntu server/router (192.168.0.1) ---> Internet
MFP get right IP and settings form DHCP server. On Windows machines disabled all firewalls and so on.
From Ubuntu I can do with MFP what I want, but why I can`t even ping it form Windows?
Thanks
Edit:
Content of /etc/sysctl.conf :
#
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables
# See sysctl.conf (5) for information.
#
#kernel.domainname = example.com
# Uncomment the following to stop low-level messages on console
#kernel.printk = 3 4 1 3
##############################################################3
# Functions previously found in netbase
#
# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
#net.ipv4.conf.default.rp_filter=1
#net.ipv4.conf.all.rp_filter=1
# Uncomment the next line to enable TCP/IP SYN cookies
# See http://lwn.net/Articles/277146/
# Note: This may impact IPv6 TCP sessions too
#net.ipv4.tcp_syncookies=1
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
# Uncomment the next line to enable packet forwarding for IPv6
# Enabling this option disables Stateless Address Autoconfiguration
# based on Router Advertisements for this host
#net.ipv6.conf.all.forwarding=1
###################################################################
# Additional settings - these settings can improve the network
# security of the host and prevent against some network attacks
# including spoofing attacks and man in the middle attacks through
# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
#net.ipv4.conf.all.accept_redirects = 0
#net.ipv6.conf.all.accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net.ipv4.conf.all.secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
#net.ipv4.conf.all.send_redirects = 0
#
# Do not accept IP source route packets (we are not a router)
#net.ipv4.conf.all.accept_source_route = 0
#net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
#net.ipv4.conf.all.log_martians = 1
#
Edit 2:
After some fixes - all, except of me in local net can use MFP.
So, new puzzle:
My local network:
http://prntscr.com/kvk5g
"Hakuhonoo" can`t see MFP, but other do.
Content of /etc/iptables.conf:
# Generated by iptables-save v1.4.12 on Fri Nov 9 01:51:58 2012
*filter
:INPUT ACCEPT [23:1420]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [20:18904]
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -d 224.0.0.0/4 -i eth0 -j ACCEPT
-A INPUT -s 224.0.0.0/4 -i eth0 -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 80:85 -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 1985 -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 25565 -j ACCEPT
-A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 60000:65000 -j ACCEPT
-A INPUT -i eth0 -j DROP
-A FORWARD -i eth0 -o br0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o eth0 -j ACCEPT
-A FORWARD -d 224.0.0.0/4 -j ACCEPT
-A FORWARD -s 224.0.0.0/4 -j ACCEPT
-A FORWARD -i eth0 -p tcp --dport 81:85 -j ACCEPT
-A FORWARD -i eth0 -j DROP
COMMIT
# Completed on Fri Nov 9 01:51:58 2012
# Generated by iptables-save v1.4.12 on Fri Nov 9 01:51:58 2012
*nat
:PREROUTING ACCEPT [377:31747]
:INPUT ACCEPT [39:3558]
:OUTPUT ACCEPT [11:872]
:POSTROUTING ACCEPT [7:570]
-A PREROUTING -i eth0 -p tcp --dport 81:85 -j DNAT --to 192.168.0.101
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Fri Nov 9 01:51:58 2012
# Generated by iptables-save v1.4.12 on Fri Nov 9 01:51:58 2012
*mangle
:PREROUTING ACCEPT [1425:140833]
:INPUT ACCEPT [762:69219]
:FORWARD ACCEPT [495:56655]
:OUTPUT ACCEPT [643:122295]
:POSTROUTING ACCEPT [1152:179096]
-A PREROUTING -d 224.0.0.0/4 -p udp -j TTL --ttl-inc 1
COMMIT
# Completed on Fri Nov 9 01:51:58 2012
Did you set your Ubuntu to forward packets?
Enable routing: (taken from here)
Configure the gateway for routing between two interfaces by enabling IP forwarding:
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
Edit /etc/sysctl.conf, and (up to 10.04) add these lines:
net.ipv4.conf.default.forwarding=1
net.ipv4.conf.all.forwarding=1
From 10.10 onwards, it suffices to edit /etc/sysctl.conf and uncomment:
net.ipv4.ip_forward=1