netstat display Huge amount of "SYN_RCVD" state - performance

netstat display Huge amount of "SYN_RCVD" state on Solaris 11, it's a nginx reverse server.
# netstat -na | awk '/^20/ {++S[$NF]} END {for(a in S) print a, S[a]}'
TIME_WAIT 515
ESTABLISHED 427
SYN_SENT 14
LAST_ACK 223
Connected 9488
FIN_WAIT_1 37
FIN_WAIT_2 167
CLOSING 48
CLOSE_WAIT 11
Idle 5
SYN_RCVD 4437
The server tcp/ip parameters:
# ndd -get /dev/tcp tcp_time_wait_interval
60000
# ndd -get /dev/tcp tcp_keepalive_interval
15000
# ndd -get /dev/tcp tcp_fin_wait_2_flush_interval
67500
# ndd -get /dev/tcp tcp_conn_req_max_q
16384
# ndd -get /dev/tcp tcp_conn_req_max_q0
16384
# ndd -get /dev/tcp tcp_xmit_hiwat
400000
# ndd -get /dev/tcp tcp_recv_hiwat
400000
# ndd -get /dev/tcp tcp_cwnd_max
2097152
# ndd -get /dev/tcp tcp_ip_abort_interval
20000
# ndd -get /dev/tcp tcp_rexmit_interval_initial
4000
# ndd -get /dev/tcp tcp_rexmit_interval_max
10000
# ndd -get /dev/tcp tcp_rexmit_interval_min
3000
# ndd -get /dev/tcp tcp_max_buf
4194304
How to tuning this nginx reverse server.
Thanks.

# of conn reqs in SYN_RCVD - "fully-established" connections -
those which have finished the 3-way handshake and
are waiting to be picked up by an "accept()" call.
You should consider tuning your accept_mutex nginx parameter and may be try different nginx processing method, just to get if it is real reason.

Related

issue with docker on a vps server

i'm new to Docker and facing with an issue for days and i don't know how to manage that. here is the situation;
i have a sudoer user in a vps server that runs such services like apache2 and mysql and ... and also serves couple application that i don't want to hurt them by upgrade or change services like php version or change web server from apache to nginx.
i want to serve a laravel application in docker and i built this docker-compose.prod.yml file in my project root folder that lives in /home/myUser/www/laravelProject (contains laravel application):
version : '3'
networks:
laravel:
volumes :
dbdata:
driver: local
services:
nginx:
image : nginx:stable-alpine
container_name: nginx
ports :
- "8080:80"
volumes :
- ./:/var/www/html ## i think it would make sens if i do like ./:/home/myuser/www/laravelProject ???!!!
- ./nginx/default.prod.conf:/etc/nginx/conf.d/default.conf
depends_on :
- php
- mysql
- redis
- artisan
networks :
- laravel
mysql :
image : mysql:5.7.29
container_name: mysql
restart : unless-stopped
tty : true
ports :
- "4406:3306"
environment :
MYSQL_DATABASE : ${DB_DATABASE}
MYSQL_USER : ${DB_USERNAME}
MYSQL_PASSWORD : ${DB_PASSWORD}
MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
SERVICE_TAGS : dev
SERVICE_NAME : mysql
volumes :
- dbdata:/var/lib/mysql
- ./mysql/my.cnf:/etc/mysql/my.cnf
networks :
- laravel
php :
build :
context : .
dockerfile: php.prod.dockerFile
container_name: php
volumes :
- ./:/var/www/html
ports :
- "9000:9000"
networks :
- laravel
artisan :
build :
context : .
dockerfile: php.dockerfile
container_name: artisan
volumes :
- ./:/var/www/html
depends_on :
- mysql
working_dir : /var/www/html
entrypoint : ['php', '/var/www/html/artisan']
networks :
- laravel
redis :
container_name: redis
image : redis
ports :
- "6379:6379"
volumes :
- ./:/data
entrypoint : redis-server --appendonly yes
restart : always
networks :
- laravel
i'm sorry i think i missed tab's and spaces. however, and this is my php.prod.dockerfile
FROM php:7.4-fpm-alpine
WORKDIR /home/www/html // again it's not supposed to be /home/myUser/www/project ???!!!
RUN docker-php-ext-install pdo pdo_mysql
RUN chown -R www-data:www-data /var/www // i don't really have any sense why i'm doing this !!
i build the containers successfully using command (in my project root):
docker-compose -f docker-compose.prod.yml up -d --build
and the default.prod.nginx file is :
server {
listen 80;
index index.php index.html;
server_name api.myUser.com www.api.myUser.com;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
root /var/www/html/public;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass php:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
}
unfortunately when i try to reach to the endpoint api.myUser.com:8080 i see
This site can’t be reached
http://api.myUser.com:8080/ is unreachable.
ERR_ADDRESS_UNREACHABLE
what i'm missing ?
(sorry for my english)
EDIT: here is the output of docker-compose up --build
Creating network "kooche-mobl_laravel" with the default driver
Building php
Step 1/4 : FROM php:7.4-fpm-alpine
---> f9f075c5a926
Step 2/4 : WORKDIR /var/www/html
---> Using cache
---> 5478beb70e23
Step 3/4 : RUN docker-php-ext-install pdo pdo_mysql
---> Using cache
---> 049d5b4134d8
Step 4/4 : RUN chown -R www-data:www-data /var/www
---> Using cache
---> b697f0fb62a1
Successfully built b697f0fb62a1
Successfully tagged kooche-mobl_php:latest
Building artisan
Step 1/4 : FROM php:7.4-fpm-alpine
---> f9f075c5a926
Step 2/4 : WORKDIR /var/www/html
---> Using cache
---> 5478beb70e23
Step 3/4 : RUN docker-php-ext-install pdo pdo_mysql
---> Using cache
---> 049d5b4134d8
Step 4/4 : RUN chown -R www-data:www-data /var/www
---> Using cache
---> b697f0fb62a1
Successfully built b697f0fb62a1
Successfully tagged kooche-mobl_artisan:latest
Creating redis ... done
Creating mysql ... done
Creating php ... done
Creating artisan ... done
Creating nginx ... done
Attaching to redis, php, mysql, artisan, nginx
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Entrypoint script for MySQL Server 5.7.29-1debian10 started.
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Switching to dedicated user 'mysql'
mysql | 2020-09-04 06:46:10+00:00 [Note] [Entrypoint]:Entrypoint script for MySQL Server 5.7.29-1debian10 started.
mysql | 2020-09-04T06:46:11.850419Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use -- explicit_defaults_for_timestamp server option (see documentation for more details).
mysql | 2020-09-04T06:46:11.857132Z 0 [Note] mysqld (mysqld 5.7.29-log) starting as process 1 ...
mysql | 2020-09-04T06:46:11.866317Z 0 [Note] InnoDB: PUNCH HOLE support available
mysql | 2020-09-04T06:46:11.866418Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql | 2020-09-04T06:46:11.866455Z 0 [Note] InnoDB: Uses event mutexes
mysql | 2020-09-04T06:46:11.866506Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mysql | 2020-09-04T06:46:11.866533Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql | 2020-09-04T06:46:11.866578Z 0 [Note] InnoDB: Using Linux native AIO
mysql | 2020-09-04T06:46:11.868019Z 0 [Note] InnoDB: Number of pools: 1
mysql | 2020-09-04T06:46:11.868769Z 0 [Note] InnoDB: Using CPU crc32 instructions
mysql | 2020-09-04T06:46:11.876267Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
mysql | 2020-09-04T06:46:11.918223Z 0 [Note] InnoDB: Completed initialization of buffer pool
mysql | 2020-09-04T06:46:11.936304Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysql | 2020-09-04T06:46:11.955456Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
mysql | 2020-09-04T06:46:12.004799Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql | 2020-09-04T06:46:12.004958Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql | 2020-09-04T06:46:12.055925Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql | 2020-09-04T06:46:12.058036Z 0 [Note] InnoDB: 96 redo
rollback segment(s) found. 96 redo rollback segment(s) are active.
mysql | 2020-09-04T06:46:12.058061Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
mysql | 2020-09-04T06:46:12.059038Z 0 [Note] InnoDB: Waiting for purge to start
mysql | 2020-09-04T06:46:12.109390Z 0 [Note] InnoDB: 5.7.29 started; log sequence number 12758905
mysql | 2020-09-04T06:46:12.110224Z 0 [Note] Plugin 'FEDERATED' is disabled.
mysql | 2020-09-04T06:46:12.115113Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql | 2020-09-04T06:46:12.126409Z 0 [Note] InnoDB: Buffer pool(s) load completed at 200904 6:46:12
mysql | 2020-09-04T06:46:12.137920Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
mysql | 2020-09-04T06:46:12.138013Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
mysql | 2020-09-04T06:46:12.139676Z 0 [Warning] CA certificate ca.pem is self signed.
mysql | 2020-09-04T06:46:12.139893Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
mysql | 2020-09-04T06:46:12.141123Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
mysql | 2020-09-04T06:46:12.141277Z 0 [Note] IPv6 is available.
mysql | 2020-09-04T06:46:12.141345Z 0 [Note] - '::' resolves to '::';
mysql | 2020-09-04T06:46:12.141398Z 0 [Note] Server socket created on IP: '::'.
mysql | 2020-09-04T06:46:12.168549Z 0 [Note] Event Scheduler: Loaded 0 events
mysql | 2020-09-04T06:46:12.169118Z 0 [Note] mysqld: ready for connections.
mysql | Version: '5.7.29-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not
empty, will attempt to perform configuration
nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
php | [04-Sep-2020 06:46:10] NOTICE: fpm is running, pid 1
php | [04-Sep-2020 06:46:10] NOTICE: ready to handle connections
nginx | 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
redis | 1:C 04 Sep 2020 06:46:10.064 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 04 Sep 2020 06:46:10.064 # Redis version=6.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 04 Sep 2020 06:46:10.064 # Configuration loaded
redis | 1:M 04 Sep 2020 06:46:10.068 * Running mode=standalone, port=6379.
redis | 1:M 04 Sep 2020 06:46:10.068 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis | 1:M 04 Sep 2020 06:46:10.068 # Server initialized
redis | 1:M 04 Sep 2020 06:46:10.068 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis | 1:M 04 Sep 2020 06:46:10.070 * Ready to accept connections
nginx | 10-listen-on-ipv6-by-default.sh: error:
/etc/nginx/conf.d/default.conf differs from the packages version
nginx | /docker-entrypoint.sh: Launching /docker-
entrypoint.d/20-envsubst-on-templates.sh
nginx | /docker-entrypoint.sh: Configuration complete; ready
for start up
i've also this line on netstat -tulnp | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 16093/docker-proxy
EDIT 2:
and about the firewall (iptables)
cat /etc/sysconfig/iptables |grep ACCEPT
its out put
:PREROUTING ACCEPT [884:49904]
:INPUT ACCEPT [262:14960]
:OUTPUT ACCEPT [60:4106]
:POSTROUTING ACCEPT [77:5110]
:INPUT ACCEPT [247:22593]
:OUTPUT ACCEPT [199:50843]
-A INPUT -p tcp -m tcp --dport 8090 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 8090 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 8080 -j ACCEPT
-A FORWARD -o br-4a4e80432e3f -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-4a4e80432e3f ! -o br-4a4e80432e3f -j ACCEPT
-A FORWARD -i br-4a4e80432e3f -o br-4a4e80432e3f -j ACCEPT
-A FORWARD -o br-8e73d0dbe1df -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-8e73d0dbe1df ! -o br-8e73d0dbe1df -j ACCEPT
-A FORWARD -i br-8e73d0dbe1df -o br-8e73d0dbe1df -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --gid-owner 209 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --gid-owner 12 -j ACCEPT
-A OUTPUT -d 127.0.0.1/32 -p tcp -m multiport --dports 25,465,587 -m owner --uid-owner 201 -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 25,465,587 -m owner --uid-owner 0 -j ACCEPT
-A DOCKER -d 192.168.32.2/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 3306 -j ACCEPT
-A DOCKER -d 192.168.32.3/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 9000 -j ACCEPT
-A DOCKER -d 192.168.32.4/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 6379 -j ACCEPT
-A DOCKER -d 192.168.32.6/32 ! -i br-4a4e80432e3f -o br-4a4e80432e3f
-p tcp -m tcp --dport 80 -j ACCEPT
i think it seems ok, but when i try telnet myPort 8080
says:
telnet: Unable to connect to remote host: No route to host

squid transparent proxy (intercept) + router

there is already a configured transparent proxy squid-3.5.27, there is an EdgeOSEdgeRouter router
I want to make a circuit
all computers on the network -> router -> squid
squid - 109.0.0.110
router - 109.0.0.1
test Windows - 109.0.0.8
configuration squid.conf, version - squid-3.5.27
# You should use the same dns resolver on squid and all clients
dns_nameservers 127.0.0.1
# acls
acl localnet src 109.0.0.0/24 # RFC1918 possible internal network
acl localnet src 192.168.1.0/24 # RFC1918 possible internal network
acl localnet src 192.168.10.0/24 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl blocked_http dstdomain "/etc/squid/blocked_sites.txt"
# http access
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access deny blocked_http
http_access allow localnet
http_access allow localhost
http_access deny all
http_port 3128 intercept
https_port 3129 intercept ssl-bump connection-auth=off cert=/etc/squid/squidCA.pem
http_port 3130
always_direct allow all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
acl blocked ssl::server_name "/etc/squid/blocked_https.txt"
acl whitelist src "/etc/squid/whitelist_ip.txt"
ssl_bump splice whitelist
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump terminate blocked
ssl_bump splice all
sslcrtd_program /opt/source/squid-3.5.27/src/ssl/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
acl YOUTUBE ssl::server_name .googlevideo.com
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 5120/5120
delay_access 1 allow YOUTUBE
coredump_dir /var/spool/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
cache_dir aufs /var/spool/squid 20000 49 256
maximum_object_size 61440 KB
minimum_object_size 3 KB
cache_swap_low 90
cache_swap_high 95
maximum_object_size_in_memory 512 KB
memory_replacement_policy lru
logfile_rotate 4
try this doc
iptables -t nat -A PREROUTING -i eth0 ! -s 109.0.0.110 -p tcp --dport 80 -j DNAT --to 109.0.0.110:3128
iptables -t nat -A POSTROUTING -o eth0 -s 109.0.0.8/32 -d 109.0.0.110/32 -j SNAT --to 109.0.0.1
iptables -A FORWARD -s 109.0.0.8/32 -d 109.0.0.110/32 -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT
prescribed iptables
I try to open a site on a test Windows (109.0.0.8) - access is denied, on Windows the gateway is 109.0.0.1, in the logs on squid
1546203601.533 0 109.0.0.110 TCP_MISS/403 4857 GET http://myip.ru/ - HIER_NONE/- text/html
1546203601.533 1 109.0.0.1 TCP_MISS/403 4977 GET http://myip.ru/ - ORIGINAL_DST/109.0.0.110 text/html
tcpdump squid server
11:00:57.141246 IP 109.0.0.8.54026 > myip.ru.http: Flags [F.], seq 1, ack 1, win 2087, length 0
11:00:57.141570 IP 109.0.0.8.54030 > myip.ru.http: Flags [S], seq 1736419147, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:00:57.141971 IP myip.ru.http > 109.0.0.8.54026: Flags [.], ack 2, win 58, length 0
11:00:57.142115 IP myip.ru.http > 109.0.0.8.54026: Flags [F.], seq 1, ack 2, win 58, length 0
11:00:57.142304 IP myip.ru.http > 109.0.0.8.54030: Flags [S.], seq 4065681746, ack 1736419148, win 29200, options [mss 1452,nop,nop,sackOK,nop,wscale 9], length 0
11:00:57.142363 IP 109.0.0.8.54026 > myip.ru.http: Flags [.], ack 2, win 2087, length 0
11:00:57.142505 IP 109.0.0.8.54030 > myip.ru.http: Flags [.], ack 1, win 260, length 0
11:00:57.144743 IP 109.0.0.8.54025 > myip.ru.http: Flags [P.], seq 2136:2856, ack 23054, win 2087, length 720: HTTP: GET / HTTP/1.1
11:00:57.146027 IP myip.ru.http > 109.0.0.8.54025: Flags [P.], seq 23054:23572, ack 2856, win 69, length 518: HTTP: HTTP/1.1 403 Forbidden
tell me what could be the problem, I will be grateful for any help?
Update
when you open a site on http by Windows, it is issued - 403, by https - an invalid certificate, the proxy server certificate is substituted in the browser, what am I doing wrong?
Update 2
when adding rules on a router
iptables -t nat -I PREROUTING -i eth0 -s 109.0.0.8 -p tcp --dport 80 -j DNAT --to 109.0.0.110:3128
iptables -t nat -I PREROUTING -i eth0 -s 109.0.0.8 -p tcp --dport 443 -j DNAT --to 109.0.0.110:3129
iptables -t nat -I POSTROUTING -o eth0 -s 109.0.0.8 -d 109.0.0.110 -j SNAT --to 109.0.0.1
proxy server rules
*nat
:PREROUTING ACCEPT [314:20555]
:INPUT ACCEPT [313:20511]
:OUTPUT ACCEPT [844:60999]
:POSTROUTING ACCEPT [2:130]
-A PREROUTING -s 109.0.0.0/24 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3129
-A PREROUTING -s 109.0.0.0/24 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -s 192.168.10.0/24 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3129
-A PREROUTING -s 192.168.10.0/24 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -s 192.168.1.0/24 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3129
-A PREROUTING -s 192.168.1.0/24 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A POSTROUTING -s 109.0.0.0/24 -j SNAT --to-source 109.0.0.110
-A POSTROUTING -s 192.168.10.0/24 -j SNAT --to-source 109.0.0.110
-A POSTROUTING -s 192.168.1.0/24 -j SNAT --to-source 109.0.0.110
COMMIT
*filter
:INPUT ACCEPT [340:18626]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1809:273786]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 109.0.0.0/24 -j ACCEPT
-A INPUT -s 192.168.1.0/24 -j ACCEPT
-A INPUT -s 192.168.10.0/24 -j ACCEPT
-A INPUT -j LOG
-A INPUT -p tcp -m multiport --dports 3128:3130 -j DROP
-A FORWARD -s 109.0.0.0/24 -p udp -m multiport --dports 80,443 -j DROP
COMMIT
when you open a site on Windows on http, in cache.log
kid1| WARNING: Forwarding loop detected for:
GET / HTTP/1.1^M
Upgrade-Insecure-Requests: 1^M
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36^M
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8^M
Accept-Encoding: gzip, deflate^M
Accept-Language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7^M
Via: 1.1 proxy.server (squid/3.5.27)^M
X-Forwarded-For: 109.0.0.1^M
Cache-Control: max-age=259200^M
Connection: keep-alive^M
Host: myip.ru^M
^M
kid1| WARNING: Forwarding loop detected for:
GET /favicon.ico HTTP/1.1^M
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36^M
Accept: image/webp,image/apng,image/*,*/*;q=0.8^M
Referer: http://myip.ru/^M
Accept-Encoding: gzip, deflate^M
Accept-Language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7^M
Via: 1.1 proxy.server (squid/3.5.27)^M
X-Forwarded-For: 109.0.0.1^M
Cache-Control: max-age=259200^M
Connection: keep-alive^M
Host: myip.ru^M
in access.log
1546711344.892 0 109.0.0.110 TCP_MISS/403 4514 GET http://myip.ru/ - HIER_NONE/- text/html
1546711344.893 0 109.0.0.1 TCP_MISS/403 4634 GET http://myip.ru/ - ORIGINAL_DST/109.0.0.110 text/html
1546711344.913 0 109.0.0.110 TCP_MISS/403 4479 GET http://myip.ru/favicon.ico - HIER_NONE/- text/html
1546711344.913 0 109.0.0.1 TCP_MISS/403 4599 GET http://myip.ru/favicon.ico - ORIGINAL_DST/109.0.0.110 text/html
WARNING: Forwarding loop detected for
how to fix it, any help ?
Don't use DNAT and change destination IP! Forward your traffic to the Squid machine as it was generated on the client machine by changing the IP routing table. See https://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute
I had the same problem which has been solved by this approach.

Output network speed in mbit/s using netcat

I'm using netcat piped thorugh pv to measure network speed
Server A
netcat -l -q -1 -p 1234 | pv > /dev/null
Server B
dd if=/dev/zero bs=10M count=1 | nc -v -n 10.10.10.2 1234
Result is returned on server B as
(UNKNOWN) [10.10.10.2] 1234 (?) open
1+0 records in
1+0 records out
10485760 bytes (10 MB) copied, 5.24922 s, 2.0 MB/s
What I need to do is convert MB/s to Mbit/s and output that so in script I can do
echo $speed
16 Mbits/sec
Also I need netcat to always be listengin on Server A and not close after the transfer has completed. I've tries the -k and -q -1 switches but no luck.

Gluster strange issue with shared mount point like seprate mount.

I have two nodes and for experiment i have install glusterfs and create volume and successfully mounted on own node, but if i create file in node1 it is not showing in node2, look like both behaving like they are separate.
node1
10.101.140.10:/nova-gluster-vol
2.0G 820M 1.2G 41% /mnt
node2
10.101.140.10:/nova-gluster-vol
2.0G 33M 2.0G 2% /mnt
volume info split brian
$ sudo gluster volume heal nova-gluster-vol info split-brain
Gathering Heal info on volume nova-gluster-vol has been successful
Brick 10.101.140.10:/brick1/sdb
Number of entries: 0
Brick 10.101.140.20:/brick1/sdb
Number of entries: 0
test
node1
$ echo "TEST" > /mnt/node1
$ ls -l /mnt/node1
-rw-r--r-- 1 root root 5 Oct 27 17:47 /mnt/node1
node2 (file isn't there, while they are shared mount)
$ ls -l /mnt/node1
ls: cannot access /mnt/node1: No such file or directory
What i am missing??
Iptable solved my problem
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 49152 -j ACCEPT

Finding the IP from the PID

I'm a newbie to shell programming and I'd like to find the IP address from the process ID. Right now, I'm able to get the PID for a specific process from :
vmname=$1
pid=`ps aux | grep $vmname | awk 'NR==1{printf("%s\n", $2) }'`
echo $pid
The above method returns the PID but how do I get the port from the pid? If I get the port, is there a command to get the IP address as well?
I'm using Ubuntu 11.04 and the above script is actually trying to find out the IP of a virtual machine running on KVM using this method.
Thanks!
You can employ the lsof utility. It gives the list of open files for a process. Use lsof -p pid . You need to grep on the output to get the port values for eg. something like this - lsof -p pid| grep TCP. This will list all the ports opened or connected to by the process. Refer to the manual of the utility. For most systems the utility comes pre-bundled with your OS. However, if it is not pre-bundled then you need to install this utility.
The PID and the computer's IP Address are two completely unrelated things.
PID stands for Process ID, and it's a handle for the OS to keep track of your program, among other things.
IP address is related to a network interface. Most computers have one or two of these (in the case of ethernet card/wireless device.)
Anyway, one way to get your computer's IP address is something similar to the following...There are quite possibly better ways to do it and I just don't know 'em...
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 60:eb:69:96:da:87
inet addr:192.168.1.112 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::62eb:69ff:fe96:da87/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:876533 errors:0 dropped:0 overruns:0 frame:0
TX packets:560999 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:229205080 (229.2 MB) TX bytes:136756800 (136.7 MB)
Interrupt:40 Base address:0x8000
$ ifconfig eth0 | grep "inet addr"
inet addr:192.168.1.112 Bcast:192.168.1.255 Mask:255.255.255.0
$ ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2
192.168.1.112 Bcast
$ ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2 | cut -d " " -f 1
192.168.1.112
So the last command will get you what you want inside your script. The rest are just there to show you how I built up to the last command.
Before I start lsof should be used as suggested by #Drona if lsof and if root/su/sudo access is available.
For completness I was investigating this for getting the IP address of currently logged in chrooted SFTP users for a nagios script I did not want to have to create a sudoers rule for.
Easy way (not as easy as lsof and needs root but for completeness)
Step 1
$ ps -ef | grep ssh
UID PID PPID C STIME TTY TIME CMD
root 2479 14186 0 17:05 ? 00:00:00 sshd: sftpuser [priv]
1008 2481 2479 0 17:06 ? 00:00:00 sshd: sftpuser#notty
1008 2482 2481 0 17:06 ? 00:00:00 sshd: sftpuser#internal-sftp
root 2483 14186 0 17:06 ? 00:00:00 sshd: ttyuser [priv]
ttyuser 2485 2483 0 17:06 ? 00:00:00 sshd: ttyuser#pts/0
Above you can see the PID for the ssh users (added the ps columns for easier interpretation)
Step 2
sudo lsof -p 2481 | grep TCP
sshd 2481 root 3u IPv4 29176657 0t0 TCP 192.168.1.2:44156 (ESTABLISHED)
Alternative (more complex has the possibility of not needing rood)
Step 2 - Requires root access but is optional
$ sudo ls -l /proc/2481/fd
total 0
lrwx------ 1 root root 64 Jul 3 17:07 0 -> /dev/null
lrwx------ 1 root root 64 Jul 3 17:07 1 -> /dev/null
lr-x------ 1 root root 64 Jul 3 17:07 11 -> pipe:[29209918]
lrwx------ 1 root root 64 Jul 3 17:07 2 -> /dev/null
lrwx------ 1 root root 64 Jul 3 17:07 3 -> socket:[29209894]
lrwx------ 1 root root 64 Jul 3 17:07 5 -> socket:[29211080]
lr-x------ 1 root root 64 Jul 3 17:07 6 -> pipe:[29209915]
l-wx------ 1 root root 64 Jul 3 17:07 7 -> pipe:[29209915]
l-wx------ 1 root root 64 Jul 3 17:07 8 -> pipe:[29209916]
lr-x------ 1 root root 64 Jul 3 17:07 9 -> pipe:[29209917]
Step 3
$ fgrep 29209894 /proc/2481/net/tcp
8: 0101A8C0:0016 0201A8C0:B0B0 ...
here fgrep uses the number on the socked and the PID to extract the information.
The important information is 0101A8C0:0016 and 0201A8C0:B0B0. The first relates to ther server and the second is the connected client where the first part (split by the colon) is the hexadecimal representation of the reversed IP address and the second is the hexadecimal representation of the port. i.e
0101A8C0 -> 1.1.168.192 -> 192.168.1.1. If you know the port the server is listening on you can skip Step 2 and use the following instead of Step 3.
Step 2 + 3 Replacement when knowing the server port - if no root is availalble
in this case as I was checking for SFTP connections on the standard ssh port of 22 (in hex 0016)
$ fgrep 0016 /proc/2481/net/tcp
8: 0101A8C0:0016 0201A8C0:B0B0 ...

Resources