execute shell script in ruby - ruby

I want to execute the following shell script
system('echo "
rdr pass on lo0 inet proto tcp from any to 192.168.99.1 port 80 -> 192.168.99.1 port 8080
rdr pass on lo0 inet proto tcp from any to 192.168.99.1 port 443 -> 192.168.99.1 port 4443
" | sudo pfctl -ef - > /dev/null 2>&1; echo "==> Fowarding Ports: 80 -> 8080, 443 -> 4443 & Enabling pf"'
)
This works fine, i now want to pass the IP address loaded from a YAML file, i tried the following
config.yaml
configs:
use: 'home'
office:
public_ip: '192.168.99.2'
home:
public_ip: '192.168.99.1'
Vagrantfile
require 'yaml'
current_dir = File.dirname(File.expand_path(__FILE__))
configs = YAML.load_file("#{current_dir}/config.yaml")
vagrant_config = configs['configs'][configs['configs']['use']]
system('echo "
rdr pass on lo0 inet proto tcp from any to '+vagrant_config['public_ip']+' port 80 -> '+vagrant_config['public_ip']+' port 8080
rdr pass on lo0 inet proto tcp from any to '+vagrant_config['public_ip']+' port 443 -> '+vagrant_config['public_ip']+' port 4443
" | sudo pfctl -ef - > /dev/null 2>&1; echo "==> Fowarding Ports: 80 -> 8080, 443 -> 4443 & Enabling pf"'
)
The second method does not work, nor it shows any error, can someone point me to the right direction, what i want is to read public_ip dynamically from config file or variable
Thanks
UPDATE 1
I get the following output
pfctl: Use of -f option, could result in flushing of rules
present in the main ruleset added by the system at startup.
See /etc/pf.conf for further details.
No ALTQ support in kernel
ALTQ related functions disabled
pfctl: pf already enabled
What can be possibly wrong?

For troubleshooting purposes, it would be wise to output the command you're going to run prior to sending it out to system.
cmd = 'echo "
rdr pass on lo0 inet proto tcp from any to '+vagrant_config['public_ip']+' port 80 -> '+vagrant_config['public_ip']+' port 8080
rdr pass on lo0 inet proto tcp from any to '+vagrant_config['public_ip']+' port 443 -> '+vagrant_config['public_ip']+' port 4443
" | sudo pfctl -ef - > /dev/null 2>&1; echo "==> Fowarding Ports: 80 -> 8080, 443 -> 4443 & Enabling pf"'
puts "Command to run:\n\n#{cmd}"
system( cmd )
Then, it would be wise to make the output from the system command visible. To make sure you get this feedback, I suggest you replace
sudo pfctl -ef - > /dev/null 2>&1
with (adding '-v' for more verbose output - pfctl man page)
sudo pfctl -efv -
and then look for the output and/or error messages.
Then, once the bugs are sorted out, you can put it back into stealthy, quiet mode :D
Also, since you are running with sudo you'll need to make sure the shell you're running within has sudo privileges and also make sure you're not being prompted for a password unknowingly.

Related

Unable to bind any program to IPv4 TCP port 80 on Mac [duplicate]

I have the following very simple docker-compose.yml, running on a Mac:
version: "3.7"
services:
apache:
image: httpd:2.4.41
ports:
- 80:80
I run docker-compose up, then I run this curl and Apache returns content:
/tmp/test $ curl -v http://localhost
* Trying ::1:80...
* TCP_NODELAY set
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.66.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sat, 26 Oct 2019 18:30:03 GMT
< Server: Apache/2.4.41 (Unix)
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
However, if I try to access the container using 127.0.0.1 instead of localhost, I get connection refused:
/tmp/test $ curl -v http://127.0.0.1
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connection failed
* connect to 127.0.0.1 port 80 failed: Connection refused
* Failed to connect to 127.0.0.1 port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to 127.0.0.1 port 80: Connection refused
Localhost does point to 127.0.0.1:
/tmp/test $ ping localhost
PING localhost (127.0.0.1): 56 data bytes
And netstat shows all local IP addresses port 80 to be forwarded:
/tmp/test $ netstat -tna | grep 80
...
tcp46 0 0 *.80 *.* LISTEN
...
I came to this actually trying to access the container using a custom domain I had on my /etc/hosts file pointing to 127.0.0.1. I thought there was something wrong with that domain name, but then I tried 127.0.0.1 and didn't work either, so I'm concluding there is something very basic about docker I'm not doing right.
Why is curl http://localhost working but curl http://127.0.0.1 is not?
UPDATE
It seems localhost is resolving to IPv6 ::1, so port forwarding seems to be working on IPv6 but not IPv4 addresses. Does that make any sense?
UPDATE 2
I wasn't able to fix it, but pointing my domain name to ::1 instead of 127.0.0.1 in my /etc/hosts serves as a workaround for the time being.
UPDATE 3
8 months later I bumped into the same issue and found my own question here, still unanswered. But this time I can't apply the same workaround, because I need to bind the port forwarding to my IPv4 address so it can be accessed from other hosts.
Found the culprit: pfctl
AFAIK, pfctl is not supposed to run automatically but my /System/Library/LaunchDaemons/com.apple.pfctl.plist said otherwise.
The Packet Filtering was configured to redirect all incoming traffic on port 80 to 8080, and 443 to 8443. And this is done without any process actually listening to port 80 and 443, that's why lsof and netstat wouldn't show anything,.
/Library/LaunchDaemons/it.winged.httpdfwd.plist has the following
<key>ProgramArguments</key>
<array>
<string>sh</string>
<string>-c</string>
<string>echo "rdr pass proto tcp from any to any port {80,8080} -> 127.0.0.1 port 8080" | pfctl -a "com.apple/260.HttpFwdFirewall" -Ef - && echo "rdr pass proto tcp from any to any port {443,8443} -> 127.0.0.1 port 8443" | pfctl -a "com.apple/261.HttpFwdFirewall" -Ef - && sysctl -w net.inet.ip.forwarding=1</string>
</array>
<key>RunAtLoad</key>
The solution was simply to listen on ports 8080 and 8443. All requests to ports 80 and 443 are now being redirected transparently.
While debugging this I found countless open questions about similar problems without answers. I hope this helps somebody.

listen tcp 127.0.4.1:2040: bind: can't assign requested address

I am using this golang code:
l, err := net.Listen("tcp", "127.0.4.1:2040")
if err != nil {
log.Fatal("d0b9184a-5248-413e-a5a8-30fea66997f5:", err)
}
log.Fatal(s.Serve(l))
I am getting this error:
listen tcp 127.0.4.1:2040: bind: can't assign requested address
anyone know why that won't work?
You have to add an additional IPv4 address to the loopback interface:
sudo ifconfig lo0 alias 127.0.4.1
Before running the command you can verify the routes using:
netstat -nr
By default (On MacOS Catalina 10.15.4), these routes are present:
127 127.0.0.1 UCS lo0
127.0.0.1 127.0.0.1 UH lo0
After you have added your desired address to the loopback interface (i.e, lo0), netstat -nr would show:
127 127.0.0.1 UCS lo0
127.0.0.1 127.0.0.1 UH lo0
127.0.4.1 127.0.4.1 UH lo0
Now try using your program, it should work.
Also, this is temporary. It won't persist on reboot. For that give this a read!

ngrep: how to exclude a match with regex

Am using ngrep command to capture the network traffic(mongodb queries) in bash terminal on CentOS.
I want to exclude a match pattern from capturing with regular expression.
Below basic ngrep is capturing everything on port 27017
ngrep -q -Wbyline -dany "^.*$" port 27017
But am facing issues while using bash special characters(! & ?) in regex.
FYI added the errors below.
[root#development ~]ngrep -q -Wbyline -dany -i "^((?!journal).)*$" port 27017
-bash: !journal: event not found
[root#development ~]# ngrep -q -Wbyline -dany "^((?\!journal).)*$" port 27017
interface: any
filter: ( port 27017 ) and (ip or ip6)
compile failed: unrecognized character after (? or (?-
In below command, there is no error but not working.
[root#development ~]# ngrep -q -Wbyline -dany -i "^((\?\!journal).)*$" port 27017
interface: any
filter: ( port 27017 ) and (ip or ip6)
match: ^((\?\!journal).)*$
How can I capture everything except, queries matching journal ?

(OS X) Port in use, however it is not shown by netstat or lsof

Sorry for my english.
I was trying to forward port 80 from my vagrant box to host machine (OS X) and got this message
"The forwarded port to 80 is already in use on the host machine."
So, in order to figure out which program uses port 80 i ran this:
➜ ~ sudo lsof -n -i:80 | grep LISTEN
➜ ~
However, as you can see, it shows nothing.I have also tried netstat, but result was the same. Then i tried to use netcat + tcpdump to look at tcp session:
➜ ~ nc -vvv 127.0.0.1 80
Connection to 127.0.0.1 80 port [tcp/http] succeeded!
➜ ~
In another window:
➜ ~ sudo tcpdump -ni lo0 port 80
Password:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo0, link-type NULL (BSD loopback), capture size 65535 bytes
00:03:47.019805 IP 127.0.0.1.50666 > 127.0.0.1.80: Flags [S], seq 2187569264, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 194193524 ecr 0,sackOK,eol], length 0
00:03:47.019834 IP 127.0.0.1.80 > 127.0.0.1.50666: Flags [R.], seq 0, ack 2187569265, win 0, length 0
So it looks like the port is closed, because it immediately sent RESET flag, but why did nc show that connection was successful and lsof show nothing.
I'm really confused. Can anyone tell me what is going on, or what am i doing wrong?
I can provide additional information if needed.
Thanks!
Looks like that's firewall reset connection.
Turn off Avast WebShield if it exists.

IPTables configuration for Transparent Proxy

I am confuse why my IPTable does not work in Router. what I'm trying to do is redirect any packets from source ip destined to port 80 and 443 to 192.168.1.110:3128. however when I tried this:
iptables -t nat -A PREROUTING -s 192.168.1.5 -p tcp --dport 80:443 -j DNAT --to-destination 192.168.1.110:3128
does not work. however when I add this,
iptables -t nat -A POSTROUTING-j MASQUARADE
it works. but the problem with masquarade is I do not get the real ip but instead the ip of the router. I need to get the source ip so my proxy server could record all ip connected to it. can some one tell me how to make it work without making POSTROUTING jump to Masquarade?
For real transparent proxying you need to use the TPROXY target (in the mangle table, PREROUTING chain). All other iptables-mechanisms like any NAT, MASQUERADE, REDIRECT rewrite the IP addresses of the packet, which makes it impossible to find out where the packet originally was intended to.
The proxy program has to bind() and listen() on a socket like any other server, but needs some specific socket flags (which requires some Linux capabilities (type of permission) or root). – Once connected, there is some way to get the “intended server” from the OS.
Sorry, I’m a little lazy about the details, but searching for “TPROXY” as keyword will get you going quickly!
If I am not wrong, the correct syntax of the rule would be:
iptables -t nat -A PREROUTING -s 192.168.1.5 -p tcp -m multiport --dports 80,443 -j DNAT --to-destination 192.168.1.110:3128
--dport 80:443 will forward all ports from 80 to 443
--dports 80,443 will forward port 80 and 443 only.
If you want traffic hitting 192.168.1.5 on port 80 and 443 to be forwarded to 192.168.1.110's 3128 port then you should use the below rule:
iptables -t nat -A PREROUTING -d 192.168.1.5 -p tcp -m multiport --dports 80,443 -j DNAT --to-destination 192.168.1.110:3128
You should also make sure the gateway on 192.168.1.110 is pointed to your router ip.
Finally you can use the masquerade rule as below.
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o eth1 -j MASQUERADE
eth1 should be your outgoing interface.
I had the same issue and the solution was to tell the transparent proxy to forward the source ip in the right header fields.
In case of my nginx proxy the rules were close to:
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://name_of_proxy;
proxy_redirect off;
}
i used the iptables -t nat -A PREROUTING -p tcp -s foreign ip to your device --dport 80:443 -j DNAT --to-destination your application or local ip:port.i think you did the prerouting the packet in your device out which never connect to port 80 or 443,these is for web server connect to device.192.168.1.5 is like my local address.
and remember to configecho 1 > /proc/sys/net/ipv4/ip_forward
I think you are doing NAT in both directions by not specifying an interface. Try adding -o eth0 to your -j MASQUERADE line. (Substitute whatever your "external" interface is, instead of eth0, depending on your setup.)

Resources