I am setting up port scanner for remote server in my application using Go. I am using DialTimeout function in Go net package to check whether a remote host port is opened or not. The result is fine with success case. But, if i/o timeout happens, I need to identify whether
The port is closed (No service is running) or
Port is blocked (Firewall filtered) or
Due to internet connectivity down in local system where the application is running.
Have tried nmap cli command, I can able to differentiate those failure 3 cases exactly.
nmap command tried: nmap -sA -p port_number host_ip
I found a Go 3rd party libray to use nmap.
But, I don't want to use nmap in my application. Are there any other alternatives in Go to exactly differentiate those 3 cases?
In the simple world
Lets assume you want to scan a Linux system.
If you get an ICMP message type 3 code 3, the firewall explicitly told you:
Hi, I am the firewall of your target host. The host is running. I hereby inform you that you (potentially amongst others) can not access this port. So now that you know you should quit your connection attempts. However, I won't tell you wether it is because there is no service running behind it (in which case my response is simply a courtesy) or because I was told to deny you access. Goodbye!
The port is closed if you do not get above answer and can not make a connection. I hence strongly advice to use context.WithTimeout to make a connection.
In the real world
However, this only applies if the admin of the target host did not change the ICMP message type to respond with - or chose just to drop any packets coming from sources which are not allowed to access the respective service. In the latter case, there is no way for you to know wether the port is closed or filtered.
All of the above only applies if we are talking of an iptables based firewall on the target system with default settings.
Now assume something which is by far more likely: A border firewall plus a local firewall. The border firewall might send other ICMP messages (or, again, simply drop your packages). Those rules apply additionally to the rules of the local firewall. So it is a misconception that you are actually scanning a host. It is more accurate to say that you scan the services reachable via a specific IP.
EDIT
Why would one send an ICMP message explicitly rejecting connection attempts?
There are various reasons to come to that decision. There is a good answer on serverfault.com
Related
I don't have any code yet, so please feel free to move this to a sister site, if you think it belongs there :)
I have a Program A ( I don't have it's source code, so I can't modify it's behavior ) running on my machine which keeps listening to a particular port on the system for TCP data. It's a Peer to Peer application.
System 1 running A ====================== System 2 sunning A
The program A is supposed to run on systems where I may not be allowed to modify Firewall settings to allow incoming connections on the port the program listens to. I have an EC2 linux server running Ubuntu 16.
So I thought I can use an existing tool or create a program that would connect to the server on port X, and fetch the data from the server, and locally throw that data to the port A is listening to.
System 1 running A ========= SERVER =========== System 2 sunning A
What kind of configuration should I have on the server ? And is there any program I can use for this, or an idea of how to make one ?
I did something similar to bypass firewalls and hotspots.
Check this out https://github.com/yarrick/iodine, with a proper configuration your would be able to send\receive packets as DNS queries which is I know is always allowed, I used my server to get usual internet access with any hotspot I found.
You would lose some time, higher latency but you will have access.
Hope I helped.
The documentation on saltstack appears to be unclear regarding what ports are required from the salt-master -> salt-minion (apparently none are required).
It suggests that ports only need to be opened from the salt-minion -> salt-master.
(See: http://docs.saltstack.com/en/latest/topics/tutorials/firewall.html)
If however commands are executed remotely on the salt-master targeted to a minion, surely the master needs to be able to push this into the minion and therefore require a network opening to allow for this.
Therefore my question is if the saltstack ports (4505 & 4506) need to be opened in both directions, or whether the remote commands are triggered over another protocol?
[A bit of background: My team want salt-stack setup to manage a server landscape in quite a restrictive network where each individual network route needs to be requested in the security concept. This is not controlled by our company and I need to explicitly request all required routes and in each direction.]
Salt uses a zeromq pub/sub interface to communicate with the minions. Indeed, you only need to open ports 4505 and 4506 on the master's firewall.
The minions listen on one port on the master, which is the "pub" port, and then return results to the master on the other port.
The master never actually "pushes" commands to the minions. The minions listen for commands published on the pub port. Which is why you don't need to open any incoming ports on your minions.
I want to have a generic shell script which will check network connectivity between two hosts.
I wrote shell script with host and nslookup command to get the more details of target host, with these command I can't determine if current host can talk to target host.
Also I can't use(restricted) ping command , I was wondering if can use some other command to check network connectivity betweenn two hosts
Please suggest
Given a target host to determine if source host can communicate to target host
This is too vague to be useful. To solve this problem, you need to nail down what you mean by "communicate." A host may be able to send ICMP but not TCP. It may be able to send TCP but not ICMP. It may be able to send TCP to port 80, but not to 22. It may be able to send HTTP to port 80, but not SSH to port 80. Packets you send may return an error, or they may be silently dropped. The endpoint may receive your packets, but not process them. It may process them but not respond to you. There are many levels of "communicate."
So the best thing to test with is the thing you actually want to do. So if you want to communicate with HTTP over port 80, the best test is to do that. In fact, the best test is to just do the thing you wanted to do and not check beforehand. You're going to have to deal with errors no matter way. Just because you checked beforehand doesn't mean your actual attempt will be successful.
But sometimes you do just want to check "connectivity" (for some value of "connectivity") for monitoring purposes. In that case, again, do the thing you want. The easiest shell tool for checking HTTP connectivity is to fetch something with curl. If you need some other port, then a very nice generic solution is netcat (often called nc). I like:
nc -G 1 <host> <port> </dev/null
A return code of 0 means it connected; 1 means it failed.
For more esoteric issues, you can use nmap or even hping to craft about anything you want.
But most of the time, you shouldn't check at all. And if you do check, check with the thing you really want to do.
Most proxy servers perform the job of forwarding data to an appropriate "real" server. However, I am in the process of designing a distributed system in which when the "proxy" receives a TCP/IP socket connection, the remote system actually connects with a real server which the proxy nominates. All subsequent data flows from remote to the real server.
So is it possible to "forward" the socket connection request so that the remote system connects with the real server?
(I am assuming for the moment that nothing further can be done with the remote system. Ie the proxy can't respond to the connection by sending the IP address of the actual server and the remote connections with that. )
This will be under vanilla Windows (not Server), so can't use cunning stuff like TCPCP.
I assume your "remote system" is the one that initiates connection attempts, i.e. client of the proxy.
If I get this right: when the "remote system" wants to connect somewhere, you want the "proxy server" to decide where the connection will really go ("real server"). When the decision is made, you don't want to involve the proxy server any further - the data of the connection should not pass the proxy, but go directly between the "remote system" and the "real server".
Problem is, if you want the connection to be truly direct, the "remote system" must know the IP address of of the "real server", and vice versa.
(I am assuming for the moment that nothing further can be done with
the remote system. Ie the proxy can't respond to the connection by
sending the IP address of the actual server and the remote connections
with that. )
Like I said, not possible. Why is it a problem to have the "proxy" send back the actual IP address?
Is it security - you want to make sure the connection really goes where the proxy wanted? If that's the case, you don't have an option - you have to compromise. Either the proxy forwards all the data, and it knows where the data is going, or let the client connect itself, but you don't have control where it connects.
Most networking problems can be solved as long as you have complete control over the entire network. Here, for instance, you could involve routers on the path between the "remote system" and the "real client", to make sure the connection is direct and that it goes where the proxy wanted. But this is complex, and probably not an option in practice (since you may not have control over those routers).
A compromise may be to have several "relay servers" distributed around the network that will forward the connections instead of having the actual proxy server forward them. When a proxy makes a decision, it finds the best (closest) relay server, tells it about the connection, then orders the client to connect to the relay server, which makes sure the connection goes where the proxy intended it to go.
There might be a way of doing this but you need to use a Windows driver to achieve it. I've not tried this when the connection comes from an IP other than localhost, but it might work.
Take a look at NetFilter SDK. There's a trial version which is fully functional up to 100000 TCP and UDP connections. The other possibility is to write a Windows driver yourself, but this is non-trivial.
http://www.netfiltersdk.com
Basically it works as follows:
1) You create a class which inherits from NF_EventHandler. In there you can provide your own implementation of methods like tcpConnectRequest to allow you to redirect TCP connections somewhere else.
2) You initialize the library with a call to nf_init. This provides the link between the driver and your proxy, as you provide an instance of your NF_EventHandler implementation to it.
There are also some example programs for you to see the redirection happening. For example, to redirect a connection on port 80 from process id 214 to 127.0.0.0:8081, you can run:
TcpRedirector.exe -p 80 -pid 214 -r 127.0.0.1:8081
For your proxy, this would be used as follows:
1) Connect from your client application to the proxy.
2) The connection request is intercepted by NetFilterSDK (tcpConnectRequest) and the connection endpoint is modified to connect to the server the proxy chooses. This is the crucial bit because your connection is coming from outside and this is the part that may not work.
Sounds like routing problem, one layer lower than TCP/IP;
You're actually looking for ARP like proxy:
I'd say you need to manage ARP packets, chekcing the ARP requests:
CLIENT -> WHOIS PROXY.MAC
PROXY -> PROXY.IP is SERVER.IP
Then normal socket connection via TCP/IP from client to server.
alt text http://img440.imageshack.us/img440/6950/problemyd1.png
(The curly lines with dots represent a network route.)
Having a process called "foo.exe", is there a way to forward everything it communicates over TCP/IP to a forwarding proxy located elsewhere? This forwarding should not reflect to other processes.
Another question: if there are multiple network adapters, is it possible to force a process to use one specific adapter.
Since in this example the targethost.com is known, I could just edit "system32\drivers\etc\hosts" to map targethost.com's IP to localhost, where on port 8765 would be the first forwarder waiting for an incoming connection and pass everything forward to proxy.foo.com. I was wondering if there's a more elegant way of doing this.
This is not for malware, I'm doing some network testing with my complex home network. Thank you for warning us.
Some free software for this would be perfect, alternatively a code idea (native or .net). Thank you very much.
It's not too hard if you make your own computer a firewall, then your app connects to a port on your own computer, and that port is forwarded to both the original destination and logged or forwarded on to your spying computer.
Alternatively you can make your other computer the firwall and have it log/forward the info.
Finally you could use a sniffer.
SocksCap will probably do the job (if you're OK with establishing a SOCKS proxy at proxy.foo.com).
You could hook into the TCP stack, for example, by using the Windows Filtering Platform or its predecessors, or you could substitute the network libraries/calls of that particular process.