I'm using Ruby to run an nmap -sP ping scan on my home network continuously to check for new hosts connecting. A couple of questions:
Is there a better way to do this?
How effective is -sP at finding new hosts?
Will running this over and over on a loop create any problems for my home network which is used just for casual web browsing?
Is there a better way to do this?
This is good enough if hosts are guaranteed to accept pings; however, sometimes nmap -sP can spew packets faster than wifi networks can deal with them. If you see issues like this, just lower the rate with nmap --scan-delay 0.1 -sP.
How effective is -sP at finding new hosts?
As good as ping is; however, fresh Windows installations often block ping by default. You could run nmap -sT -P0 just to be sure you got everything...
The ultimate host detection scheme is to poll your ethernet switch for new mac-address entries.
Will running this over and over on a loop create any problems for my home network which is used just for casual web browsing?
Nope
A better solution would be to use your switch ("router") to check for new clients on the network. Most home wireless routers will have a Web page listing the current DHCP leases, which you could poll from your Ruby script. There may also be a page listing all MAC addresses associated with the network (essentially the switch's ARP table).
Related
My knowledge of networking is very basic so please bear with what I am sure are some very fundamental questions.
Is it possible to determine which interface/adapter (on a machine with multiple physical and virtual interfaces/adapters) is directly connected to a device with a known static IPv4 address without changing the network (i.e. the first 3 octets of its IPv4 address) of the interface/adapter beforehand?
I am asking this because I am trying to automate the setting of interface/adapter static IPv4 addresses on multiple machines in order for them to be able to communicate with the device and for it to work even if the machines do not use the same interface/adapter.
My original idea was to brute-force it by
Getting the device's IPv4 address as user input
Getting all physical interfaces/adapters connected to an Unidentified.network
Saving their previous network settings somewhere and loop through them while configuring them to use an appropriate static IPv4 address based on input in step 1
Attempting to ping device for a specified duration and if timeout occurs, revert the network settings of the interface/adapter to its original one otherwise, keep the new network settings and break out of the loop
My current worry is that the above process may inadvertently cause original network settings to be lost if some unpredictable crash were to occur during it.
So, it would be ideal if I could skip steps 2 and 3 and immediately detect which interface/adapter needs to have its IP address changed but based on what I read, this does not seem possible at all since if you could detect the device, then there would be no need to set a static IP address for the interface/adapter in the first place (cause you can already detect it).
I've also seen that using the ARP cache would work if the first-time connection on all machines are done manually and then setting the appropriate entry to persistent so that it would survive across reboots but I would ideally want even first-time connections to be automated if possible.
Can anyone provide any insight as to whether I am attempting the impossible?
Or is there a better way to achieve what I want?
I have seen some recommendations to use wireshark to sniff for incoming ARP packets but that would require installing it on all the machines I would be deploying my automated approach on which may not be feasible.
For context, I am automating this on Windows 10 and would prefer using powershell/bash/python.
Edit 1
Thanks to the link provided in the comment, I have gained a better understanding of IPv4 terminology.
So I guess I can reformulate my question.
Device
IPv4: 192.168.0.216/24
Network: 192.168.0.0
Machine
First interface/adapter
IPv4: 169.254.19.133/16
Network: 169.254.0.0
Second interface/adapter
IPv4: 169.254.27.245/16
Network: 169.254.0.0
If there is already a physical connection between the second interface/adapter and the device, is there a way to detect (not communicate with) the device without having to change the network of the second interface/adapter beforehand?
Or is brute-forcing it the only way to achieve my goal?
My problem is that I have two computers in different networks which can "communicate" only over a common file share mounted on both systems (typically Windows SMB server on third-party host). All other ports are blocked! I would like to use this file share for a direct communication between these two machines. I'm root on both machines, it's possible to use Windows or Linux on them. There is also no need to use SSH for the communication, 'netcat' or other command-line oriented tools are available.
What has been done so far to solve it: I started a (long ;-) discussion about tunneling SSH, which is on hold now, SSH: TCP-over-File?, but includes many valuable information (especially from Ch. Duffy et al.). I have understood that SSH is not suitable for that, but the question, if this Windows tunneling tool could by useful, is still open: https://labs.mwrinfosecurity.com/tools/tcp-over-file-tunnel/
How to establish a communication using common tools coming with the operating systems (Ubuntu or Windows 7) in this case?
Your requirement of common tools excludes any possible Windows solutions.
netcat is a great tool, if you have direct TCP/IP connectivity, which is not the case here.
File redirection is not a concern, since you can redirect locally and pickup on remote; I will assume you are looking for a more elegant terminal as your definition of communication.
You are on the right track using tail -f, but attempting to emulate a network connection would require rewriting SSH to use a file for communication.
My improvement to your code as described in "SSH: TCP-over-file" is to simulate an actual terminal. Run the following code on the local machine in single window:
tail -f /mnt/fileshare/my_ssh_out >>/dev/`ps eax | grep $$ |cut -d ' ' -f 2 | tail -n 1` &
cat >> /mnt/fileshare/my_ssh_in
The line: /dev/ps eax | grep $$ |cut -d ' ' -f 2 | tail -n 1 & gets the current terminal, so the output can be redirected there.
Note: you cannot redirect the output locally.
So, if you're root on both machines, I'm assuming there's an intermediate firewall you do not control? Between that and the very vague requirement of "communications", I see a few options that aren't terribly fast but could work.
Create 2 folders, one for each machine to drop 'message' files into. Each node would run an app listening for new files (from the other computer) being queued up, download and process new ones, and delete processed messages.
Have one machine running a server (you said they were networked, so I am assuming internet here) with web sockets, and have the other connect to it as a client. You can then pass messages and trigger actions over the web.
Neither of these will be fast or perfectly reliable, and probably wouldn't provide full control over the other computer. You may be limited to pre-defined actions triggered by specific keywords, or you might be crazy enough to pull in arbitrary commands from the messages and blindly execute them under your user permissions.
If the only segmentation between the computers is IP subnets, for example: no VLANs or physical separation. Try to connect using IPv6 by pinging the link-local addresses.
Use the following Windows command in a cmd window:
ipconfig /all
Look for the "FE80:"... address on the physical adapter.
On the other computer:
ping -6 fe80:...
If this works you may use that IP address to communicate between computers. Each of the VMs will have a different address. In Linux use ifconfig to find the addresses.
If that does not work, you can always use two cheap USB wired ethernet adapters and a crossover cable.
If You have ssh connection from your workstation to both servers - just use putty for tunneling or something like that.
Can I write a Windows Task or some kind of configuration script that will choose between two Local Area Connections according to the website I am visiting?
It may not be the best answer to my problem, but if it is possible, at least I know it will work.
The issue in full:
My main ISP currently has an issue routing me to my own websites (all hosted on the same server). It also has a 'sticky IP address' (note, not static) - it will only change your IP address once a fortnight, and they can't (won't) even force a change. Their second line support are working on the issue, but so far, no good, and I cannot access my own websites via their internet connection.
So, currently, I am switching from my main network to my mobile 3G network (tethered) any time I want to work on or view my own websites.
I would like to write a script that will make Windows automatically choose my mobile network for FTP, email and browsing my own websites, but use my main ISP for all other online activity.
Haven't a clue where to start - any help appreciated! Thanks,
Sarah
Well, that was simple, no scripting required. If you're using IPv4 (you can check here https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&ved=0CDoQFjAB&url=http%3A%2F%2Fwww.amiusingipv6.com%2F&ei=Bq4_UpakM4vv0gXboIDwCQ&usg=AFQjCNHiUnyKvVcUe8Z966YwoycLI28urw&bvm=bv.52434380,d.d2k)
Turn Windows features on or off.
Turn on RIP Listener.
Disable your internet connection that cannot connect to the website you are trying to access, and 'tracert' to the website: make a note of the IP address of the website, and the first IP address in the hop list ([FIRST IP]).
Type in 'route print' and make a note of the Interface number of your secondary internet connection.
Type in 'route add -p [WEBSITE IP ADDRESS] mask 255.255.0.0 [FIRST IP] IF [INTERFACE NUMBER] metric 1'
Reboot.
All traffic to the website IP address (be it mail, FTP, whatever) will go through the secondary connection, all other traffic goes through the main connection.
IPv6 instructions here:
http://windows.microsoft.com/en-gb/windows7/configuring-multiple-gateways-on-a-network
My setup is just an iPhone connected via USB as secondary connection and a Home Hub connected via ethernet as primary.
Hope this is useful to someone else - but of course, no one should ever think of using this to get around IP blocks on message boards...
Sarah
Is there a Mac/Unix commands that lets you see the local network in terms of machines/IP addresses? If there's something on the Mac that is gui-based that would be great too.
ping the broadcast address (the broadcast address is printed as part of the output to ifconfig en0)
The hosts answering are on your local network. (You may also try arp -a but that only keeps track of recently contacted hosts so you may want to run it after the broadcast.)
There is a program called Bonjour Browser that will list well known services that have registered on your local network. I believe that most Macs have one or more registered protocols by default.
The only way to reliably do this is to scan the network using ping sweeps and similar techniques looking for open ports etc on various addresses. You can do that with nmap which is available for OS X. See http://www.netadmintools.com/art406.html for an example.
EDIT: Just to clarify, as diciu pointed out, you can usually ping the broadcast address and/or use your arp cache as well. This will probably work for most home networks where directed broadcast is allowed.
If not, then you would need to run a ping sweep with a tool like nmap to individually check each address for an available host. Many network discovery/scanning tools can check for more than just ping, looking for listening ports, SNMP, etc. as well.
in the days of tiger (10.4) every mac broadcast a 'presence' service on bonjour, which made finding macintoshes on the network a snap. alas, no more...
You can use netdiscover on Mac OS, which is based on ARP packets. It will send ARP requests and scan the response.
For example, run netdiscover command sudo netdiscover -i en0 -r 10.106.0.0/16 in my local network would bring the following result:
Try IP Scanner 2.5 for OS X. http://10base-t.com/ Looking for others, but that's the only real one I've found for OS X.
You can use Nmap but that seems to be a bit much for your stated goals.
OS X ships with netstat, or open Up /Applications/Utilities/Network Utility.app, perhaps this will work for you?
EDIT: oops. netstat doesn't do what I thought.
angry IP scanner for a no mus - no fuss IP scanner with basic port scanning.
NMap and Zenmap for the big power scans.
I actually use both together, as nmap can get easily sidetracked by certain reverse proxy boxes.
I often run into problems where I can't get something to connect to something else. I usually forget to check something obvious. Can you help with:
A tip/technique for diagnosing a connection issue
The name of a tool or application that can help (and the situation in which it's useful)
I know the question is a little non-specific, but hopefully the answers can form a useful starting point for anybody who's stuck trying to get computers/programs talking to each other.
Please can you give one answer per answer so the best ones can be voted up.
Simple checks to run when debugging network problems:
Has each machine got an IP address, Go to command prompt and run ipconfig. Key things to check here are the interfaces and ensuring the appropriate ones have IP addresses.
Check both machines IP addresses are in the same range and subnet if you are running it on an internal or Virtual network.
Try pinging each machine from the other to see if they can communicate with each other. Note that some firewalls will block ping requests.
If Pinging fails then check to see if firewalls are active. If the communication is within a 'safe' internal network then try disabling the firewalls and re-pinging.
If the connections are over a wireless network then check signal strength.
If pinging fails and you are connecting through several networks then try running a tracert to see at which will may show you where on the network the connection is failing.
If you are able to ping but not connect then check firewall settings and network connection settings. Windows 2000+ has the capability of setting port an ip access on a connection properties.
Try drawing a network diagram of the connections to help in visualising the problem.
If you are connecting through routers, firewalls and loadbalancers then check that all devices are not tied to any specific ip addresses and that the IP address redirection (if in place) is correct. Also check any NAT logs to see if connections are being received and properly re-directed.
Wireshark
Latest versions of ProcMon
netstat
Wireshark www.wireshark.org
Wireshark is a network protocol analyzer for Unix and Windows.
Features:
Deep inspection of hundreds of protocols, with more being added all the time
Live capture and offline analysis
Standard three-pane packet browser
Multi-platform: Runs on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many others
Captured network data can be browsed via a GUI, or via the TTY-mode TShark utility
The most powerful display filters in the industry
Rich VoIP analysis
Read/write many different capture file formats: tcpdump (libpcap), Catapult DCT2000, Cisco Secure IDS iplog, Microsoft Network Monitor, Network General Sniffer® (compressed and uncompressed), Sniffer® Pro, and NetXray®, Network Instruments Observer, Novell LANalyzer, RADCOM WAN/LAN Analyzer, Shomiti/Finisar Surveyor, Tektronix K12xx, Visual Networks Visual UpTime, WildPackets EtherPeek/TokenPeek/AiroPeek, and many others
Capture files compressed with gzip can be decompressed on the fly
Live data can be read from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and others (depending on your platfrom)
Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2
Coloring rules can be applied to the packet list for quick, intuitive analysis
Output can be exported to XML, PostScript®, CSV, or plain text.
work the OSI model from the bottom up
Physical (Do you have a network adapter/connection)
Link layer (arp, ethernet port blocked by network team (I've seen this where locked down environments see two MAC addresses coming from one workstation port and shut down the port)
Network layer (ipconfig, tracert, ping,)
Do you have a network address (DHCP, fixed)
Are you on a proper subnet/have routing between subnets
Is something in the middle blocking you
firewalls, routing tables
When in doubt, check to see if the windows firewall is messing with your communications. 8 times out of 10, it's at fault.
Using tracert is a good start to see how far along the chain you are getting.
For virtual machines it's usally a good idea to make sure you have the loopback adapter set correctly in the Host os.
Most frequently used tool is the ping. It can be used both to test your connection and the availability of a target
Second tool is the tracert if you want to see where the packets get lost.
For more advanced debugging I use the following tools: nmap, wireshark, etc.
Windows has a netstat utility which is pretty similar to the Unix netstat and can do a number of different things that might help you solve network issues.
Random example:
netstat -r displays routing information
netstat /? for usage information
Since you said you're using 2 virtual machines I would hazard a guess that both machines are setup in a NAT configuration (rather than a unique network device) -- In the NAT configuration, neither machine would (typically) be able to ping the other.
If you're familiar with the command line, you can try the "netstat" command.
You can also try "arp -a" to list all the IP/MAC addresses known to your PC.
The "tracert [ip address]" command will show you how many gateways/routers your packets jump through on their way to their destination. (This is probably not helpful if both machines are on the same network, though.)
And don't forget to check your Windows firewall settings.
Otherwise, if you want to get down and dirty, you can try the packet sniffer known as Wireshark: http://www.wireshark.org/ (aka. Ethereal)
Pull the network cable out
If you can get some communications to a device (eg a ping), but can't get your program to talk to a service on the computer. Then, try pulling the network cable out and see if the ping stops. This will verify you're communicating with the computer you really think you are.
On windows i user PortQueryUI : http://www.microsoft.com/en-us/download/details.aspx?id=24009
DNS activity: Portable DNS Cache and Firewall;
General network activity: Wireshark, Network Monitor;
Windows utilities: ping, netstat, nslookup.
You need to be use the process of elimination, for example if you can ping the ip address but not the hostname then there's DNS issues. If you can ping the system but not connect to a share etc.
DNS out of sync
If you're using a virtual machine and you perform a roll-back on it, then it could become out of sync with the DNS (Domain name server). Try to remove and re-add the machine to the domain, or if you've got access to the DNS machine, then get it to flush its cache.