I want to collect only MAC addresses from arp-scan -l, omitting IP addresses and the devices name. How do we do it? I know that these in the middle column are all MAC addresses.
192.168.1.1 bc:98:89:47:20:f8 Fiberhome Telecommunication Technologies Co.,LTD
192.168.1.3 70:18:8b:5e:01:fd Hon Hai Precision Ind. Co.,Ltd.
192.168.1.5 90:e7:c4:da:80:76 HTC Corporation
192.168.1.6 b8:27:eb:b0:4d:25 Raspberry Pi Foundation
I want like this
bc:98:89:47:20:f8
70:18:8b:5e:01:fd
90:e7:c4:da:80:76
b8:27:eb:b0:4d:25
Edit:
arp-scan -l gives the following result
Interface: wlp5s0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.9.5 with 256 hosts (https://github.com/royhills/arp-scan)
192.168.1.1 bc:98:89:47:20:f8 Fiberhome Telecommunication Technologies Co.,LTD
192.168.1.5 90:e7:c4:da:80:76 HTC Corporation
192.168.1.3 70:18:8b:5e:01:fd Hon Hai Precision Ind. Co.,Ltd.
192.168.1.5 90:e7:c4:da:80:76 HTC Corporation (DUP: 2)
192.168.1.6 b8:27:eb:b0:4d:25 Raspberry Pi Foundation
192.168.1.6 b8:27:eb:b0:4d:25 Raspberry Pi Foundation (DUP: 2)
192.168.1.4 80:35:c1:4a:a5:dc (Unknown)
9 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.9.5: 256 hosts scanned in 3.017 seconds (84.85 hosts/sec). 7 responded
I am assuming that you are using a linux-like shell and awk utility is installed( it is mostly pre-installed )-
arp-scan -l | awk '/.*:.*:.*:.*:.*:.*/{print $2}'
Learn basic Unix shell!
In particular the cut command can be used to extract columns from text. Set the field separator to space (probably default) and select the second field.
Using --plain option of arp-scan can make things easier. It will display plain output showing only responding hosts. The hosts' information are separated by tab, MAC address (2nd column) can be extracted easily through cut, as Anony-Mousse described:
arp-scan -l --plain | cut -f 2
Related
I have run an nmap scan using the --script ssl-enum-ciphers -p443 192.168.0.0/24 options against multiple IP addresses. I have also run an extremely similar scan using the --script ssh2-enum-algos -p22 options, that produces output in the same format.
I want to quickly analyze this data and zero in on specific matches of specific ciphers or algorithms. The overarching goal is to run an ad-hoc internal vulnerability assessment without access to fancy tools such as Nessus or Rapid7 InsightVM.
While nmap supports the -oX option to output to XML, I have found that neither Microsoft Word, Excel, or a web browser know how to open the file. The Microsoft products keep producing an error that the xml format is incorrect.
So then I tried the nmap-parse-output code on Github. While it will easily group IP addresses by ports, it doesn't appear to be able to take that a step further and analyze the ciphers or algorithms for me.
So now I'm trying to figure out a way to manually parse these blobs of data.
A typical result might contain data for multiple IP addresses in the following format (the following example stdout is edited for brevity):
Nmap scan report for 192.168.1.1
Host is up (0.00064s latency).
PORT STATE SERVICE VERSION
443/tcp open ssl/http lighttpd
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
| TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
Nmap scan report for 192.168.1.2
Host is up (0.00048s latency).
PORT STATE SERVICE VERSION
443/tcp open ssl/http nginx (reverse proxy)
| ssl-enum-ciphers:
| TLSv1.2:
| ciphers:
| TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
| compressors:
| NULL
| cipher preference: server
|_ least strength: A
Nmap scan report for 192.168.1.3
....
Nmap scan report for 192.168.1.4
....
How can write a script, using standard applications available to bash, to loop through each blob of text? We don't know the length of each blob, so I need to match on the string "Nmap scan report" or something similar, and extract the data that is in between each of those matches.
Something like this would get me started, but it isn't complete, and doesn't actually separate each blob individually:
for i in $(cat scan-results | grep "Nmap scan report for"); do more data analysis here; done
For example, I might want to search for any IP address that support RC4 ciphers on port 443, so in the "do more data analysis", I would like to run: grep -i rc4
Or in the case of ssh algorithms, I want to ensure all cbc algorithms are disabled, so I could run: grep -i cbc
The resulting goal would be to list anything that matches so that I can quickly attribute the match to the specific IP address. I don't care how the results look, I just care about finding the results quickly.
Any help would be appreciated!
Without more sample data, and going solely on the limited examples, and keeping in mind that output format isn't of importance ...
NOTE: My sample data file - nmap.dat - is a cut-n-paste copy of the sample nmap data provided by the OP.
I'm thinking a multi-pattern grep may suffice, eg:
# search for any IP address that support RC4 ciphers on port 443
$ grep -i "Nmap scan report for|443|RC4" nmap.dat
Nmap scan report for 192.168.1.1
443/tcp open ssl/http lighttpd
Nmap scan report for 192.168.1.2
443/tcp open ssl/http nginx (reverse proxy)
Nmap scan report for 192.168.1.3
Nmap scan report for 192.168.1.4
# want to ensure all cbc algorithms are disabled
$ egrep -i "Nmap scan report for|cbc" nmap.dat
Nmap scan report for 192.168.1.1
Nmap scan report for 192.168.1.2
| TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A
| TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A
Nmap scan report for 192.168.1.3
Nmap scan report for 192.168.1.4
I am writing a bash script that runs an Nmap scan of the network. After this the scan needs to be examinned and the relevant bits need to be extracted.
I need to extract the IP, MAC and OS from the completed scan. The problem is that Nmap does not always get the OS from the scan and therefore does not put it in the results. I need to associate the IP, MAC and OS in the end result.
Here is an example of a test scan:
Nmap scan report for 192.168.0.1
Host is up (0.0029s latency).
Not shown: 990 closed ports
PORT STATE SERVICE
PORT# STATE XXXXXXX
MAC Address: MA:CA:DR:ES:S0:03 (Unknown)
Device type: general purpose
Running: Linux 2.6.X|3.X
OS CPE: cpe:/o:linux:linux_kernel:2.6 cpe:/o:linux:linux_kernel:3
OS details: Linux 2.6.32 - 3.13
Network Distance: 1 hop
Nmap scan report for 192.168.0.102
Host is up (0.0044s latency).
Not shown: 999 closed ports
PORT STATE SERVICE
PORT# STATE XXXXXXX
MAC Address: MA:CA:DR:ES:S0:02 (Sony Mobile Communications AB)
Too many fingerprints match this host to give specific OS details
Network Distance: 1 hop
Nmap scan report for 192.168.0.104
Host is up (0.00024s latency).
Not shown: 995 filtered ports
PORT STATE SERVICE
PORT# STATE XXXXXX
MAC Address: MA:CA:DR:ES:S0:01 (Micro-star Intl)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running (JUST GUESSING): Microsoft Windows 2008 (91%)
OS CPE: cpe:/o:microsoft:windows_server_2008::sp1 cpe:/o:microsoft:windows_server_2008:r2
Aggressive OS guesses: Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2 (91%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 1 hop
Also note how the last one in the example above could not find the OS, in this case the aggress guess is wanted
The end result needs to be a text file that has has something like the following:
192.168.0.1 - MA:CA:DR:ES:S0:03 - Linux 2.6.32 - 3.13
192.168.0.102 - MA:CA:DR:ES:S0:02 - Not found
192.168.0.104 - MA:CA:DR:ES:S0:01 - Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2
I did some research but could not find anything that explains how I can associate the IP with the mac addresses and the os in the text blocks.
I have the following commands that work with a simple scan where the IP and Mac addresses are next to each other
while read line; do
Mac="$(grep -oE '[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}:[A-Z0-9]{2}' <<< "$line")"
ip="$(grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' <<< "$line")"
echo -e $ip'\t-\t '$Mac >>/path/to/results.txt
done </path/to/testscan.txt
I am fairly new to bash scripting so apologies if I am missing something obvious.
The nmap command for anyone interested is:
nmap -O --osscan-guess 192.168.0.0/24 -oN /path/to/testscan.txt
Sorry for the wall of text, I figured the more information the better!
This would be pretty easy to parse with awk:
BEGIN {os_details="Not found"}
/^Nmap scan report/ {target=$5}
/^MAC Address/ {mac_address=$3}
/^OS details/ {os_details=substr($0, length("OS details: "))}
/^Aggressive OS guesses/ {
os_details=substr($0, length("Aggressive OS guesses: "))
}
# This matches the blank lines between hosts
/^$/ {
printf "%s - %s - %s\n", target, mac_address, os_details
target=""
mac_address=""
os_details="Not found"
}
END {
printf "%s - %s - %s\n", target, mac_address, os_details
}
Running this on your sample data gets you:
192.168.0.1 - MA:CA:DR:ES:S0:03 - Linux 2.6.32 - 3.13
192.168.0.102 - MA:CA:DR:ES:S0:02 - Not found
192.168.0.104 - MA:CA:DR:ES:S0:01 - Microsoft Windows Server 2008 SP1 or Windows Server 2008 R2 (91%)
I had to make one correct to what I believe was an error in your sample data...I removed the blank line before the MAC Address line here:
Nmap scan report for 192.168.0.104
Host is up (0.00024s latency).
Not shown: 995 filtered ports
PORT STATE SERVICE
PORT# STATE XXXXXX
MAC Address: MA:CA:DR:ES:S0:01 (Micro-star Intl)
Using option -oX of the nmap (output to the XML format) the parsing could be more accurate:
nmap -oX /path/to/testscan.xml ...
# or
nmap -oX - ... > /path/to/testscan.xml
Then you could to use, for example, xmllint to parse this XML with XPath:
file="/path/to/testscan.xml"
get_details() {
local file addr mac os
file="$1"
addr=$2
mac=$(xmllint --xpath "string(//address[../address[#addr='$addr']][#addrtype='mac']/#addr)" "$file")
os=$(xmllint --xpath "string(//os[../address[#addr='$addr']]/osmatch/#name)" "$file")
: ${mac:="No data"}
: ${os:="No data"}
printf "%s - %s - %s\n" "$addr" "$mac" "$os"
}
for a in $(xmllint --xpath "//address[#addrtype='ipv4']/#addr" "$file" | grep -Po '\d+\.\d+\.\d+\.\d+'); do
get_details "$file" $a
done
In order to check that all the servers across a fleet aren't supporting deprecated algorithms, I'm (programmatically) doing this:
telnet localhost 22
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_8.0p1 Ubuntu-6build1
SSH-2.0-Censor-SSH2
4&m����&F �V��curve25519-sha256,curve25519-sha256#libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1Arsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519lchacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.comlchacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.com�umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1�umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1none,zlib#openssh.comnone,zlib#openssh.comSSH-2.0-Censor-SSH2
Connection closed by foreign host.
Which is supposed to be a list of supported algorithms for the various phases of setting up a connection. (kex, host key, etc). Every time I run, I get a different piece of odd data at the start - always a different length.
There's an nmap plugin - ssh2-enum-algos - which returns the data in it's complete form, but I don't want to run nmap; I have a go program which opens the port, and sends the query, but it gets the same as telnet. What am I missing, and how do I fix it?
For comparison, here's the top few lines from the output of nmap script:
$ nmap --script ssh2-enum-algos super
Starting Nmap 7.80 ( https://nmap.org ) at 2019-12-27 22:15 GMT
Nmap scan report for super (192.168.50.1)
Host is up (0.0051s latency).
rDNS record for 192.168.50.1: supermaster
Not shown: 998 closed ports
PORT STATE SERVICE
22/tcp open ssh
| ssh2-enum-algos:
| kex_algorithms: (12)
| curve25519-sha256
| curve25519-sha256#libssh.org
| ecdh-sha2-nistp256
| ecdh-sha2-nistp384
| ecdh-sha2-nistp521
Opening a tcp connection to port 22, (in golang, with net.Dial) then accepting and sending connection strings leaves us able to Read() from the Reader for the connection. Thence the data is in a standard format described by the RFC. From this, I can list the algorithms supported in each phase of an ssh connection. This is very useful for measuring what is being offered, rather than what the appears to be configured (it's easy to configure sshd to use a different config file).
It's a useful thing to be able to do from a security POV.
Tested on every version of ssh I can find from 1.x on a very old solaris or AIX box, to RHEL 8.1.
In some cases you can specify an algorithm to use, and if you specify one that is not supported the server will reply with a list of supported algorithms.
For example, to check for supported key exchange algorithms you can use:
ssh 127.0.0.1 -oKexAlgorithms=diffie-hellman-group1-sha1
diffie-hellman-group1-sha1 is insecure and should be missing from most modern servers. The server will probably respond with something like:
Unable to negotiate with 127.0.0.1 port 22: no matching key exchange method found. Their offer: curve25519-sha256,curve25519-sha256#libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
Exit 255
Typing: "ssh -Q cipher | cipher-auth | mac | kex | key"
will give you a list of the algorithms supported by your client
Typing: "man ssh"
will let you see what options you can specify with the -o argument, including Cipher, MACs, and KexAlgorithms
I need your help again.
I've downloaded the speedtest-cli via apt-get on my Raspberry PI 3B (Running Raspbian Stretch with Kernel 4.14.81.v7+) because I want to monitor my network speed throughout the day(s).
Problem is the following: I want to get only the http:// address which speedtest-cli --share prints on the terminal. (I want to display the latest speedtest on my website) I tried
speedtest-cli --share | grep 'Share results: '
but it prints
Share results: http://www.speedtest.net/result/*********
instead of
http://www.speedtest.net/result/********
Also another question: I've written this code in an crontab -e:
*/60 * * * * (date >> /home/pi/speedtest/speed.txt; /usr/bin/speedtest-cli --server 3645 --simple >> /home/pi/speedtest/speed.txt; echo "\n" >> /home/pi/speedtest/speed.txt)
It hasn't printed something yet, is it correct?
Edit:
The standard output of speedtest-cli --share is sth. like that:
Retrieving speedtest.net configuration...
Testing from ISP (IP Address)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Speedtest Provider (Location of ST Provider) [distance in km]: Ping
Testing download speed................................................................................
Download: ***.** Mbit/s
Testing upload speed....................................................................................................
Upload: ***.** Mbit/s
Share results: http://www.speedtest.net/result/***********.png
Try this with GNU grep:
speedtest-cli --share | grep -Po 'Share results: \K.*'
or
speedtest-cli --share 2>&1 | grep -Po 'Share results: \K.*'
I'm attempting to determine if the Auto Proxy URL on a Mac as been configured.
First, I want to get the port number
port=$(route get example.com | grep interface | sed 's/.*\(...\)/\1/')
Then use the port number to obtain the active network service
service=$(networksetup -listnetworkserviceorder | grep $port | sed 's/,.*$//; s/^.*: //')
And finally, I use the active network service to obtain the auto proxy info
autoproxy=$(networksetup -getautoproxyurl \"$service\")
I'm running into a problem with any network service that includes spaces. For example Wi-Fi works fine, but Apple USB Ethernet Adapter does not.
I thought the solution would be to escape the quotes (\"$service\")
The odd thing is that if I echo $service (where $service is Apple USB Ethernet Adapter) it returns a properly quoted result of "Apple USB Ethernet Adapter" If I then copy and paste this exact result as a replacement for the variable $service, I get the result I was expecting (URL: (null) Enabled: No)
However, running the command as it was originally written returns a parameter error (Error: The parameters were not valid).
This is the result of running the script as is:
autoproxy=$(networksetup -getautoproxyurl "$service")
echo $autoproxy
Error: The parameters were not valid.
However, if I copy and paste the output of $service, then it returns the result I was expecting.
service=$(networksetup -listnetworkserviceorder | grep $port | sed 's/,.*$//; s/^.*: //')
echo \"$service\"
"Apple USB Ethernet Adapter"
autoproxy=$(networksetup -getautoproxyurl "Apple USB Ethernet Adapter")
echo $autoproxy
URL: (null) Enabled: No
You are using "" for service variable in autoproxy expression.