Yesterday, I received a pac file which defines only proxy but not port number,
like this one:
function FindProxyForURL(url, host) {
if (shExpMatch(host, "vpn.domain.com"))
return "PROXY proxy.mydomain.com";
}
Then I used
ipconfig /displaydns
netstat -n
cmds to test this pac file, found that,
[proxy.mydomain.com] always uses port 80 in either HTTP or HTTPS.
My question is, why does it use port 80, is it some kind of default definition?
Related
Objective is to extract the ip address from the 11th column and feed it to whois.
The sourcefile it's first line are the headers so they should be ignored.
then I try to select with awk the 11th column.
Since skipping first line seems to be too hard (for me) right now I left it out for now. Any good suggestion is welcome.
The code so far:
while IFS= read -r p
do
DESTIP=$(awk 'BEGIN{FS=OFS=";"} {print $11}' $p)
echo "$DESTIP; $p"
ORGNAME=$(whois $DESTIP|grep 'OrgName')
COUNTRY=$(whois $DESTIP|grep 'Country')
echo "$p;$ORGNAME;$COUNTRY" >>whois-results.txt
done < working-sorted.csv
The first lines of the sourcefile:
timestamp (UTC);ID;Threat Level;Category;Exporter IP address;Observation domain ID (ODID);Source MAC;Manufacturer;Source IP;Source Port;Destination IP;Destination Port;Protocol;Description
2020-03-14 13:54:10;20810;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;118.25.123.42;49420;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short fo
r Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 118.25.123.42 has made a TCP connection towards the dest
ination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we suggest
to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monitore
d network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist
2020-03-14 13:53:45;20809;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;144.217.92.167;55134;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short f
or Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 144.217.92.167 has made a TCP connection towards the de
stination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we sugges
t to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monito
red network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist
Result for now:
awk: cmd. line:1: fatal: cannot open file `2020-01-19' for reading (No such file or directory)
DESTINATION IP=
Variable P= 2020-01-19 20:42:56;43;3;Remote Administration Tool;::ffff:ac8:c8d0/128;101;00:0c:29:4c:20:37;Vmware;172.16.16.100;54552;52.174.64.84;443;TCP;Connection to blacklisted destination
After adjusting the awk to:
DESTIP=$(awk -v TEST='$p' 'BEGIN{FS=OFS=";"} {print $9;}')
I do get the ip addresses from the correct column, but the are in one list and not line by line, nor passed to the whois commands
Desired output:
timestamp (UTC);ID;Threat Level;Category;Exporter IP address;Observation domain ID (ODID);Source MAC;Manufacturer;Source IP;Source Port;Destination IP;Destination Port;Protocol;Description;OrgName;Country;
2020-03-14 13:54:10;20810;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;118.25.123.42;49420;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short fo
r Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 118.25.123.42 has made a TCP connection towards the dest
ination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we suggest
to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monitore
d network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist;SomeName;SomeCountry
For now I'm a bit stuck.
Help would be appriciated.
Consider this approach instead of your shell loop:
$ cat tst.awk
BEGIN {
numFlds = split("OrgName Country",nr2name)
FS=OFS=";"
}
{ delete name2val }
NR == 1 {
for (fldNr=1; fldNr<=numFlds; fldNr++) {
fldName = fldVal = nr2name[fldNr]
name2val[fldName] = fldVal
}
}
NR > 1 {
cmd = "whois \047" $9 "\047"
while ( (cmd | getline line) > 0 ) {
fldName = fldVal = line
sub(/[[:space:]]*:.*/,"",fldName)
sub(/[^:]+:[[:space:]]*/,"",fldVal)
name2val[fldName] = fldVal
}
close(cmd)
}
{
printf "%s%s", $0, OFS
for (fldNr=1; fldNr<=numFlds; fldNr++) {
fldName = nr2name[fldNr]
fldVal = name2val[fldName]
printf "%s%s", fldVal, (fldNr<numFlds ? OFS : ORS)
}
}
.
$ awk -f tst.awk file
timestamp (UTC);ID;Threat Level;Category;Exporter IP address;Observation domain ID (ODID);Source MAC;Manufacturer;Source IP;Source Port;Destination IP;Destination Port;Protocol;Description;OrgName;Country
2020-03-14 13:54:10;20810;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;118.25.123.42;49420;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short for Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 118.25.123.42 has made a TCP connection towards the dest ination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we suggest to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monitore d network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist;;
2020-03-14 13:53:45;20809;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;144.217.92.167;55134;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short for Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 144.217.92.167 has made a TCP connection towards the de stination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we sugges t to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monito red network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist;OVH Hosting, Inc.;CA
since it gives you exactly the output you wanted, it won't fail when Country, for example, appears in one of the values (your current shell script will fail due to a false match given a company name of "Big Country", for example), and with that you can access any of the values output from whois by just referring to their name. So if you wanted to additionally print the "OrgAbuseEmail" all you have to do is change this:
numFlds = split("OrgName Country",nr2name)
to this:
numFlds = split("OrgName Country OrgAbuseEmail",nr2name)
Alternatively, this avoids spawning a shell once per IP address and so MAY be a bit more efficient than the above:
$ cat tst.sh
#!/bin/env bash
file="$1"
awk 'BEGIN{FS=OFS=";"} {print $9, $0}' "$file" |
while IFS=';' read -r ip all; do
whois "$ip"
printf '%s\n---\n' "$all"
done |
awk '
BEGIN {
numFlds = split("OrgName Country",nr2name)
for (fldNr=1; fldNr<=numFlds; fldNr++) {
fldName = nr2name[fldNr]
name2val[fldName] = fldName
}
FS = OFS = ";"
}
/^[[:alpha:]]+:/ {
fldName = fldVal = $0
sub(/[[:space:]]*:.*/,"",fldName)
sub(/[^:]+:[[:space:]]*/,"",fldVal)
name2val[fldName] = fldVal
}
/^---$/ {
printf "%s%s", prev, OFS
for (fldNr=1; fldNr<=numFlds; fldNr++) {
fldName = nr2name[fldNr]
fldVal = name2val[fldName]
printf "%s%s", fldVal, (fldNr<numFlds ? OFS : ORS)
}
delete name2val
}
{ prev = $0 }
'
.
$ ./tst.sh file
timestamp (UTC);ID;Threat Level;Category;Exporter IP address;Observation domain ID (ODID);Source MAC;Manufacturer;Source IP;Source Port;Destination IP;Destination Port;Protocol;Description;OrgName;Country
2020-03-14 13:54:10;20810;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;118.25.123.42;49420;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short fo r Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 118.25.123.42 has made a TCP connection towards the dest ination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we suggest to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monitore d network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist;;
2020-03-14 13:53:45;20809;5;Ingress Traffic;::ffff:ac8:c8d0/128;101;00:1a:8c:f0:c2:c0;Sophos;144.217.92.167;55134;172.16.16.150;22;TCP;Ingress connection to common SSH port: 100% CertaintyHigh Severity Category: SSH Description: Short f or Secure Shell Description: This connection represents an encrypted channel (SSH), which is commonly used in IT environments to connect to remote machines. Observations: Source IP 144.217.92.167 has made a TCP connection towards the de stination IP 172.16.16.150 (Private) on destination port 22. Advice:We recommend to investigate the following conditions: 1) Verify if it is expected for your network environment to generate SSH connections. If it is expected, we sugges t to disable this category. A network where developers and sysadmins often host their machines is an example of a network where a significant amount of SSH connections is expected. 2) If you are not expecting SSH traffic from the monito red network, it is recommended to investigate the endpoint according to your company security policies. If the destination 172.16.16.150 is trusted, it is recommended to add that SSH destination IP to the whitelist;OVH Hosting, Inc.;CA
I found a workaround. maybe not so nice, but at least working:
(edit: the workaround is now a bit nicer and less "workaround" :)
The code below is adjusted on advice of Cyrus.)
while IFS= read -r p
do
DESTIP=$(echo "$p" | awk 'BEGIN{FS=OFS=";"} {print $9;}')
echo "DESTINATION IP= $DESTIP"
echo "Variable P= $p"
ORGNAME=$(whois $DESTIP|grep 'OrgName')
COUNTRY=$(whois $DESTIP|grep 'Country')
echo "$p;$domain;$ORGNAME;$COUNTRY" >>working-whois.csv
done < working-sorted.csv
Thanks to Cyrus in this by making clear it needs an inputfile.
I have setup openvpn client on a asus router, it is running padavan firmware, which is similar to tomato and other.
The VPN client works, but I would like to limits it's use to one or 2 ips on my LAN (i.e. AppleTV) and all other clients bypass the VPN connection.
The padavan vpn client has a custom script that is executed with the interface goes up and down on tun0 which is the interface.
I have attempted to route the IP address of the client that I want to use, but it does not prevent access via all of the other clients:
#!/bin/sh
### Custom user script
### Called after internal VPN client connected/disconnected to remote VPN server
### $1 - action (up/down)
### $IFNAME - tunnel interface name (e.g. ppp5 or tun0)
### $IPLOCAL - tunnel local IP address
### $IPREMOTE - tunnel remote IP address
### $DNS1 - peer DNS1
### $DNS2 - peer DNS2
# private LAN subnet behind a remote server (example)
peer_lan="192.168.0.130"
peer_msk="255.255.255.253"
### example: add static route to private LAN subnet behind a remote server
func_ipup()
{
# route add -net $peer_lan netmask $peer_msk gw $IPREMOTE dev $IFNAME
# route add -net $peer_lan gw $IPREMOTE dev $IFNAME
route add default dev tun0 table 200
rule add from 192.168.0.130 table 200
return 0
}
func_ipdown()
{
# route del -net $peer_lan netmask $peer_msk gw $IPREMOTE dev $IFNAME
return 0
}
logger -t vpnc-script "$IFNAME $1"
case "$1" in
up)
func_ipup
;;
down)
func_ipdown
;;
esac
I realise that this is very specific to the padavan firmware, but I think that the commands that are executed when it goes up should be universal, and my routing skills are very limited !
Maybe I need to block / allow using ip tables instead?
Any suggestions or help gratefully appreciated !
I'm using WFPSampler to redirect all traffic to a specific interface by using command:
WFPSampler.exe -s PROXY -l FWPM_LAYER_ALE_BIND_REDIRECT_V4 -pla 10.0.2.15 -v -in
This works just fine, traffic from all of the processes is redirected as expected. The only problem is that it binds 127.0.0.1 to 10.0.2.15 as well and then some applications fail to connect.
For example, I've created simple Python HTTP server on 127.0.0.1:8000 and I can not access it over the browser using this address.
I know that on FWPM_LAYER_ALE_BIND_REDIRECT_V4 it is only possible to filter by local address, but I have somehow to filter by remote address at this point to avoid binding localhost to 10.0.2.15
You could redirect the outgoing traffic from 10.0.2.15 to 127.0.0.1 at the same time, with the command like:
WFPSampler.exe -s PROXY -l FWPM_LAYER_ALE_CONNECT_REDIRECT_V4 -ipra 10.0.2.15 -pra 127.0.0.1 -v -in
The comments of this answer has metioned it.
This ist my current setup:
Host config:
define host{
use generic-host ; Inherit default values from a template
host_name A+A ; The name we're giving to this host
alias A+A Objektausstattung Router ; A longer name associated with the host
address https://87.139.203.190:444 ; IP address of the host
hostgroups Router ; Host groups this host is associated with
}
Service config:
define service{
use generic-service ; Inherit default values from a template
host_name A+A
service_description HTTP
check_command check_http
}
I´ll get this error from Nagios:
check_icmp: Failed to resolve https://87.139.203.190:444
What am I doing wrong here ?
Nagios tries to resolve to ip-address and port. Try ip-address only.
address https://87.139.203.190 ; IP address of the host
Your host definition should only specify an IP address for the 'address'. The URL is not an attribute of the host, but of the HTTP check your want to perform.
The Service definition specifies the check_command, which is in turn defined in the checkcommands.cfg file. This will specify exactly what command is to be run, possibly using additional parameters passed.
You will probably want to pass the port number as a parameter, and that you are to use HTTPS. How to do this will depend on your settings. For example, you could use this in your checkcommands.cfg:
define command{
command_name check_https
command_line $USER1$/check_http -t 12 -H $HOSTADDRESS$ -f ok --ssl=1 -u "$ARG1$" -p "$ARG2$" -w $ARG3$ -c $ARG4$
}
Then you could configure your service with a checkcommand thus:
check_command check_https!/!444!1!5
This would check for the url http://87.139.203.190:444/, giving a warning if it takes over 1s and a critical if it takes over 5s to complete. TLSv1 would be used (else you might get a false positive on web servers with Poodle protection).
I am using LFTP to transfer files from a server, which unfortunately does not recognize the PORT command. I do not have control over the server (do not know in detail what server is) and I have to use the active mode.
This is the command line as:
lftp -e 'debug 10;set ftp:passive-mode off; set ftp:auto-passive-mode no; ls; bye;' -u user,password ftp://ftp.site.com
This is the debug output:
<--- 200 Using default language en_US
---> OPTS UTF8 ON
<--- 200 UTF8 set to on
---> OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
<--- 200 OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
---> USER xxxxx
<--- 331 Password required for xxxxx
---> PASS xxxxxx
<--- 230 User xxxxx logged in
---> PBSZ 0
<--- 200 PBSZ 0 successful
---> PROT P
<--- 200 Protection set to Private
---> PORT 172,16,133,11,146,168
<--- 500 Illegal PORT command
---> LIST
---> ABOR
---- Closing aborted data socket
---- Chiusura del socket di controllo
It seems that LFTP renounces to connect to data socket because the remote server does not support the PORT command. Is there a way to convince LFTP can still connect to port 20? By FTP manual obviously no problem.
The issue, I think, is not that the FTP server doesn't support the PORT command (it does), but rather, it doesn't like the IP address/port that your FTP client is sending in the PORT command.
PORT 172,16,133,11,146,168
...tells the server to connect to address 172.16.133.11, port 37544*. The interesting part here is the IP address: it's an RFC 1918 address (i.e. it's a private network address). That, in turn, suggests that your FTP client is in a LAN somewhere, and is connecting to an FTP server using a public IP address.
That remote FTP server cannot connect to a private network address; by definition, RFC 1918 address are not publicly routable.
Thus it very well could be that the FTP server is trying to make a connection to the address/port given in your PORT command, fails, thus that is why the FTP server fails the command, saying:
500 Illegal PORT command
To make a PORT command work with that FTP server, you would need to discover the public IP address that that server can connect to, to reach your client machine. Let's say that this address is 1.2.3.4. Then you would need to tell lftp to use that address in its PORT command, using the ftp:port-ipv4 option.
Chances are, though, that public IP address is the address of a NAT/router/firewall, and that that NAT/router/firewall will not allow connections, from the outside world to a high numbered port (e.g. 37544), to be routed to a machine within the LAN. This is one of the issues with active FTP data transfers, i.e. FTP data transfers which use the PORT (or EPRT) commands: they are not considered "firewall-friendly".
Hope this helps!
* - why 146,168 translates to port 37544?
According to FTP's RFC959 those parameters are:
(...) 16-bit TCP port address. This address information is broken into
8-bit fields and the value of each field is transmitted as a decimal
number (in character string representation).
146 dec = 10010010 bin = A
168 dec = 10101000 bin = B
A B
10010010 10101000 bin = 37544 dec