I'm trying to learn the ropes on packet queuing, so I thought I'd set up a limitation on traffic coming into port 80 from known Tor Exit nodes. This is on FreeBSD 9, so OpenBSD-specific solutions might not apply (syntax/etc).
# Snipped to mainly the relevant parts
table <torlist> persist file "/var/db/torlist"
# ...
set block-policy return
scrub in all
scrub out on $ext_if all
# What I *want* to do is create a cue for known tor exit nodes
# no single one IP should be able to do more than 56k/sec
# but the combined bandwidth of all tor visitors should not
# exceed 512k/sec, basically limiting Tor visitors to something
# like dialup
altq on $ext_if cbq bandwidth 512k queue { qin-tor }
queue qin-tor bandwidth 56Kb cbq ( default rio )
# ...
block in log all
antispoof for { $ext_if, $tun_if }
antispoof quick for $int_if inet
### inbound web rules
# Main Jail ($IP4_PUB3 is my webserver IP)
pass in on $ext_if inet proto tcp from <torlist> to $IP4_PUB3 port www synproxy state queue qin-tor
pass in on $ext_if inet proto tcp to $IP4_PUB3 port www synproxy state
The problem is, when the altq, queue, and pass line specific for torlist are enabled, all connections are extremely slow. I've even tested my own IP against pfctl -t torlist -T test , and got back "0/1 addresses match", and if I test one from the list it's "1/1 addresses match"
So I'm not really educated in the matter of what exactly I'm doing wrong, I was assuming the pass in line with in it would only be applied to the IPs listed in that table, as such my own IP wouldn't validate on that rule and would pass onto the next one.
Getting it working isn't urgent, but any help in understanding where I'm failing would be greatly appreciated.
Turns out that I didn't quite understand how altq works. When I created a queue on my external interface with only one queue I created a default for all connections. As a result I had to define my top speed plus create a default queue for everything else.
For example if my system has 100Mb top
altq on $ext_if cbq bandwidth 100Mb queue { qin-www, qin-tor }
queue qin-www bandwidth 98Mb priority 1 cbq ( default borrow )
queue qin-tor bandwidth 56Kb priority 7 cbq ( rio )
...
pass in on $ext_if inet proto tcp to $IP4_PUB3 port www synproxy state
pass in on $ext_if inet proto tcp from <torlist> to $IP4_PUB3 port www synproxy state queue qin-tor
(doesn't need to be on top since pf parses all the rules unless you use 'quick')
In this way only those IPs matching in gets throttled down to the qin-tor queue, everything else not defined defaults to the qin-www queue.
The FAQ on OpenBSD's pf didn't seem to make this clear to me until I thought about why there would be an error for a "default", then I figured maybe it applies to the whole interface, so need to define a default for rules not marked to a specific queue.
So there it is... the solution to my 'simple' problem. Hopefully anyone else who has this problem comes accross this.
This is the FAQ I was going by for packet queueing: http://www.openbsd.org/faq/pf/queueing.html
Related
What I have discovered is that there are a number of ports on my Windows 10 box which (1) are not in use by any process and (2) I cannot listen on.
I discovered this problem trying to run a node server which used port 3000. I found a number of questions on this topic. This one is typical: Node.js Port 3000 already in use but it actually isn't?
All the respondents of this question and similar questions all suggest using "netstat -ano" to find the process which is using the port and killing it.
What I have found is that there are large number of ports blocked which are not tied to processes. This is not related to AV or firewall. I turned off the firewall and I have only Windows Defender AV.
I wrote a program to listen on the ports between 3000 and 5000 inclusive on 127.0.0.1.
int port = 3000;
while(port <= 5001)
{
try
{
ListenOnPort(port);
++port;
}
catch (Exception ex)
{
Console.WriteLine($"Listen on {port} failed: {ex.Message}");
++port;
}
}
Where ListenOnPort is...
private static void ListenOnPort(int v)
{
var uri = new UriBuilder("http", "127.0.0.1", v);
HttpListener listener = new HttpListener();
listener.Prefixes.Add(uri.Uri.ToString());
Console.WriteLine($"Listening on {v}");
listener.TimeoutManager.IdleConnection = new TimeSpan(0, 0, 1);
listener.Start();
var task = listener.GetContextAsync();
if(task.Wait(new TimeSpan(0,0,1)))
{
HttpListenerResponse response = task.Result.Response;
// Construct a response.
string responseString = "<HTML><BODY> Hello world!</BODY></HTML>";
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
// Get a response stream and write the response to it.
response.ContentLength64 = buffer.Length;
System.IO.Stream output = response.OutputStream;
output.Write(buffer, 0, buffer.Length);
// You must close the output stream.
output.Close();
}
listener.Stop();
}
The program produced output similar to this...
Listening on 3000
Listen on 3000 failed: The process cannot access the file because it is being used by another process
Listening on 3001
Listen on 3001 failed: The process cannot access the file because it is being used by another process
Listening on 3002
Listen on 3002 failed: The process cannot access the file because it is being used by another process
Listening on 3003
Listen on 3003 failed: The process cannot access the file because it is being used by another process
Listening on 3004
Listen on 3004 failed: The process cannot access the file because it is being used by another process
Listening on 3005
Listen on 3005 failed: The process cannot access the file because it is being used by another process
Listening on 3006
Listening on 3007
Listening on 3008
Listening on 3009
Listening on 3010
What I discovered is that between the ranges of 3000 and 5000, there are 624 ports which are blocked. Meanwhile "netstat -ano" shows that there are exactly 5 ports in use in that range. So what is blocking the 619 other ports?
Right...
While looking for something else I found the answer (at least the source of the problem). The reason I cannot connect to these ports is because they are all part of excluded port ranges on windows. To see the excluded ports use...
$ netsh int ipv4 show excludedportrange tcp
And there, magically, is a list of all the ports I cannot connect to. These excluded port ranges apparently originate from HyperV and Docker, both of which I have installed. There is apparently a way to get the ports back...not easy since it involves uninstalling Docker and HyperV, then reserving the port ranges for yourself and then reinstalling HyperV and Docker. Not worth it. Now that I simply know how to find the ports I cannot use, I will simply not use them!
Set the Windows "Dynamic Port Range" in a non conflicting place
We managed to contain this problem for the case where you can not change your port numbers (like a non configurable application).
When you issue the command:
netsh int ip show excludedportrange protocol=tcp
You get an output with a list of port ranges reserved:
Protocol tcp Port Exclusion Ranges
Start Port End Port
---------- --------
33474 33573
50000 50059 *
58159 58258
58259 58358
58359 58458
58459 58558
58559 58658
58659 58758
58759 58858
* - Administered port exclusions.
The most likely reason for this is Windows Hyper-V (Microsoft's hardware virtualization product) that reserves random port ranges (usually blocks of 100 ports).
This becomes a pain, because if you are developing an application or larger solution that uses multiple ports, some times you get a conflict and some times not after rebooting your system.
To find the "Dynamic Port Range" you can issue the command:
netsh int ipv4 show dynamicport tcp
The answer:
Protocol tcp Dynamic Port Range
---------------------------------
Start Port : 1024
Number of Ports : 64511
You can instruct Windows to modify this range out of the conflicting area. Let's say your development is under and up to port 60000, you can issue the following command to restrict the dynamic port range out of it (you must have administrator privilegs):
netsh int ipv4 set dynamic tcp start=60001 num=5534
To make Hyper-V (and Windows in general) use this new dynamic range you have to reboot your system.
Now if we request the excluded port range:
netsh int ip show excludedportrange protocol=tcp
The response has changed:
Protocol tcp Port Exclusion Ranges
Start Port End Port
---------- --------
50000 50059 *
63904 64003
64004 64103
64105 64204
64205 64304
64305 64404
64405 64504
64505 64604
64605 64704
* - Administered port exclusions.
Only the "Administered port exclusions" remains below port 60001
I can't seem to extend my pool for my additional users. I don't have any problem with my connection when the ip address handed is 192.168.10.xxx but when it reaches to 192.168.11.xxx to 192.168.16.xxx, it can no longer use the internet. What am I missing with my setup?
Updated:
If you want to add more IPs in your DHCP server, just increase the range of your current network. You currenlty have 253 clients (192.168.10.0/24)
Don't add more /24 networks, it's useless, just use for example 192.168.8.0/21 (range .8.1 to .15.254) to get 2046 IPs. See http://www.subnet-calculator.com/subnet.php to test various network ranges.
So I suggest this:
remove parasites /24 networks and NAT rules (192.168.11.0/24 to .16/24)
increase the range of your current network: change IP address from 192.168.10.1/24 to 192.168.10.1/21, subnet 255.255.248.0, network 192.168.8.0
change NAT/masquerade rule: src-address=192.168.8.0/21
change dhcp network range 192.168.8.0/21
change dhcp pool size with two segments: 192.168.8.1-192.168.9.254 and 192.168.10.100-192.168.15.254
and, normally, it should work
I would like to use five TCP and five UDP streams (sending and receiving) in parallel on a single host, where the UDP traffic consists of video, and the TCP traffic is arbitrary. How do I use both transport layers in parallel on the same node?
In INET 3.0.0 there is an example nclients in examples\inet directory. It could be a good start point to prepare your model.
As long as the TCP and UDP traffic is independent, you can easily install several UDP and TCP applications at the same time in the same host. Something like this:
**.cli[*].numTcpApps = 2
**.cli[*].tcpApp[0].typename = "TelnetApp"
**.cli[*].tcpApp[1].typename = "TCPBasicClientApp"
**.cli[*].numUdpApps = 2
**.cli[*].udpApp[0].typename = "UDPVideoStreamSvr"
**.cli[*].udpApp[1].typename = "UDPVideoStreamSvr"
// ... further cofiguration of the applications
Using Andrew Rapp's XBee-API, how can I sample I/O data via a coordinator from more than two endpoints?
I have 17 Series 1 XBees. I have programmed one to be a coordinator (API mode = 2) and the rest to be endpoints. Using XBee-API I am sending a Force I/O Sample ("IS") remote AT command, unicast to each endpoint. This works perfectly well when there are up to two endpoints, but as soon as a third is added, one of the three always becomes non-responsive (times out with XBeeTimeoutException). It's not always the same physical unit that stops responding, but it is always the third one (for example, if I send Force I/O Sample to Device1, Device2, and Device3, Device3 will time out, and if I change the order to Device3, Device1, Device2, Device2 will time out.
If I set up more than three XBees, about 1 out of 3 will time out - but not every third one.
I've verified that the XBees themselves are fine. I've searched the Internet and Stack Overflow in particular to no avail. I've tried using a simple ZNetRemoteAtRequest. I've tried opening and closing the XBee coordinator serial connection once for all three devices, once per device, and once per program run. I've tried varying the distance between the coordinator and endpoints (never more than five feet apart). I've tried different coordinator configuration parameters (from the Digi documentation). I've tried changing out the XBee for the coordinator.
This is the code I'm using to send the Force I/O Sample request to each endpoint and read the response:
xbee = new XBee(); // Coordinator
xbee.open("/dev/ttyUSB0, 115200)); // Happens before any of the endpoints are contacted
... // Loop through known endpoint addresses
XBeeRequest request = new ZBForceSampleRequest(new XBeeAddress64(endpointAddress));
ZNetRemoteAtResponse response = null;
response = (ZNetRemoteAtResponse) xbee.sendSynchronous(request, remoteXBeeTimeout);
if (response.isOk()) {
// Process response payload
}
... // End loop and finally close coordinator connection
What might help polling I/O samples from more than two endpoints?
EDIT: I found that Andrew Rapp's XBee-API library fakes multithreaded behavior, which causes the synchronization issues described in this question. I wrote a replacement library that is actually multithreaded and correctly maps responses from multiple XBee endpoints: https://github.com/steveperkins/xbee-api-for-java-1-4. When I wrote it Java 1.4 was necessary for use on the BeagleBone, Plug, and Zotac single-board PCs but it's an easy conversion to 1.7+.
Are you using hardware flow control on your serial port? Is it possible that you're sending requests out when the local XBee has deasserted CTS (e.g., asking you to stop sending)? I assume you're running at 115200 bps, so the XBee serial port can keep up with the network data rate.
Can you turn on debugging information, or connect some port monitoring hardware/software to display the data going over the serial port to the local XBee?
I am setting up a grid-enabled cluster. I plan to assign 2 IP to my head node: one for local connection (LAN for distributing jobs to compute nodes) and one for public (internet for user access). So, my /etc/hosts file looks something like this:
111.111.111.111 myserver.whatever.com myserver #for public IP
11.11.11.11 myserver.whatever.com myserver #for local LAN
22.22.22.22 computenode01
33.33.33.33 computenode03
My concern here is will the hostname of myserver get messed up since it is mapped to two IPs?
I fear the system will always choose the first entry (111.111.111.111) if you want to resolve "myserver" address.
It will simply ignore the second entry, as I guess. Choose different hostnames for each entry, e.g. myserver.local and myserver.remote.