I have implemented a small program which uses Net-SNMP to get network related information. It uses asynchronous mode to query multiple hosts.
Since Net-SNMPv3 does not support multi-threading, is it beneficial to shift to SNMP++?
I have a lot of hosts or routers in the network where I need to monitor the traps and general network information of ~2000 routers.
Will using SNMP++ be much faster that Net-SNMP as it uses multi-threading?
I guess you are implementing in C++ and maybe need to, but I you can use snmp4j it has the possibility to use multi-threading. Even if you have to wrap the code to using Java.
Are at the same issue, where 1-2000 units need both SNMP Get / Walks and due to slow units, there needs to be multiple threads fetching data in parallel.
Related
I am testing a custom FPGA NIC and I need to send management information (such as header info for matching) and traffic data to it using a traffic generator from within the user space.
The driver built for the FPGA is a modified version of IXGBE with DMA support for management, and also supports DPDK for kernel bypass to achieve high throughput.
I am trying to understand how the various software (driver, userspace application, etc) should be stacked/connected to each-other so I can achieve the objective of reading and writing to PCIe on the NIC using set of scripts from user space?
I have also been looking at this project
https://github.com/CospanDesign/python-pci
which is useful however based on Xilinx XDMA.
Would appreciate any help, pointers on this.
Sorry, the question is too broad. For such a broad question there is a generic answer: have a look at Inter Process Communication:
https://en.wikipedia.org/wiki/Inter-process_communication
There are variety of methods, like Unix sockets, shared memory, netlink etc, to communicate between user space processes. As well as a variety of methods to communicate between user space and kernel space.
Just pick the best for you and try to do something. If it fails, come again on SO and ask ;)
I would like to know if it is possible for a Ruby program to possess multiple IP addresses? I am trying to download a lot of data from a site, but it is very slow with only 1 connection at a time.
I intend to multi-thread my program with each thread using its own IP address, but I do not know if it is possible in the first place, any help or hints would be greatly appreciated.
It is definitely possible for a machine or a program to have multiple IP addresses. You can even have multiple network adapters, and tie each of them to different physical connections.
However, it can get really hairy to maintain. The challenge for that is partly in the code, partly in the system maintenance, and partly in the networking required to make that happen.
A better approach that you can take is to design your program so that it can run distributed. As such, you can have several copies of it synchronized and doing the work in parallel. You can then scale it horizontally (build more copies) as required, and over different machines and connections if required.
EDIT: You mentioned that you cannot scale horizontally, and that you prefer to use multiple connections from the same machine.
It's very likely that for this you'll have to go a little bit lower in the network stack, developing yourself the connection through sockets in order to use specific network interfaces.
Check out an introduction to Ruby sockets.
Also, check out these related questions:
How does a socket know which network interface controller to use?
Binding to networking interfaces in ruby
Ruby: Binding a listening socket to a specific interface
Can I make ruby send network traffic over a specific iface?
It is unbelievable that ZeroMQ uses select() on Windows, I didn't know that until I have completes my code and started performance test. They should present this information on their web site with big red font.
Is there anyway to replace ZeroMQ's select()?
IOCP is proactor model and can't be easily integrated into it, how about WSAEventSelect, this is also a reactor model and have a near performance like poll.
Another choice for me is http://nanomsg.org/, but it is still alpha.
One of the main objectives in Zeromq is to provide a consistent API for communication between threads, processes, nodes, and clusters. Protocol specific optimization is outside of this scope because of the ways that it can effect other areas of communication. For example, shared memory would be a better form of IPC, but UNIX domain sockets make a consistent API easier. It would also be nice to know when an endpoint disconnects, but how would you implement such behavior between threads?
Their main goal is to allow every pattern to work the same way regardless of topology, protocol, system, or language, to the point that any mixture can be used regardless of how odd it may seem (node.js Websockets communicating with C# brokers passing messages to Ruby and PHP workers which share work with java threads, etc.)
Each of it's features would be enhanced greatly if optimised for each specific protocol and system, but that would also make uniform patterns close to impossible.
BTW, they might accept a pactch if you could find a way to implement iocp while still maintaining this versatility and neutrality.
PPS, nanomsg is made by one of the main original developers of Zeromq. Crossroads.IO is a direct fork of Zeromq, by original Zeromq developers as well and including some developers of nanomsg. if I'm not mistaken, Nano will likely become the core of crossroads when complete.
I am curious about whether my server would work better on Linux or Windows, from what I have read Windows only supports around 2,000 connections/threads while I have not seen much information about how many threads / connections Linux can handle.
Is there any advantages to using Linux over Windows other than stability / security for my TCP /IP server?
Thanks.
Threads and sockets are different resources, the limits for each will depend not just on Linux vs Windows but also which versions of each OS you are using. Also, if you're using a class library instead of raw socket or thread APIs, those might impose a specific limit. As an example early versions of CSocket in MFC created a hidden window for each socket, so you were effectively limited to the number of GDI resources on the system.
Either platform will be fine, and most apps will never get big enough to need more than a single server to run them anyway. Get your project done in whichever way is easier for you.
I would imagine that the primary concern when building a high-scale application is the experience of the engineers on your team, including operations engineers. By all means consider performance when selecting a platform, but the experience and preference of your development and operations engineers is probably more important - after all, they will need to maintain and operate the service respectively.
In any case, if you have a real need for a service with 2000 concurrent clients, it probably has some high availability requirement which means it can't be run on a single server anyway.
I'm developing an application where, to satisfy the performance requirements, tuning of low-level network stuff (such as TCP window size etc) seems to be required.
I found the magnitude of my knowledge to be a bit better than "there's TCP and there's UDP", which is far from enough for this task.
What resources might I study to get a better knowledge of which aspects of TCP influence which performance characteristics in which usage scenarios (for example, how to decrease latency while transmitting 100kb packets to 1000 clients simultaneously on a 10gbit LAN), etc.?
What tools might help me? (I already know about Wireshark, but most probably I am not using it to its full potential)
First understand what you're doing:
http://en.wikipedia.org/wiki/Transmission_Control_Protocol
http://www.tcpipguide.com/free/t_TCPWindowSizeAdjustmentandFlowControl.htm
Then understand how to look at what you're about to change:
http://www.wireshark.org/
http://wiki.wireshark.org/TCP_Analyze_Sequence_Numbers
http://www.microsoft.com/downloads/details.aspx?familyid=983b941d-06cb-4658-b7f6-3088333d062f&displaylang=en
Then understand how to change things:
http://msdn.microsoft.com/en-us/library/ms819736.aspx
http://blogs.msdn.com/b/wndp/archive/2007/07/05/receive-window-auto-tuning-on-vista.aspx
Since you're currently on a windows platform and you mention that you're "developing an application"... You probably also want to know about I/O Completion Ports and how you maximise data flow whilst conserving server resources using write completion driven flow control. I've written about these quite a bit and the last link is a link to my free high performance C++ client/server framework which may give you some pointers as to how to use IOCP efficiently.
http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html
http://www.lenholgate.com/blog/2008/07/write-completion-flow-control.html
http://www.serverframework.com/products---the-free-framework.html
And google is your friend...