I've been having a very confusing discussion with a colleague about the use of Network Direct in Windows to provide RDMA, and Linux where we have a working implementation using the verbs api and rdmacm.
Said collegue developed his Network Direct implementation after he couldn't get the verbs one to work. I did the rdmacm implementation under Linux. We both got them talking linux=>linux and windows=>windows but never got around to testing Linux=>windows or windows=>linux. During the development there was some hint of it possibly working, they appeared to start talking but of course things were still in development.
The project was put on hold for months and the framework that it was built into has had significant redesign. I'm left with pulling what was working from the Windows side into the new framework and I'm having a hard time getting it working or seeing any part of it connect from Linux.
My understanding is that at the lower layer they will both talk verbs. But I'm now not so sure with colleague saying things like "Remember this is not verbs", just to add to the confusion.
So the question is. will an application that uses rdmacm be able to establish a connection to an app that uses Windows NetworkDirect interface or are the two completely different and incompatible?
According to Mellanox support, Network Direct should be compatible with InfiniBand verbs. We've had good luck doing RDMA transfers using rdmacm Linux-Linux and Windows-Linux. We used the windows libraries available at open fabrics alliance (look for ofed_sdk). It was not easy to figure out what libraries to get in order to do the Windows part.
However, Microsoft does not want people to use the ibverbs approach on MS-Windows any more. If you are having trouble getting ND on Windows to talk to Linux, are you sure that you are putting the values in network order before shipping them over?
Related
What follows is really a learning exercise and not necessarily a search for a production solution. I've recently done a bunch of reading about Windows driver development and am looking for a first good exercise in practical application before potentially proceeding toward a future win10 mobile device family project.
Just to get my feet wet with win10 driver development, I was thinking it would be interesting to develop (and then publish) a filesystem driver project that implemented a dumb Windows equivalent of /dev/null or /dev/random - really, any virtual device that I can read a stream of data from.
I haven't done much Windows development in the past, but after reading through a couple books on Safari I've landed on MS's driver samples GitHub page, which seems like a good start, but doesn't seem to provide a clear way forward from what I've read so far.
Right now I'm still casting about with web searches and would sure appreciate some guidance in how to proceed toward this goal (references, reading materials, etc) I'll be happy to publish whatever silly project(s) I wind up generating for others to learn from in the future.
One of the best way to start Windows driver development is with toaster sample . It will provide you basic workable understanding of Windows driver development. You will be able to understand basics for writing bus driver, function driver and filter drivers(upper, lower) in Windows.
I'm new to SNMP. I have to develop a manager. I've been searching on how to program for a manager. I've come across WinSNMP, SNMP++ and Net-SNMP. The plus point for Net-SNMP is that it can translate the trap content from the MIB (it can parse the MIB). But would Net-SNMP be a good choice of library for developing a manager? I was wondering if it would be a good idea to use snmptrapd as a basis to develop a manager. At the moment I'm focusing on receiving traps but eventually I'll have to program for the others too (Get, Set, ...).
What is the general outline on how to program for a manager? I have been reading on SNMP for weeks but still haven't got a clear understanding of how to go about doing it.
Any help would be much appreciated. Thanks.
Which toolkit you want will greatly depend on your final-end-goals and preferred language (eg, C vs C++). Make sure you get one with SNMPv3 support, which the native windows library doesn't have (but mg-soft and snmp research's implementation does). You'll likely have to pay money, eventually, if you want SNMPv3 support on windows using the winsnmp APIs.
Net-SNMP and SNMP++ are both well respected open libraries. Net-SNMP has pretty much cornered the market in terms of deployment and comes pre-installed or easily available on every OS except windows (where it also works fine, but MS just doesn't package it for you).
If you want to receive traps, Net-SNMP's snmptrapd is certainly an easy place to start and offers some very simply ways to get started through extensible commands (see the snmptrapd.conf man page to help you get started) or via C (see the apps/snmptrapd*.c files for some example C-registrations).
To get started with Net-SNMP and programming outgoing management operations, you should start with the tutorials about writing management applications. If your management application is ever going to talk to multiple hosts at once, make sure you get a toolkit that does asynchronous support (see the second item in the above link).
There is also JDMK 5.1 API from Sun microsyatem, it is very easy to develop the SNMP Agent as well Manager.
I would like to find a good (up-to-date) Java Trail type beginning tutorial on Websockets, with both Javascript client code and Java server code (that actually works). On the server side, if all I have to do is create a simple socket server or something like that, please tell. I have so far found nothing (although I've googled for hours) that provides the essential basic information like this (except on the client side, which is apparently very easy).
I'm not interested in using existing Websocket server systems (other than starting with a simple copy-paste from a simple tutorial). If you're interested, I'll tell you why below.
I have an open-source project that's been begging for HTML5 since long before it began (from its initial conception many years ago). Getting the two-way communication from Websockets as opposed to long-polling, forever-frames etc. looks like a winner to me. The current version of the (framework) software is HLL "light" - with my typical emphasis on not dragging in extra code. It uses Java SE only, although I intend to build version(s) using EE communications. I can at present easily install the whole framework with room to spare for application components on small devices like cell phones.
I want minimum code.
I pride myself on understanding all the code in the system.
I think it's death to a new open-source project if you have to tell people who want to try it that they have to download and install all sorts of other open-source code to do it.
I don't want my code subject to changes in somebody else's code.
This really shouldn't be all that difficult - if I just find the right documentation.
UPDATE: I no longer need a simple tutorial on websockets. I'm well on my way to having built a production class websocket server and have a very nice browser-based client to use for testing. I am now building a non-browser-based client to go with it. Building a multi-client websocket server that implements the full standard isn't simple, btw. (In hindsight - it's no wonder I didn't get any answers.) And if it's going to work as a stand-alone server for browser-based clients, it needs http (to provide the web pages at least) and other stuff. Not a project for the faint of heart or someone with little time to put into it.
Since first posting, I've been building my own WebSocket server. Dreams that all this would be simple quickly passed. I am however, getting close enough to mention the word "release" as in free Beta for anyone who wants a websocket server. (Notice I'm just "mentioning" the word - I'm still at least a couple of weeks away.) It's a multi-user server that's light-weight and screamingly fast. I'll eventually add http support so no other server will be needed for delivery of web pages (WebSockets are upgraded http connections anyway). I'll also be adding HLL functionality. First thing about that is that it will simplify application. Even though I've worked consistently to maintain "light-weight" status, this whole combination has been a life's ambition and I believe developers will find it amazing for building anything from simple and direct to large scale complex, distributed, intelligent systems.
I've used the most recent version of the standard, which is the final draft version, and will probably be very close if not the same as the final standard; version 1. So far, it works with Chromium (Chrome dev) and Firefox. There is however, apparently some code in their WebKit that provides support for other browsers. I haven't tried that yet. With the browser, I've so far made the connection and pass text messages (large and small) back and forth. That's all I've discovered so far that the browser handles. I take it support for very large text "messages" would really only be used for file transfer in conjunction with HTML5 local storage capabilities. I haven't tried that yet either. Binary transfers will likely be connected to HTML5 streaming video and audio at some point, but I haven't run across any mention of it so far.
I'm currently working on a Java client that will support the full standard, including pings and binary transfers, file transfers - the whole ball of wax. Most of what I need has already been built for the server and its "echo" application which includes message sending just like a client. I will then revisit some of my earliest work to generalize in a nice way when adding the new functionality. Having one or more Java websocket clients in applications will be a good thing in many cases, I think. And it'll give the the full capabilities for web browsers as well, whenever they're ready for it.
I also know that Opera is quite popular among HTML5 enthusiasts. I actually started using the version of the standard that they support so it shouldn't be difficult to add it. (Just fooling around trying to find my way - and got the stuff specific to that version working.) I'm seriously tempted to spend the time putting the support in for that version even though it should become totally obsolete within a few months. It's just that Opera is very good on the other HTML5 stuff and there's a large crowd of enthusiasts using it. On the other hand, they mostly work on the browser-client side, and I don't know if they'll get involved much on building server-side applications.
Blog articles related to my work on WebSockets.
WebSocket Server Demontration, SEPTEMBER 26, 2011
WebSocket Demonstration on Microsoft Internet Explorer, NOVEMBER 16, 2011
WebSockets with Apache Tomcat and HLL, JANUARY 12, 2013
I am developing socket reading on an ActiveX EXE (i.e on a seperate thread).
How many sockets i can safely read independently?
I am working on windows XP OS.
I think this might be an operating system limit - I think I remember running up against a limit of 80 sockets on the XP machine I was using several years ago.
I would recommend that you abandon your effort and go with a commercial solution. I remember going down this path back in the 90s and running into a brick wall with ActiveX EXEs as far as threading goes. The thing is that ActiveX EXEs are apartment threaded, not free-threaded, so you don't get completely independent threads.
And doing server side threads properly is hard enough in modern languages, let alone ones that weren't designed for this purpose.
I ended up purchasing Server Sockets from Dart. Easily the best investment for that project. The performance is truly great - you are only limited by the system resources.
MSWINSCK.OCX is a very old way of doing things; it came with Visual Basic 6.0 and i remember using it way back when. i'm not sure the licensing on it... apparently it registers fine under 32-bit win7, but not 64-bit; here's a link to how to get it to register on 64-bit systems: http://angrybyte.com/windows-hacks/mswinsck-ocx-for-64-bit-windows-7-vista/
if you have an MSDN subscription or similar that gives you the ability to download the developer tools (bizSpark, etc. will do it too) then i believe that will also give you a license to redistribute the .ocx.
(btw, i don't actually remember the interface, but i seem to remember it being at least slightly more intuitive than the berkeley socket() interfaces.)
however, personal recommendation given your requirements: learn the APIs, there are lots of examples out there, and just write yourself a class that encapsulates them in a similar way as, say, the .NET Socket class... the APIs aren't that hard and i'm sure there's lots of help to be had here as well, and that's probably better than relying on something that's out-of-date like the control...
I don't mean for this question to be a flame bait but I'll be using Microsoft and their win32 API as a example of a legacy API.
Now what I am wondering here is Microsoft is spending a lots of their money and energy in maintaining their legacy API, including all of the "glitches/bugs/workaround" that are needed to keep the API functioning the same. Now I'm aware that in Windows 7 they are providing a way for the customer to run their application in a "Windows XP" VM which would be one such way for them to start cleaning up their win32 API because they could then push all of the application into the "Windows XP" VM.
So now what I am wondering is, is it possible to virtualization a legacy API in such way that an customer/program can still access and use it, yet at the same time be able to take advantage of the newer version/API? Because as far as I understand it, if the application is ran in the "Windows XP" VM, it won't be able to access any of the newer API/feature of Windows 7.
The thing that puzzles me about this question when it comes up is that Windows has been doing this since NT came out in the mid nineties. This is how NT runs DOS and Win16 programs, and how it always has. The NTVDM virtualization layer runs 16-bit apps under Win32 with very little special support from the core OS. This is just one example - another is WINE, which as I understand it does a pretty reasonabl job of running windows apps on top of an API set which is very different from that of windows. So it is definitely possible.
The more pertinent question would be why Microsoft would consider it. In order for you to think it is necessary you have to think two things. 1) There is something better to replace the win32 API with and 2) Maintaining the Win32 API is a burden.
Both of these are questionable. In the case of kernel duties, such as accessing hardware and synchronizing and doing threads and processes and memory the Win32 API does a pretty good job, and is ultimately quite close to what the kernel really does. If you think there is a better API then that must mean there is also a better kernel. I personally don't think that NT needs replacing right now. For graphics and windowing, admitedly gdi32 is a bit long in the tooth. But Microsoft solved that problem by building WPF right alongside it. This then brings in the burden question. Well, sure there are two APIs to maintain, but if you virtualized GDI on top of WPF you'd still have to maintain both anyway so there is no benefit there. The advantage of running both in parallel is that GDI already exists and is already tested. All you have to do is to fix the occasional bug, whereas a new virtualization layer would have to be written and tested all over again, which takes time away from making WPF better.
In terms of maintaining back compat, that isn't as much of a burden as it sounds. It is mainly a test question - you have to test that the API behaviour doesn't change, but again - those tests have already been written, so it isn't really any extra work.
So, to answer a question with a question, why would they bother?
This is an interesting question, at least to me, here are some of my thoughts.
Your understanding is correct, an application running in the XP VM only has access to the Win32 APIs provided by XP in the VM. One of the many ways that I have seen Microsoft's approach to enhancing specific APIs is to create new functions with the enhanced/fixed functionality and name the new function by append Ex and even ExEx to the original name, for example
GetVersion
GetVersionEx
For functions that accept pointers to structures, the structures are 'versioned' by using the size of the structure to determine the functionality required, so older code would be passing a previous size of the structure while newer code would be passing in the newer larger strucure and the API functions accordingly.
I guess, the problem has become that it is no longer just differences in how an API works, but more integral to the functioning of the operating system and the internal structures which have changes significantly enough that arguably badly written code is effectively broken.
As to your actual question, I guess it would be quite tough. Even if one thought to let the OS adjust how it executes code based on a target OS version in the PE header of the executable, what would happen if a newer DLL was loaded into the process that targeted the latest OS, now how should the OS handle this when the code is executing? IMHO, I think this would be very challenging, one frought with pitfalls that would ultimately fail.
Of course that is just my raw thoughts on the topic so I might be 100% wrong and there is some simple approach that just did not come to mind.