In its simplest form, COM allows you to instantiate C++-like classes from DLL in your application. Basically it's a glorified wrapper around LoadLibrary and some conventions regarding the interface. This is called using an in-process component.
But COM also supports out-of-process components. If you instantiate a class from such a component, COM starts a new process. Your objects live in said process, and are marshalled transparently over to you, so you don't care too much about where they live. They might even be on a different computer (DCOM). You can also fetch objects from already running applications. A well-known example is controlling MS Office via a script. This is called Automation (formerly OLE Automation, and there is a bit of confusion around what exactly this term encompasses).
There are a couple of nice articles explaining how (in-process) COM works low-level (e.g. COM from scratch. I'd like to know how it works when your component is out-of-process. Especially, what IPC does COM use beneath the hood to communicate between the processes? Window messages, shared memory, sockets, or something else? MSDN lists COM as an IPC method by itself, but I'm guessing it has to use something else underneath. Are different IPC methods used in different cases (instantiating an OOP component from C++, accessing an Excel document from VBScript, embedding a document in another via OLE)? It seems like it is all the same underlying technology. And lastly, how does marshalling fit in the picture? I believe it is neccessary to serialize method parameters for transmitting between processes, correct?
According to this MSDN article, it's RPC.
When you instantiate an OOP component, the COM subsystem generates an in-process proxy. This proxy is responsible for packing parameters and unpacking return values. It also generates a stub in the server process, which, expectably, unpacks parameters and packs return values.
Interestingly enough, the whole marshaling process can be customized, by implementing IMarshal.
DCOM was originally added as an extension to COM, precisely for cross apartment calls. Note cross apartment calls are not always from process to process. A process can have many apartments (0 or 1 MTA and/or 0 to n STAs, etc.) . There is at least one apartment per process, etc.
DCOM, some kind of a "middleware", needed a technology for all this low-level work: data representation, caller/callee convention, memory management, wire marshaling, session handling, security, error handling, etc. so Microsoft naturally used the in-house implementation of DCE/RPC: MSRPC. Note that as Microsoft says on its site,
"With the exception of some of its advanced features, Microsoft RPC is
interoperable with other vendors’ implementations of OSF RPC."
There was some tentative work to have all this implemented by other vendors, but they were basically killed by the rise of the internet and HTTP.
Also, note this RPC uses Windows Messages for STA apartement messages. I suggest you read carefully this document (not available any more on Microsoft site, shame on them :-) for more details:
DCOM Architecture by Markus Horstmann and Mary Kirtland - July 23, 1997 .
See also this interesting case study about a DCOM/RCP issue that should tell you a lot of how RPC over Windows message works under the scene: Troubleshooting a DCOM issue: Case Study
Related
I need to do something relatively simple, and I don't really want to install a MOM like RabittMQ etc.
There are several programs that "register" with a central
"service" server through TCP. The only function of the server is to
call back all the registered clients when they all in turn say
"DONE". So it is a kind of "join" (edit: Barrier) for distributed client processes.
When all clients say "DONE" (they can be done at totally different times), the central server messages
them all saying "ALL-COMPLETE". The clients "block" until asynchronously called back.
So this is a kind of distributed asynchronous Observer Pattern. The server has to keep track of where the clients are somehow. It is ok for the client to pass its IP address to the server etc. It is constructable with things like Boost::Signal, BOOST::Asio, BOOST::Dataflow etc, but I don't want to reinvent the wheel if something simple already exists. I got very close with ZeroMQ, but non of their patterns support this use-case very well, AFAIK.
Is there a very simple system that does this? Notice that the server can be written in any language. I just need C++ bindings for the clients.
After much searching, I used this library
https://github.com/actor-framework
It turns out that doing this with this framework is relatively straightforward. The only real "impediment" to using it is that the library seems to have gotten an API transition recently and the documentation .pdf file has not completely caught up with the source. No biggie since the example programs and the source (.hpp) files get you over this hump. However, they need to bring the docs in sync with the source. In addition, IMO they need to provide more interesting examples on how to use c++ Actors for extreme performance. For my case it is not needed, but the idea of actors (shared nothing) in this use-case is one of the reasons people use it instead shared memory communication when using threads.
Also, getting used to the syntax that the library enforces (get used to lambdas!) if one is not used to state of the art c++11 programs it can be a bit of a mind-twister at first. Then, the triviality of remembering all the clients that registered with the server was the only other caveat.
STRONGLY RECOMMENDED.
Microsoft RPC provides an IPC mechanism that can be done in a function-calling manner. This has been extremely helpful for my project where my main service delegates tasks to a child process, and functions in the child process can be called as if they were implemented in the main service. That takes away the burden of having to serialize abstract data and define custom protocols when using other IPC mechanisms such as named pipes, sockets, protobuf, etc. I'm aware that RPC does use them internally.
I've read an article on implementing COM for Mac OS X which is probably the closest thing to what I need. If I find no other no other way of implementing the type of IPC I need, I'm probably going to go with COM, but I thought I'd make sure that I'm not missing anything.
Have a look at "XPC Services". From the documentation:
XPC services are managed by launchd and provide services to a single
application. They are typically used to divide an application into
smaller parts. This can be used to improve reliability by limiting the
impact if a process crashes, and to improve security by limiting the
impact if a process is compromised.
And later in that guide:
The NSXPCConnection API is an Objective-C-based API that provides a
remote procedure call mechanism, allowing the client application to
call methods on proxy objects that transparently relay those calls to
corresponding objects in the service helper and vice-versa.
What is the best way for multiple client programs to
communicate with a single server program, all running
on a single Windows computer? All written in VB6.
I'd appreciate recommendations of how you might solve
this problem.
NOTE: we are working on transition to .NET, but have to
add a capability to the V6B version before the .NET will
be ready.
The possibilities include TPC connections, named pipes,
shared memory, messages, files, and more.
A client passes the server a string as input, and the server
combines it with data known only to the server, to generate
another string which is returned to the client. Both strings
are only about 100 characters long. The server is contacted
only when a new file needs to be opened, and so it is a very
low volume of communication... probably a flurry of 10 calls
within 15 seconds, followed by an hour of idle time.
But it is possible that two clients would choose about the
same time to request information. Blocking/Locking are certainly
acceptable, as the server will be done with each request in
well under a second, and several seconds of delay is unimportant
to any of the programs.
The server's algorithm is complex, and for several reasons important
to the application should not be replicated in each helper program.
That is the reason for needing a server.
Background:
I am adding capability to a large existing legacy program.
This single program has several other legacy programs which
act as helpers and are run when the user makes certain
choices. These programs are started with a shell command,
and are not just separate threads. For instance, one helper
loads new data from a DVD drive onto the hard drive. Another
helper just displays a chart of the current positions of
the planets.
This is a LARGE commercial legacy program that happens to be
written in VB6. We are working to convert it and all the
helper programs to .NET, but must first release a new version
under vb6 with this added capability. (Please don't tell me
to not use VB6, as we are already moving elsewhere.)
We need a temporary VB6 solution.
VB6 does TCP and UDP extremely well via the standard Winsock Control component included in Pro and Enterprise Editions. A lot of shadetree coders do seem to struggle with it though. This is probably the most obvious route since the only other native IPC in VB6 would be COM/DCOM and DDE, however MSMQ provided excellent support for VB6 as well.
The downside of IP-based protocols is their limited namespace and resulting high probability of collisions (64K port numbers, many set aside for standard applications, ephemeral port ranges, etc.). They're also somewhat "heavyweight" but considering the vast resources of even the oldest PCs still in service and your light traffic requirements you can ignore that in deciding.
Another option you've considered is Named Pipes.
This offers a number of advantages in your situation. For one thing the namespace is much larger requiring only a unique name, which in the post-Win9x era can be up to 256 characters long making uniqueness fairly easy to achieve. For another, as long as your firewalls permit "File and Print Sharing" you're all set on that front.
Also, for your application you only seem to require an RPC-style mechanism rather than arbitrary bidirectional streams or messages. TransactNamedPipe() calls in your clients might be ideal. Named Pipes work over a LAN, but within one PC they are quite fast and light weight.
While VB6 doesn't come with a Named Pipe component such a thing is fairly easy to create as long as extremely high performance isn't required. You can use Timer-based polling in the server instead of trying to implement overlapped I/O to get asynchronicity. I put one together a couple of years ago and have had good luck with this approach.
I published a fairly stable rendition of this a while back at PipeRPC - RPC Over Named Pipes. There is an older and a somewhat newer version there with examples of use and documentation. As designed, clients make "calls" passing a Byte array of request parameters and receiving back a Byte array of response results. You can also shove Unicode Strings though with no changes, letting the compiler coerce the types.
Just one "drop in" UserControl for both clients and servers.
Looking back at this question:
The server's algorithm is complex, and for several reasons important
to the application should not be replicated in each helper program.
That is the reason for needing a server.
If that's really the concern why not just create a shared DLL that all programs use?
For a one-off upgrade release to an existing VB6 application being moved to a newer platform, I would stress keeping the modification as simple and straightforward as possible. As a result, I wouldn't go down any routes involving shared memory or anything relatively unusual.
A few options, none perfectly simple, but at least some ideas:
Expose a COM object in the server code that performs the translation, and can be consumed by the client apps. The clients instantiate the object from the server as an out-of-process object, and let COM handle all the marshalling, etc.
Does the server have any network awareness? VB6 doesn't do sockets/tcp natively very well, but if you've had a reason to add that in, you might be able to leverage it to perform a socket-based connection and data exchange.
The server and client could each poll a common resource folder for the presence of a specific file that constituted inbound/outbound requests for the translation service you describe. Not very elegant, but it might be the simplest.
Just a few ideas to give you some things to think about. Hope that's helpful in some way. Good luck!
When a COM+ application is created the wizard offers to choose between a library and a server application.
A server application is activated in a separate process and this can be used to cheaply interop 64-bit consumers with 32-bit in-proc COM components.
What's the use of library applications that are activated right in the caller process? Why use them instead of plain old in-proc COM servers?
There are several:
Performance - it is a bit faster as you don't have to go through the message automation (marshalling and unmarshalling)
Isolation - if many different Applications are using the Library, then each will have it's own copy. This point is most important when dealing with the differences between an MTA (Multi Threaded Apartment) and a STA (Single Threaded Apartment Model)
THE IN-PROC Server (which is really an out of processes, out of the caller's process) is shared by all different callers (this is a great way to have cheap IPC/RPC)
Ok I am editing with a few more definitions, and a bit more references:
Context is really all the state around the use of an object.
causality is really a thread like concept indicating the use of an object in a context. ("A causality is a distributed chain of COM method calls that spans any number of contexts in any number of processes" - from ISBN: 0-201-61594-0)
Those to concepts are discussed in about 30 pages of chapter 2 from Tim Ewald's excellent book "Transactional COM+" ISBN: 0-201-61594-0
So taking a direct quote from the summary of chapter 2:
"An object can interact with its context using object context and with a given causality using call context. These two objects provide interfaces for interacting with COM+ runtime services. This style of coding, 'reaching into context' makes COM+ development very different from classic COM development."
Finally, Chapter 2 has a discussion "Why Library Applications?",
(which is different from your question, Why not just plain old COM?)
His arguments mainly indicate the same reasons from using a COM object,
1. Each application has it's own instance.
2. Load into non- DLLhost.exe process.
3. Much Less Overhead.
4. Simple Deploy of common Objects.
So the bottom line is that if you are not Distributed, and Not Transactional In Nature, there may be no real advantage to using COM+ over COM. But if you write a COM+ application and deploy it as a LIBRARY application, it will behave a little bit more like a COM component.
Hope that helps.
The main purpose is to benefit from COM+ application contexts.
CoGetObjectContext for IObjectContext or IObjectContextActivity will return E_NOTINTERFACE from pure in-process component, while it will successfully work in a COM+ library application (and a server application of course).
The security context is also available through CoGetCallContext for ISecurityCallContext.
It has nothing to do with performance or isolation.
As a site note, one way to check what's available to COM+ library applications is to run dcomcnfg.exe navigate to Component Services, Computers, My Computer, COM+ application, create a new library application and check what's still enabled (as opposed to a server application).
My company is looking at implementing a new VPN solution, but require that the connection be maintained programatically by our software. The VPN solution consists of a background service that seems to manage the physical connection and a command line/GUI utilty that initiates the request to connect/disconnect. I am looking for a way to "spy" on the API calls between the front-end utilty and back-end service so that our software can make the same calls to the service. Are there any recommended software solutions or methods to do this?
Typically, communications between a front-end application and back-end service are done through some form of IPC (sockets, named pipes, etc.) or through custom messages sent through the Service Control Manager. You'll probably need to find out which method this solution uses, and work from there - though if it's encrypted communication over a socket, this could be difficult.
Like Harper Shelby said, it could be very difficult, but you may start with filemon, which can tell you when certain processes create or write to files, regmon, which can do the same for registry writes and reads, and wireshark to monitor the network traffic. This can get you some data, but even with the data, it may be too difficult to interpret in a manner that would allow you to make the same calls.
I don't understand why you want to replace the utility, instead of simply running the utility from your application.
Anyway, you can run "dumpbin /imports whatevertheutilitynameis.exe" to see the static list of API function names to which the utility is linked; this doesn't show the sequence in which they're called, nor the parameter values.
You can then use a system debugger (e.g. Winice or whatever its more modern equivalent might be) to set breakpoints on these API, so that you break into the debugger (and can then inspect parameter values) when the utility invokes these APIs.
You might be able to glean some information using tools such as Spy++ to look at Windows messages. Debugging/tracing tools (Windbg, or etc.) may allow you to see API calls that are in process. The Sysinternals tools can show you system information to some degree of detail of usage.
Although I would recommend against this for the most part -- is it possible to contact the solution provider and get documentation? One reason for that is fragility -- if a vendor is not expecting users to utilize that aspect of the interface, they are more likely to change it without notice.