I need to provide pluggable functionality to a Go program.
The idea is that a 3rd party can add functionality for a given path, i.e.
/alive maps to http://localhost:9876, or
/branding maps to http://localhost:9877 and so on.
I first tried to think of it as adding a JSON config file, where each such plugin would have an entry, e.g.:
{
"Uri": "alive",
"Address":"http://localhost:9876",
"Handler":"github.com/user/repo/path/to/implementation"
},
This though blatantly reveals Java thinking - and feels like utterly inadequate for Go - there is no notion of Class loaders in Go, and loading this would mean to have to use the loader package from golang's tools.
Proposals on how to do this in a more Go- idiosyncratic way? In the end I just need to be able to map a URI to a port and to an implementation.
Compile-time configuration
If you can live with compile-time configuration, then there is no need for a JSON (or any other) configuration file.
Your main package can import all the involved "plugins", and map their handler to the appropriate path. There is also no need to create multiple servers, although you may do so if that fits better your (or the modules') needs.
Run-time configuration
Run-time configuration and plugging in a new module requires code to be loaded at run-time. This is supported by the plugin package, but currently only under linux.
For this you may use a JSON config file, where you would list the compiled plugins (path to the compiled plugins) along with the paths you need to map them.
In the main package you can read the config file, and load the plugins, which should expose a variable or a function that returns you the handler that handles the traffic (requests). This is preferred to the plugins themselves firing up an http server for performance reasons, but both can work (plugins returning a handler for you to register, and the plugins launching their servers).
Note that there is also no need to make the configuration "static", the main app could receive and load new modules at runtime too (e.g. via a dedicated handler, which could receive the (file) path to the new module and the path to map it to, optionally maybe even the binary plugin code too; but don't forget about security!).
Note that while you can load plugins at runtime, there is no way to "unload" them. Once a plugin is loaded, it will stay in memory until the app exists.
Separate, multi-apps
There is a third solution, in which your main app would act as a proxy. Here you may start the additional "modules" as separate apps, listening on localhost at specific ports, and the main app would act as a proxy, forwarding requests coming in to the other independent apps listening on different ports #localhost (or even on other hosts).
The standard library provides you the httputil.ReverseProxy doing just this.
This does not require runtime code loading, as the "modules" are separate apps which can be launched separately. Still, this gives runtime configuration flexibility, and this solution also works on all platforms. Moreover this setup supports taking down modules at runtime, as you can just as easily un-map / close the apps of the independent modules.
The separate apps can also be launched separately, or from / by the main app, both solutions are viable.
Related
I would like to know if it is possible to create isolated modules with golang, the application always starts in a file, can i separate several REST modules and upload them depending on the conditions?
For example. I have a products module with Rest and database, and I have a users module with Rest and database. Can I create a way to choose if I will run only one or both dynamically? Without having to import everything into a main.go file?
Go is compiled language and import happens at the build time, not at the run time. Each module source is imported and compiled into the final binary and that's what you run then.
To achieve what you are asking for you'd probably need to split your program into multiple binaries each containing only relevant portion of code for its responsibility. Then you would need also to provide some communication between them (e.g. having gateway that intercepts incoming request and make relevant request(s) internally to e.g. Users service or Products service, etc.). This is often called microservice architecture.
For smaller project I recommend just put everything together first; and as it grows (and causes problems, if ever) eventually refactor to microservices.
By which mechanism is apache module loaded during runtine, or during startup ?Are in that process used mechanisms like interprocess communication ? Is apache actively calling methods in module, or module itself calls apache method, or both.
For example, get request commes to apache and mod_spnego (kerberos authentication) is loaded. How does apache know when to call main function in module code ?
Apache and its modules are written in C. The modules are loaded during the startup, from shared object files (.so on Unix, .dll on Windows or as Dynamic Shared Objects). When building the Apache HTTPD, it's also possible to compile the modules statically into the httpd binary. Both way, they work from the same process.
How does Apache know when to call main function in module code?
Take a look at these articles, and see what's said about Hooks.
Apache 2.2, Converting Modules from Apache 1.3 to Apache 2.0
The new architecture uses a series of hooks to provide for calling
your functions. These you'll need to add to your module by way of a
new function, static void register_hooks(void). The function is really
reasonably straightforward once you understand what needs to be done.
Each function that needs calling at some stage in the processing of a
request needs to be registered, handlers do not.
Developing modules for the Apache HTTP Server 2.4
When handling requests in Apache HTTP Server 2.4, the first thing you
will need to do is create a hook into the request handling process. A
hook is essentially a message telling the server that you are willing
to either serve or at least take a glance at certain requests given by
clients. All handlers, whether it's mod_rewrite, mod_authn_*,
mod_proxy and so on, are hooked into specific parts of the request
process. As you are probably aware, modules serve different purposes;
Some are authentication/authorization handlers, others are file or
script handlers while some third modules rewrite URIs or proxies
content.
I have application which has core website, api and admin area. I wanted to know is it bad idea to have everything in one app or should I create different Symfony2 project or should I split them into different kernels?
I'm not sure if adding lots of bundles on same kernel will effect performance a lot or is just a little bit, which does not matter?
Following are options:
keep everything on same kernel, it wont make much difference
have multiple kernel for different part of application (api, admin and core website)
create different Symfony2 project for admin area and api.
or your wise words :)
You can define more "environments".
For example :
In AppKernel.php
public function registerBundles()
{
$bundles = array(
new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
new Symfony\Bundle\SecurityBundle\SecurityBundle(),
new Symfony\Bundle\TwigBundle\TwigBundle(),
new Symfony\Bundle\MonologBundle\MonologBundle(),
new Symfony\Bundle\SwiftmailerBundle\SwiftmailerBundle(),
new Doctrine\Bundle\DoctrineBundle\DoctrineBundle(),
new Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle(),
//new AppBundle\AppBundle()
);
if (in_array($this->getEnvironment(), array('api'), true)) {
$bundles[] = new ApiBundle\ApiBundle();
//-- Other bundle
}
//-- Other environments
return $bundles;
}
}
It mostly depends on bundles quality. And this how much connected they are.
I would reject point 3 at start (create different Symfony2 project for admin area and api.) - as probably you don't build two separate applications.
Have multiple kernel for different part of application (api, admin and core website)
Common problem is created by Listeners and services in container. Especially when your listener should work only in one of app contexts (api/frontend/backend). Even if you remember to check it at very beginning of listener method (and do magic only in wanted context) then still listener can depend on injected services which need to be constructed and injected anyway. Good example here is FOS/RestBundle: even if you configure zones then still on frontend (when view_listener is activated for api) view_handler is initialized and injected to listener - https://github.com/FriendsOfSymfony/FOSRestBundle/blob/master/Resources/config/view_response_listener.xml#L11 I'm not sure for 100% here but also disabling translations and twig (etc.) for API (most of api's don't need it) will speed it up.
Creating separate Kernel for API context would solve that issue (in our project we use one Kernel and we had to disable that listener - as blackfire.io profiles were telling us that it saves ~15ms on every fronted request).
Creating new Kernel for API would make sure that none of API-only services/listeners will not interfere with frontend/backend rendering (it work both ways). But it will create for you additional work of creating shared components used in many bundles inside project (those from different kernels) - but in world with composer it's not a huge task anymore.
But it's case only for people who measure every millisecond of response time. And depends on your/3dparty bundles quality. If all there is perfectly ok then you don't need to mess with Kernels.
It's personal choice, but I have a similar project and I have a publicBundle, adminBundle and apiBundle all within the same project.
The extra performance hit is negliable but organisation is key ... that is why we're using an MVC package (Symfony) in the first place, is it not? :)
NB: You terminology is a little confusing, I think by Kernel you mean Bundle.
Have several kernels could not necessarily help.
Split your application in bundles and keep all advantages of sharing your entities (and so on) through the different parts of your application.
You can define separated routing/controllers/configuration that are loaded depending on the host/url.
Note :
If you are going to separate your app in two big bundles (i.e. Admin & Api),
and that the two share the same entities, you will surely have to do a choice.
This choice may involves that one of your bundles contains too much (and non related) logic and will need to be refactored in several bundles later.
Create a bundle per section of your application that corresponds to a set of related resources and make difference between the two parts through different contexts from configuration.
Also, name your classes/namespaces sensibly.
I'm doing some in memory Caching for some Plugins in Microsoft CRM. I'm attempting to figure out if I need to be concerned about different orgs populating the same cache:
// In Some Plugin
var settings = Singleton.GetCache["MyOrgSpecificSetting"];
// Use Org specific cached Setting:
or do I need to do something like this to be sure I don't cross contaminate settings:
// In Some Plugin
var settings = Singleton.GetCache[GetOrgId() + "MyOrgSpecificSetting"];
// Use Org specific cached Setting:
I'm guessing this would also need to be factored in for Custom Activities in the AsyncWorkflowService as well?
Great question. As far as I understand, you would run into the issue you describe if you set static data if your assemblies were not registered in Sandbox Mode, so you would have to create some way to uniquely qualify the reference (as your second example does).
However, this goes against Microsoft's best practices in Plugin/Workflow Activity development. Every plugin should not rely on state outside of the state that is passed into the plugin. Here is what it says on MSDN found HERE:
The plug-in's Execute method should be written to be stateless because
the constructor is not called for every invocation of the plug-in.
Also, multiple system threads could execute the plug-in at the same
time. All per invocation state information is stored in the context,
so you should not use global variables or attempt to store any data in
member variables for use during the next plug-in invocation unless
that data was obtained from the configuration parameter provided to
the constructor.
So the ideal way to managage caching would be to use either one or more CRM records (likely custom) or use a different service to cache this data.
Synchronous plugins of all organizations within CRM front-end run in the same AppDomain. So your second approach will work. Unfortunately async services are running in separate process from where it would not be possible to access your in-proc cache.
I think it's technically impossible for Microsoft NOT to implement each CRM organization in at least its own AppDomain, let alone an AppDomain per loaded assembly. I'm trying to imagine how multiple versions of a plugin-assembly are deployed to multiple organizations and loaded and executed in the same AppDomain and I can't think of a realistic way. But that may be my lack of imagination.
I think your problem lies more in the concurrency (multi-threading) than in sharing of the same plugin across organizations. #BlueSam quotes Microsoft where they seem to be saying that multiple instances of the same plugin can live in one AppDomain. Make sure multiple threads can concurrently read/write to your in-mem cache and you'll be fine. And if you really really want to be sure, prepend the cache key with the OrgId, like in your second example.
I figure you'll be able to implement a concurrent cache, so I won't go into detail there.
R-Osgi provides us a way to call service from a remote OGSi container. WebSite: http://r-osgi.sourceforge.net.
I'm new to R-OSGi and now I want to split my OSGi container into small ones and interact each other by R-Osgi because it's too huge. But it seems R-OSGi only provides a way for Registered Service. We know that the most popular way to interaction between 2 bundles besides Service, "exported-package" is also widely used.
So, is there anyone familiar with R-OSGi and know how to use "exported-package" from remote OSGi container?
Thanks for any response.
If you think about it, attempting to handle the remote importing/exporting of packages is very complex, fragile and prone to error; you'd need to send all bundle lifecycle events over the wire, and honour them in the importing system (which would require caching).
Additionally the framework would need to know ahead of time to use these class definitions (you cannot instantiate a class that references classes that aren't available to your classloader). A remote bundle's classloader may depend on classes from another classloader, this chain could go round and round a network making classloading take ages.
Put another way; your local bundles will never resolve without the class definitions they depend on, and considering there could be thousands+ of potential remote exporters on a network/hardware with very poor SLA, this wouldn't scale well or be very robust considering the fallacies of distributed computing.
If we tried to do remote packages, the framework would need to import all exported packages from all available remote nodes and then select just one to import each bundle export from (this would be arbitrary and if the select node goes down, the whole import remote package process would have to triggered again).
What you need to do is separate you api/interfaces from your implementation, you then distribute the api bundle to all nodes that need it and then use dOSGi to import services.
Apologies if this unclear or waffly but it should explain why it's simply not practical to have remote exported packages.
On a side note; I'm not sure if r-osgi is being actively maintained or is up-to-date with the latest Remote Services Admin spec, from looking at the last commit to SVN trunk was 14/02/2011. There's some alternative dOSGi implementations listed here (but avoid CXF).
EDIT: In terms of deployment, distributing your bundles (and configuration) can be done from an OBR (there are number of public ones and several implementations Felix/Eclipse), or a maven repository can be reappropriated with pax url handler.