I'm trying to find out whether we should use Akka or Reactor for our next project. One of the most important questions is if the future framework of choice will provide remoting. As i saw, Akka offers this just in the way we'd like to have it.
In the GitHub wiki, unfortunately the TCP-server/client sections are blank and i couldn't find other informations about that yet.
Will reactor provide remoting?
I don't think Akka and Reactor are Apples to Apples. Reactor is purposely minimal, with only a couple external dependencies. It gives you a basic set of tools in which to write event-driven applications but it by design does not enforce a particular model. It would actually not take that long to implement a Dynamo system using Reactor components. Everything that would be needed is there and it would likely only take writing a tutorial on it to show how to wire things together.
The Dynamo model that Akka uses is a proven system. Basho has done a fantastic implementation of it in Riak. It's great to see Akka following their lead in that respect. If we were to implement a Reactor clustering system, it would likely be the Dynamo model. But since a Reactor is basically just event handlers and topic pub/sub, your Consumers can do any remote communication you want. They can integrate with HTTP, AMQP, Redis, whatever. There's no need to have special APIs for doing this sort of thing because they're just events. You can code up an AMQP client application in about 10 minutes and be publishing data from RabbitMQ into a Reactor application.
We might very well at some point have different implementations of clustering for different purposes. The Dynamo model might work well for some while others would want a simple Redis-based system. Or maybe one could leverage the components already in Reactor to work with the Java Chronicle to do disk-based clustering--something you can do right now just by wiring up the right Consumers. But those will be in external modules that could be added to Reactor. reactor-core itself will likely never have an opinionated clustering solution simply because it doesn't fit the purpose of those core components: a foundational framework for event-driven applications on the JVM.
(I'm working on the TcpClient/TcpServer wiki docs right now, so hopefully those will be filled in for the M2 of Reactor which will be happening very soon.)
Related
I'm trying to implement suppliers using Spring Cloud Function and Kafka. I need that one supplier should publish after every 10 secs and other should publish after every 30 secs. I could see from documentation, I can change delay using spring.cloud.stream.poller.fixed-delay property. Reference
But I need to set different delay for each topic. Is there any way to do it?
From the spring-cloud-function perspective there isn't any kind of polling as it is not the responsibility of the framework.
From the spring-cloud-stream perspective that uses spring-cloud-function indeed there is a mechanism that you have described. However, keep in mind that spring-cloud-stream is primarily designed to support concept of microservices (not your general messaging framework) and in microservices we embrace do one thing but do it well without affecting others approach. So having more then one supplier kind of goes against this model.
If you are building a general purpose messaging app, then i'd suggest to use Spring Integration framework which provides all the necessary hooks to accomplish what you need, but will require a bit more configuration details.
Spring boot 2.1.5
Project Reactor 3.2.9
I am setting up a bunch of rest reactive APIs using the above-mentioned frameworks and I am running into an annoying problem with MDC (mapped diagnostic context). My applications are in JAVA.
MDC relies on thread locals to store the current query's mapped context to put in the logs. That system, obviously, is not perfect and contradicts the reactive pattern since the different steps of your execution will be executed through different threads.
I have run into the same problem with the Play Reactive framework but found a workaround there by copying the mapped context transparently from one actor to another.
For spring and reactor, I could not find a satisfying solution yet.
Some random examples found on the internet:
First - It works but forces you to use a bunch of utility methods
Same thing
Second - It tries to copy the context during the onNext publisher event but seems to lose some features on the way of doing that. The signal context, for example, is lost.
I am in need of a proper solution to deal with this:
A library which would make the link between MDC and reactor?
A way to tweak reactor/spring to achieve it transparently?
Any advice?
"I could not find a satisfying solution yet."
Working with contexts is the only solution for the moment. Since as you said threadlocals goes against everything that has to do with reactive programming. Using thread local as a storage point during a request is a resource heavy way of solving things and in my opinion poor design. Unless logging frameworks themselves come up with a better solution to the problem we developers must pass the data through the context to accommodate for the logging frameworks blocking nature.
Reactive programming is a paradigm shift in the programming world. Other things like database drivers, that use threadlocal to rollback transactions are also in big trouble. the JDBC database driver spec is defined as blocking in nature, and atm. there has been attempts by spring and the R2DBC project to define a new JDBC driver spec that is inherently non/blocking. This means that all vendors must rewrite ther database driver implementations from scratch.
Reactive program is so new that lots of libraries need to rewrite entire codebases. The logging frameworks as we know it needs to be rewritten from the ground up which is a huge task. And the context in reactive is actually something that should not even be in reactive programming, it was implemented just to accommodate for MDC problems.
It's actually a lot of overhead needing to pass data from thread to thread.
So what can we do?
push on logging frameworks, and/or help logging frameworks to rewrite their codebase
Accept that there is no "tweak" that will magically fix this
use the context and the way suggested in the blogposts
Project reactor context
For my master thesis I'm developing an application framework for selling tickets for large events. My main requirements are modifiability, scalability and performance. My clients (event organisers) should be able to easily replace a component at runtime and add functionality. An example of such a component could be the seat assignment component.
My mentors said to look at OSGi. The idea of loosely coupled bundles is certainly appealing. When looking for alternatives I discovered Akka. This framework promises a lot of things, like scalability and high performance. I wondered if Akka's concept of actors suits my modifiability requirements. Akka seems more productive than OSGi, so development would be faster. Akka also seems more fit for scalability. With OSGi I would have a harder time.
If you have experience in both OSGi and Akka, which would you recommend for me? What are the pro and cons of both technologies when comparing them? And finally, are there are any good alternatives to OSGi or Akka that cover my requirements?
EDIT
First, thank you for the replies so far, you're a great help.
As mentioned below, I'm trying to compare apples and pears. A more logical question would be: How can OSGi and Akka be used together and benefit from each other? How is this structured? Do all your actors reside in one OSGi bundle, do they each get a separate bundle, is there a hybrid solution or isn't there really a 'right' way to do it?
EDIT bis
I posted a follow up question here, asking how to combine OSGi and Akka.
As Peter says they are not directly comparable. In fact you can use them together and they should be quite complementary.
Akka provides an asynchronous communications API. OSGi provides a modular, service-oriented framework. There is nothing in Akka, for example, that would solve the problem of isolating modules so that they cannot have visibility of each others' internals. Likewise there is nothing in OSGi quite like the async communications provided by Akka. So use them together and you get the best of both worlds...
OSGi does have synchronous Services, which are the principal method of communication between modules in a single JVM. OSGi also has a Remote Services layer that can be used for communication between remote machines. This is probably the area where OSGi and Akka most directly overlap, I suppose. But even here I think there is potential for cooperation. For example, OSGi Remote Services has a really powerful discovery mechanism that allows us to advertise capabilities on the network. You could possibly use this discovery to find Akka actors that are available for you to communicate with.
I'm not aware of anybody actually working on this as present, so I think that exploring and expanding on this idea would be an excellent topic for a master's thesis!
Which university do you attend? The OSGi Alliance is very interested in fostering links with the academic community, perhaps we could set up an online meeting with you and your professor?
I think you compare apple and pears. You can run Scala code on OSGi (though their binary compatibility is horrible).
Scala is a programming language, and Akka a messaging library. OSGi is a dynamic component system. So not sure how you can compare them
I agree with both Neil and Peter, you're asking us to make a comparison between apples and oranges. You can use both frameworks together. I'm currently working on a project with the same main requirements you specified. I'm creating a prototype that demonstrates using both technologies together, OSGi to provide modularity and updatability, and Akka to provide scalability and performance.
If you would like to see both frameworks working together you can play with the sample application I posted on github.
They aren't an apples to apples comparison. They are Orthogonal to each other and, if anything, complement each other. Use both.
I have been trying to understand a bit more about the wider picture of OSGi without reading thru the entire specification. As with so many things, the introduction to what OSGi actually is was probably written by someone who had been working on it for a decade and perhaps wasn't best placed to put themselves in the mindset of someone who knows nothing about it :-)
Looking at Felix's example DictionaryService, I don't really understand what is going on. Is OSGi a distinct instance of a JVM into which you load bundles which can then find each other?
Obviously it is not just this because other answers on StackOverflow are explicit that OSGi can solve the dependency problem of a distributed system containing modules deployed within distinct JVMs (plus the FAQ keeps talking about networks).
In this latter case, how does a component running in one JVM interact with another component in a separate JVM? Can the two components "use" each other as if they were running within the same JVM (i.e. via local method calls), and how does OSGi manage the marshalling of data across a network (do you have to use Serializable for example)?
Or does the component author have to use some other distinct mechanism (either provided by OSGi or written themselves) for communication between remote components?
Any help much appreciated!
Yes, OSGi only deals with bundles and services running on the same VM. However, one should note that it is a distinct feature of OSGi that it facilitates running multiple applications (in a controlled way and sharing common modules) on the same JVM at all.
When it comes to accessing services outside the clients JVM, there is currently no standardized solution. Paremus Infiniflow and the derived open-source project Newton use an SCA approach. The upcoming 4.2 release of the OSGi specs will address one side of the problem, namely how to use generic distribution software in such a way that it can bring remote services into the client's JVM.
As somebody mentioned R-OSGi, this approach also deals with the other side of the problem, being how to manage dependencies between distributed OSGi frameworks. Since R-OSGi is not generic distribution software but explicitly deals with the lifecycle issues and dependency management of OSGi bundles.
As far as I know, OSGi does not solve this problem out of the box. There are OSGi-bundles, for example Remote OSGi, which allow the programmer to distribute services across a network.
Not yet, i think it's being worked on for the next release.
But some companies have already implemented distributed osgi. One i'm aware of is Paremus' Infiniflow (http://www.paremus.com/products/products.html). At linkedin they are also working on this. More info here: Building Linkedin next gen architecture with osgi and here: Matt raible: building linkedin next gen architecture
Here's a summary of the changes for OSGI 4.2: Some thoughts on the OSGi R4.2 draft, There's a section on RFC-119 dealing with distributed OSGi.
AFAIK, bundles are running in the same JVM, but are not loaded using the same class loader (that why you can use two different versions of the same bundle at the same time).
To interact with components in another JVM, you must use a network protocol such as rmi.
The OSGi alliance is working on a standard for distributed OSGi:
http://www.osgi.org/download/osgi-4.2-early-draft2.pdf
There even is an early Apache implementation of this new standard:
http://cxf.apache.org/distributed-osgi.html
#Patriarch24
The accepted answer to this question would seem to indicate otherwise (unless I'm misreading it). Also, taken from the FAQ:
The OSGi Service Platform provides the functions to change the composition dynamically on the device of a variety of networks, without requiring a restart
(Emphasis my own). Although in the same FAQ it describes OSGi as in-VM.
Why am I so confused about this? Why is such a basic question about a decade-old technology not clear?
The original problem of OSGI was more related to distribution of code (and then configuration of bundle) than to distribution of execution.
People looking at distributed components are rather looking towards SCA
The "introduction" link is not really an intro, it is a FAQ entry. For more information, see http://www.osgi.org/About/WhatIsOSGi Not hard to find I would think.
Anyway, OSGi is an in-VM SOA. That is, the OSGi Framework is about what happens inside the VM, it provides a framework for structuring your application inside the VM so you can built it too a large extent from components. So the core has nothing to do with distribution, it is completely oblivious of who implements the services, it just provides a mechanism for modules to meet each other in a loosely coupled way.
That said, the µService model reifies the joints between the modules and it turns out that you can build support on top of the framework that provides distribution to the other components. In the last releases we specified some mechanisms that make this standardized in the core and provide a special service Remote Service Admin that can manage a distributed topology.
If you are looking for a distributed OSGi centric Cloud runtime - then the Paremus Service Fabric ( https://docs.paremus.com/display/SF16/Introduction ) provides these capabilities.
One or more Systems each consisting of a number of OSGi assemblies (Blueprint or Declarative Services) can be dynamically deployed and maintained across a population of OSGi runtime Frameworks (Knopflerfish, Felix or Equinox).
A light weight RSA remote framework is provided which provides Service discovery by default using DDS (a seriously good middleware messaging technology) - (thought ZooKeeper and other approach can be used). Currently supported re-moting protocols include RMI and Avro.
Regards
Richard
Does anybody has an experience with Spring Integration project as embedded ESB?
I'm highly interesting in such use cases as:
Reading files from directory on schedule basis
Getting data from JDBC data source
Modularity and possibility to start/stop/redeploy module on the fly (e.g. one module can scan directory on schedule basis, another call query from jdbc data source etc.)
repeat/retry policy
UPDATE:
I found answers on all my questions except "Getting data from JDBC data source". Is it technically possible?
Remember, "ESB" is just a marketing term designed to sell more expensive software, it's not a magic bullet. You need to consider the specific jobs you need your software to do, and pick accordingly. If Spring Integration seems to fit the bill, I wouldn't be too concerned if it doesn't look much like an uber-expensive server installation.
The Spring Integration JDBC adapters are available in 2.0, and we just released GA last week. Here's the relevant section from the reference manual: http://static.springsource.org/spring-integration/docs/latest-ga/reference/htmlsingle/#jdbc
This link describes the FileSucker with Spring Integration. Read up on your Enterprise Integration patterns for more info I think.
I kinda think you need to do a bit more investigation your self, or do a couple of tries on some of your usecases. Then we can discuss whats good and bad
JDBC Adapters appear to be a work in progress.
Even if there is no specific adapter available, remember that Spring Integration is a thin wrapper around POJOs. You'll be able to access JDBC in any component e.g. your service activators.
See here for a solution based on a polling inbound channel adapter too.