I found many examples of using spring boot reactive with document databases, but none with SQL databases.
I see that it may not support sql databases yet, probably because some missing feature on the jpa/jdbc stack.
I also see that there is no point to use reactive services that depend on the a sql database with no reactive support.
The question here is: Is there any ongoing development to make this happen (reactive jpa)?
There is a reactive feature built into many RDBMSs called "Change Data Capture" which writes data to an async transaction log for an enabled table. Usually, reactive systems built to stream this data are built on top of that feature. For example, a well-known open source tool that does this is Debezium. You can find other open source projects online that do something similar, or to write your own using the simple CDC functions that are usually provided to support it.
Related
I want to use the JOOQ DSL in Quarkus to build my SQL (and hopefully execute them).
Therefore I added the following Quarkus JOOQ extension.
Since I want to use the reactive PG SQL Client in my project, I'm asking myself if e.g. the fetch() method of JOOQ will block the thread ? Is it compatible with the reactive vertx client under the hood or does it use a blocking one ? Looks like the latter one since it doesn't return a future or anything like that.
In that case I propably should only use JOOQ for creating the SQL string.
Which parts of the jOOQ API can be used reactively
jOOQ's ResultQuery<R> extends Publisher<R>, so you can just place a jOOQ query in any reactive stream implementation. There are 3 main Publisher subtypes in jOOQ:
ResultQuery<R> extends Publisher<R>
RowCountQuery extends Publisher<Integer>
Batch extends Publisher<Integer>
And starting with jOOQ 3.17, there will also be a way to create transactional Publisher types.
With this in mind, in the reactive world, you will never need to call any of the traditional jOOQ blocking execution methods. You'll always implicitly execute jOOQ queries via some reactive streams integration.
Avoiding calls to blocking API
Starting with jOOQ 3.17, all the blocking API (e.g. ResultQuery.fetch()) will be annotated as org.jetbrains.annotations.Blocking, so you get IDE support to warn you that you're about to do something that might not make sense in your non-blocking context.
Backing implementation
For any of this to work, you need to provide jOOQ with an R2DBC connection. R2DBC is an SPI that enables interoperability between client libraries like jOOQ and R2DBC drivers, like r2dbc-postgres. Just like JDBC, it works as an SPI, not strictly an API. Besides, it integrates also directly with the reactive streams SPI, which has been integrated in the JDK 9 via the Flow API.
There might be future work to support alternative non-blocking drivers in the future, however R2DBC seemed to be the most interoperable choice at the time the reactive support was added, and I do hope that the Vert.x and R2DBC teams will find ways to cooperate more tightly in the future. The Vert.x SQL client, for example, does not implement the reactive streams SPI directly, Red Hat does not seem too interested (yet) in moving forward with this issue here: https://github.com/eclipse-vertx/vertx-sql-client/issues/249
So, for now, this means that you have to either:
Use jOOQ with R2DBC, which is what jOOQ supports (not sure if Quarkus will support R2DBC, though I don't see any reason why it shouldn't)
Use jOOQ to generate SQL only and run the SQL with Vert.x (you'll lose a lot of type safety and convenience, as well as access to advanced features like MULTISET, which relies on jOOQ executing your query)
A side note on reactive execution
Of course, it's always important to think about whether you really need to go reactive. In my personal experience, this is mostly a matter of programming style, not actual performance and/or load requirements. Sticking with the blocking paradigm and JDBC will greatly simplify your every day work, and I doubt you'll notice a measurable difference in production.
I'm looking for a solution to do the same thing, I haven't tested it yet but I came across this repo:
https://github.com/jklingsporn/vertx-jooq
https://github.com/jklingsporn/quarkus-jooq-reactive-example
It may help to be fully reative using vert.x in quarkus.
I need to attach a listener to a table in db
which should call a spring boot method, once CRUD operation is performed in the table(pre listeners and post listeners)
the entry can be made from any source
how can i do that in spring boot?
If the entity can be created from any source - e.g. manual insert - this is something which is outside of the scope and context of your running application.
What you're describing is known as the CDC (change data capture) pattern.
To implement CDC in this case you need to use the instrumentation of the underlying database - for example triggers.
As I see this is tagged with MongoDb - triggers are not an option as mongodb doesn't have support for triggers.
If you are using MongoDb v3.6+ you can leverage the new Change Streams feature. This is the official example with Java.
Change streams allow applications to access real-time data changes
without the complexity and risk of tailing the oplog. Applications can
use change streams to subscribe to all data changes on a single
collection, a database, or an entire deployment, and immediately react
to them. Because change streams use the aggregation framework,
applications can also filter for specific changes or transform the
notifications at will.
If you are using earlier versions of MongoDb you can monitor the oplog or use tailable cursors with capped collections.
Another approach would be to look into a 3rd party solution that turns everything happening in the DB as event streams - like for example debezium.
This article explains how to call any program from DB-Trigger.
Therefore, you can just create a Spring Boot java app and make the sys call to your app.
Similar mechanism is also available in Oracle and other DB.
Is it possible to use Hibernate and RDBMS(Mysql, Postgres etc) with ReactiveCrudRepository instead of CrudRepository? I have tried some samples with Spring Data Jpa and Hibernate, but couldn't get it done. I was only able to find a few samples on ReactiveCrudRepository for MongoDB and cassandra.
Is it possible to use Hibernate and Mysql with ReactiveCrudRepository instead of CrudRepository?
TL;DR:
Not with Hibernate and MySQL, but with R2DBC and Postgres, Microsoft SQL Server or H2.
Take a look at Spring Data R2DBC.
Long Version
Why not JPA?
With Hibernate/JPA included this won't happen in the foreseeable future.
JPA is based on the idea that you load part of your data model into memory, manipulate the resulting object model and let JPA transform these changes.
All this within a single transaction.
This is kind of the opposite how one deals with a reactive store where you try to make atomic changes and try to decouple the loading, processing and storing and all this without blocking.
Why not JDBC?
So we have to look at the technology level below JPA: JDBC.
But JDBC is still blocking: You send a SQL statement to your database and then JDBC will block until you get the result.
And again this goes against the idea of reactive: Never block.
One could wrap this in a thread pool to mitigate this to some extent, but that is more of a workaround than a solution.
Why R2DBC?
There are some suitable drivers for some databases that could be used for reactive repositories.
But they are proprietary and thereby not a good basis for something that really should eventually work across all (relevant) relational databases.
For some time the Spring Data team hoped that ADBA would fill that gap.
But discussions on the mailing list made it clear that ADBA was not aiming for reactive but only for asynchronous.
Again not what we needed for a reactive repository abstraction.
So early in 2018 various people living at the intersection or reactive and relational decided that we need a standard for reactive database access.
R2DBC (Reactive Relational Database Connectivity)
is a proposal for such a standard.
The hope is that it either helps convincing Oracle to move ADBA to a reactive approach or if that doesn't happen it becomes the standard itself.
And with already three implementations available chances for the second option look promising.
R2DBC itself is mainly an SPI, i.e. an API that is to be implemented by database providers.
The SPI is designed in a way that puts minimal requirements on implementers.
But this also makes R2DBC somewhat cumbersome to use.
The idea is that other libraries will step up and build libraries designed for usability on top of that SPI, as it happened with JDBC.
Spring Data R2DBC
Spring Data R2DBC is one such library and it offers what you asked for: Support for ReactiveCrudRepository although it is independent of JPA/Hibernate and there is no support for MySQL yet.
State of the projects
Both R2DBC and Spring Data R2DBC didn't have a production release yet and it will take at least several months to get there.
Spring Data R2DBC just released the first milestone.
See the release article for its current capabilities.
R2DBC is on its 6th milestone. See the release article for details.
See also this answer: Why does Spring not provide reactive (non-blocking) clients for relational databases?
Original answer as a reference for archeologists:
As of now (Jan 2017) it is not possible.
The currently relevant release for the reactive part of Spring Data is Spring Data Kay M1 (You can check if there is a newer version available on the project home page)
And a blog post from the Spring Data team about that release and specifically the reactive parts in it starts with (emphasis mine):
Spring Data Kay M1 is the first release ever that comes with support for reactive data access. Its initial set of supported stores — MongoDB, Apache Cassandra, and Redis — all ship reactive drivers already, which made them very natural candidates for such a prototype.
The reason is that there is no standard non-blocking way to access a relational database. So only those that support this kind of API are supported right now.
One could implement a ReactiveCrudRepository using JPA or JDBC and delegate the work to a thread pool. This would provide an async API on the outside, but would still consume the resources for the Threads and block between independent data accesses, so only a small part of the benefits of the reactive approach would get realized.
Hibernate started a new Hibernate Reactive subproject for reactive streams support which provides Hibernate/JPA similar APIs to access RDBMS. But unfortunately at the moment, Spring Data does not support it. So there is no a ReactiveCrudRepoisoty for Hibernate Reactive.
But you can integrate Hibernate with Spring yourself and get reactive support.
Define a persistence.xml file, note the provider class must be specified as the one in Hibernate Reactive.
Declare a Mutiny.SessionFactory bean.
Then inject it in your repository class.
I have created a complete example demos Hibernate Reactive + Spring.
Update: Till now Spring team has no plan to support it, if you are willing to taste other framework, check Quarkus and Micronaunt, both have seamless Hibernate Reactive support. Check my Quarkus Hibernate Reactive example and Micronaut Hibernate Reactive example.
According to quote from previous answer
One could implement a ReactiveCrudRepository using JPA or JDBC and delegating the work to a thread pool. This would provide an async API on the outside, but would still consume the resources for the Threads and block between independent data accesses, so only a small part of the benefits of the reactive approach would get realized.
James Ward claims it can be non-blocking. I mean I asked him:
yeah ok, but isn't ScalikeJDBC-Async doing exactly the same? just putting query invocation into another thread pool?
and he replied
No because ScalalikeJDBC-Async uses https://github.com/mauricio... which is actually a non-blocking (NIO) JDBCish database driver.
source
So you can be reactive by replacing hibernate + spring data with postgresql-async (should work with mysql).
you could try with quarkus framework and panache mongo hibernate reactive repositories. https://quarkus.io/guides/mongodb-panache .It is easy manage a reactive repository over mongoDB, It is later but hope helps.
Currently We have an enterprise application that works with spring and JPA.
Today we are planning our next generation server.
We are debating whether to use spring-data in our project? It seems to increase productivity and development times.
Are there any alternatives to spring-data to consider? Why not using spring and JPA alone?
What do you suggest?
Bear in mind we are starting to develop from scratch so no constraints are available other than:
we use mysql and mongoDB
we code in java
we will develop client side code in GWT.
Currently we have a layered architecture.
We have a Service layer and a manager layer, which takes care for persisting and business logic. Whoever built that didn't see a good reason to insert the third DAO layer.
There are some technical benefits of Spring Data over Spring + JPA, which in a pure SQL environment, I think give Spring Data an advantage:
Spring Data uses the same CrudRepository interface for all implementations, so you'll have less effort to switch between JPA to MongoDB
Spring Data saves you writing the same methods again and again. You just add the method to the interface and it'll generate it for you (e.g. UserRepository.findByUsername())
You can save boilerplate on REST implementations for JPA, MongoDB and others (see http://projects.spring.io/spring-data-rest/)
If you wanted to experiment with other persistence or indexing services, then there are Spring Data implementations for both mature and newer technologies such as for Neo4j, Hadoop, Solr, ElasticSearch, fuzzydb.
Given that you use MySQL and MongoDB, I think Spring Data is a strong candidate, as it allows developers to code to a single data access API (Spring Data) instead of two (JPA and the MongoDB Java Client).
Regarding the existing architecture, it sounds as though your manager layer is implementing either a Rich Domain pattern, or Active Record.
Spring Data is in my view very well suited to Rich Domain when combined with injection of services using Spring's #Configurable.
Lastly, I'd say that Spring Data also gives a significant advantage when needing to implement services for things like Spring Security and Spring Social, which use MongoDB or others instead of SQL.
We did this in the fuzzydb sample webapp that can be found here. (Disclaimer: I'm the currently sole recent committer on fuzzydb, and haven't touched it for a number of years, but we did have a live service, www.fridgemountain.com, based on that code, but neglected to promote it)
Does anybody has an experience with Spring Integration project as embedded ESB?
I'm highly interesting in such use cases as:
Reading files from directory on schedule basis
Getting data from JDBC data source
Modularity and possibility to start/stop/redeploy module on the fly (e.g. one module can scan directory on schedule basis, another call query from jdbc data source etc.)
repeat/retry policy
UPDATE:
I found answers on all my questions except "Getting data from JDBC data source". Is it technically possible?
Remember, "ESB" is just a marketing term designed to sell more expensive software, it's not a magic bullet. You need to consider the specific jobs you need your software to do, and pick accordingly. If Spring Integration seems to fit the bill, I wouldn't be too concerned if it doesn't look much like an uber-expensive server installation.
The Spring Integration JDBC adapters are available in 2.0, and we just released GA last week. Here's the relevant section from the reference manual: http://static.springsource.org/spring-integration/docs/latest-ga/reference/htmlsingle/#jdbc
This link describes the FileSucker with Spring Integration. Read up on your Enterprise Integration patterns for more info I think.
I kinda think you need to do a bit more investigation your self, or do a couple of tries on some of your usecases. Then we can discuss whats good and bad
JDBC Adapters appear to be a work in progress.
Even if there is no specific adapter available, remember that Spring Integration is a thin wrapper around POJOs. You'll be able to access JDBC in any component e.g. your service activators.
See here for a solution based on a polling inbound channel adapter too.