im working in spring boot API and im connected to Postgres And MongoDB, the idea of the app is Postgres is used to store data and MongoDB for reading data(full-text search) , my issue is that i want to know what's the best way to synchronize data between Postgres and MongoDB. for example, when I persist my user object in Postgres i want to replicate this raw in MongoDB . what I plan to do is to save the same object twice using spring data (Post api) but I'm not sure if it will a good idea to synch data in my java code.
this is an example of my model :
#Table("users")
public class User{
#Id
#Column("user_id")
private Integer id;
//..
#Document(collection = "users")
public class UserDocument {
#Id
private String id;
//...
anyone can guide me what's the best way to achieve this data sync please?
Best regards.
Personally, I wouldn't handle this in code, I would probably configure a kafka sink to take writes into Postgres and send them to Mongo.
With that said, I assume that you have a service layer that's performing these writes to Postgresql. I don't see any reason why immediately after writing to Postgres, you can't persist it to Mongo as well. Placing them in the same service method allows both operations to participate in the transaction too.
Related
In a Spring Boot Application, there is a method mapped with a POST API for posting certain data in database. The issue is based on API URL parameter, data source will change.
Like the API is: {baseURL}/api/{someIdentifier}/addUser
Now, there is another file or consider a database which maps Database Connection Strings (like Datasource, Username, password, driver) to this {someIdentifier}. There could be a lot of such identifiers (corresponding to which there could be multiple databases and their parameters).
Now when this API gets hit, based on this identifier there will be a method which will fetch connection strings, make the connection and then it should save the data in that database. On every API, creating a connection is not feasible.
Can anyone please suggest which tool or technology can be helpful for solving this problem, especially using Spring Boot.
Thanks in advance!
You are looking for the AbstractRoutingDataSource.
From its documentation:
Abstract DataSource implementation that routes getConnection() calls to one of various target DataSources based on a lookup key. The latter is usually (but not necessarily) determined through some thread-bound transaction context.
we use in our backend AWS Aurora which has two instances, one for read+write-access and one only for read access.
In our backend we use Spring data JPA to manage our Entities in the database. Would it now be possible, that spring will use the read node for selecting data and the read/write node only for write operations?
How could we configure that? Therefore the read instance has another hostname this instance do nothing.
Thx.
I have a particular instance of a class that loads some data from the database, so every time the database is updated the system should recreate the instance of that class to get the updated data
There are trivial several solutions:
(1) Use scheduling to load the latest data from DB periodically.
(2) Provide a web service such as RESTful API to load the latest data from DB.
(3) If your DB supports event-driven listeners, you can trigger your application to achieve this either by invoking a service described in (2) or send a message to queue and handle it by consumer.
I have been researching Spring Data Rest especially for cassandra and one of the questions my coworkers and I had was when does Spring Data connect to the database. We don't always want a rest controller to connect to the database so when does spring establish a connection if say we had a class extend the CRUDRepository? Does it connect to the database during the start of application itself? Is that something we can control?
For example, I implemented this example on Spring's website:
https://spring.io/guides/gs/accessing-data-rest/
At what point in the code does spring connect to the database?
Spring will connect to the DB as soon as the Datasource get initialized. Basically, Spring contexts will become alive somehow (Web listeners, manually calling them) and start creating beans. As soon as it reaches the Datasource, connection will be made and the connection pool will be populated.
Of course the above is based on a normal out of the box configuration and everything can be setup up to your taste.
So unless, you decide to control the connections yourself, DB connections will be sitting there waiting to be used.
Disagree with the above answer.
As part of research i initiated the datasource using a bean configuration and then changed my database password(not in my spring application but the real db username password)
The connection stays for a while and then in some point of time (maybe idle time) it stops working and throws credential exception.
This is enough to say the JPA does not keep the connection sitting and waiting to be used but uses some mechanism to occupy/release the db connection as per the need.
I have a requirement for multiple RedisLabs databases for my application as described in their home page:
multiple dedicated databases in a plan
We enable multiple DBs in a single plan, each running in a dedicated process and in a non-blocking manner.
I rely on Spring Cloud Connectors in order to connect to Heroku (or Foreman in local) and it seems the RedisServiceInfoCreator class allows for a single RedisLabs URL i.e. REDISCLOUD_URL
Here is how I have configured my first redis connection factory:
#Configuration
#Profile({Profiles.CLOUD, Profiles.DEFAULT})
public class RedisCloudConfiguration extends AbstractCloudConfig {
#Bean
public RedisConnectionFactory redisConnectionFactory() {
PoolConfig poolConfig = ...
return connectionFactory().redisConnectionFactory("REDISCLOUD", new PooledServiceConnectorConfig(poolConfig));
}
...
How I am supposed to configure a second connection factory if I intend to use several redis labs databases?
Redis Cloud will set for you an env var only for the first resource in each add-on that you create.
If you create multiple resources in an add-on, you should either set an env var yourself, or use the new endpoint directly in your code.
In short the answer is yes, RedisConnectionFactory should be using Jedis in order to connect to your redis db. it is using jedis pool that can only work with a single redis endpoint. in this regard there is no different between RedisLabs and a basic redis.
you should create several connection pools to work with several redis dbs/endpoints.
just to extend, if you are using multiple dbs to scale, there is no need with RedisLabs as they support clustering with a single endpoint. so you can simple create a single db with as much memory as needed, RedisLabs will create a cluster for you and will scale your redis automatically.
if you app does require logical seperation, then creation multiple dbs is the right way to go.