I am exploring ways to achieve reads from slave and writes to master on a JPA based webapp using a MySQL Master/Slave.
I think I have an option to pursue and like some opinions about past experiences achieving the same thing.
To start with , I would like to -
create an additional persistence Unit called readOnlyPU
create an additional readOnlyEntityManagerFactory that uses the readOnlyPU
the readOnlyPU and writeOnlyPU both share same entity objects meaning 2 different Hibernate Cache with same copy of objects.
when needed, use the readOnlyEntityManager in my code.
I think this approach should work, but I would like to ask about your opinion about the approach and if such approach is common for enabling Master/Slave replication .
Related
According to my understanding
We should only have one service connecting to a database
With CQRS you will be keeping two databases in sync, hypothetically using some “service” glueing them together
Doesn’t that now mean there’s a service which only purpose is to keep the two in sync, and another service to access the data.
Questions
Doesn’t that go against rule number above? Or does this pattern only apply when native replication is being used?
Also, other than being able to independently scale the replicated database for more frequent reads, does the process of keeping both in sync kind of take away from that? Either way we’re writing the same data to both in the end.
Ty!
We should only have one service connecting to a database
I would rephrase this to: each service should be accessible via that service's api. And all internals, like database, should be completely hidden. Hence, there should be no (logical) database sharing between services.
With CQRS you will be keeping two databases in sync, hypothetically using some “service” glueing them together
CQRS is a pattern for splitting how a service talks to a data layer. Typical example would be something like separating reads and writes; as those are fundamentally different. E.g. you do rights as commands via a queue and reads as exports via some stream.
CQRS is just an access pattern, using it (or not using it) does nothing for synchronization. If you do need a service to keep two other ones in sync, then you still should use services' api's instead of going into the data layer directly. And CQRS could be under those api's to optimize data processing.
The text from above might address your first question. As for the second one: keeping database incapsulated to a service does allow that database (and service) to be scaled as needed. So if you are using replication for reads, that would be a reasonable solutions (assuming you address async vs sync replication).
As for "writing data on both ends", I am actually confused what does that mean...
I'm new to the Spring Eco System and I was wondering, if there is anything that should be done if you wanna scale the application layer horizontally and preserve only one database instance (in this case Postgresql).
Speaking of, should I worry about inconsistency because of a default second or third level cache in the applications or any other things?
Based on your setup I could see 2 things
You want to scale horizontally at application level but with just single DB. Which might turn out to be the Single Point of Failure. You can run your DB in a cluster as well.
Multiple instances connected to the same Database. So there may be a case where Multiple reads at the same time will not be a problem but multiple writes to the same table work differently so watch out for that.
You can read more about Postgresql blocking here.
Regarding your concern on the inconsistency
There may be some inconsistency at some point of time but its effect would not be noticeable, just add few ms overheads.
Note:
You haven't mentioned how are you planning to scale at application level generait's its done using LoadBalancer (eg: Nginx)
I Have two mico-serives A and B where they connect to seperate database, From Mico-serives A i need to persist(save) objects of both A and B in same transtation how to achive this.
I am using Spring micro-servies with netflix-oss.Please give suggestions on best way to do achive 2 phase commit.
you can not implement traditional transaction system in micro-services in a distributed environment.
You should you Event Sourcing + CQRS technique and because they are atomic you will gain something like implementing transactions or 2PC in a monolithic system.
Other possible way is transaction-log-mining that I think linked-in is using this way but it has its own cons and pros. for e.g. binary log of different databases are different and event in same kind of database there are differences between different versions.
I suggest that you use Event Sourcing + CQRS and string events in an event-store then try reaching eventual consistency base on CAP theorem after transferring multiple events between micro-service A and B and updating domain states in each step.
It is suggested that you use a message broker like ActiveMQ, RabbitMQ or Kafka for sending event-sourced events between different microservices and string them in an event store like mysql or other systems.
Another benefit of this way beside mimicking transactions is that you will have a complete audit log.
It is an architecture(microservices) problem. Spring boot or netflix-oss do not offer a direct solution. You have to implement your own solution. Check with event driven architecture. It can give you some ideas.
You could try the SAGA pattern https://microservices.io/patterns/data/saga.html
I am working on one application using EJB1.2. previously running fine but from past few days I am getting following exception
Exception in ejbLoad:: weblogic.ejb20.cache.CacheFullException: size=85783, target=5000, incr=1 at weblogic.ejb20.cache.EntityCache$SizeTracker.shrinkNext(JI)Lweblogic.ejb20.cache.EntityCache$MRUElement;(EntityCache.java:438) at weblogic.ejb20.cache.EntityCache.put
(Ljavax.transaction.Transaction;Lweblogic.ejb20.cache.CacheKey;Ljavax.ejb.EntityBean;Lweblogic.ejb20.interfaces.CachingManager;)V(EntityCache.java:141) at weblogic.ejb20.manager.DBManager.getReadyBean(Ljavax.transaction.Transaction;Ljava.lang.Object;)Ljavax.ejb.EntityBean;(DBManager.java:332) at
weblogic.ejb20.manager.DBManager.preInvoke(Lweblogic.ejb20.internal.InvocationWrapper;)Ljavax.ejb.EnterpriseBean;(DBManager.java:249) at
weblogic.ejb20.internal.BaseEJBLocalObject.preInvoke(Lweblogic.ejb20.internal.InvocationWrapper;)Lweblogic.ejb20.internal.InvocationWrapper;(BaseEJBLocalObject.java:228) at weblogic.ejb20.internal.EntityEJBLocalObject.preInvoke(Lweblogic.ejb20.internal.MethodDescriptor;Lweblogic.security.service.ContextHandler;)Lweblogic.ejb20.internal.InvocationWrapper;(EntityEJBLocalObject.java:72) at com.nextjet.enterprise.locationcode.locationcode.LocationCode_v2epgs_ELOImpl.getLocationCodeData()Lcom.nextjet.enterprise.locationcode.LocationCodeData;(LocationCode_v2epgs_ELOImpl.java:28) at com.nextjet.enterprise.locationcode.locationcodemanager.LocationCodeManagerBean.loadShippingAddress(Ljava.lang.Long;Ljava.lang.String;)Lcom.nextjet.enterprise.locationcode.LocationCodeView;(LocationCodeManagerBean.java:538) at com.nextjet.enterprise.locationcode.locationcodemanager.LocationCodeManagerBean.doSearchShippingAddresses(Ljava.lang.String;)Lcom.nextjet.enterprise.locationcode.LocationCodeSearchResult;(LocationCodeManagerBean.java:514) at com.nextjet.enterprise.locationcode.locationcodemanager.LocationCodeManagerBean.lookupAccountShipping.....
For now I am changing value of <max-beans-in-cache> in weblogic-ejb-jar.xml
I am changing the above value to <max-beans-in-cache>100000</max-beans-in-cache>
is it the only solution for this kind of exception or could there be a data related issue from database?
10000 is quite a high value for max-beans-in-cache and from the log it seems the application tried to make a call for up to 85785 instances of the EJB.
I would suggest some refactoring in your code.
Your code is doing
com.nextjet.enterprise.locationcode.locationcode.LocationCode_v2epgs_ELOImpl
.getLocationCodeData()
Is this is mainly a read operation ? Or are you doing simultaneous writes and reads?
You could refactor this in 2 ways to reduce the EJB overheads if it is mainly doing read operations.
1) Read Oracle's recommendation on tuning the EJB settings and database options for concurrency, especially the Read-Mostly pattern
http://docs.oracle.com/cd/E13222_01/wls/docs81/ejb/entity.html#ChoosingaConcurrencyStrategy
2) If you are mainly doing reads - then dont use Entity EJBs at all. Use the FastLaneReader pattern which is using a direct JDBC call to fetch the data for SELECT, and you can do writes using the EJBs as at present. In this way, the max-beans-in-cache can be reduced
A very detailed example is given on the Sun Design Patterns site
http://java.sun.com/blueprints/patterns/FastLaneReader.html
I have an application with about 20 models and controllers and am not using any particular framework. What is the best practice for using multiple remote objects in Flex performance-wise?
1) Method 1 - One per Component - Each component instantiates a RemoteObject for itself
2) Method 2 - Multiple in Application Root - Each controller is handled by a RemoteObject in the root
3) Method 3 - One in Application Root - Combine all controllers into one class and handle them with one RemoteObject
I'm guessing 3 will have the best performance but will be too messy to maintain and 1 would be the cleanest but would take a performance hit. What do you think?
Best practice would be "none of the above." Your Views should dispatch events that a controller or Command component would use to call your service(s) and then update your model on return of the data. Your Views would be bound to the data, and then the Views would automatically be updated with the new data.
My preference is to have one service Class per different piece or type of data I am retrieving--this makes it easier to build mock services that can be swapped for real services as needed depending on what you're doing (for instance if you have a complicated server setup, a developer who is working on skinning would use the mocks). But really, how you do that is a matter of personal preference.
So, where do your services live, so that a controller or command can reach them? If you use a Dependency Injection framework such as Robotlegs or Swiz, it will have a separate object that handles instantiating, storing, and and returning instances of model and service objects (in the case of Robotlegs, it also will create your Command objects for you and can create view management objects called Mediators). If you don't use one of these frameworks, you'll need to "roll your own," which can be a bit difficult if you're not architecturally minded.
One thing people who don't know how to roll their own (such as the people who wrote the older versions of Cairngorm) tend to fall back on is Singletons. These are not considered good practice in this day and age, especially if you are at all interested in unit testing your work. http://misko.hevery.com/code-reviewers-guide/flaw-brittle-global-state-singletons/
A lot depends on how much data you have, how many times it gets refreshed from the server, and of you have to support update as well as query.
Number 3 (and 2) are basically a singletons - which tends to work best for large applications and large datasets. Yes, it would be complex to maintain yourself, but that's why people tend to use frameworks (puremvc, cairgorm, etc). much of the complexity is handled for you. Caching data within the frameworks also enhances performance and response time.
The problem with 1 is if you have to coordinate data updates per component, you basically need to write a stateless UI, always retrieving the data from the server on each component visibility.
edit: I'm using cairgorm - have ~ 30 domain models (200 or so remote calls) and also use view models. some of my models (remote object) have 10's of thousands of object instances (records), I keep a cache with/write back. All of the complexity is encapsulated in the controller/commands. Performance is acceptable.
In terms of pure performance, all three of those should perform roughly the same. You'll of course use slightly more memory by having more instances of RemoteObject and there are a couple of extra bytes that get sent along with the first request that you've made with a given RemoteObject instance to your server (part of the AMF protocol). However, the effect of these things is negligible. As such, Amy is right that you should make a choice based on ease of maintainability and not performance.