We are building around 20+ mule projects and we are thinking if DB connector should be at domain level or individual project level. Please suggest. One drawback we can think of is that if one of the services is taking time or slow, it will impact other services as data base connection is shared.
Thanks,
You can share the connector libraries at the domain, or also the database configurations. A database configuration in a domain will be sharing connection with all applications that use it. If the connection is shared and one of the applications misbehaves then yes, it can affect the others by not releasing connections for example. On the other hand, deploying each app with the connector adds the library once per app, and you have to maintain all configurations separately. Shared connectors might make it easier to upgrade to a new version. Note that deploying a new version of a domain requires restarting all its applications.
It is up to you to determine if the trade offs are worth it or not. No one else can know the impact for your applications, the time to maintain, the effort to change configurations in 20 applications, etc.
Related
I'm working on a Spring Boot application, using AbstractRoutingDatasource in order to provide multitenancy feature. New tenants can be added dynamically, and each one has it's own datasource and pool configuration, and everything is working well, since we only have around 10 tenants right now.
But I'm wondering: since the application is running on a docker container, with limit resources, as the number of tenants grows, also more and more threads will be allocated for each connection (considering a pool from 1 to 30 threads for each tenant) and the container, at some point (with 50 tenants, for example), will be killed due to memory limit defined at container startup.
It appears to me that, this multitenancy solution (using AbstractRoutingDatasource) is not suitable to an application designed to be containerized since I can't simply scale it horizontally to deal with more tenants.
Am I missing something? Should I be worried about that?
The point made in the post is about the system resource exhaustion that might arise with the increasing volume of requests as a result of increased tenants in the system. I would like to address few points
The whole infrastructure can be managed efficiently using ECS & AWS Fargate so that when there is a huge load, there are automatically new containers spun up to take the load. In case of having separate servers, ELB might be of help. There will be no issues when spinning up new containers / servers as your services are stateless
Regarding the number of active connections to a database from your application, you should profile your app and understand the DAP data access patterns. Any master data or static information should NOT be taken always from the database (Except for the 1st time), instead they should be cached. There are many managed cache services that can help you scale better.
In regards to the database, it is understood that tenants have their own databases, in case of a very large tenant, try to scale out the databases as well.
Focus on building the entire suite of features using async features in JAVA or using RxJava so that the async nature will help managing the threads.
Since you have not mentioned what cloud your applications will be deployed, I have cited sample using AWS. However most of the features can be used across Azure , GCP or AWS.
There are lot of strategies to scale, the right understanding of the business needs and data usage patterns etc... could help us decide the right approach.
Hope this clarifies.
I want to create a production management system to be used by a small manufacturing firm. The system will allow to document different stages in manufacturing of equipment. The requirements are as follows:
1.Non browser based interface.Need something like Swing or AWT based.While i understand the convenience of implementing a browser based solution,the business owner insists on a non browser interface
2.Accessed from multiple systems.These systems will allow CRUD operations on the central system (Thin Client?)
3.The application will not have more than 3 concurrent users.
I need some advice regarding what would be a good path for this kind of application.Currently, i'm thinking of using Griffon with RMI. However, i don't have much development experience.Read a bit about Apache River (Jini) too.Would it be a good idea to use Griffon with RMI?
Please provide some advice. Thanks.
EDIT:after some reading, i've decided to use more mainstream frameworks.So, Griffon is not an option. How about Jini(Apache River) or OSGI (Apache Felix)?
Hmm how is that a project which recently moved out of the incubation phase be considered mainstream vs a project that's been used in production for more than 3 years now? Anyway, Apache River gives you access to Jini technology and nothing more; meaning you can't achieve item #1 of your list with River alone. River may use RMI for accessing remote resources, however you can use RMI directly, or try out DRMI, Kryonet, Hessian/Burlap, Spring's HTTP Invoker, Protocol Buffers, Avro/Thrift, REST, SOAP, ZMQ and many more.
Even if you choose one of these options and/or River you still have to define the following things
application structure (file structure and runtime behavior)
build setup
dependency management
testing profiles
packaging
deployment strategies
These things and more are what Griffon brings to the table. As you may have noticed the framework allows you to build up applications by adding plugins, reducing thew amount of time you must allot for hunting down dependencies, setting up bootstrap mechanism and getting things done. On the subject of remoting technologies have a look at the different options Griffon has to offer http://artifacts.griffon-framework.org/tags/plugin/remoting
Even more, you can also combine OpenDolphin (http://open-dolphin.org/dolphin_website/Home.html) with Griffon. There's even an example application found at the opendolhpin repository showing a full client-server application (build with Griffon, Grails and OpenDolphin) https://github.com/canoo/open-dolphin/tree/master/dolphin-griffon-crud
With what seems to be your current understanding of the problem, I would not recommend OSGI, especially for a small manufacturing firm (Possible maintenance issues, depending on the "personel").
The main reason why I wouldn't advocate JINI or OSGI in your case is because of what you said
However, i don't have much development experience.
JINI (Apache River) is a viable option as long as you fully understand the concepts of LookupService and service registrations, etc. There's tons of RMI going on here with possible firewall implications...
OSGI is not difficult but you may have issues deciding how to structure your applications as well as interacting with services, etc.
Try to stick to the simplest approach that you can handle for the implementation (Flexible design in mind): Make it work and then improve it.
There are simple Web Services options such as Spring Remoting (over http/https for example), unless Spring introduces too many concepts and headaches for your app.
What's the most proper and best practice driven way of configuration my transformations?
In other words let's imagine I have a big ETL solution based on kettle that does stuff by connecting to different data source, I would like to store these data sources in a centralized location and have each transformation look it up everytime it needs to connect somewhere.
In SSIS there is package configuration what is the alternative that I have with pentaho?
Ps: I do not want to install any 3rd party framework.
Thank you
This can be done in various ways.
Parameterising the database connections, and configuring the properties via kettle.properties. You could still access that kettle.properties from a shared area or something.
As above, but configuring the connections by reading credentials from a database. Has to be hand crafted, but can be made to work with some caveats.
If you use the repository, then the database connections are stored centrally anyway. So if you have a dev and a prd repo, when you promote, dont promote the db connection itself. Trickier than it sounds though.
As for all of that, the new 4.4(?) release should have proper lifecycle management to make dealing with all this stuff a lot easier!
What should we take care of before moving an application from a single Websphere Application Server to a Websphere cluster
This is my list from experience. It is not complete but should cover the most common problem areas:
Plan head the distributed session management configuration (ie. will you use memory-to-memory or database based replicaton). Make a notice that if you are still on 32-bit platform the resource requirement overhead from clustering might cause you instability issues if your application uses already lots of memory.
Make sure that everything you put into user sessions can be serialized with the default serializer (implements Serializable). You might otherwise run into problems with distributed sessions.
The same goes for everything you put into DynaCache. Make sure everything serializes properly.
Specify and make sure all the resource definitions (JDBC providers etc) will be made to a proper scope. I would usually recommend using the actual Cluster scope for everything that your applications installed to cluster use. That ensures the testing features work properly from proper points, and that you don't make conflicting definitions.
Make sure your application uses relative paths for resources in web interfaces. Once you start load balancing and stuff you can run into some serious problems if you have bolted down a lot of stuff.
If you had any sort of timers make sure they work well with clusters. With Quartz that means probably that you should use the JDBC store for timer tasks. With EJB Timers make sure you register the timers only once (it is possible to corrupt the timer database of WAS if you have several nodes attempting the registering at the exactly same time) and make sure you install them to Cluster scope.
Make sure you use the WAS provided SSO mechanisms. If you have a custom implementation please make sure it handles moving the user between servers in cluster well.
Keep it simple, depending on your requirements, try configuring your load balancer to use sticky sessions and not hold state in your HTTP Session. That way you don't need to use resource hungry in memory session replication.
Single Sign On isn't an issue for a single cluster as your HTTP clients will not be moving off the same http://server.acme.com/... host domain name.
Most of your testing should focus on database contention. If you have a highly transactional application (i.e. many writes to the same table) make sure you look at your database Isolation levels so that locks are not held unecessarily. Same goes for your transaction demarkaction. Keep transactions as brief as possible. If you dont have database skills yourself make sure you get a Database Analyst to help you monitor the database while you test.
Also a good advice to raise a PMR to IBM Support up front of any major changes, such as this one or upgrading to new versions etc. Raise it as a "Software Usage Question" and they can provide you with feedback from their knowledge database based on other customers input. Same would apply for any type of product which you have a support agreement for - ask support before problems occur.
Lets say you're starting a new web project that required the website to run on and MVC framework on Mono. A couple major requirements are that it has to scale easy, be stable and work with multiple servers that may or may not be in the same place or even on the same local network.
The first thing I thought of was a sort of cluster communication between servers. Each server would act as a node and be its own standalone application and would query other nodes in a known list for session information and things like that.
But one major design questions I have is should this functionality be built into the supporting framework or should the application handle the synchronization of the data?
Or am I just way off and this would never work?
Normaly clustering rather belongs to some kind of middleware layer, thus on your framework level. However it can also be implemented on the application level.
It depends on your exact use, if you want load balancing, scalability etc.