I'm junior backend developer and now I'm working on a project about bank, which is a distributed system. What I knew before was that there were some message library such as ZeroMQ to realize the communication between components in a distributed system. But now, in the project, they used oracle queuing.
My colleague told me that this was better because we had no risk to lose any message to send even if processes die accidently.
My questions:Q1: If Oracle queuing is better, when should we use things like ZeroMQ?
andQ2: What is the disadvantage of Oracle queuing, comparing with ZeroMQ?
Your colleague is right here, because Oracle AQ comes with persistence and zeroMQ is in-memory. You'd use zeroMQ if you need max messages per second (millions). Price isn't a thing, because Oracle doesn't charge extra for AQ, it's even free with Oracle XE.
If your application relies on Oracle already, there are no disadvantages putting the messaging into Oracle.
Related
I'm actually working in a situation where a .NET stack is managing an Oracle Database. Now, because the legacy code is consistently based on PL SQL stored procedures that deal with the majority of the work, the correct choice of driver to connect to the database is of primary importance.
For this reason, knowing that Oracle provides a large number of driver for the most known programming languages, I was trying to find a documented benchmark (even with all the problem and the influence of the context in which the tests are made) that could compare the different Oracle drivers for the different programming languages, just to support the hypothesis that the best choice in terms of performance for an isolated test microservice would be to use the Java driver in combination with Scala (Java is now property of Oracle now, after all).
Are there any on the internet that could support (or not) this hypothesis?
EDIT
I didn’t provide all the information. What I’m trying to achieve is develop a series of microservices focused on fetching data from the database and convert them into json to be consumed by the front end. .NET driver behaves seamlessly until the numbers become really overwhelming (> 1000 rows).
That’s why I was wondering if it could make any sense using JDBC to increase performance, having heard that, for instance, .NET driver for SQL server, both made by the same Company, performs 5 times better than the oracle one when it comes to gathering data from a cursor.
Your choice of drives may not give you the milage if most of the work is in PL/SQL or stored procedures. Having said that, jdbc is a good choice. However do not fight if developers are more familiar with other drivers like Oracle ODBC, oracle provider for .NET, ADO,etc.. all protocols have a Oracle drive to access Oracle DB.
You don’t have to convert to json. Oracle DB can convert it. Your network latency is more impacted by how big the pipe is ,array size,and packet size than protocols.
I am trying to understand the persistence model of IBM MQ. I realize that the messages are stored in file based system but is there a way I can store the files in DB. I have come across article that states that the messages can be stored in the DB2 database, but I would like to know if I can store in other databases such as Oracle DB or SQLite or No-SQL Db's.
You may be misinterpreting the article. (A link would be nice.) Early versions of MQ were closely intertwined with IBM's database at the time and much of the product retains traces of that heritage in names, sizes of data structures, and some functional constraints.
However modern MQ's primary use of database is in the coupling facility in support of shared queues, and this is z/OS only. However that's the exception and for the vast majority of MQ versions and platforms you don't have the ability to select the queue persistence mechanism.
Remember that IBM's MQ was never intended to be a queueing front-end database client. MQ was invented to address problems of synchronous communication in which the outage of a single component caused entire systems to fail. The purpose of MQ was so that the application could treat the network as if it was 100% reliable (which was far from the case back in the 1980's). In order to achieve that MQ kept all critical operations local to the host on which it was installed and minimized external dependencies to things that were the most reliable: POSIX IPC and locally-mounted filesystems.
Some more recent message transports, especially JMS providers, give you the option of selecting a persistence store and even allowing it to be remote. Some are barely more than a JMS API over a database client. Though each of these approaches has valid use cases, IBM MQ has retained its focus on ultimate speed and enterprise-grade reliability.
Other IBM products such as the WAS Messaging Engine, MQTT, MQ Light, Sterling and others exist to fill some of the other requirement spaces. As for MQ, no there's no option to pick your persistence store other than the on the z/OS platform where your options are mostly related to the coupling facility.
AppHarbor looks very appealing for our .NET solution. But I have some questions I could not find on internet.
Our major concern is reliability of dedicated SQL Server:
Is it clustered / mirrored / replicated?
What happens when they upgrade / patch / maintain server or. hosted server and when hardware fails?
Are upgrades scheduled?
Can we set time interval when they do upgrades?
Which version and edition of Sql Server is used?
Can I use full text search?
Can I use Reporting service?
Is communication with SQL database reliable? For example in Azure SQL it is recommended to build in retry logic - if command does not succeed, retry.
Is AppHarbor reliable? Every cloud provider has occasionally some blackouts (Amazon, MS Azure ...). Is AppHarbor any less reliable compare to them? I know AppHarbor runs on top of Amazon.
Are there a lot of hidden issues you run into? What are the most common?
Did anybody decide to leave appHarbor for a good reason?
As far I can see Azure is a real cloud system with all the downside and upside - more scalable, but with modified infrastructure like customized SQL server .... AppHarbor mimics more on-premises solution. Is my understanding correct?
How is documentation?
How is support?
Thank you for your help.
Yes AppHarbor offers redundant/replicated dedicated SQL Server databases. These plans are available upon request.
This depends on the type of maintenance/update and your SQL Server database plan. If the database server is replicated, downtime can be minimized by failing over to the replica while performing maintenance. In the event of a server failure the database will be attached to a new instance and the application's configuration will be automatically updated. Should a hard drive fail leading to corrupted/lost data AppHarbor make daily backups that will be used to restore your database. It should be noted that hard drive failures are very rare.
We generally coordinate planned maintenance that requires downtime with customers whenever possible. Dedicated SQL Server customers can also select their own maintenance window.
Not really, but AppHarbor will reach out and coordinate with you when it is necessary.
Different SQL Server versions and editions are used depending on the plan. For single-instance dedicated SQL Servers we generally use SQL Server 2008 R2 Web Edition. Dedicated SQL Server 2012 instances are available upon request. Replicated setups require other and more expensive SQL Server editions. You may also want to consider our dedicated MySQL service if you'd like to reduce costs and don't rely on SQL Server specific features - since AppHarbor doesn't have to pay license costs these are less expensive, particularly for a replicated setup.
Yes.
Not by default, but we can work with you to support reporting services on your dedicated SQL Server instance.
Yes. In fact the primary reason customers upgrade from shared to dedicated SQL Server is for consistent, reliable performance.
I'd say so. The last major outage occurred on July 29th, 2012 due to an electrical storm that affecting multiple availability zones in AWS's North Virginia region. As an example, our blog has been available 99.997% of the time since then. In the event of an application instance failure applications are rapidly moved to healthy instances. We recommend running with at least two workers to ensure redundancy in those cases.
I'm admittedly not the best person to answer this question. The most common request/limitation we hear about is that you can't currently trigger a backup yourself. This will be available at a later time, but we do keep daily backups of your databases.
-
AppHarbor's cloud application platform is relatively similar to Azure in terms of scalability. We support rapid "elastic scaling" of application workers both vertically and horizontally. With regards to the dedicated SQL Server service your understanding is correct: It is very similar to an on-premise solution. While the scaling story is different compared to SQL Azure this allows for much greater flexibility. We can tailor a database plan and server that suits your requirements whether you need high CPU, RAM and/or I/O performance. Similarly we can offer database sizes that are 10x larger than SQL Azure's current 150GB database size limitation.
Most documentation is available in the knowledge base. We try and keep this as up-to-date and comprehensive as possible, but if you find yourself missing some information you're of course more than welcome to let us know and we'll add it. Third party add-on providers typically maintain their own AppHarbor-specific documentation.
This is another question where I may be a little biased, but I can tell a little about our goals: Our goal is to always answer non-critical support requests related to apps on both free and paid plans within the day. Critical support requests and supports requests related to applications or databases on paid plans take priority. Support is included in the plans, but we're working on offering premium support options as well. We generally try to exceed your expectations and are always happy to help out and give advice on issues you experience - whether they're related to the AppHarbor platform or not.
Disclaimer: I'm a co-founder of AppHarbor.
I'm trying to figure out what's the best way to design a active-active cluster that uses a replicated database. For network load balancing and failover, I can use Windows NLB. For database, I can use MySQL which can do master-master replication out of the box. This is the simple part.
Now my problem is how to program the messaging service, which is connected to a replicated database. What is the best way to go about designing it so that both services work with the same tables without conflict? On failure, the uncompleted transactions from the failed node must be assumed by the other node.
Here is how the messaging service works. Web clients will call the web service with a recipient and a message. The web service will insert the message into the database queue. When a specific condition is met, the message will be transmitted. This could happen within seconds or after a couple of days.
I've done extensive searches on the Internet but to no avail. Has anyone done something similar? Thanks.
If mysql isnt a requirement, i'd suggest a nosql memory optimized db like http://ravendb.net/, it is much more suited for high-availability than mysql. Just a thought.
Chat applications don't usually require transactions / intense reliability.
TSQL is more suited for financial applications, where reliability of data is the focus, for a chat you might be better off with a db where speed is the focus.
Just my 2 cents, since this seems more like a request for an opinion.
EDIT
Looks like raven supports ACID transactions, so even better.
I am developing some integration software for a client using amongst other things, C#, NServiceBus and Oracle 10g (client and server). The requirement is that I need to develop a new plugin for NServiceBus to create an implementation of ITransport which is the queuing mechanism for the messages. So Oracle Advanced Queuing is used for this. I have done quite a bit of work writing code for advanced queuing in Oracle 11g (client and server), but looking at ODP.Net 10g it seems that the queuing support is lacking or non-existent so that may prove to be problematic.
My question is this:
I know that you can use the 11g client against a 10g database server, but is it a good idea for Oracle Advanced Queuing and are there any gotchas I need to know about?
Many thanks.
Is it a good idea for Advanced Queueing? Well, I don't see why not, since 11g client connecting to 10g server is supported.
I don't imagine that AQ would pose any unique problem, specific to AQ. If you think about it, AQ is just PL/SQL calls that interact with tables under the covers. There's really nothing different at the client side, than there is with any other Oracle code.
So, I say go for it. But, as always: Test, test test. And then test some more.
But, in principle, I don't see a problem.
You may want to check out the NServiceBus-Contrib project, as there is an AQS transport there for 2.x.