Is there a practical way for my app to get notified when Heroku Connect adds records to a table?
I currently have a Flask app connected to a Salesforce org via Heroku Connect. I have event listeners for before_insert, after_insert, before_update, after_update. Additionally, SQLALCHEMY_ECHO is set to True. When I create a record in Salesforce, none of the event listeners fire, and no SQL statements are printed. However, if I query the model that matches the mapped sObject, I can see the new record. Therefore, Heroku Connect must be updating the table, but in a way that doesn't trigger event listeners. I did read up a bit on pg_notify (LSITEN), but all solutions seem to involve a select loop, which is much less elegant than db.event.listens_for decorators.
Related
You have a command/operation which means you both need to save something in database end send an event/message to another system. For example you have an OrderService and when a new order is created you want to publish an "OrderCreated"-event for another system/systems to react on (either direct message or using a message broker) and do something.
The easiest (and naive) implementation is to save in db and if successful then send message. But of course this is not bullet proof because the other service/message broker is down or your service crash before sending message.
One (and common?) solution is to implement "outbox pattern", i.e. instead of publish messages directly you save the message to an outbox table in your local database as part of your database transaction (in this example save to outbox table as well as order table) and have a different process (polling db or using change data capture) reading the outbox table and publish messages.
What is your solution to this dilemma, i.e. "update database and send message or do neither"? Note: I am not talking about using SAGAs (could be part of a SAGA though but this is next level).
I have in the past used different approaches:
"Do nothing", i.e just try to send the message and hope it will be sent. Which might be fine in some cases especially with a stable message broker running on same machine.
Using DTC (in my case MSDTC). Beside all the problem with DTC it might not work with your current solution.
Outbox pattern
Using an orchestrator which will retry process if you have not got a "completed" event.
In my current project it is not handled well IMO and I want to change it to be more resilient and self correcting. Sometimes when a service is calling another service and it fails the user might retry and it might work ok. But some operations might require out support to fix it (if it is even discovered).
ATM it is not a Microservice solution but rather two large (legacy) monoliths communicating and is running on same server but moving to a Microservice architecture in the near future and might run on multiple machines.
I have no knowledge on Oracle EBS and Oracle Alert mechanism.
My understanding is that Oracle Alert works just like database trigger.
Will Oracle Alert fire when database updates/inserts happen from the back-end?
We have observed alert is only firing for transactions front end and not running for back-end updates?
Is it guaranteed that just like a Trigger, EBS Alert will fire on every update to the record?
My understanding is that Oracle Alert works just like database trigger.
Yes, it is somewhat like a database trigger created from the Front-End Application. To explain further, there are two types of Oracle Alerts, Periodic and Event Alerts.
Periodic Alerts are alerts have a specific schedule and run according to a set period and time.
Event Alerts are alerts that only send notifications whenever inserts or updates have been performed on a table from the Front-End Application.
Take note that for Event Alerts, the triggering table must be setup in Oracle EBS' Application Object Library (called an Application Table).
Will Oracle Alert fire when database updates/inserts happen from the backend?
No. Taking this line from Krishna Reddy:
Oracle Alerts can only be triggered from an application that has been
registered in Oracle Applications. Alerts cannot be triggered via SQL
updates or deletes to an Alert activated trigger.
To add more context, Oracle Alert is a simple and efficient way to give you an immediate view of the critical activities in your Oracle Application. It helps Business Users / Administrators be on top of important or unusual business events you need to know about via E-Mail. It can also automate a process depending on the user’s response.
Some weaknesses and limitations though, is that Oracle Alert cannot process rows up to more than around 50, and its Report Layout has a text-based design and does not support HTML. Also, the text width is also limited.
Check out the Oracle documentation and this good article about Oracle Alerts.
Can somebody help me how to send Notifications using Azure SQL Server ?
Lets say I want to create an Application where the user adds, updates, or deletes a schedule .
If the scheduler runs and finds that it has to send notifications for a particular time, say 6:00 pm . I was wondering if there is any way I can use SQL Server so it can send the notification when called by the scheduled job. ?
I believe, it is not possible with the SQL Azure. In SQL Server, you have Query Notifications. If you want to use SQL Azure, then i would offer you to implement the notifications functionality in your own application. So, application, when making the changes to the database, sends the notification as well. Somehow, for example, using Azure Queues - with the information about columns updated.
Is it possible to somehow subscribe to 'add to db' event using Node.js? Database is currently populated via Ruby on Rails. Thanks.
MongoDB wise
There's an ongoing discussion about triggers on mongodb Jira.
But for now you're stuck with storing auto-increment values alongside with your data, and using indexed queries to check if there's anything new.
Rails wise
I'm assuming you're using Mongoid. Use callbacks or observers to send messages to a fast capped collection / unix socket / whatever. Other ODMs shouldn't be too different.
You need to notify the Node.js process from the Rails app when you insert something in the DB.
Listen on a socket/port from Node.js process
From Rails write on that socket when record added
From Node, process each message on the socket
This is from the Stream - AQ docs.
You can register system events, user events, and notifications on queues with Oracle Internet Directory. System events are database startup, database shutdown, and system error events. User events include user log on and user log off, DDL statements (create, drop, alter), and DML statement triggers. Notifications on queues include OCI notifications, PL/SQL notifications, and e-mail notifications.
Sounds interesting. What does this get me?
I mean these things look like DDL Triggers... So it's a matter of not building the DDL trigger in a database but instead building it in OID and letting OID manage the firing of the trigger?
Having never used it, this is my guess.
Imagine you have a hundred databases, and you want to log every time people log into each one, you could do it on each individual server, but that would make answering questions like "Which databases did 'Mark' Login to" difficult".
So, instead, you have each database register its "user logon" events with OID (via AQ), you then have a process receiving these events from OID and logging them. You then have a single point where you can audit system wide logins.
You can likely also use it to propagate messages from one AQ to another, and to lookup what queues exist within the system which can be subscribed to.