I have a database in Azure SQL Server, which stores messages from IoT devices. Those devices send periodic messages to a listener which is set up (along with a lot of logic before it ends up in SQL DB) and runs in node-red. Everything works well for a couple of weeks now.
Enter Logic Apps. I have a simple trigger that executes two stored procedures on a schedule (one issues major update on one table that is also used in trigger (on a table in SQL) firing when node-red inserts data, second inserts data into a table feeding PowerBI dashboard. Total runtime for two executions is less than 20 seconds.
The moment this trigger is enabled, node-red starts experiencing time-outs when connecting to the server. And I am literal: for 3 days no issues, enabled Logic Apps trigger and I can see incomplete data saved in DB. It doesn't even have to run, just enabled causes issues.
It was disabled last week due to same issues, though we didn't know then this was the cause. We assumed neighbors were crowding us and tier was low on that db. But we forked out some cash and upgraded. Yet problem is back. And I can definitely state this is the ONLY change that was made on Azure in regards to that setup in 3 days.
I am kinda stumped - not sure what is happening, so not sure how to fix it except disabling trigger again. What is going on?
Related
My application currently polls database every n seconds to see if there are any new records.
To reduce network round trips, and CPU cycles of this polling i was thinking to replace it with CQN based approach where database will itself update subscribed application if there is any Commit to database.
The only problem is what if Oracle was NOT able to notify application due to any connection issue between oracle and subscribed application or if the application was crashed or killed due to any reason? ... Is there a way to know if application have missed any CQN notification?
Is polling database via application code itself the only way for mission critical applications?
You didn't say whether every 'n' seconds means you're expecting data every few seconds, or you just need your "staleness" to as low as that. That has an impact on the choice of CQN, because as per docs, https://docs.oracle.com/en/database/oracle/oracle-database/12.2/adfns/cqn.html#GUID-98FB4276-0827-4A50-9506-E5C1CA0B7778
"Good candidates for CQN are applications that cache the result sets of queries on infrequently changed objects in the middle tier, to avoid network round trips to the database. These applications can use CQN to register the queries to be cached. When such an application receives a notification, it can refresh its cache by rerunning the registered queries"
However, you have control over how persistent you want the notifcations to be:
"Reliable Option:
By default, a CQN registration is stored in shared memory. To store it in a persistent database queue instead—that is, to generate reliable notifications—specify QOS_RELIABLE in the QOSFLAGS attribute of the CQ_NOTIFICATION$_REG_INFO object.
The advantage of reliable notifications is that if the database fails after generating them, it can still deliver them after it restarts. In an Oracle RAC environment, a surviving database instance can deliver them.
The disadvantage of reliable notifications is that they have higher CPU and I/O costs than default notifications do."
I have a Heroku server with about $250.00 worth of monthly addons (due to upgrades Heroku Postgres and Heroku Redis). I'm no longer using the server for the foreseeable future, but would like to be able to boot the server back up at a later date with the same configuration.
Is there a way to temporarily halt all server functionality to prevent myself from getting billed, with the possibility of rebooting the server at a later date?
Well, you can step down the dynos to hobby-dev tier if you've less than 2 process types. Or you can simply shut them down. Just go to https://dashboard.heroku.com/, click on your app and then go to the 'resources' tab to control the dynos.
Stepping down heroku-redis should be easy too. It's anyway temporary storage, that you can restart/scale up later. Also see this
The only sticking point might be your Postgres DB. If it has more than 10,000 rows, you'll have to pay atleast $9 per month, and if you've more than 1Mn rows in the DB, you'll have to pay atleast $50 per month. Many times DBs collect a lot of logs data. You can consider cleaning and compacting the data if that's possible. Or you can take a local Database dump and decommission the DB and when you decide to start the app again upload the DB (this is a bit of an extreme step though, so be doubly sure that you've everything backup up.)
My VB6 program uses ADODB to do a lot of SQL (2000) CRUD.
Sometimes the network connection between the remote clients and the data center somehow "drops" resulting in the impossibility to establish new connections (so users launching the program can't use it).
The issue is the following:
Anyone who is using the program at the moment of the "drop" can continue using it with no issues whatoever, perform every operation, update data, read data, and everything seems like is working normally.
User then proceeds to fire up a "sum-up" report which lists everything that was done (before or after the "drop").
If we then check the database, all data regarding whatever was done after the network drop is not there. User goes back into the program and everything is as it was before the network drop.
It seems like all queries where somehow performed in-memory ? I'm at a loss about how to even approach the issue (I'm familiar enough with VB6 to work with the source code but I don't know a lot about ADODB).
I haven't yet tried to replicate the behavior due to limited customer's availability (development environment is housed in their offices), I'll try starting up the program from the IDE then rip out the network cable.
Provided I can replicate the issue, how do I fix this ? Is there some setting I'm not aware of ?
On a side note: the issue is sporadic (it happened a handful of times during the last year, and the software is being used heavily and on a daily basis by mutiple concurrent users).
After reading up on Disconnected Recordsets, it seems that's what's behind this odd behavior I'm experiencing.
This is not something that can be simply "turned off".
I am experiencing very low performance in my web application in which trivial HTTP requests take dozens of seconds to be processed. Tracing down the application code I discovered the majority of time is spent executing the first DB query, even if it is as simple as a SELECT on a single row-single column table. This happens for every HTTP request, independently from the query performed. After this first pathological DB interaction the remaining queries go smoothly.
I am using Hibernate on top of an Oracle DB (using jdbc).
It is not a problem of connection pool since I am successfully using Hibernate-c3p0, neither it seems to be related to Oracle itself, because all query returns immediately if performed directly on DB.
Furthermore, Hibernate SessionFactory is correctly created only once, at application start up time and concurrency is not a problem at all since tests have been done with single user.
Finally, my DB IP address is correctly resolved in my application server /etc/hosts so that even DNS related issues can be discarded (I am using two distinct virtual machines, DB and APP server).
I do not know what to look for, any help?
This sounds like your session factory object is being spun up on the first query. Generally I try to initialize the session factory on application startup to avoid this when issuing the first query because generally the user can see this slowdown. When doing it up front in application startup you will avoid this.
I am in process of developing an MFC based Windows based application, using PostgreSQL which would perform
Fetches information from the UI
Performs some logic and store related information to the database
The stored information has to be send immediately OR at schedule interval (ex. at 5:00 on xyz date) over the network
Currently, we have developed a dispacher mechanism (thread ), which constantly polls the database for new information inserted in the database. The thread fetches the information and send to the network module.
But, I feel this is not the correct approach as
Polling every time is a overhead. There can be times when there is nothing to execute
It is not real time , because we poll after every 5 seconds
So
Is there a way to send a trigger to my network module as soon as information is updated in database?
Or any better way to achieve this task?
Thanks in advance.
You can use the listen/notify feature of PostgreSQL for this.
http://www.postgresql.org/docs/current/static/sql-listen.html
http://www.postgresql.org/docs/current/static/sql-notify.html
The clients interested in the messages would execute a listen statement and the trigger would then notify them.
I don't use C# so, but according to the manual you can retrieve the messages in an asynchronous manner - which still involves some "lightweight" polling as the notification message is only sent as part of the answer of the server. The manual claims that running an "empty" statement (such as ;) will be enough. Using Java/JDBC I used a simple select 42 which doesn't impose a big workload on the server as no tables are touched.
This polling is defintely faster and more scalable than actually retrieving the table's data.
Yes you are right #RDX, you shouldnt poll it every time rather you could write a trigger in Postgres and from that trigger try calling a java program which could be seen in the below thread.
Calling java pgm from Postgres trigger