I am new to DCN, can I use it to detect updates on a column in my table along with inserts in that table ?
I am referring to this
Yes, you can - Change Notifications made for that. You need to register СN listener with query to watch (it can a whole table select * from your_table or part of it select column1 from your_table where column2='xxx') and callback function . You should understand that it is async mechanism changes will not detect immediately, but after some time.
Your documentation's link shows way how to implement it using JDBC.
Read it if you want to use Oracle PL/SQL for that.
Related
MySQL has an "on update" feature e.g.
CREATE TABLE t1 (
ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
dt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
I need a similar behavior in snowflake where I can update a column say "lastupdated" every time there is an update on the row.
Is this possible in snowflake?
Checkout "Snowflake Stream" option. You can create a stream on top of your table, and stream will have a couple of columns which will give you exactly what you're looking for!
Its not very well explored feature unfortunately!
In other database implementations this is achieved through triggers.
Snowflake doesn't support triggers.
I wonder if you could create a stored procedure in Snowflake to accomplish what you are trying to do.
If you are trying to update a row with a timestamp
or you could just update the field in your copy or replace statement.
Similarly done this way: https://community.snowflake.com/s/question/0D50Z00006uSiEKSA0/syntax-for-adding-a-column-with-currenttimestamp-default-constraint
Example 1:
> UPDATE <target_table>
SET Lastupdate = current_timestamp()
[ FROM <additional_tables> ]
[ WHERE <condition> ]
Example 2:
>create or replace table x(i int, t timestamp default current_timestamp());
>insert into x(i) values(1);
borrowed from this link
as noted above, triggers are not supported - you'll have to do this explicitly in sql. note that your process should also be handling data in some type of batches; if you are trying to do anything a single record at a time in snowflake - at least for any real volume - you're going to have a bad time.
That's a pretty nice feature request. I've been using MS SQL Server for years... any "updated" columns were either done in the code or, as already indicated, using triggers.
I checked the snowflake docs and found this reference, which only applies to INSERTs and CTAS:
DEFAULT ... or AUTOINCREMENT ...
Specifies whether a default value is automatically inserted in the column if a value is not explicitly specified via an **INSERT or CREATE TABLE AS SELECT** statement:
https://docs.snowflake.net/manuals/sql-reference/sql/create-table.html
you can do something like this:
CREATE or REPLACE TABLE t1 (
ts TIMESTAMP_LTZ(9) as CURRENT_TIMESTAMP,
dt DATE as CURRENT_DATE,
NAME VARCHAR(200)
);
insert into t1 (NAME) VALUES ('Jerry Smith');
insert into t1 (NAME) VALUES ('Gazorpazorp Smith');
select * from t1;
just means that your values change every time you select from the table
You can use a combination of streams, external or internal stages and eventing , to record the DML CRUD changes. actually this combination is very elegant because your simulated triggers can trigger external events.
1) Create a stream
create stream supplierStream on table SupplierTable before(statement => 'yourGUID `statementID');`
2) Configure your event grid topic if using Azure. let's say your topic name is "SupplierTopic"
MS event grid
3) create your notification integration
CREATE NOTIFICATION INTEGRATION supplierIntegration
ENABLED = true
TYPE = QUEUE
NOTIFICATION_PROVIDER = AZURE_STORAGE_QUEUE
4) create your stage
create or replace stage supplierStage
url='azure://your account container ID'
storage_integration = SupplierIntegration;
5) consume the event grid event , in server or serverless system.
Have you tried MERGE combined with UPDATE?
https://docs.snowflake.com/en/sql-reference/sql/merge.html
Long time user, first time "asker".
I am attempt to construct an Oracle procedure and/or trigger that will compare two tables with the MINUS operation and then insert any resulting rows into another table. I understand how to do the query in standard SQL, but I am having trouble coming up with an efficient way to do this using PL/SQL.
Admittedly, I am very new to Oracle and pretty green with SQL in general. This may be a silly way to go about accomplishing my goal, so allow me to explain what I am attempting to do.
I need to create some sort of alert that will be triggered when the V_$PARAMETER view is changed. Apparently triggers can not respond to changes to view but, instead, can only replace actions on views...which I do not wish to do. So, what I did was create a table that to mirror that view to essentially save it as a "snapshot".
create table mirror_v_$parameter as select * from v_$parameter;
Then, I attempted to make a procedure that would minus these two so that, whenever a change is made to v_$parameter, it will return the difference between the snapshot, mirror_v_$parameter. I trying to create a cursor with the command:
select * from v_$parameter minus select * from mirror_v_$parameter;
to be used inside a procedure, so that it could be used to fetch any returned rows and insert them into another table called alerts_v_$parameter. The intent being that, when something is added to the "alert" table, a trigger can be used to somehow (haven't gotten this far yet) notify my team that there has been a change to the v_$parameter table, and that they can refer to alerts_v_$parameter to see what has been change. I would use some kind of script to run this procedure at a regular interval. And maybe, some day down the line when I understand all this better, manipulate what goes into the alerts_v_$parameter table so that it provides better information such as specifically what column was changed, what was its previous value, etc.
Any advice or pointers?
Thank you for taking the time to read this. Any thoughts will be very appreciated.
I would create a table based on the exact structure of v_$parameter with an additional timestamp column for "last_update", and periodically (via DBMS_Scheduler) merge into it any changes from the real v_$parameter table and capture the timestamp of any detected change.
You might also populate a history table at the same time, either using triggers on update of your table or with SQL.
PL/SQL is unlikely to be required, except as a procedural wrapper to the SQL code.
Examples of Merge are in the documentation here: http://docs.oracle.com/cd/E11882_01/server.112/e26088/statements_9016.htm#SQLRF01606
I have a self referencing table in Oracle 9i, and a view that gets data from it:
CREATE OR REPLACE VIEW config AS
SELECT c.node_id,
c.parent_node_id,
c.config_key,
c.config_value,
(SELECT c2.config_key
FROM vera.config_tab c2
WHERE c2.node_id = c.parent_node_id) AS parent_config_key,
sys_connect_by_path(config_key, '.') path,
sys_connect_by_path(config_key, '->') php_notation
FROM config_tab c
CONNECT BY c.parent_node_id = PRIOR c.node_id
START WITH c.parent_node_id IS NULL
ORDER BY LEVEL DESC
The table stores configuration for PHP application. Now I need to use same config in oracle view.
I would like to select some values from the view by path, but unfortunately this takes 0,15s so it's unacceptable cost.
SELECT * FROM some_table
WHERE some_column IN (
SELECT config_value FROM config_tab WHERE path = 'a.path.to.config'
)
At first I thought of a function index on sys_connect_by_path, but it is impossible, as it needs also CONNECT BY clause.
Any suggestions how can I emulate an index on the path column from the 'config' view?
If your data doesn't change frequently in the config_tab, you could use a materialized view with the same query as your view. You could then index the path column of your materialized view.
CREATE MATERIALIZED VIEW config
REFRESH COMPLETE ON DEMAND
AS <your_query>;
CREATE INDEX ix_config_path ON config (path);
Since this is a complex query, you would need to do a full refresh of your materialized view every time the base table is updated so that the data in the MV doesn't become stale.
Update
Your column path will be defined as a VARCHAR2(4000). You could limit the size of this column in order to index it. In your query, replace sys_connect_by_path(...) by SUBSTR(sys_connect_by_path(..., 1, 1000) for example.
You won't be able to use REFRESH ON COMMIT on a complex MV. A simple trigger won't work. You will have to modify the code that updates your base table to include a refresh somehow, I don't know if this is practical in your environment.
You could also use a trigger that submits a job that will refresh the MV. The job will execute once you commit (this is a feature of dbms_job). This is more complex since you will have to check that you only trigger the job once per transaction (using a package variable for example). Again, this is only practical if you don't update the base table frequently.
Can I find out when the last INSERT, UPDATE or DELETE statement was performed on a table in an Oracle database and if so, how?
A little background: The Oracle version is 10g. I have a batch application that runs regularly, reads data from a single Oracle table and writes it into a file. I would like to skip this if the data hasn't changed since the last time the job ran.
The application is written in C++ and communicates with Oracle via OCI. It logs into Oracle with a "normal" user, so I can't use any special admin stuff.
Edit: Okay, "Special Admin Stuff" wasn't exactly a good description. What I mean is: I can't do anything besides SELECTing from tables and calling stored procedures. Changing anything about the database itself (like adding triggers), is sadly not an option if want to get it done before 2010.
I'm really late to this party but here's how I did it:
SELECT SCN_TO_TIMESTAMP(MAX(ora_rowscn)) from myTable;
It's close enough for my purposes.
Since you are on 10g, you could potentially use the ORA_ROWSCN pseudocolumn. That gives you an upper bound of the last SCN (system change number) that caused a change in the row. Since this is an increasing sequence, you could store off the maximum ORA_ROWSCN that you've seen and then look only for data with an SCN greater than that.
By default, ORA_ROWSCN is actually maintained at the block level, so a change to any row in a block will change the ORA_ROWSCN for all rows in the block. This is probably quite sufficient if the intention is to minimize the number of rows you process multiple times with no changes if we're talking about "normal" data access patterns. You can rebuild the table with ROWDEPENDENCIES which will cause the ORA_ROWSCN to be tracked at the row level, which gives you more granular information but requires a one-time effort to rebuild the table.
Another option would be to configure something like Change Data Capture (CDC) and to make your OCI application a subscriber to changes to the table, but that also requires a one-time effort to configure CDC.
Ask your DBA about auditing. He can start an audit with a simple command like :
AUDIT INSERT ON user.table
Then you can query the table USER_AUDIT_OBJECT to determine if there has been an insert on your table since the last export.
google for Oracle auditing for more info...
SELECT * FROM all_tab_modifications;
Could you run a checksum of some sort on the result and store that locally? Then when your application queries the database, you can compare its checksum and determine if you should import it?
It looks like you may be able to use the ORA_HASH function to accomplish this.
Update: Another good resource: 10g’s ORA_HASH function to determine if two Oracle tables’ data are equal
Oracle can watch tables for changes and when a change occurs can execute a callback function in PL/SQL or OCI. The callback gets an object that's a collection of tables which changed, and that has a collection of rowid which changed, and the type of action, Ins, upd, del.
So you don't even go to the table, you sit and wait to be called. You'll only go if there are changes to write.
It's called Database Change Notification. It's much simpler than CDC as Justin mentioned, but both require some fancy admin stuff. The good part is that neither of these require changes to the APPLICATION.
The caveat is that CDC is fine for high volume tables, DCN is not.
If the auditing is enabled on the server, just simply use
SELECT *
FROM ALL_TAB_MODIFICATIONS
WHERE TABLE_NAME IN ()
You would need to add a trigger on insert, update, delete that sets a value in another table to sysdate.
When you run application, it would read the value and save it somewhere so that the next time it is run it has a reference to compare.
Would you consider that "Special Admin Stuff"?
It would be better to describe what you're actually doing so you get clearer answers.
How long does the batch process take to write the file? It may be easiest to let it go ahead and then compare the file against a copy of the file from the previous run to see if they are identical.
If any one is still looking for an answer they can use Oracle Database Change Notification feature coming with Oracle 10g. It requires CHANGE NOTIFICATION system privilege. You can register listeners when to trigger a notification back to the application.
Please use the below statement
select * from all_objects ao where ao.OBJECT_TYPE = 'TABLE' and ao.OWNER = 'YOUR_SCHEMA_NAME'
I tried to put it in a sentence but it is better to give an example:
SELECT * FROM someTable WHERE id = someID;
returns no rows
...
some time passes (no inserts are done to the table and no ID updates)
...
SELECT * FROM someTable WHERE id = someID;
returns one row!
Is it possible that some DB mechanism prevents first SELECT to return row?
Oracle log has no errors.
No transactions are rolled back when two selects are executed.
You can't see uncommitted data in another session. When did the commit happen?
EDIT1: Are you the only one using this database? Or did/do you have multiple sessions?
I think in another session you or someone else has inserted this row, you do your select and you don't see this row. After that a commit happens in the other session (maybe implicit because a session is closed) and then you see this row when you select again.
I can think of other explanations, but I first want to know are you only one using this database.
With read consistency as provided by Oracle, you should not see a row appear like that. If you are running in some mode with automatic commits, so that each statement is a self-contained transaction, then read consistency is not being violated. Which program are you using to access the database? I agree with the other observations; the row should not appear if your session is not inserting it and no other session is active at the same time. I don't know of a DBMS that indulges in spontaneous data generation.
Don't you have scheduled jobs in that Oracle?