I am using table stream for my ETL pipeline. I am just wondering is there any possibility of flushing the data from table stream manually without saving it anywhere in other table?
Snowflake will advance the stream when it sees the stream after FROM in a DML statement. It doesn't care how or if you use the rows, so you can do this:
insert into JUNK_TABLE select ANY_COLUMN from MY_STREAM where false;
If you run just this part of the insert, you can see that nothing will get inserted:
select ANY_COLUMN from MY_STREAM where false;
The where clause will evaluate to false for every row because that's what it's set to return. This means the insert statement won't insert a single row, but it will consume the stream.
Here's a mini script to test this:
-- Quick setup:
create or replace table MY_TABLE(COL1 varchar);
create or replace stream MY_STREAM on table MY_TABLE;
--Create a junk table so the syntax works:
create table JUNK_TABLE like MY_TABLE;
insert into MY_TABLE(COL1) values ('Row1'), ('Row2'), ('Row3');
select * from MY_STREAM; --The stream has three rows
insert into JUNK_TABLE select COL1 from MY_STREAM where false; --Consume the stream
select * from MY_STREAM; -- The stream has no rows
select * from JUNK_TABLE; -- Neither does the junk table because "where false" on the insert
Have you considered recreating the Stream with a CREATE OR REPLACE STREAM ... ?
If you had to do thatperiodically, you could also make it part of a stored procedure to run within a scheduled Task.
Related
Can I create an AFTER TRIGGER on a table and using that table in my SELECT query without getting mutating table error?
Example to a query I want to use.
This query will update number of times a certain status name is showing up in alert life cycle:
CREATE OR REPLACE TRIGGER COUNT_STEP
AFTER INSERT
ON STEPS
FOR EACH ROW
DECLARE
V_COUNT_SETP VARCHAR (10000);
BEGIN
SELECT COUNT (STATUS_NAME)
INTO V_COUNT_SETP
FROM (SELECT A.ALERT_ID, S.STATUS_NAME
FROM ALERTS A, ALERT_STATUSES S, STEPS ST
WHERE :NEW.ALERT_INTERNAL_ID = A.ALERT_INTERNAL_ID
AND ST.ALERT_STATUS_INTERNAL_ID = S.STATUS_INTERNAL_ID
AND S.STATUS_NAME IN ('Auto Escalate'))
GROUP BY ALERT_ID;
UPDATE ALERTS A
SET A.COUNT = V_COUNT_ESC
WHERE A.ALERT_INTERNAL_ID = :NEW.ALERT_INTERNAL_ID;
END;
/
The table I'm inserting a record to is also needed for counting the number of step occurrences since it's stores the alert id and all the steps id it had.
You need to be a bit more clearer in your questions. But, from what i understood, you need to create a trigger on a table, and perform a select for that same table. That gives you a mutanting table error. To bypass that, you need to perform a compound trigger on that table. Something like this:
create or replace trigger emp_ct
for insert on employees compound trigger
v_count number; -- Add variable here
before statement is
begin
-- PERFORM YOUR SELECT AND SEND TO A VARIABLE
end before statement;
after each row is
begin
-- DO WANT YOU WANTED TO DO. USE THE VARIABLE
end after each row;
end;
basically, with a compound trigger, you can capture every trigger event. By doing that, allows to query the table you're capturing.
I am new to Oracle. I have a table in Oracle which has 4 columns Period, Open_Flag,Creation_Dt,Updated_By.
The Period column is the Primary key of the table. I have created a proc which will check the value of period from input parameter in the table, if its existing, the value of Open_flag has to be updated else a new record shall be inserted.
create or replace
PROCEDURE PROC_REF_SAP_PERIOD(
V_PERIOD IN NUMBER,V_OPEN_FLAG IN VARCHAR2,V_CREATION_DT IN DATE,V_UPDATED_BY IN VARCHAR2)
AS
BEGIN
MERGE INTO REF_SAP_PERIOD T
USING (SELECT * FROM REF_SAP_PERIOD WHERE PERIOD=V_PERIOD )S
ON (T.PERIOD=S.PERIOD )
WHEN MATCHED THEN UPDATE SET OPEN_FLAG = V_OPEN_FLAG --WHERE PERIOD=V_PERIOD AND CREATION_DT=V_CREATION_DT AND UPDATED_BY=V_UPDATED_BY
WHEN NOT MATCHED THEN INSERT (PERIOD,OPEN_FLAG,CREATION_DT,UPDATED_BY) VALUES (V_PERIOD,V_OPEN_FLAG,V_CREATION_DT,V_UPDATED_BY);
END;
The issue is that the Update is working well in this case, however, the insert is not working. Please help.
You are merging table with itself, filtered by period. Obviously, it will never see your non-existent values in itself.
Try this line instead of your USING line:
using (select V_PERIOD "period" from dual)S
I have to create a procedure which searches any recently added records and if there are then move them to ARCHIVE table.
This is my statement which filters recently added records
SELECT
CL_ID,
CL_NAME,
CL_SURNAME,
CL_PHONE,
VEH_ID,
VEH_REG_NO,
VEH_MODEL,
VEH_MAKE_YEAR,
WD_ID,
WORK_DESC,
INV_ID,
INV_SERIES,
INV_NUM,
INV_DATE,
INV_PRICE
FROM
CLIENT,
INVOICE,
VEHICLE,
WORKS,
WORKS_DONE
WHERE
Client.CL_ID=Invoice.INV_CL_ID and
Invoice.INV_CL_ID = Client.CL_ID and
Client.CL_ID = Vehicle.VEH_CL_ID and
Vehicle.VEH_ID = Works_Done.WD_VEH_ID and
Works_done.WD_INV_ID = Invoice.INV_ID and
WORKS_DONE.WD_WORK_ID = Works.WORK_ID and
Works_done. Timestamp >= sysdate -1;
You may need something like this (pseudo-code):
create or replace procedure moveRecords is
vLimitDate timestamp := systimestamp -1;
begin
insert into table2
select *
from table1
where your_date >= vLimitDate;
--
delete table1
where your_date >= vLimitDate;
end;
Here are the steps I've used for this sort of task in the past.
Create a global temporary table (GTT) to hold a set of ROWIDs
Perform a multitable direct path insert, which selects the rows to be archived from the source table and inserts their ROWIDs into the GTT and the rest of the data into the archive table.
Perform a delete from the source table, where the source table ROWID is in the GTT of rowids
Issue a commit.
The business with the GTT and the ROWIDs ensures that you have 100% guaranteed stability in the set of rows that you are selecting and then deleting from the source table, regardless of any changes that might occur between the start of your select and the start of your delete (other than someone causing a partitioned table row migration or shrinking the table).
You could alternatively achieve that through changing the transaction isolation level.
O.K. may be something like this...
The downside is - it can be slow for large tables.
The upside is that there is no dependence on date and time - so you can run it anytime and synchronize your archives with live data...
create or replace procedure archive is
begin
insert into archive_table
(
select * from main_table
minus
select * from archive_table
);
end;
In Oracle, I have a requirement where in I need to insert records from Source to Target and then update the PROCESSED_DATE field of source once the target has been updated.
1 way is to use cursors and loop row by row to achieve the same.
Is there any other way to do the same in an efficient way?
No need for a cursor. Assuming you want to transfer those rows that have not yet been transfered (identified by a NULL value in processed_date).
insert into target_table (col1, col2, col3)
select col1, col2, col3
from source_table
where processed_date is null;
update source_table
set processed_date = current_timestamp
where processed_date is null;
commit;
To avoid updating rows that were inserted during the runtime of the INSERT or between the INSERT and the update, start the transaction in serializable mode.
Before you run the INSERT, start the transaction using the following statement:
set transaction isolation level SERIALIZABLE;
For more details see the manual:
http://docs.oracle.com/cd/E11882_01/server.112/e26088/statements_10005.htm#i2067247
http://docs.oracle.com/cd/E11882_01/server.112/e25789/consist.htm#BABCJIDI
A trigger should work. The target table can have a trigger that on update, updates the source table's column with the processed date.
My preferred solution in this sort of instance is to use a PL/SQL array along with batch DML, e.g.:
DECLARE
CURSOR c IS SELECT * FROM tSource;
TYPE tarrt IS TABLE OF c%ROWTYPE INDEX BY BINARY_INTEGER;
tarr tarrt;
BEGIN
OPEN c;
FETCH c BULK COLLECT INTO tarr;
CLOSE c;
FORALL i IN 1..tarr.COUNT
INSERT INTO tTarget VALUES tarr(i);
FORALL i IN 1..tarr.COUNT
UPDATE tSource SET processed_date = SYSDATE
WHERE tSource.id = tarr(i).id;
END;
The above code is an example only and makes some assumptions about the structure of your tables.
It first queries the source table, and will only insert and update those records - which means you don't need to worry about other sessions concurrently inserting more records into the source table while this is running.
It can also be easily changed to process the rows in batches (using the fetch LIMIT clause and a loop) rather than all-at-once like I have here.
Got another answer from some one else. Thought that solution seems much more reasonable than enabling isolation level as all my new records will have the PROCESSED_DATE as null (30 rows which inserted with in the time the records got inserted in Target table)
Also the PROCESSED_DATE = NULL rows can be updated only by using my job. No other user can update these records at any point of time.
declare
date_stamp date;
begin
select sysdate
into date_stamp
from dual;
update source set processed_date = date_stamp
where procedded_date is null;
Insert into target
select * from source
where processed_date = date_stamp;
commit;
end;
/
Let me know any further thoughts on this. Thanks a lot for all your help on this.
I'm looking for the best way to change a data type of a column in a populated table. Oracle only allows changing of data type in colums with null values.
My solution, so far, is a PLSQL statement which stores the data of the column to be modified in a collection, alters the table and then iterates over the collection, restoring the original data with data type converted.
-- Before: my_table ( id NUMBER, my_value VARCHAR2(255))
-- After: my_table (id NUMBER, my_value NUMBER)
DECLARE
TYPE record_type IS RECORD ( id NUMBER, my_value VARCHAR2(255));
TYPE nested_type IS TABLE OF record_type;
foo nested_type;
BEGIN
SELECT id, my_value BULK COLLECT INTO foo FROM my_table;
UPDATE my_table SET my_value = NULL;
EXECUTE IMMEDIATE 'ALTER TABLE my_table MODIFY my_value NUMBER';
FOR i IN foo.FIRST .. foo.LAST
LOOP
UPDATE my_table
SET = TO_NUMBER(foo(i).my_value)
WHERE my_table.id = foo(i).id;
END LOOP;
END;
/
I'm looking for a more experienced way to do that.
The solution is wrong. The alter table statement does an implicit commit. So the solution has the following problems:
You cannot rollback after alter the alter table statement and if the database crashes after the alter table statement you will loose data
Between the select and the update users can make changes to the data
Instead you should have a look at oracle online redefinition.
Your solution looks a bit dangerous to me. Loading the values into a collection and subsequently deleting them fom the table means that these values are now only available in memory. If something goes wrong they are lost.
The proper procedure is:
Add a column of the correct type to the table.
Copy the values to the new column.
Drop the old column.
Rename the new column to the old columns name.