1/ Does FLASHBACK and SELECT AS OF/ VERSION BETWEEN use the same source of history to fall back to ? This question is related to the second question.
2/ I am aware that FLASHBACK cannot go back before a DDL change.
My question is for SELECT AS OF, would it be able to select something before a DDL change.
Take for example
CREATE TABLE T
(col1 NUMBER, col2 NUMBER)
INSERT INTO T(col1, col2) VALUES('1', '1')
INSERT INTO T(col1, col2) VALUES('2', '2')
COMMIT;
SLEEP(15)
ALTER TABLE T DROP COLUMN col2;
SELECT * FROM T
AS OF SYSTIMESTAMP - INTERVAL '10' SECOND;
Would the select return 2 columns or 1 ?
Pardon me I do not have a database at hand to test.
Any DDL that alter the structure of a table invalidates any existing undo data for the table. So you will get the error 'ORA-01466' unable to read data - table definition has changed.
Here is a simple test
CREATE TABLE T
(col1 NUMBER, col2 NUMBER);
INSERT INTO T(col1, col2) VALUES('1', '1');
INSERT INTO T(col1, col2) VALUES('2', '2');
COMMIT;
SLEEP(15)
ALTER TABLE T DROP COLUMN col2;
SELECT * FROM T
AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '60' SECOND);
ERRROR ORA-01466 upon executing the above select statement.
However DDL operations that alter the storage attributes of a table do no invalidate undo data so that you can still use flashback query.
1) FLASHBACK TABLE and SELECT .. AS OF use the same source, UNDO. There is also FLASHBACK DATABASE - although it uses the same mechanism it uses a separate source, flashback logs that must be optionally configured.
2) Flashback table and flashback queries can go back before a DDL change if you enable a flashback archive.
To use that feature, add a few statements to the sample code:
CREATE FLASHBACK ARCHIVE my_flashback_archive TABLESPACE users RETENTION 10 YEAR;
...
ALTER TABLE t FLASHBACK ARCHIVE my_flashback_archive;
Now this statement will return 1 column:
SELECT * FROM T;
And this statement will return 2 columns:
SELECT * FROM T AS OF SYSTIMESTAMP - INTERVAL '10' SECOND;
Related
From the Vertica Docs:
DROP_PARTITION ( table_name , partition_value [ , ignore_moveout_errors, reorganize_data ])
Can the partition_value predicate be dynamic by any method?
I want to drop the partition based on MAX(partition_col_val) condition from another staging table.
Vertica also does not support variable creation as such, where I can keep the MAX(partition_col_val).
Is there any workaround possible?
You cannot use sub-querys to generate a dynamic predicate value for you drop_partition.
Normally i treat the lack of the dynamic feature with a script that will generate the drop_partition sql expresion into a .sql file and i execute in the next step.
See an example here :
create table tblone (id int not null) partition by id;
insert into tblone values (1);
commit;
-- flush the output of the query into a file
\o /tmp/file.sql
select
'SELECT DROP_PARTITION(''public.tblone'','||
max(id)||
');' from tblone;
-- execute the content of the file
\i /tmp/file.sql
--this is the content.
SELECT DROP_PARTITION('public.tblone',1);
This is when you have partition based on non dates and data that needs to be derived from other data sets.
In case you have date as your partition key or a derived value from a data column you could use a internal function to dynamically populate the drop_partition key value:
drop table tblone cascade;
create table tblone (id date not null) partition by
(((date_part('year', id) * 100) + date_part('month', id)));
insert into tblone values (getdate());
commit;
dbadmin=> select * from tblone;
id
------------
2017-01-04
(1 row)
dbadmin=> SELECT DROP_PARTITION('tblone',(date_part('year', getdate()) * 100) + date_part('month', getdate()));
DROP_PARTITION
-------------------
Partition dropped
(1 row)
dbadmin=> select * from tblone;
id
----
(0 rows)
-you can always play with the getdate() to get the current or last month or any periods you wish to.
Another option is to use the vsql cmd line variables
Example
dbadmin=> drop table tblone cascade;
DROP TABLE
dbadmin=> create table tblone (id int not null) partition by id;
CREATE TABLE
dbadmin=> insert into tblone values (1);
1
dbadmin=> commit;
COMMIT
dbadmin=> select * from tblone;
id
----
1
(1 row)
-- show only tuples
dbadmin=> \t
Showing only tuples.
-- spit the max value into a file
dbadmin=> \o /tmp/file
dbadmin=> select max(id) from tblone;
dbadmin=> \o
dbadmin=> \t
Tuples only is off.
-- set the value of the variable to the file content(your max value)
dbadmin=> \set maxvalue `cat /tmp/file`
dbadmin=> \echo :maxvalue
1
-- run the drop partition using the variable
dbadmin=> SELECT DROP_PARTITION('tblone',:maxvalue);
DROP_PARTITION
-------------------
Partition dropped
(1 row)
dbadmin=> select * from tblone;
id
----
(0 rows)
i hope this helped :)
An easy way to drop many partitions from a table using a single line of code would be using MOVE_PARTITIONS_TO_TABLE into a dummy table and then drop the dummy table
- this will require no lock on the main table and dropping the dummy table is an cheap task for the database (will be a bulk drop_partition's).
create dummy table from base table (including projections).
generate dynamic MOVE_PARTITIONS_TO_TABLE('source','from
partition','to partition','target/dummy table').
drop dummy table.
see small example (is not 100 complete- you can adapt it)
is the same approach (generate & execute)
\o /tmp/file.sql
select 'create dummy table as source table including projections;';
select
'SELECT MOVE_PARTITIONS_TO_TABLE(''source'','''||
:minpartition()||
''','''||
:maxpartition()||
''',''target/dummy table'')'
from tblone;
select 'drop table dummy cascade';
-- execute the content of the file
\i /tmp/file.sql
--make sure the content of the file is correct content
**
BTW - if you look for Vertica Database Articles and Scripts from time
to time i post at http://wwww.aodba.com
**
I have to create a procedure which searches any recently added records and if there are then move them to ARCHIVE table.
This is my statement which filters recently added records
SELECT
CL_ID,
CL_NAME,
CL_SURNAME,
CL_PHONE,
VEH_ID,
VEH_REG_NO,
VEH_MODEL,
VEH_MAKE_YEAR,
WD_ID,
WORK_DESC,
INV_ID,
INV_SERIES,
INV_NUM,
INV_DATE,
INV_PRICE
FROM
CLIENT,
INVOICE,
VEHICLE,
WORKS,
WORKS_DONE
WHERE
Client.CL_ID=Invoice.INV_CL_ID and
Invoice.INV_CL_ID = Client.CL_ID and
Client.CL_ID = Vehicle.VEH_CL_ID and
Vehicle.VEH_ID = Works_Done.WD_VEH_ID and
Works_done.WD_INV_ID = Invoice.INV_ID and
WORKS_DONE.WD_WORK_ID = Works.WORK_ID and
Works_done. Timestamp >= sysdate -1;
You may need something like this (pseudo-code):
create or replace procedure moveRecords is
vLimitDate timestamp := systimestamp -1;
begin
insert into table2
select *
from table1
where your_date >= vLimitDate;
--
delete table1
where your_date >= vLimitDate;
end;
Here are the steps I've used for this sort of task in the past.
Create a global temporary table (GTT) to hold a set of ROWIDs
Perform a multitable direct path insert, which selects the rows to be archived from the source table and inserts their ROWIDs into the GTT and the rest of the data into the archive table.
Perform a delete from the source table, where the source table ROWID is in the GTT of rowids
Issue a commit.
The business with the GTT and the ROWIDs ensures that you have 100% guaranteed stability in the set of rows that you are selecting and then deleting from the source table, regardless of any changes that might occur between the start of your select and the start of your delete (other than someone causing a partitioned table row migration or shrinking the table).
You could alternatively achieve that through changing the transaction isolation level.
O.K. may be something like this...
The downside is - it can be slow for large tables.
The upside is that there is no dependence on date and time - so you can run it anytime and synchronize your archives with live data...
create or replace procedure archive is
begin
insert into archive_table
(
select * from main_table
minus
select * from archive_table
);
end;
I need to truncate a table that has around 40 million records in it. i guess it is ok to REUSE STORAGE for the table since new data will be inserted after the truncate.
I need to know
approximately how much time the command
TRUNCATE TABLE <tablename> REUSE STORAGE; would take.
Can this be done offline, so that the operations/ commands on the DB
don't get affected?
Based on "so that the operations/ commands don't get affected", it sounds like you'd like to replace the contents of a table with "good" values, without anyone seeing an empty table. If users/processes need to be able to continue making changes to the table while you switch out the contents, you'll need to research DBMS_REDEFINITION. If this is a read-only table (in other words, you can do this operation while nobody is inserting/updating/deleting its contents), there's a nice DIY approach that takes advantage of partition exchanging:
1) Create a table which has identical partitioning and indexing as original, and has the desired "good" rows.
2)
IF YOUR TABLE IS PARTITIONED:
CREATE TABLE <tablename>_TMP AS SELECT * FROM <tablename> WHERE 1=0;
-- Add any local indexes on <tablename> as standard indexes on <tablename_tmp>
BEGIN
FOR R IN (SELECT * FROM USER_TAB_PARTITIONS WHERE TABLE_NAME = <tablename>)
LOOP
EXECUTE IMMEDIATE
'INSERT INTO <tablename>_TMP'
||' SELECT * FROM <good_data_tablename> PARTITION ('||R.PARTITION_NAME||')';
EXECUTE IMMEDIATE
'ALTER TABLE <tablename>_TMP'
||' EXCHANGE PARTITION WITH TABLE <tablename> PARTITION ' || R.PARTITION_NAME
||' INCLUDING INDEXES WITHOUT VALIDATION UPDATE GLOBAL INDEXES';
EXECUTE IMMEDIATE
'TRUNCATE TABLE <tablename>_TMP';
END LOOP
END;
/
DROP TABLE <tablename>_TMP ;
IF YOUR TABLE IS NOT PARTITIONED:
CREATE TABLE <tablename>_TMP
PARTITION BY RANGE (<non-null-column>)
(PARTITION ALL_DATA values less than (maxvalue))
AS SELECT * FROM <good_data_tablename>;
-- Add any <tablename> indexes as local indexes on <tablename>_TMP
ALTER TABLE <tablename> EXCHANGE PARTITION WITH TABLE <tablename>_TMP INCLUDING INDEXES WITHOUT VALIDATION;
DROP TABLE <tablename>_TMP ;
In Oracle, I have a requirement where in I need to insert records from Source to Target and then update the PROCESSED_DATE field of source once the target has been updated.
1 way is to use cursors and loop row by row to achieve the same.
Is there any other way to do the same in an efficient way?
No need for a cursor. Assuming you want to transfer those rows that have not yet been transfered (identified by a NULL value in processed_date).
insert into target_table (col1, col2, col3)
select col1, col2, col3
from source_table
where processed_date is null;
update source_table
set processed_date = current_timestamp
where processed_date is null;
commit;
To avoid updating rows that were inserted during the runtime of the INSERT or between the INSERT and the update, start the transaction in serializable mode.
Before you run the INSERT, start the transaction using the following statement:
set transaction isolation level SERIALIZABLE;
For more details see the manual:
http://docs.oracle.com/cd/E11882_01/server.112/e26088/statements_10005.htm#i2067247
http://docs.oracle.com/cd/E11882_01/server.112/e25789/consist.htm#BABCJIDI
A trigger should work. The target table can have a trigger that on update, updates the source table's column with the processed date.
My preferred solution in this sort of instance is to use a PL/SQL array along with batch DML, e.g.:
DECLARE
CURSOR c IS SELECT * FROM tSource;
TYPE tarrt IS TABLE OF c%ROWTYPE INDEX BY BINARY_INTEGER;
tarr tarrt;
BEGIN
OPEN c;
FETCH c BULK COLLECT INTO tarr;
CLOSE c;
FORALL i IN 1..tarr.COUNT
INSERT INTO tTarget VALUES tarr(i);
FORALL i IN 1..tarr.COUNT
UPDATE tSource SET processed_date = SYSDATE
WHERE tSource.id = tarr(i).id;
END;
The above code is an example only and makes some assumptions about the structure of your tables.
It first queries the source table, and will only insert and update those records - which means you don't need to worry about other sessions concurrently inserting more records into the source table while this is running.
It can also be easily changed to process the rows in batches (using the fetch LIMIT clause and a loop) rather than all-at-once like I have here.
Got another answer from some one else. Thought that solution seems much more reasonable than enabling isolation level as all my new records will have the PROCESSED_DATE as null (30 rows which inserted with in the time the records got inserted in Target table)
Also the PROCESSED_DATE = NULL rows can be updated only by using my job. No other user can update these records at any point of time.
declare
date_stamp date;
begin
select sysdate
into date_stamp
from dual;
update source set processed_date = date_stamp
where procedded_date is null;
Insert into target
select * from source
where processed_date = date_stamp;
commit;
end;
/
Let me know any further thoughts on this. Thanks a lot for all your help on this.
I have a question regarding a unified insert query against tables with different data
structures (Oracle). Let me elaborate with an example:
tb_customers (
id NUMBER(3), name VARCHAR2(40), archive_id NUMBER(3)
)
tb_suppliers (
id NUMBER(3), name VARCHAR2(40), contact VARCHAR2(40), xxx, xxx,
archive_id NUMBER(3)
)
The only column that is present in all tables is [archive_id]. The plan is to create a new archive of the dataset by copying (duplicating) all records to a different database partition and incrementing the archive_id for those records accordingly. [archive_id] is always part of the primary key.
My problem is with select statements to do the actual duplication of the data. Because the columns are variable, I am struggling to come up with a unified select statement that will copy the data and update the archive_id.
One solution (that works), is to iterate over all the tables in a stored procedure and do a:
CREATE TABLE temp as (SELECT * from ORIGINAL_TABLE);
UPDATE temp SET archive_id=something;
INSERT INTO ORIGINAL_TABLE (select * from temp);
DROP TABLE temp;
I do not like this solution very much as the DDL commands muck up all restore points.
Does anyone else have any solution?
How about creating a global temporary table for each base table?
create global temporary table tb_customers$ as select * from tb_customers;
create global temporary table tb_suppliers$ as select * from tb_suppliers;
You don't need to create and drop these each time, just leave them as-is.
You're archive process is then a single transaction...
insert into tb_customers$ as select * from tb_customers;
update tb_customers$ set archive_id = :v_new_archive_id;
insert into tb_customers select * from tb_customers$;
insert into tb_suppliers$ as select * from tb_suppliers;
update tb_suppliers$ set archive_id = :v_new_archive_id;
insert into tb_suppliers select * from tb_suppliers$;
commit; -- this will clear the global temporary tables
Hope this helps.
I would suggest not having a single sql statement for all tables and just use and insert.
insert into tb_customers_2
select id, name, 'new_archive_id' from tb_customers;
insert into tb_suppliers_2
select id, name, contact, xxx, xxx, 'new_archive_id' from tb_suppliers;
Or if you really need a single sql statement for all of them at least precreate all the temp tables (as temp tables) and leave them in place for next time. Then just use dynamic sql to refer to the temp table.
insert into ORIGINAL_TABLE_TEMP (SELECT * from ORIGINAL_TABLE);
UPDATE ORIGINAL_TABLE_TEMP SET archive_id=something;
INSERT INTO NEW_TABLE (select * from ORIGINAL_TABLE_TEMP);