Is there any major performance issues if I use virtual columns in an Oracle table?
We have a scenario where the db has fields stored as strings. Since other production apps run off those fields we can't easily convert them.
I am tasked with generating reports from the same db. Since I need to be able to filter by dates (which are stored as strings) it was brought to my attention that we could create a virtual date field so that I can query against that.
Has anyone ran into any roadblocks with this approach?
A virtual column is defined using an expression that is evaluated when you select from the table. There is no performance hit on inserts/updates on the table.
For example:
create table t1 (
datestr varchar2(100),
datedt date generated always as (to_date(datestr,'YYYYMMDD'))
);
Table created.
SQL> insert into t1 (datestr) values ('20160815');
1 row created.
SQL> insert into t1 (datestr) values ('xxx');
1 row created.
SQL> commit;
Commit complete.
Note that I was able to insert an invalid date value into datestr. Now we can try to select the data:
SQL> select * from t1 where datedt = date '2016-08-15';
ERROR:
ORA-01841: (full) year must be between -4713 and +9999, and not be 0
This could be a problem for you if you can't guarantee all the strings hold valid dates.
As for performance, when you run the above query what you are really running is:
select * from t1 where to_date(datestr,'YYYYMMDD') = date '2016-08-15';
So the query will not be able to use an index on the datestr column (probably), and you may want to add an index on the virtual column. Again, this won't work if any of the strings don't contain valid dates.
Another consideration is potential impact on existing code. Hopefully you won't have any code like insert into t1 values (...); i.e. not specifying the column list. If you do you will get the error:
ORA-54013: INSERT operation disallowed on virtual columns
Related
I have queries that take an existing large table and build tables off of them for reporting. The problem is that the source tables are 60-80MM+ records and it takes a long time to recreate. I'd like to be able to identify which records are new so I can build just add the new records to the reporting tables.
To me, the best way to identify this is to have an identity column. Is there any significant cost to creating this and adding it to the table?
Separately, is it possible to create a materialized view that takes data from one of these tables but add a sequence as part of the materialized view? That is, something like
create materialized view some_materialized_view as
select somesequence.nextval, source_table.*
from source_table?
You can add a sequence based column to your table, but as Gary suggests I wouldn't do that.
The task you are about to solve is so common that other solutions have been already implemented.
The first built-in option that comes to mind is the system change number SCN, a kind of Oracle internal clock. By default, tables are set up to record the SCN of the whole (usually 8K) block, containing usually many rows, but you can set a table to keep a record of the SCN that changed every row. Then you can track the columns that are new or change and have not been copied to your reporting tables.
CREATE TABLE t (c1 NUMBER) ROWDEPENDENCIES;
INSERT INTO t VALUES (1);
COMMIT;
SELECT c1, ora_rowscn FROM t;
Secondly, I would think of adding a date column. With 60-80 mio rows I wouldn't do this with ALTER TABLE xxx ADD (d DATE DEFAULT SYSDATE), but with rename, create as select, drop:
CREATE TABLE t AS SELECT * FROM all_objects;
RENAME t TO told;
CREATE TABLE t AS SELECT sysdate AS d, told.* FROM told;
ALTER TABLE t MODIFY d DATE DEFAULT SYSDATE;
DROP TABLE told;
Thirdly, I would read up on materialized views. I never had the chance to use this a work, but in theory, you should be able to set up a materialized view log on your 80 m table that records changes and updates dependent materialized views.
And forthly, I'd look into partitioning your large table on the (newly introduced) date column, so that identifying the new rows will become faster. That sadly depends on your version and Oracle license, though.
I have an Interface which's created with Oracle Forms. It has a base table block in which there's a field (namely col3"has the same name with the table's column that's derived from").
Form has two other fields col1 and col2 as of numeric type, and those fields are also members of the same table having col3 column mentioned above.
I converted col3 to a virtual column as the sum of the columns col1 and col2 in the table's definition( col3 number generated always as (nvl(col1,0)+nvl(col2,0)) virtual )
When I try to commit the changes in the base table it hurls the error message
ORA-54013: INSERT operation disallowed on virtual columns
I know applying a DML to a virtual column has no sense, but even I changed the col3 field's
Query Only property from No to Yes
Update Allowed property from Yes to No
Insert Allowed property from Yes to No
I get the same error message. I don't want to write explicit DML statements, since the form has lots of other fields, and they should also be listed in those statements.
Do you have any idea how I can overcome this problem without giving up base table block structure ? ( I'm using the version Fusion Middleware 11g )
As everything you do with that column is query, how about setting it to be a non-database column? It would require writing a POST-QUERY trigger, though - otherwise you wouldn't see its value after executing a query.
:block.col3 := :block.col1 + :block.col2;
Put the same code into WHEN-VALIDATE-ITEM triggers on both :block.col1 and :block.col2 items, so that you'd calculate the result when inserting/updating values.
Alternatively, create a procedure (within the form), put the above code into it, and then call the procedure from those triggers - for a long term, that's probably a better choice as you'd have to maintain code changes only in the procedure.
If I want to run a report daily and store the report's date as one of the column headers. Is this possible?
Example output (Counting the activities of employees for that day):
SELECT EMPLOYEE_NAME AS EMPLOYEE, COUNT(ACTIVITY) AS "Activity_On_SYSDATE" FROM EMPLOYEE_ACCESS GROUP BY EMPLOYEE_NAME;
Employee Activity_On_17042016
Jane 5
Martha 8
Sam 11
You are looking to do a reporting job with a data storing tool. The database (and SQL) is for storing and retrieving data, not for creating reports. There are special tools for creating reports.
In database design, it is very unhealthy to encode actual data in table or column name. Neither a table name nor a column name should have, as part of the name (and of the way they are used), an employee id, a date, or any other bit of actual data. Actual data should only be in fields, which in turn are in columns in different tables.
From what you describe, your base table should have columns for employee, activity and date. Then on any given day, if you want the count for the "current" day, you can query with
select employee, count(activity) ct
from table_name
where activity_date = SYSDATE
group by employee
If you want, you can also include the "activity_date" column in the output, that will show for which date the report was run.
Note that I assumed the column name for "date" is "activity_date." And in the output I used "ct" for a column alias, not "count." DATE and COUNT are reserved words, like SYSDATE, and you should NOT use them as table or column name. You could use them as aliases, as long as you don't need to refer to these aliases anywhere else in SQL, but it is still a very bad idea. Imagine you ever need to refer to a column (by name or by alias) and the name or alias is SYSDATE. What would a where clause like this mean?
where sysdate = sysdate
Do you see the problem?
Also, I can't tell from your question - were you thinking of storing these reports back in the database? To what end? It is better to store just one query and run it whenever needed (and make the "activity_date" for which you want the counts) an input parameter, so you can run the query for any date, at any time in the future. There is no need to store the actual daily reports in the database, as long as the base table is properly maintained.
Good luck!
I have a query to delete some records from a table, but take too much time.
The table is use it in a stored procedure to match another table.
Every time that the SP is executed the table is truncated and filled with 2 or 3 millions of records depending of the received parameters.
The table doesn't have any FK or constraints
The query to delete the records that I am using is:
DELETE FROM TABLE1
WHERE (fecha,hora_ini,origen,destino,tipo,valor,rowsm1) IN (
SELECT fecha_t,hora_t,origen_t,destino_t,tipo,valor,id_t
FROM TABLE2)
I try to decrease the time in execute the query creating an index based in the same columns of the query
CREATE INDEX smb1 ON table1 (fecha,hora_ini,origen,destino,tipo,valor,rowsm1);
And the query take more time to execute.
How can improve the performance of this "DELETE" query.
UPDATE
EXPLAIN PLAN OUTPUT
DELETE TABLE1
TABLE ACCESS TABLE1
TABLE ACCESS FULL TABLE1
TABLE ACCESS FULL TABLE2
TABLE ACCESS FULL TABLE2
The index you created looks like a quite big index:
CREATE INDEX smb1
ON table1 (fecha,hora_ini,origen,destino,tipo,valor,rowsm1);
Sure, this depends on the amount of data but generally I would rather look for one or two selective columns - if possible.
Don't forget, that the index data has to be read as well and if it doesn't help to speed up the query, you even loose performance.
This might for instance happen, if the table is very small, because the database reads data block by block (I think it was about 8K). A small table can be read in one step - no need to use an index here.
Or, if more or less all records are selected. In this case the table has to be read anyway.
If you want to speed up the query you should create the same index (with a good selectivity) on table2. This way the EXPLAIN PLAN will look somewhat lie this:
DELETE STATEMENT
DELETE
NESTED LOOPS SEMI
INDEX FULL SCAN
INDEX RANGE SCAN
You can switch off logging and delete the rows,
Here is an example, you can do it 2 ways,
1.) Physically chaning the table to Nologging
2.) Using Nologging hint in the delete statement.
1.) First approach
both testemp and testemp2 are same tables with same data while testemp takes over a minute , testemp2 takes only 1 second
SQL> delete from testemp;
14336 rows deleted.
Elapsed: 00:01:04.12
SQL>
SQL>
SQL> alter table testemp2 nologging;
Table altered.
Elapsed: 00:00:02.86
SQL>
SQL> delete from testemp2;
14336 rows deleted.
Elapsed: 00:00:01.26
SQL>
The table needs to be put back to logging only when we physically change the table using "Alter" command, if you are using as hint not required please see the example below
2.) Second approach
SQL> set timing on;
SQL> delete from testemp2;
14336 rows deleted.
Elapsed: 00:00:01.51
Deleting data after reinserting same data into table now with nologging;
SQL> delete /*+NOLOGGING*/ from testemp2;
14336 rows deleted.
Elapsed: 00:00:00.28
SQL> select logging from user_Tables where table_name='TESTEMP2';
LOG
---
YES
In my application I have created a column with sequence to have unique id for each and every record that I add,I doubt will there be any unique row-number created by oracle itself for every record we updated in table,if yes then how to access that row number like
SELECT row-number from table where employee_name='name';
here I want to get unique row number created by oracle,
I have searched on net but haven't got proper information
Oracle does maintain a ROWID for each row; however, in my opinion using ROWID in user-written code is both poor practice and dangerous. ROWID is only guaranteed to be constant for the duration of a single transaction. ROWID is not guaranteed to be constant forever and the database can change it if and when it determines that a change is necessary. If your data does not supply a value or combination of values which are unique and unchanging I strongly suggest you learn how to create an artificial key which is automatically set using sequences and triggers. I believe 12c supplies auto-increment columns which you can use if you're using the latest version of Oracle.
Share and enjoy.
There are 2 unique identifiers in Oracle named as ROWNUM and ROWID. You can use them in such ways:-
SELECT ROWNUM
FROM table
WHERE employee_name = 'name';
and
SELECT ROWID
FROM table
WHERE employee_name = 'name';
You can read further about them.
Rownum -
http://docs.oracle.com/cd/B12037_01/server.101/b10759/pseudocolumns008.htm
Rowid - http://docs.oracle.com/cd/B19306_01/server.102/b14200/pseudocolumns008.htm