Below is the major part of my script which interacts with an Oracle database through SQL*Plus.
#--------------- Now connecting to sqlplus
`$SQLPLUS \#${basePath}/VoucherQuery.sql $startdate> ${basePath}/logs/QueryResult.$currentDate.log`;
if ( $? == 0) {
logger("Processing with the sqlplus is completed. For more details check ${basePath}/logs/QueryResult.$currentDate.log ", 0); }
else {
logger("Not able to fetch data from sqlplus. Please check", 1); exit;}
#print "select * from sample where SERIALNUMBER = $serial\n";
open (FH, "${basePath}/logs/QueryResult.$currentDate.log") or die "Can't open query ${basePath}/logs/QueryResult.$currentDate.log file: $!\n";
my ($serial_number, $state, $at, $operator_id, $old_state);
while (my $data = <FH>) {
chomp ($data);
#print $data."\n";
my #data = split (/\s+/, $data);
my ($serial_number, $state, $at, $operator_id, $old_state) = #data[0..4];
my ($date, $time) = split (/T/, $at);
$date =~ s/(\d{4})(\d{2})(\d{2})/$3-$month{$2}-$1/;
$date =~ s/(.*)(\d{2})(\d{2})/$1$3/;
print WFH "$circle,$date,$time,$operator_id,$serial_number,$old_state,$state\n";
}
close(FH);
close(WFH);
-------------------------------------------------------------
>cat VoucherQuery.sql
SELECT * FROM (SELECT serialnumber, state, at, operatorid, lag(state) OVER ( PARTITION BY serialnumber ORDER BY at) AS previous FROM VOUCHER) WHERE at LIKE '&1';
But the database table contains millions of records and even a simple select count(*) query isn't able to generate output. Now the problem is that no constraints have been defined while creating the database.
I have experience of scripting, but I am quite a novice in SQL queries as far as performance is concerned.
I want to ask
How much difference will it make if I define the primary key constraint within the table? (It is a third-party server so I have to be sure before making any changes.)
Will an index improve the performance? How could they help in this specific query?
Should I break this query into more, simpler queries?
Here is the table description
SQL> DESC VOUCHER;
Name Null? Type
-------------------------- -------- -------------------------------
SERIALNUMBER VARCHAR2(20)
STATE VARCHAR2(4000)
AT VARCHAR2(4000)
OPERATORID VARCHAR2(4000)
SUBSCRIBERID VARCHAR2(20)
TRANSACTIONID VARCHAR2(20)
One more thing. I have to deal with SQL*Plus only I can't use DBI with DBD::Oracle module because of Solaris issues. I want to solve this on my own but need your advice on these performance issues as I can't use hit and miss methods on them.
I am far less experienced in Oracle than other databases, so this may well not be correct. But if it works it should certainly be faster than your existing SQL because it limits the data it is working on first before adding the previous column.
SELECT
serialnumber,
state,
at,
operatorid,
LAG(state) OVER (PARTITION BY serialnumber ORDER BY at) AS previous
FROM
voucher
WHERE
serialnumber in (SELECT DISTINCT serialnumber FROM voucher WHERE at LIKE '&1')
Note that it won't produce exactly the same result as your own query because it will include all records relating to a given serialnumber as long as at least one of those records has an at column that matches &1.
Related
i'm working with stored procedures, i have a query that should bring one specific id
SELECT * into SID_INCOMING FROM
(
SELECT SID
FROM TBL_INCOMING
WHERE XKEY = ''||nro_tarjetatmp||'_'||fecha_hr_trasacfinal||'_'||datos_referencia_adquirentetmp||''
AND CODIGO_AUTORIZACION IN(''||codigo_autorizacion||'')
AND MIT IN(''||mit||'')
ORDER BY SID ASC
) WHERE ROWNUM <= 1;
the variables values are in order
4946110112060005_200116_74064350165099586691985
536018
05
when it's executed i get this result
but when i execute the same query with the same parameters i get this result
and this is the one i sould get with the procedure, so why is this happening?
it seems to me, that the sp is not considering the second and third parameter
any help is welcome, thanks
You need to use aliases to avoid your problem with AND MIT IN(''||mit||''): this predicate compares column MIT with itself.
SO just add aliases:
SELECT * into SID_INCOMING FROM
(
SELECT SID
FROM TBL_INCOMING ti
WHERE ti.XKEY = ''||your_proc_name.nro_tarjetatmp||'_'||your_proc_name.fecha_hr_trasacfinal||'_'||your_proc_name.datos_referencia_adquirentetmp||''
AND ti.CODIGO_AUTORIZACION IN(''||your_proc_name.codigo_autorizacion||'')
AND ti.MIT IN(''||your_proc_name.mit||'')
ORDER BY ti.SID ASC
) WHERE ROWNUM <= 1;
Also why do you add ''||? It doesn't add anything into your variables, but forces impicit type conversion in cases when they are not varchar2/char data types.
Could you show the results of "describe TBL_INCOMING" please? Since looks like CODIGO_AUTORIZACION is a number data type and probably MIT is number too.
Is there a hint to generate execution plan ignoring the existing one from the shared pool?
There is not a hint to create an execution plan that ignores plans in the shared pool. A more common way of phrasing this question is: how do I get Oracle to always perform a hard parse?
There are a few weird situations where this behavior is required. It would be helpful to fully explain your reason for needing this, as the solution varies depending why you need it.
Strange performance problem. Oracle performs some dynamic re-optimization of SQL statements after the first run, like adaptive cursor sharing and cardinality feedback. In the rare case when those features backfire you might want to disable them.
Dynamic query. You have a dynamic query that used Oracle data cartridge to fetch data in the parse step, but Oracle won't execute the parse step because the query looks static to Oracle.
Misunderstanding. Something has gone wrong and this is an XY problem.
Solutions
The simplest way to solve this problem are by using Thorsten Kettner's solution of changing the query each time.
If that's not an option, the second simplest solution is to flush the query from the shared pool, like this:
--This only works one node at a time.
begin
for statements in
(
select distinct address, hash_value
from gv$sql
where sql_id = '33t9pk44udr4x'
order by 1,2
) loop
sys.dbms_shared_pool.purge(statements.address||','||statements.hash_value, 'C');
end loop;
end;
/
If you have no control over the SQL, and need to fix the problem using a side-effect style solution, Jonathan Lewis and Randolf Geist have a solution using Virtual Private Database, that adds a unique predicate to each SQL statement on a specific table. You asked for something weird, here's a weird solution. Buckle up.
-- Create a random predicate for each query on a specific table.
create table hard_parse_test_rand as
select * from all_objects
where rownum <= 1000;
begin
dbms_stats.gather_table_stats(null, 'hard_parse_test_rand');
end;
/
create or replace package pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2;
end pkg_rls_force_hard_parse_rand;
/
create or replace package body pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2
is
s_predicate varchar2(100);
n_random pls_integer;
begin
n_random := round(dbms_random.value(1, 1000000));
-- s_predicate := '1 = 1';
s_predicate := to_char(n_random, 'TM') || ' = ' || to_char(n_random, 'TM');
-- s_predicate := 'object_type = ''TABLE''';
return s_predicate;
end force_hard_parse;
end pkg_rls_force_hard_parse_rand;
/
begin
DBMS_RLS.ADD_POLICY (USER, 'hard_parse_test_rand', 'hard_parse_policy', USER, 'pkg_rls_force_hard_parse_rand.force_hard_parse', 'select');
end;
/
alter system flush shared_pool;
You can see the hard-parsing in action by running the same query multiple times:
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
Now there are three entries in GV$SQL for each execution. There's some odd behavior in Virtual Private Database that parses the query multiple times, even though the final text looks the same.
select *
from gv$sql
where sql_text like '%hard_parse_test_rand%'
and sql_text not like '%quine%'
order by 1;
I think there is no hint indicating that Oracle shall find a new execution plan everytime it runs the query.
This is something we'd want for select * from mytable where is_active = :active, with is_active being 1 for very few rows and 0 for maybe billions of other rows. We'd want an index access for :active = 1 and a full table scan for :active = 0 then. Two different plans.
As far as I know, Oracle uses bind variable peeking in later versions, so with a look at the statistics it really comes up with different execution plans for different bind varibale content. But in older versions it did not, and thus we'd want some hint saying "make a new plan" there.
Oracle only re-used an execution plan for exactly the same query. It sufficed to add a mere blank to get a new plan. Hence a solution might be to generate the query everytime you want to run it with a random number included in a comment:
select /* 1234567 */ * from mytable where is_active = :active;
Or just don't use bind variables, if this is the problem you want to address:
select * from mytable where is_active = 0;
select * from mytable where is_active = 1;
I am using scala jdbc to check whether a partition exists for an oracle table. It is returning wrong results when an aggregate function like count(*) is used.
I have checked the DB connectivity and other queries are working fine. I have tried to extract the value of count(*) using an alias, But it failed. Also tried using getString. But it failed.
Class.forName(jdbcDriver)
var connection = DriverManager.getConnection(jdbcUrl,dbUser,pswd)
val statement = connection.createStatement()
try{
val sqlQuery = s""" SELECT COUNT(*) FROM USER_TAB_PARTITIONS WHERE
TABLE_NAME = \'$tableName\' AND PARTITION_NAME = \'$partitionName\' """
val resultSet1 = statement.executeQuery(sqlQuery)
while(resultSet1.next())
{
var cnt=resultSet1.getInt(1)
println("Count="+cnt)
if(cnt==0)
// Code to add partition and insert data
else
//code to insert data in existing partition
}
}catch(Exception e) { ... }
The value of cnt always prints as 0 even though the oracle partition already exists. Can you please let me know what is the error in the code? Is this giving wrong results because I am using scala jdbc to get the result of an aggregate function like count(*)? If yes, then what would be the correct code? I need to use scala jdbc to check whether the partition already exists in oracle and then insert data accordingly.
This is just a suggestion or might be the solution in your case.
Whenever you search the metadata tables of the oracle always use UPPER or LOWER on both side of equal sign.
Oracle converts every object name in to the upper case and store it in the metadata unless you have specifically provided the lower case object name in double quotes while creating it.
So take an following example:
-- 1
CREATE TABLE "My_table_name1" ... -- CASE SENSISTIVE
-- 2
CREATE TABLE My_table_name2 ... -- CASE INSENSITIVE
In first query, we used double quotes so it will be stored in the metadata of the oracle as case sensitive name.
In second query, We have not used double quotes so the table name will be converted into the upper case and stored in the metadata of the oracle.
So If you want to create a query against any metadata in the oracle which include both of the above cases then you can use UPPER or LOWER against the column name and value as following:
SELECT * FROM USER_TABLES WHERE UPPER(TABLE_NAME) = UPPER('<YOUR TABLE NAME>');
Hope, this will help you in solving the issue.
Cheers!!
wondering if there is way to validate a query before executing
Is there way to check/validate Query without executing it?
One way that we validate SQL is to add a condition to the SQL that could never be true.
Example:
long ll_rc
long ll_result
string ls_sql, ls_test
string ls_message
//Arbitrary SQL
ls_sql = "SELECT * FROM DUAL"
//This SQL when executed will always return 0 if successful.
ls_test = "select count(*) from ( " + ls_sql + " WHERE 1 = 2 )"
DECLARE l_cursor DYNAMIC CURSOR FOR SQLSA ;
PREPARE SQLSA FROM :ls_test;
OPEN DYNAMIC l_cursor;
ll_rc = SQLCA.SQLCODE
choose case ll_rc
case 0
//Success
ls_message = "SQL is properly formed"
case 100
//Fetched row not found. This should not be the case since we only opened the cursor
ls_message = SQLCA.SQLERRTEXT
case -1
//Error; the statement failed. Use SQLErrText or SQLDBCode to obtain the detail.
ls_message = SQLCA.SQLERRTEXT
end choose
CLOSE l_cursor ; //This will fail if open cursor failed.
messagebox( "Result", ls_message )
Note: If your SQL is VERY complicated, which I suspect it isn't, the database optimizer may take several seconds to prepare your SQL. It will be significantly less time than if you run the entire query.
Since the database is the final arbitrator for what is "valid" (table and column names and such) the general answer is no. Now you could come up with a class in PB which checks statement syntax, object names, etc. so you wouldn't have to touch the db but it would be obsolete as soon as any changes were made to the db.
Put the select statement in any script and compile it. Part of the work will be to check the SQL syntax against the database you are connected to.
Watch out: you need at least one bound variable in the column list of your SQL statement. This is not the case for other DML statements.
Example:
in my case:
select noms into :ls_ttt from contacts;
results in a message Unknown columns 'noms' in 'field list'.
However,
select nom into :ls_ttt from contacts;
does not show any error.
Hope this helps.
Using h2 in embedded mode, I am restoring an in memory database from a script backup that was previously generated by h2 using the SCRIPT command.
I use this URL:
jdbc:h2:mem:main
I am doing it like this:
FileReader script = new FileReader("db.sql");
RunScript.execute(conn,script);
which, according to the doc, should be similar to this SQL:
RUNSCRIPT FROM 'db.sql'
And, inside my app they do perform the same. But if I run the load using the web console using h2.bat, I get a different result.
Following the load of this data in my app, there are rows that I know are loaded but are not accessible to me via a query. And these queries demonstrate it:
select count(*) from MY_TABLE yields 96576
select count(*) from MY_TABLE where ID <> 3238396 yields 96575
select count(*) from MY_TABLE where ID = 3238396 yields 0
Loading the web console and using the same RUNSCRIPT command and file to load yields a database where I can find the row with that ID.
My first inclination was that I was dealing with some sort of locking issue. I have tried the following (with no change in results):
manually issuing a conn.commit() after the RunScript.execute()
appending ;LOCK_MODE=3 and the ;LOCK_MODE=0 to my URL
Any pointers in the right direction on how I can identify what is going on? I ended up inserting :
Server.createWebServer("-trace","-webPort","9083").start()
So that I could run these queries through the web console to sanity check what was coming back through JDBC. The problem happens consistently in my app and consistently doesn't happen via the web console. So there must be something at work.
The table schema is not exotic. This is the schema column from
select * from INFORMATION_SCHEMA.TABLES where TABLE_NAME='MY_TABLE'
CREATE MEMORY TABLE PUBLIC.MY_TABLE(
ID INTEGER SELECTIVITY 100,
P_ID INTEGER SELECTIVITY 4,
TYPE VARCHAR(10) SELECTIVITY 1,
P_ORDER DECIMAL(8, 0) SELECTIVITY 11,
E_GROUP INTEGER SELECTIVITY 1,
P_USAGE VARCHAR(16) SELECTIVITY 1
)
Any push in the right direction would be really appreciated.
EDIT
So it seems that the database is corrupted in some way just after running the RunScript command to load it. As I was trying to debug to find out what is going on, I tried executing the following:
delete from MY_TABLE where ID <> 3238396
And I ended up with:
Row not found when trying to delete from index "PUBLIC.MY_TABLE_IX1: 95326", SQL statement:
delete from MY_TABLE where ID <> 3238396 [90112-178] 90112/90112 (Help)
I then tried dropping and recreating all my indexes from within the context, but it had no effect on the overall problem.
Help!
EDIT 2
More information: The problem occurs due to the creation of an index. (I believe I have found a bug in h2 and I have working on creating a minimal case that reproduces it). The simple code below will reproduce the problem, if you have the right set of data.
public static void main(String[] args)
{
try
{
final String DB_H2URL = "jdbc:h2:mem:main;LOCK_MODE=3";
Class.forName("org.h2.Driver");
Connection c = DriverManager.getConnection(DB_H2URL, "sa", "");
FileReader script = new FileReader("db.sql");
RunScript.execute(c,script);
script.close();
Statement st = c.createStatement();
ResultSet rs = st.executeQuery("select count(*) from MY_TABLE where P_ID = 3238396");
rs.next();
if(rs.getLong(1) == 0)
System.err.println("It happened");
else
System.err.println("It didn't happen");
} catch (Throwable e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I have reduced the db.sql script to about 5000 rows and it still happens. When I attempted to go to 2500 rows, it stopped happening. If I remove the last line of the db.sql (which is the index creation), the problem will also stop happening. The last line is this:
CREATE INDEX PUBLIC.MY_TABLE_IX1 ON PUBLIC.MY_TABLE(P_ID);
But the data is an important player in this. It still appears to only ever be the one row and the index somehow makes it inaccessible.
EDIT 3
I have identified the minimal data example to reproduce. I stripped the table schema down to a single column, and I found that the values in that column don't seem to matter -- just the number of rows. Here is the contents of (snipped with obvious stuff) of my db.sql generated via the SCRIPT command:
;
CREATE USER IF NOT EXISTS SA SALT '8eed806dbbd1ea59' HASH '6d55cf715c56f4ca392aca7389da216a97ae8c9785de5d071b49de5436b0c003' ADMIN;
CREATE MEMORY TABLE PUBLIC.MY_TABLE(
P_ID INTEGER SELECTIVITY 100
);
-- 5132 +/- SELECT COUNT(*) FROM PUBLIC.MY_TABLE;
INSERT INTO PUBLIC.MY_TABLE(P_ID) VALUES
(1),
(2),
(3),
... snipped you obviously have breaks in the bulk insert here ...
(5143),
(3238396);
CREATE INDEX PUBLIC.MY_TABLE_IX1 ON PUBLIC.MY_TABLE(P_ID);
But that will recreate the problem. [Note that my numbering skips a number every time there was a bulk insert line. So there really is 5132 rows, though you see 5143 select count(*) from MY_TABLE yields 5132]. Also, I seem to be able to recreate the problem in the WebConsole directly now by doing:
drop table MY_TABLE
runscript from 'db.sql'
select count(*) from MY_TABLE where P_ID = 3238396
You have recreated the problem if you get 0 back from the select when you know you have a row in there.
Oddly enough, I seem to be able to do
select * from MY_TABLE order by P_ID desc
and I can see the row at this point. But going directly for the row:
select * from MY_TABLE where P_ID = 3238396
Yields nothing.
I just realized that I should note that I am using h2-1.4.178.jar
The h2 folks have already apparently resolved this.
https://code.google.com/p/h2database/issues/detail?id=566
Just either need to get the code from version control or wait for the next release build. Thanks Thomas.