This is on 11g. I have an INSERT ALL statement which uses a SELECT to build up the values to insert. The select has some subqueries that check that the record doesn't already exist. The problem is that the insert is taking over 30 minutes, even when there are zero rows to insert. The select statement on its own runs instantly, so the problem seems to be when it is used in conjunction with the INSERT ALL. I rewrote the statement to use MERGE but it was just as bad.
The table has no triggers. There is a primary key index, and a unique constraint on two of the columns, but nothing else that looks like it might be causing an issue. It currently has about 15000 rows, so definitely not big.
Has anyone a suggestion for what might be causing this, or how to go about debugging it?
Here's the INSERT ALL statement.
insert all
into template_letter_merge_fields (merge_field_id, letter_type_id,table_name,field_name,pretty_name, tcl_proc)
values (template_let_mrg_fld_sequence.nextval,letter_type_id,table_name,field_name, pretty_name, tcl_proc)
select lt.letter_type_id,
i.object_type as table_name,
i.interface_key as field_name,
i.pretty_name as pretty_name,
case
when w.widget = 'dynamic_select' then
'dbi::'||i.interface_key||'::get_name'
when w.widget = 'category_tree' and
i.interface_key not like '%_name' and
i.interface_key not like '%_desc' then
'dbi::'||i.interface_key||'::get_name'
else
'dbi::'||i.interface_key||'::get_value'
end as tcl_proc
from template_letter_types lt,
dbi_interfaces i
left outer join acs_attributes aa on (aa.object_type||'_'||aa.attribute_name = i.interface_key
and decode(aa.object_type,'person','party','aims_organisation','party',aa.object_type) = i.object_type)
left outer join flexbase_attributes fa on fa.acs_attribute_id = aa.attribute_id
left outer join flexbase_widgets w on w.widget_name = fa.widget_name
where i.object_type IN (select linked_object_type
from template_letter_object_map lom
where lom.interface_object_type = lt.interface_object_type
union select lt.interface_object_type from dual
union select 'template_letter' from dual)
and lt.interface_object_type = lt.interface_object_type
and not exists (select 1
from template_letter_merge_fields m
where m.sql_code is null
and m.field_name = i.interface_key
and m.letter_type_id = lt.letter_type_id)
and not exists (select 1
from template_letter_merge_fields m2
where m2.pretty_name = i.pretty_name
and m2.letter_type_id = lt.letter_type_id)
Related
I'm trying to execute a left join where multiple conditions must be met with the inclusion of pulling in the MAX sequence number that meets those conditions.
The left join is on the unique identifier in both tables. Table acaps_history has several rows for each app_id. I need to pull in only one row with the highest seq_number and activity_code of 'XU'. If the code 'XU' doesn't exist for the given app_id, then the case statement above should return 'N' for that row. The code I have currently just isn't working - returning the error "a column may not be outer-joined to a subquery":
create table orig_play3 as
(select
x.*,
case when xa.activity_code in 'XU' then 'Y' else 'N' end as cpo_flag
from
dfs_tab_orig_play_x x
left join cf.acaps_history xa on
x.APP_ID = xa.FOC_APPL_ID
and xa.activity_code in 'XU'
and xa.seq_number = (select max(seq_number) from cf.acaps_history where FOC_APPL_ID=x.app_id)
)
Given your error, it seems that the issue is the last part of your query:
and xa.seq_number = (select max(seq_number) from cf.acaps_history where FOC_APPL_ID=x.app_id)
This is still operating in the context of the ON clause, so the sub-query to find the max sequence number is the issue.
You should be able to avoid this by moving that sub-query out of the ON clause:
LEFT JOIN (
SELECT FOC_APPL_ID, activity_code, seq_number
FROM cf.acaps_history
WHERE activity_code in 'XU'
) xa
ON x.APP_ID = xa.FOC_APPL_ID
WHERE xa.seq_number = (select max(ah.seq_number) from cf.acaps_history ah where ah.FOC_APPL_ID=x.app_id and ah.activity_code in 'XU')
This may be the most inefficient way to execute this query, but it worked... It took like 3 minutes to run (table size is over 600K rows), but again, it returned the results I needed:
create table test as (
select x.*,
case when xb.activity_code in 'XU' then 'Y' else 'N' end as cpo_flag
from dfs_tab_orig_play_x x
left join
(select
xa.FOC_APPL_ID, xa.activity_code, xa.seq_number
from dfs_tab_orig_play_x x, cf.acaps_history xa
where x.app_id = xa.FOC_APPL_ID (+)
and xa.seq_number = (select max(seq_number) from cf.acaps_history where
x.app_id=FOC_APPL_ID(+) and activity_code in 'XU')) xb
on x.app_id = xb.FOC_APPL_ID (+)
)
If you are on 12c, I like OUTER APPLY for this sort of thing, because it lets you sort the rows for each app_id descending by seq_number and then just pick the highest one.
SELECT
x.*,
CASE
WHEN xa.activity_code IN 'XU' THEN 'Y'
ELSE 'N'
END
AS cpo_flag
FROM
dfs_tab_orig_play_x x
OUTER APPLY ( SELECT *
FROM cf.acaps_history xa
WHERE xa.foc_appl_id = x.app_id
AND xa.activity_code = 'XU'
ORDER BY xa.seq_number DESC
FETCH FIRST 1 ROW ONLY ) xa
Note: this logic is a little different from what you posted. In this version, it will join to the acaps_history row having the highest seq_number from among 'XU' records for the given app_id. Your version was joining to the row having the highest seq_number for the given app_id, whether that row was an 'XU' row or not. I am assuming (with little reason) that that was a bug on your part. But, if it wasn't, my version won't work as given.
Think of my two tables have the same columns. One column is the ID, and the other one is the text. Is it possible to implement the following pseudo code in PLSQL?
Compare each row (They will have the same ID)
If anything is different about them
Run a couple of queries: an Update, and an Insert
ElseIf they are the same
Do nothing
Else the row does not exist
So add the row to the table compared on
Is it easy to do this using PLSQL or should I create a standalone application to do do this logic.
As your table have the same columns, by using NATURAL JOIN you can easily check if two corresponding rows are identical -- without need to update your code if a column is added to your table.
In addition, using OUTER JOIN allow you to find the rows present in one table but not in the other.
So, you can use something like that to achieve your purpose:
for rec in (
SELECT T.ID ID1,
U.ID ID2,
V.EQ
FROM T
FULL OUTER JOIN U ON T.ID = U.ID
FULL OUTER JOIN (SELECT ID, 1 EQ FROM T NATURAL JOIN U) V ON U.ID = V.ID)
loop
if rec.id1 is null
then
-- row in U but not in T
elsif rec.id2 is null
then
-- row in T but not in U
elsif rec.eq is null
-- row present in both tables
-- but content mismatch
end if
end loop
Else the row does not exist
So add the row to the table compared on
Is this condition means that rows can be missed in both tables? If only in one, then:
insert into t1 (id, text)
select id, text
from t2
minus
select id, text
from t1;
If missed records can be in both tables, you need the same query that inserts into table t2 rows from t1.
If anything is different about them
If you need one action for any amount of different rows, then use something like this:
select count(*)
into a
from t1, t2
where t1.id = t2.id and t1.text <> t2.text;
if a > 0 then
...
otherwise:
for i in (
select *
from t1, t2
where t1.id = t2.id and t1.text <> t2.text) loop
<do something>
end loop;
A 'merge' statement is what u needed.
Here is the syntax:
MERGE INTO TARGET_TABLE
USING SOURCE_TABLE
ON (CONDITION)
WHEN MATCHED THEN
UPDATE SET (DO YOUR UPDATES)
WHEN NOT MATCHED THEN
(INSERT YOUR NEW ROWS)
Google MERGE syntax for more about the statement.
Just use MINUS.
query_1
MINUS
query_2
In your case, if you really want to use PL/SQL, then select count into a local variable. Write a logic, if count > 0 then do other stuff.
select col_1,
col_2
from tbl_a A,
tbl_b B,
( select col_1,col_2 from tbl_c where clo_c_1 <> '1' ) C
where A.col_1 = B.col_1
and A.col_2 = B.col_2
and (A.col_2 = B.col_2
or C.col_2 = A.col_2);
My environment is Oracle,when I run this SQL,if the sub SQL C hasn't got a result,then the entire SQL returns NULL.Whereas if C has a result(not null) which fits other condions,there could be a result.Would somebody explain why sub SQL at the from area need to be not NULL?Thanks very much.
You need to bring yourself into the 90s and start using standard joins:
select col_1,
col_2
from tbl_a A
inner join
tbl_b B
on A.col_1 = B.col_1
and A.col_2 = B.col_2
left join
( select col_1,col_2 from tbl_c where clo_c_1 <> '1' ) C
on
C.col_2 = A.col_2
As a guess. I'm not entirely sure what your join conditions should be but that's my first attempt.
This is expected behaviour. When you join two result sets, you only want to get results where the join criteria is satisfied. If the criteria are not satisfied, you should get no results.
If I run the query "get me all the employees older than 65, and get their departments", if there are no employees older than 65, you would not expect to get any results, even if there are some departments in the database.
SELECT emp.name, dept.dept_name
FROM emp
JOIN dept
ON (emp.dept_no = dept.dept_no)
WHERE emp.age > 65;
As the others said, if you actually want to get rows regardless of whether the subquery has any results, you need to use an outer join.
I've faced with a weird problem now. The query itself is huge so I'm not going to post it here (I could post however in case someone needs to see). Now I have a table ,TABLE1, with a CHAR(1) column, COL1. This table column is queried as part of my query. When I filter the recordset for this column I say:
WHERE TAB1.COL1=1
This way the query runs and returns a very big resultset. I've recently updated one of the subqueries to speed up the query. But after this when I write WHERE TAB1.COL1=1 it does not return anything, but if I change it to WHERE TAB1.COL1='1' it gives me the records I need. Notice the WHERE clause with quotes and w/o them. So to make it more clear, before updating one of the sub-queries I did not have to put quotes to check against COL1 value, but after updating I have to. What feature of Oracle is it that I'm not aware of?
EDIT: I'm posting the tw versions of the query in case someone might find it useful
Version 1:
SELECT p.ssn,
pss.pin,
pd.doc_number,
p.surname,
p.name,
p.patronymic,
to_number(p.sex, '9') as sex,
citiz_c.short_name citizenship,
p.birth_place,
p.birth_day as birth_date,
coun_c.short_name as country,
di.name as leg_city,
trim( pa.settlement
|| ' '
|| pa.street) AS leg_street,
pd.issue_date,
pd.issuing_body,
irs.irn,
irs.tpn,
irs.reg_office,
to_number(irs.insurer_type, '9') as insurer_type,
TO_CHAR(sa.REG_CODE)
||CONVERT_INT_TO_DOUBLE_LETTER(TO_NUMBER(SUBSTR(TO_CHAR(sa.DOSSIER_NR, '0999999'), 2, 3)))
||SUBSTR(TO_CHAR(sa.DOSSIER_NR, '0999999'), 5, 4) CONVERTED_SSN_DOSSIER_NR,
fa.snr
FROM
(SELECT pss_t.pin,
pss_t.ssn
FROM EHDIS_INSURANCE.pin_ssn_status pss_t
WHERE pss_t.difference_status < 5
) pss
INNER JOIN SSPF_CENTRE.file_archive fa
ON fa.ssn = pss.ssn
INNER JOIN SSPF_CENTRE.persons p
ON p.ssn = fa.ssn
INNER JOIN
(SELECT pd_2.ssn,
pd_2.type,
pd_2.series,
pd_2.doc_number,
pd_2.issue_date,
pd_2.issuing_body
FROM
--The changed subquery starts here
(SELECT ssn,
MIN(type) AS type
FROM SSPF_CENTRE.person_documents
GROUP BY ssn
) pd_1
INNER JOIN SSPF_CENTRE.person_documents pd_2
ON pd_2.type = pd_1.type
AND pd_2.ssn = pd_1.ssn
) pd
--The changed subquery ends here
ON pd.ssn = p.ssn
INNER JOIN SSPF_CENTRE.ssn_archive sa
ON p.ssn = sa.ssn
INNER JOIN SSPF_CENTRE.person_addresses pa
ON p.ssn = pa.ssn
INNER JOIN
(SELECT i_t.irn,
irs_t.ssn,
i_t.tpn,
i_t.reg_office,
(
CASE i_t.insurer_type
WHEN '4'
THEN '1'
ELSE i_t.insurer_type
END) AS insurer_type
FROM sspf_centre.irn_registered_ssn irs_t
INNER JOIN SSPF_CENTRE.insurers i_t
ON i_t.irn = irs_t.new_irn
OR i_t.old_irn = irs_t.old_irn
WHERE irs_t.is_registration IS NOT NULL
AND i_t.is_real IS NOT NULL
) irs ON irs.ssn = p.ssn
LEFT OUTER JOIN SSPF_CENTRE.districts di
ON di.code = pa.city
LEFT OUTER JOIN SSPF_CENTRE.countries citiz_c
ON p.citizenship = citiz_c.numeric_code
LEFT OUTER JOIN SSPF_CENTRE.countries coun_c
ON pa.country_code = coun_c.numeric_code
WHERE pa.address_flag = '1'--Here's the column value with quotes
AND fa.form_type = 'Q3';
And Version 2:
SELECT p.ssn,
pss.pin,
pd.doc_number,
p.surname,
p.name,
p.patronymic,
to_number(p.sex, '9') as sex,
citiz_c.short_name citizenship,
p.birth_place,
p.birth_day as birth_date,
coun_c.short_name as country,
di.name as leg_city,
trim( pa.settlement
|| ' '
|| pa.street) AS leg_street,
pd.issue_date,
pd.issuing_body,
irs.irn,
irs.tpn,
irs.reg_office,
to_number(irs.insurer_type, '9') as insurer_type,
TO_CHAR(sa.REG_CODE)
||CONVERT_INT_TO_DOUBLE_LETTER(TO_NUMBER(SUBSTR(TO_CHAR(sa.DOSSIER_NR, '0999999'), 2, 3)))
||SUBSTR(TO_CHAR(sa.DOSSIER_NR, '0999999'), 5, 4) CONVERTED_SSN_DOSSIER_NR,
fa.snr
FROM
(SELECT pss_t.pin,
pss_t.ssn
FROM EHDIS_INSURANCE.pin_ssn_status pss_t
WHERE pss_t.difference_status < 5
) pss
INNER JOIN SSPF_CENTRE.file_archive fa
ON fa.ssn = pss.ssn
INNER JOIN SSPF_CENTRE.persons p
ON p.ssn = fa.ssn
INNER JOIN
--The changed subquery starts here
(SELECT ssn,
type,
series,
doc_number,
issue_date,
issuing_body
FROM
(SELECT ssn,
type,
series,
doc_number,
issue_date,
issuing_body,
ROW_NUMBER() OVER (partition BY ssn order by type) rn
FROM SSPF_CENTRE.person_documents
)
WHERE rn = 1
) pd --
--The changed subquery ends here
ON pd.ssn = p.ssn
INNER JOIN SSPF_CENTRE.ssn_archive sa
ON p.ssn = sa.ssn
INNER JOIN SSPF_CENTRE.person_addresses pa
ON p.ssn = pa.ssn
INNER JOIN
(SELECT i_t.irn,
irs_t.ssn,
i_t.tpn,
i_t.reg_office,
(
CASE i_t.insurer_type
WHEN '4'
THEN '1'
ELSE i_t.insurer_type
END) AS insurer_type
FROM sspf_centre.irn_registered_ssn irs_t
INNER JOIN SSPF_CENTRE.insurers i_t
ON i_t.irn = irs_t.new_irn
OR i_t.old_irn = irs_t.old_irn
WHERE irs_t.is_registration IS NOT NULL
AND i_t.is_real IS NOT NULL
) irs ON irs.ssn = p.ssn
LEFT OUTER JOIN SSPF_CENTRE.districts di
ON di.code = pa.city
LEFT OUTER JOIN SSPF_CENTRE.countries citiz_c
ON p.citizenship = citiz_c.numeric_code
LEFT OUTER JOIN SSPF_CENTRE.countries coun_c
ON pa.country_code = coun_c.numeric_code
WHERE pa.address_flag = 1--Here's the column value without quotes
AND fa.form_type = 'Q3';
I've put separating comments for the changed subqueries and the WHERE clause in both queries. Both versions of the subqueries return the same result, one of them is just slower, which is why I decided to update it.
With the most simplistic example I can't reproduce your problem on 11.2.0.3.0 or 11.2.0.1.0.
SQL> create table tmp_test ( a char(1) );
Table created.
SQL> insert into tmp_test values ('1');
1 row created.
SQL> select *
2 from tmp_test
3 where a = 1;
A
-
1
If I then insert a non-numeric value into the table I can confirm Chris' comment "that Oracle will rewrite tab1.col1 = 1 to to_number(tab1.col1) = 1", which implies that you only have numeric characters in the column.
SQL> insert into tmp_test values ('a');
1 row created.
SQL> select *
2 from tmp_test
3 where a = 1;
ERROR:
ORA-01722: invalid number
no rows selected
If you're interested in tracking this down you should gradually reduce the complexity of the query until you have found a minimal, reproducible, example. Oracle can pre-compute a conversion to be used in a JOIN, which as your query is complex seems like a possible explanation of what's happening.
Oracle explicitly recommends against using implicit conversion so it's wiser not to use it at all; as you're finding out. For a start there's no guarantees that your indexes will be used correctly.
Oracle recommends that you specify explicit conversions, rather than rely on implicit or automatic conversions, for these reasons:
SQL statements are easier to understand when you use explicit data type conversion functions.
Implicit data type conversion can have a negative impact on performance, especially if the data type of a column value is converted to that of a constant rather than the other way around.
Implicit conversion depends on the context in which it occurs and may not work the same way in every case. For example, implicit conversion from a datetime value to a VARCHAR2 value may return an unexpected year depending on the value of the NLS_DATE_FORMAT
parameter.
Algorithms for implicit conversion are subject to change across software releases and among Oracle products. Behavior of explicit conversions is more predictable.
If you do only have numeric characters in the column I would highly recommend changing this to a NUMBER(1) column and I would always recommend explicit conversion to avoid a lot of pain in the longer run.
It's hard to tell without the actual query. What I would expect is that TAB1.COL1 is in some way different before and after the refactoring.
Candidates differences are Number vs. CHAR(1) vs. CHAR(x>1) vs VARCHAR2
It is easy to introduce differences like this with subqueries where you join two tables which have different types in the join column and you return different columns in your subquery.
To hunt that issue down you might want to check the exact datatypes of your query. Not sure how to do that right now .. but an idea would be to put it in a view and use sqlplus desc on it.
I am attempting to figure out if a column in Oracle is populated from a sequence. My impression of how Oracle handles sequencing is that the sequence and column are separate entities and one needs to either manually insert the next sequence value like:
insert into tbl1 values(someseq.nextval, 'test')
or put it into a table trigger. Meaning that it is non-trivial to tell if a column is populated from a sequence. Is that correct? Any ideas about how I might go about figuring out if a column is populated from a sequence?
You are correct; the sequence is separate from the table, and a single sequence can be used to populate any table, and the values in a column in some table may mostly come from a sequence (or set of sequences), except for the values manually generated.
In other words, there is no mandatory connection between a column and a sequence - and therefore no way to discover such a relationship from the schema.
Ultimately, the analysis will be of the source code of all applications that insert or update data in the table. Nothing else is guaranteed. You can reduce the scope of the search if there is a stored procedure that is the only way to make modifications to the table, or if there is a trigger that sets the value, or other such things. But the general solution is the 'non-solution' of 'analyze the source'.
If the sequence is used in a trigger, it is possible to find which tables it populates:
SQL> select t.table_name, d.referenced_name as sequence_name
2 from user_triggers t
3 join user_dependencies d
4 on d.name = t.trigger_name
5 where d.referenced_type = 'SEQUENCE'
6 and d.type = 'TRIGGER'
7 /
TABLE_NAME SEQUENCE_NAME
------------------------------ ------------------------------
EMP EMPNO_SEQ
SQL>
You can vary this query to find stored procedures, etc that make use of the sequence.
There are no direct metadata links between Oracle sequences and any use in the database. You could make an intelligent guess if a column's values are related to a sequence by querying the USER_SEQUENCES metadata and comparing the LAST_NUMBER column to the data for the column.
select t.table_name,
d.referenced_name as sequence_name,
d.REFERENCED_OWNER as "OWNER",
c.COLUMN_NAME
from user_trigger_cols t, user_dependencies d, user_tab_cols c
where d.name = t.trigger_name
and t.TABLE_NAME = c.TABLE_NAME
and t.COLUMN_NAME = c.COLUMN_NAME
and d.referenced_type = 'SEQUENCE'
and d.type = 'TRIGGER'
As Jonathan pointed out: there is no direct way to relate both objects. However, if you "keep a standard" for primary keys and sequences/triggers you could find out by finding the primary key and then associate the constraint to the table sequence.
I was in need of something similar since we are building a multi-db product and I tried to replicate some classes with properties found in a DataTable object from .Net which has AutoIncrement, IncrementSeed and IncrementStep which can only be found in the sequences.
So, as I said, if you, for your tables, use a PK and always have a sequence associated with a trigger for inserts on a table then this may come handy:
select tc.table_name,
case tc.nullable
when 'Y' then 1
else 0
end as is_nullable,
case ac.constraint_type
when 'P' then 1
else 0
end as is_identity,
ac.constraint_type,
seq.increment_by as auto_increment_seed,
seq.min_value as auto_increment_step,
com.comments as caption,
tc.column_name,
tc.data_type,
tc.data_default as default_value,
tc.data_length as max_length,
tc.column_id,
tc.data_precision as precision,
tc.data_scale as scale
from SYS.all_tab_columns tc
left outer join SYS.all_col_comments com
on (tc.column_name = com.column_name and tc.table_name = com.table_name)
LEFT OUTER JOIN SYS.ALL_CONS_COLUMNS CC
on (tc.table_name = cc.table_name and tc.column_name = cc.column_name and tc.owner = cc.owner)
LEFT OUTER JOIN SYS.ALL_CONSTRAINTS AC
ON (ac.constraint_name = cc.constraint_name and ac.owner = cc.owner)
LEFT outer join user_triggers trg
on (ac.table_name = trg.table_name and ac.owner = trg.table_owner)
LEFT outer join user_dependencies dep
on (trg.trigger_name = dep.name and dep.referenced_type='SEQUENCE' and dep.type='TRIGGER')
LEFT outer join user_sequences seq
on (seq.sequence_name = dep.referenced_name)
where tc.table_name = 'TABLE_NAME'
and tc.owner = 'SCHEMA_NAME'
AND AC.CONSTRAINT_TYPE = 'P'
union all
select tc.table_name,
case tc.nullable
when 'Y' then 1
else 0
end as is_nullable,
case ac.constraint_type
when 'P' then 1
else 0
end as is_identity,
ac.constraint_type,
seq.increment_by as auto_increment_seed,
seq.min_value as auto_increment_step,
com.comments as caption,
tc.column_name,
tc.data_type,
tc.data_default as default_value,
tc.data_length as max_length,
tc.column_id,
tc.data_precision as precision,
tc.data_scale as scale
from SYS.all_tab_columns tc
left outer join SYS.all_col_comments com
on (tc.column_name = com.column_name and tc.table_name = com.table_name)
LEFT OUTER JOIN SYS.ALL_CONS_COLUMNS CC
on (tc.table_name = cc.table_name and tc.column_name = cc.column_name and tc.owner = cc.owner)
LEFT OUTER JOIN SYS.ALL_CONSTRAINTS AC
ON (ac.constraint_name = cc.constraint_name and ac.owner = cc.owner)
LEFT outer join user_triggers trg
on (ac.table_name = trg.table_name and ac.owner = trg.table_owner)
LEFT outer join user_dependencies dep
on (trg.trigger_name = dep.name and dep.referenced_type='SEQUENCE' and dep.type='TRIGGER')
LEFT outer join user_sequences seq
on (seq.sequence_name = dep.referenced_name)
where tc.table_name = 'TABLE_NAME'
and tc.owner = 'SCHEMA_NAME'
AND AC.CONSTRAINT_TYPE is null;
That would give you the list of columns for a schema/table with:
Table name
If column is nullable
Constraint type (only for PK's)
Increment seed (from the sequence)
Increment step (from the sequence)
Column comments
Column name, of course :)
Data type
Default value, if any
Length of column
Index (column id)
Precision (for numbers)
Scale (for numbers)
I'm pretty sure that code can be optimized but it works for me, I use it to "load metadata" for tables and then represent that metadata as entities on my frontend.
Note that I'm filtering only primary keys and not retrieving compound key constraints since I don't care about those. If you do you'll have to modify the code to do so and make sure that you filter duplicates since you could get one column twice (one for the PK constraint, another for the compound key).