I use Oracle 11g (on Red Hat). I have simple regular table with XMLType column:
CREATE TABLE PROJECTS
(
PROJECT_ID NUMBER(*, 0) NOT NULL,
PROJECT SYS.XMLTYPE,
);
Using Oracle SQL Developer (on Windows) I do:
select T1.PROJECT P1 from PROJECTS T1 where PROJECT_ID = '161';
It works. I get one cell. I can double click and download whole XML file.
Then I tried to get result as CLOB:
select T1.PROJECT.getClobVal() P1 from PROJECTS T1 where PROJECT_ID = '161';
It works. I get one cell. I can double click and see whole text and copy it. BUT there is a problem. When I copy it to clipboard I get only first 4000 characters. It seems that there is 0x00 character at position 4000 and the rest of CLOB is not copied.
To confirm this, I wrote check in java:
// ... create projectsStatement
Reader reader = projectsStatement.getResultSet().getCharacterStream( "P1" );
BufferedReader bf = new BufferedReader( reader );
char buffer[] = new char[ 1024 ];
int count = 0;
int globalPos = 0;
while ( ( count = bf.read( buffer, 0, buffer.length ) ) > 0 )
for ( int i = 0; i < count; i++, globalPos++ )
if ( buffer[ i ] == 0 )
throw new Exception( "ZERO at " + Integer.toString(globalPos) );
Reader returns full XML but my exception is thrown because there is null character at position 4000. I could remove this single byte but this would be rather strange workaround.
I don't use VARCHAR2 there but maybe this problem is related to VARCHAR2 limitation (4000 bytes) somehow ? Any other ideas ? Is this an Oracle bug or am I missing something ?
-------------------- Edit --------------------
Value was inserted using following stored procedure:
create or replace
procedure addProject( projectId number, projectXml clob ) is
sqlstr varchar2(2000);
begin
sqlstr := 'insert into projects ( PROJECT_ID, PROJECT ) VALUES ( :projectId, :projectData )';
execute immediate sqlstr using projectId, XMLTYPE(projectXml);
end;
Java code used to call it:
try ( CallableStatement cs = connection.prepareCall("{call addProject(?,?)}") )
{
cs.setInt( "projectId", projectId );
cs.setCharacterStream( "projectXml", new StringReader(xmlStr) , xmlStr.length() );
cs.execute();
}
-------------------- Edit. SIMPLE TEST --------------------
I will use all I learned from your answers. Create simplest table:
create table T1 ( P XMLTYPE );
Prepare two CLOBs with XMLs. First with null character, second without.
declare
P1 clob;
P2 clob;
P3 clob;
begin
P1 := '<a>';
P2 := '<a>';
FOR i IN 1..1000 LOOP
P1 := P1 || '0123456789' || chr(0);
P2 := P2 || '0123456789';
END LOOP;
P1 := P1 || '</a>';
P2 := P2 || '</a>';
Check if null is in the first CLOB and not in the second one:
DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P1, chr(0) ) );
DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P2, chr(0) ) );
We will get as expected:
14
0
Try to insert first CLOB into XMLTYPE. It will not work. It is not possible to insert such value:
insert into T1 ( P ) values ( XMLTYPE( P1 ) );
Try to insert second CLOB into XMLTYPE. It will work:
insert into T1 ( P ) values ( XMLTYPE( P2 ) );
Try to read inserted XML into third CLOB. It will work:
select T.P.getClobVal() into P3 from T1 T where rownum = 1;
Check if there is null. There is NO null:
DBMS_OUTPUT.put_line( DBMS_LOB.INSTR( P3, chr(0) ) );
It seams that there is no null inside database and as long as we are in the PL/SQL context, there is no null. But when I try to use following SQL in SQL Developer ( on Windows ) or in Java ( on Red Hat EE and Tomcat7 ) I get null character at position 4000 in all returned CLOBs:
select T.P.getClobVal() from T1 T;
BR,
JM
It's not an Oracle bug (it stores and retrieves the \0 just fine. It's a client/windows bug (Different clients behave differently in regards to "NUL" as does windows)
chr(0) is not a valid character in non-blobs really (I'm curious how you ever get the XMLType to accept it in the first place as usually it wouldn't parse).
\0 is used in C to denote the end of a string (NUL terminator) and some GUIs would stop processing the string at that point. For example:
![SQL> select 'IM VISIBLE'||chr(0)||'BUT IM INVISIBLE'
2 from dual
3 /
'IMVISIBLE'||CHR(0)||'BUTIM
---------------------------
IM VISIBLE BUT IM INVISIBLE
SQL>
yet toad fails miserably on this:
sql developer fares better, as you can see it:
but if you copy it, the clipboard will only copy it up to the nul character. this copy paste error isn't SQL developers fault though, it's a problem with windows clipboard not allowing NUL to paste properly.
you should just replace(T1.PROJECT.getClobVal(), chr(0), null) to get round this when using sql developer/windows clipboard.
I also was experiencing this same issue exactly as described by Mikosz (seeing an extra 'NUL' character around the 4000th character when outputting my XMLType value as a Clob). While playing around in SQLDeveloper I noticed an interesting workaround. I was trying to see the output of my XMLType, but was tired of scrolling to the 4000th character, so I started wrapping the Clob output in a substr(...). Much to my surprise, the issue actually disappeared. I incorporated this into my Java app and confirmed that the issue was no longer present and my Clob could be retrieved without the extra character. I know that this isn't an ideal workaround, and I'm still not sure why it works (would love if someone could explain it to me), but here's an abbreviated example of what I've currently got working:
// Gets the xml contents
String sql = "select substr(x.xml_content.getClobVal(), 0) as xml_content from my_table x";
ps = con.prepareStatement(sql);
if(rs.next()) {
Reader reader = new BufferedReader(rs.getCharacterStream("xml_content"));
...
}
Bug:14781609 XDB: XMLType.getclobval() returns a temporary LOB when XML is stored in a CLOB.
fix in patchset 11.2.0.4
and another solution
if read as blob, then no error like
T1.PROJECT.getBlobVal(nls_charset_id('UTF8'))
Easy enough to verify if it's the .getClobVal() call or not - perform an INSTR test in PL/SQL (not Java) on your resultant CLOB to see if the CHR(0) exists or not.
If it does not, then I would point the finger at your Oracle client install.
Related
The UPSERT operation either updates or inserts a row in a table, depending if the table already has a row that matches the data:
if table t has a row exists that has key X:
update t set mystuff... where mykey=X
else
insert into t mystuff...
Since Oracle doesn't have a specific UPSERT statement, what's the best way to do this?
The MERGE statement merges data between two tables. Using DUAL
allows us to use this command. Note that this is not protected against concurrent access.
create or replace
procedure ups(xa number)
as
begin
merge into mergetest m using dual on (a = xa)
when not matched then insert (a,b) values (xa,1)
when matched then update set b = b+1;
end ups;
/
drop table mergetest;
create table mergetest(a number, b number);
call ups(10);
call ups(10);
call ups(20);
select * from mergetest;
A B
---------------------- ----------------------
10 2
20 1
The dual example above which is in PL/SQL was great becuase I wanted to do something similar, but I wanted it client side...so here is the SQL I used to send a similar statement direct from some C#
MERGE INTO Employee USING dual ON ( "id"=2097153 )
WHEN MATCHED THEN UPDATE SET "last"="smith" , "name"="john"
WHEN NOT MATCHED THEN INSERT ("id","last","name")
VALUES ( 2097153,"smith", "john" )
However from a C# perspective this provide to be slower than doing the update and seeing if the rows affected was 0 and doing the insert if it was.
An alternative to MERGE (the "old fashioned way"):
begin
insert into t (mykey, mystuff)
values ('X', 123);
exception
when dup_val_on_index then
update t
set mystuff = 123
where mykey = 'X';
end;
Another alternative without the exception check:
UPDATE tablename
SET val1 = in_val1,
val2 = in_val2
WHERE val3 = in_val3;
IF ( sql%rowcount = 0 )
THEN
INSERT INTO tablename
VALUES (in_val1, in_val2, in_val3);
END IF;
insert if not exists
update:
INSERT INTO mytable (id1, t1)
SELECT 11, 'x1' FROM DUAL
WHERE NOT EXISTS (SELECT id1 FROM mytble WHERE id1 = 11);
UPDATE mytable SET t1 = 'x1' WHERE id1 = 11;
None of the answers given so far is safe in the face of concurrent accesses, as pointed out in Tim Sylvester's comment, and will raise exceptions in case of races. To fix that, the insert/update combo must be wrapped in some kind of loop statement, so that in case of an exception the whole thing is retried.
As an example, here's how Grommit's code can be wrapped in a loop to make it safe when run concurrently:
PROCEDURE MyProc (
...
) IS
BEGIN
LOOP
BEGIN
MERGE INTO Employee USING dual ON ( "id"=2097153 )
WHEN MATCHED THEN UPDATE SET "last"="smith" , "name"="john"
WHEN NOT MATCHED THEN INSERT ("id","last","name")
VALUES ( 2097153,"smith", "john" );
EXIT; -- success? -> exit loop
EXCEPTION
WHEN NO_DATA_FOUND THEN -- the entry was concurrently deleted
NULL; -- exception? -> no op, i.e. continue looping
WHEN DUP_VAL_ON_INDEX THEN -- an entry was concurrently inserted
NULL; -- exception? -> no op, i.e. continue looping
END;
END LOOP;
END;
N.B. In transaction mode SERIALIZABLE, which I don't recommend btw, you might run into
ORA-08177: can't serialize access for this transaction exceptions instead.
I'd like Grommit answer, except it require dupe values. I found solution where it may appear once: http://forums.devshed.com/showpost.php?p=1182653&postcount=2
MERGE INTO KBS.NUFUS_MUHTARLIK B
USING (
SELECT '028-01' CILT, '25' SAYFA, '6' KUTUK, '46603404838' MERNIS_NO
FROM DUAL
) E
ON (B.MERNIS_NO = E.MERNIS_NO)
WHEN MATCHED THEN
UPDATE SET B.CILT = E.CILT, B.SAYFA = E.SAYFA, B.KUTUK = E.KUTUK
WHEN NOT MATCHED THEN
INSERT ( CILT, SAYFA, KUTUK, MERNIS_NO)
VALUES (E.CILT, E.SAYFA, E.KUTUK, E.MERNIS_NO);
I've been using the first code sample for years. Notice notfound rather than count.
UPDATE tablename SET val1 = in_val1, val2 = in_val2
WHERE val3 = in_val3;
IF ( sql%notfound ) THEN
INSERT INTO tablename
VALUES (in_val1, in_val2, in_val3);
END IF;
The code below is the possibly new and improved code
MERGE INTO tablename USING dual ON ( val3 = in_val3 )
WHEN MATCHED THEN UPDATE SET val1 = in_val1, val2 = in_val2
WHEN NOT MATCHED THEN INSERT
VALUES (in_val1, in_val2, in_val3)
In the first example the update does an index lookup. It has to, in order to update the right row. Oracle opens an implicit cursor, and we use it to wrap a corresponding insert so we know that the insert will only happen when the key does not exist. But the insert is an independent command and it has to do a second lookup. I don't know the inner workings of the merge command but since the command is a single unit, Oracle could execute the correct insert or update with a single index lookup.
I think merge is better when you do have some processing to be done that means taking data from some tables and updating a table, possibly inserting or deleting rows. But for the single row case, you may consider the first case since the syntax is more common.
A note regarding the two solutions that suggest:
1) Insert, if exception then update,
or
2) Update, if sql%rowcount = 0 then insert
The question of whether to insert or update first is also application dependent. Are you expecting more inserts or more updates? The one that is most likely to succeed should go first.
If you pick the wrong one you will get a bunch of unnecessary index reads. Not a huge deal but still something to consider.
Try this,
insert into b_building_property (
select
'AREA_IN_COMMON_USE_DOUBLE','Area in Common Use','DOUBLE', null, 9000, 9
from dual
)
minus
(
select * from b_building_property where id = 9
)
;
From http://www.praetoriate.com/oracle_tips_upserts.htm:
"In Oracle9i, an UPSERT can accomplish this task in a single statement:"
INSERT
FIRST WHEN
credit_limit >=100000
THEN INTO
rich_customers
VALUES(cust_id,cust_credit_limit)
INTO customers
ELSE
INTO customers SELECT * FROM new_customers;
UPDATED: The code is working as expected but the performance is very slow. When I do a search without including CLOB data then the query runs very fast but if I include CLOB variable in my search the query is very slow. I am using CLOB to pass large string data('aaaaaaa,bbbb,c,ddddd...') and store those data in global table for better performance, I thought doing such will maximize query performance. How can I improve/utilize my CLOB variable for better perfomance? Please look at the code below for more information. Appreciated for any help. I am still struggling with performance can anyone help/provide any suggestions please.
GLOBAL TT GlobalTemp_EMP( //this already exists
emp_refno (30 byte);
)
Create or replace PROCEDURE Employee(
emp_refno IN CLOB
)
AS
Begin
OPEN p_resultset FOR
with inputs ( str ) as ( //red error line here
select to_clob(emp_refno )
from dual
),
prep ( s, n, token, st_pos, end_pos ) as (
select ',' || str || ',', -1, null, null, 1
from inputs
union all
select s, n+1, substr(s, st_pos, end_pos - st_pos),
end_pos + 1, instr(s, ',', 1, n+3)
from prep
where end_pos != 0
)
INSERT into GlobalTemp_EMP //red error line here
select token from prep;
select e.empname, e.empaddress, f.department
from employee e
join department f on e.emp_id = t.emp_id
and e.emp_refno in (SELECT emp_refno from GlobalTemp_EMP) //using GTT In subquery
put this code between BEGIN and OPEN p_resultset FOR : this might have some performance issue though.
INSERT into GlobalTemp_EMP
with inputs ( str ) as (
select to_clob(emp_refno )
from dual
),
prep ( s, n, token, st_pos, end_pos ) as (
select ',' || str || ',', -1, null, null, 1
from inputs
union all
select s, n+1, substr(s, st_pos, end_pos - st_pos),
end_pos + 1, instr(s, ',', 1, n+3)
from prep
where end_pos != 0
)
select token from prep where token is not NULL;
The below doesn't seem to be valid syntax:
GLOBAL TT GlobalTemp_EMP( //this already exists
emp_refno (30 byte);
)
I don't know the reason for using byte semantics, or whether you defined it as a clob or char or varchar2.
If it is currently a clob, then perhaps you could define the column as emp_refno varchar2(30 char) and add a unique index, changing the Employee procedure to only insert new IDs. An index would help the insertions more than when you read it out.
If you want to insert a huge amount of data into GlobalTemp_EMP faster, I would recommend making it a regular table, pre-processing the data (such as in Perl or other language) to split IDs outside Oracle and then use SQL*Loader. Or perhaps an external table.
I don't think using a global temporary table will improve your performance at all (at least without indexes). Are you sure these are CLOBs? At a glance, these seem to be varchars.
To compare CLOBs, you should be using dbms_lob.compare. I think = will do an implicit conversion to a varchar (and truncate), then do the comparison.
I have declared a value in parameter #Data as ACCOUNT_NO|none|M=ACCOUNT_NO,ADD1|none|M=ADD1
I need to get a result as ACCOUNT_NO=ACCOUNT_NO|none|M,ADD1=ADD1|none|M.
Which means I need to swap between the values before and after =
I have the SQL Server Query for achieving this but I need Oracle query.
Declare #Data varchar(100)='ACCOUNT_NO|none|M=ACCOUNT_NO,ADD1|none|M=ADD1';
WITH
myCTE1 AS
(
SELECT CAST('<root><r>' + REPLACE(#Data,',','</r><r>') + '</r></root>' AS XML) AS parts1
)
,myCTE2 AS
(
SELECT CAST('<root><r>' + REPLACE(p1.x.value('.','varchar(max)'),'=','</r><r>') + '</r></root>' AS XML) as parts2
FROM myCTE1
CROSS APPLY parts1.nodes('/root/r') AS p1(x)
)
SELECT STUFF
(
(
SELECT ',' + parts2.value('/root[1]/r[2]','varchar(max)') + '=' + parts2.value('/root[1]/r[1]','varchar(max)')
FROM myCTE2
FOR XML PATH(''),TYPE
).value('.','varchar(max)'),1,1,'');
Expected Output if I execute the query ACCOUNT_NO=ACCOUNT_NO|none|M,ADD1=ADD1|none|M. Can anyone give an idea to do this one?
Sounds like a job for REGEXP_REPLACE:
WITH datatab as (select 'ACCOUNT_NO|none|M=ACCOUNT_NO,ADD1|none|M=ADD1' info from dual)
select info,
regexp_replace(info, '([^=]+)=([^=,]+),([^=]+)=([^=,]+)', '\2=\1,\4=\3') new_info
from datatab;
INFO NEW_INFO
--------------------------------------------- ---------------------------------------------
ACCOUNT_NO|none|M=ACCOUNT_NO,ADD1|none|M=ADD1 ACCOUNT_NO=ACCOUNT_NO|none|M,ADD1=ADD1|none|M
(as a complete aside, that's the first time I've ever written a regular expression and had it work first time. Apparently, I have gone over to the dark side... *{;-) )
ETA: If you need this in a procedure/function, you don't need to bother selecting the regular expression, you can do it in PL/SQL directly.
Here's an example of a function that returns the swapped over result:
create or replace function swap_places (p_data in varchar2)
return varchar2
is
begin
return regexp_replace(p_data, '([^=]+)=([^=,]+),([^=]+)=([^=,]+)', '\2=\1,\4=\3');
end swap_places;
/
-- example of calling the function to check the result
select swap_places('ACCOUNT_NO|none|M=ACCOUNT_NO,ADD1|none|M=ADD1') col1 from dual;
COL1
-------------------------------------------------
ACCOUNT_NO=ACCOUNT_NO|none|M,ADD1=ADD1|none|M
I'm trying to pull data from a complex query (it's been simplified here for review) using custom RECORD and TABLE OF RECORD data types, but I can't get data into the table due to a "PLS-00308: This construct is not allowed as the origin of an assignment" Error. I've followed the examples carefully and don't understand the problem. Can anyone point me in a direction.
here's the code
TYPE CORE_REC IS RECORD
(
OrgID CHAR(20 BYTE)
, StoreNumber VARCHAR2(200 BYTE)
, StoreName VARCHAR(200 BYTE)
, AssociateName VARCHAR2(300 BYTE)
);
TYPE CORE_REC_CURSOR IS REF CURSOR RETURN CORE_REC;
TYPE CORE_REC_TABLE IS TABLE OF CORE_REC INDEX BY BINARY_INTEGER;
FUNCTION CORE_GETCURRS (
OrgID IN CHAR
) RETURN HDT_CORE_MAIN.CORE_REC AS
CurrTable HDT_CORE_MAIN.CORE_REC;
i BINARY_INTEGER := 0;
CURSOR CurrCursor IS
WITH
CoreCurrs AS
(SELECT
busSTR.id AS OrgID
, busSTR.name AS StoreNumber
, busSTR.name2 AS StoreName
, emp.lname || ', ' || emp.fname || ' ' || emp.mname AS AssociateName
FROM tp2.tpt_company busSTR
INNER JOIN tp2.cmt_person emp
ON busSTR.ID = emp.company_id
WHERE
busSTR.id = OrgID
)
SELECT
CoreCurrs.OrgID
, CoreCurrs.StoreNumber
, CoreCurrs.StoreName
, CoreCurrs.AssociateName
FROM CoreCurrs
;
BEGIN
DBMS_OUTPUT.ENABLE(1000000);
OPEN CurrCursor;
LOOP
i := i + 1;
FETCH CurrCursor INTO CurrTable(i);
EXIT WHEN CurrCursor%NOTFOUND;
END LOOP;
CLOSE CurrCursor;
RETURN CurrTable;
END CORE_GETCURRS;
The error gets thrown at the FETCH statement.
Your variable is the wrong type, it should be:
CurrTable HDT_CORE_MAIN.CORE_REC_TABLE;
At the moment you're trying to select into an element of a record, rather than element of a table, which doesn't make sense. When it's defined as CORE_REC, referring to CurrTable(i) doesn't mean anything.
I am fairly new to PowerBuilder Classic 12.
I need a sample code to check whether a record is available and if not insert from a textbox.
I would probably need a DataStore since someone suggested a preference to SQL statements.
Thanks.
I'm not sure if this is answering your Q, but here you go:
string ls_CheckRecord
string ls_InsertRecord
SELECT record2Check
INTO :ls_Check
FROM myTable
WHERE conditionsMet;
If ( SQLCA.SQLCode <> 0 ) Then
Messagebox("SQL Error",SQLCA.SQLErrText)
Return
End if
If (NOT IsNull(ls_Check) ) Then
ls_InsertRecord = sle_ToInsert.Text
INSERT myTable
(myColumn)
VALUES
(:ls_InsertRecord) ;
If ( SQLCA.SQLCode <> 0 ) Then
Messagebox("SQL Error",SQLCA.SQLErrText)
Return
End if
End if