How to convert CLOB to UTF8 in an Oracle query? - oracle

I've got a query with a CLOB field which I want to return her value in UTF8 format. The next query works fine if the field are varchar, for example, but if it is CLOB doesn't return a correct UTF8 string.
select convert(field, 'AL32UTF8', 'WE8ISO8859P15') from table;
How can I do to return a UTF8 string from a CLOB in a query?

Use DBMS_LOB.CONVERTTOBLOB.
From the oracle documentation:
Oracle discourages the use of the CONVERT function in the current
Oracle Database release. The return value of CONVERT has a character
datatype, so it should be either in the database character set or in
the national character set, depending on the datatype. Any
dest_char_set that is not one of these two character sets is
unsupported. …
If you need a character datatype like CLOB in a character set that differs from those the database is setup with it should be converted into a BLOB.
This is where DBMS_LOB.CONVERTTOBLOB comes in.
If you need a function that returns a BLOB you have to wrap CONVERTTOBLOB into your own function.
For example:
CREATE OR REPLACE FUNCTION clob_to_blob (p_clob CLOB, p_charsetname VARCHAR2)
RETURN BLOB
AS
l_lang_ctx INTEGER := DBMS_LOB.default_lang_ctx;
l_warning INTEGER;
l_dest_offset NUMBER := 1;
l_src_offset NUMBER := 1;
l_return BLOB;
BEGIN
DBMS_LOB.createtemporary (l_return, FALSE);
DBMS_LOB.converttoblob (
l_return,
p_clob,
DBMS_LOB.lobmaxsize,
l_dest_offset,
l_src_offset,
CASE WHEN p_charsetname IS NOT NULL THEN NLS_CHARSET_ID (p_charsetname) ELSE DBMS_LOB.default_csid END,
l_lang_ctx,
l_warning);
RETURN l_return;
END;
This allows queries like:
SELECT clob_to_blob (field, 'UTF8') FROM t;
To get a list of supported values for the character set name use:
SELECT *
FROM v$nls_valid_values
WHERE parameter = 'CHARACTERSET'

use dbms_lob package for it
for example
select convert(dbms_lob.substr(field,dbms_lob.getlength(field), **0**),
'AL32UTF8',
'WE8ISO8859P15')
from table;
Fixed it:
select convert(dbms_lob.substr(field,dbms_lob.getlength(field)),
'AL32UTF8',
'WE8ISO8859P15')
from table;

Related

Returning a static CLOB from an Oracle Stored Function

I am trying to return a long paragraph from an Oracle Stored Function. I can do this in SQL Server, but I am new to oracle. How do I return long texts, like a User Agreement long paragraphs.
Note that it is only static texts and requires no DMLs.
If your text is going to be over 32767 characters, then you will want to use the CLOB datatype. The DETERMINISTIC clause is added to help with performance since there is no queries in the function, just string concatenation.
CREATE OR REPLACE FUNCTION return_clob
RETURN CLOB
DETERMINISTIC
IS
l_return_value CLOB;
BEGIN
l_return_value := 'Some text.';
l_return_value := l_return_value || ' Some more text.';
RETURN l_return_value;
END;
/

numeric or value error: raw variable length too long ORA-06512: at "SYS.UTL_RAW"

I am facing raw variable length too long issue when select the BLOB(Stored JSON string) filed value from the table.
Query:
select utl_raw.cast_to_varchar2(dbms_lob.substr(TRANSACTION_DATA)) from PS_ISA_INB_PAYLOAD_LOG;
This is my SP i have used to insert the JSON object into BLOB Field:
create or replace PROCEDURE SDIX_TICK_LOG
(
ORGANIZATIONNAME IN VARCHAR2
, TRANSACTION_TYPE IN VARCHAR2
, TRANSACTION_DATA IN BLOB
, TRANSACTION_STATUS IN VARCHAR2
) AS
l_r RAW(32767);
l_blob blob;
l_clob clob :='"ItemMasterTransfer":[{"ORACLE_UNIQUE_REC_ID":"123assd4434","CUSTOMER_ID":"PMC","ORGANIZATION_CODE":"BMftrdsM","ITEM":"696738","INVENTORY_ITEM_ID":"0000000000000000000000000000546","ORGANIZATION_ID":" ","SUBINVENTORY_CODE":"000000000000000000","LOCATOR_ID":" ","ISA_MATERIAL_GROUP":" ","TAX_GROUP":" ","CATEGORY_ID":"1956","NOUN":" ","MODIFIER":" ","MANUFACTURER_ID":" ","MFG_ITEM_ID":" ","UNIT_OF_MEASURE":"BOX","ITEM_TYPE":"P","STOCK_ENABLED_FLAG":"Y","INVENTORY_ITEM_STATUS_CODE":"A","LIST_PRICE_PER_UNIT":"0","FULL_LEAD_TIME":"0","MAX_MINMAX_QUANTITY":"10","MIN_MINMAX_QUANTITY":"10","SAFETY_LEVEL":"0","REPLENISH_TO_ORDER_FLAG":"N","UTILIZ_CD":"","CURRENCY_CD":"USD","DESCRIPTION":"","ATTRIBUTE1":" ","ATTRIBUTE2":" ","ATTRIBUTE3":" ","ATTRIBUTE4":" ","ATTRIBUTE5":" ","ATTRIBUTE6":" ","ATTRIBUTE7":" ","ATTRIBUTE8":" ","ATTRIBUTE9":" ","ATTRIBUTE10":" ","TRANSACTION_STATUS":" ","TRANS_STATUS_DESCRIPTION":" "},{"ORACLE_UNIQUE_REC_ID":"123assd4434","CUSTOMER_ID":"PMC","ORGANIZATION_CODE":"BMftrdsM","ITEM":"696738","INVENTORY_ITEM_ID":"0000000000000000000000000000546","ORGANIZATION_ID":" ","SUBINVENTORY_CODE":"000000000000000000","LOCATOR_ID":" ","ISA_MATERIAL_GROUP":" ","TAX_GROUP":" ","CATEGORY_ID":"1956","NOUN":" ","MODIFIER":" ","MANUFACTURER_ID":" ","MFG_ITEM_ID":" ","UNIT_OF_MEASURE":"BOX","ITEM_TYPE":"P","STOCK_ENABLED_FLAG":"Y","INVENTORY_ITEM_STATUS_CODE":"A","LIST_PRICE_PER_UNIT":"0","FULL_LEAD_TIME":"0","MAX_MINMAX_QUANTITY":"10","MIN_MINMAX_QUANTITY":"10","SAFETY_LEVEL":"0","REPLENISH_TO_ORDER_FLAG":"N","UTILIZ_CD":"","CURRENCY_CD":"USD","DESCRIPTION":"","ATTRIBUTE1":" ","ATTRIBUTE2":" ","ATTRIBUTE3":" ","ATTRIBUTE4":" ","ATTRIBUTE5":" ","ATTRIBUTE6":" ","ATTRIBUTE7":" ","ATTRIBUTE8":" ","ATTRIBUTE9":" ","ATTRIBUTE10":" ","TRANSACTION_STATUS":" ","TRANS_STATUS_DESCRIPTION":" "},{"ORACLE_UNIQUE_REC_ID":"123assd4434","CUSTOMER_ID":"PMC","ORGANIZATION_CODE":"BMftrdsM","ITEM":"696738","INVENTORY_ITEM_ID":"0000000000000000000000000000546","ORGANIZATION_ID":" ","SUBINVENTORY_CODE":"000000000000000000","LOCATOR_ID":" ","ISA_MATERIAL_GROUP":" ","TAX_GROUP":" ","CATEGORY_ID":"1956","NOUN":" ","MODIFIER":" ","MANUFACTURER_ID":" ","MFG_ITEM_ID":" ","UNIT_OF_MEASURE":"BOX","ITEM_TYPE":"P","STOCK_ENABLED_FLAG":"Y","INVENTORY_ITEM_STATUS_CODE":"A","LIST_PRICE_PER_UNIT":"0","FULL_LEAD_TIME":"0","MAX_MINMAX_QUANTITY":"10","MIN_MINMAX_QUANTITY":"10","SAFETY_LEVEL":"0","REPLENISH_TO_ORDER_FLAG":"N","UTILIZ_CD":"","CURRENCY_CD":"USD","DESCRIPTION":"","ATTRIBUTE1":" ","ATTRIBUTE2":" ","ATTRIBUTE3":" ","ATTRIBUTE4":" ","ATTRIBUTE5":" ","ATTRIBUTE6":" ","ATTRIBUTE7":" ","ATTRIBUTE8":" ","ATTRIBUTE9":" ","ATTRIBUTE10":" ","TRANSACTION_STATUS":" ","TRANS_STATUS_DESCRIPTION":" "}],"Organization":"PMC Biogenix","SharedSecret":"sTc1QowIu5Iy1Qt8iilnmQ==","TimeStamp":"09/28/2018 00:19:21","RowsSent":"1"}';
l_amt integer := dbms_lob.lobmaxsize;
l_dest_offset integer := 1;
l_src_offset integer := 1;
l_csid integer := dbms_lob.default_csid;
l_ctx integer := dbms_lob.default_lang_ctx;
l_warn integer;
BEGIN
dbms_lob.createTemporary( l_blob, false );
dbms_lob.convertToBlob( l_blob,
l_clob,
l_amt,
l_dest_offset,
l_src_offset,
l_csid,
l_ctx,
l_warn );
INSERT INTO PS_ISA_INB_PAYLOAD_LOG Values(ORGANIZATIONNAME,TRANSACTION_TYPE,l_blob,SYSDATE,TRANSACTION_STATUS);
END SDIX_TICK_LOG;
Your problem lies here: DBMS_LOB.SUBSTR()
DBMS_LOB is using VARCHAR2 internally, and VARCHAR2 has limit of 2000 chars. Your blob has the size of 2829 chars, therefore it is too long to be processed by DBMS_LOB.SUBSTR() at once.
You can test this by these commands:
Take only first 2000 chars from BLOB:
select utl_raw.cast_to_varchar2(dbms_lob.substr(TRANSACTION_DATA), 2000, 1) from PS_ISA_INB_PAYLOAD_LOG;
OK.
Take 2001 chars from BLOB:
select utl_raw.cast_to_varchar2(dbms_lob.substr(TRANSACTION_DATA, 2001, 1)) from PS_ISA_INB_PAYLOAD_LOG;
Error report -
SQL Error:
ORA-06502: PL/SQL: numeric or value error: raw variable length too long
ORA-06512: at line 1
06502.00000 - "PL/SQL: numeric or value error%s"
Is that possible to select entire BLOB Field value?
Basically, no. You'll need a PL/SQL function similar to the one you've already described, except this time, it will go BLOB-to-CLOB. You can call that from SQL, if it returns a CLOB or a VARCHAR2(4000).
As a side note, I question why you're taking JSON, which is character data, and storing it as a BLOB, then wanting to get character data back. Why not just store it as a CLOB in the first place?

PLSQL decode NVARCHAR2 from BASE64 to UTF-8

I have a database which stores usernames only in English at the moment.
I would like to incorporate BASE64 & UTF-8 in order to store in other languages as well; I want to store it in a column of type NVARCHAR2.
The database procedure receives the name in BASE64, I'm decoding it via UTL_ENCODE.BASE64_DECODE & converting the string to VARCHAR2 using UTL_RAW.CAST_TO_VARCHAR2. But I get gibberish back and not the actual word.
For example I get 'алекс' as the name in BASE64. I'm able to decode it but the cast to VARCHAR2/NVARCHAR2 does not return the value: I get only gibberish.
I'm running on Oracle 12c using NLS_CHARACTERSET WE8ISO8859P1
Here is the code I use to decode:
DECLARE
lv_OrgUserName VARCHAR2(2000);
lv_encodedUserName VARCHAR2(2000);
lv_UserName VARCHAR2(2000);
BEGIN
lv_OrgUserName := 'алекс';
lv_encodedUserName := UTL_RAW.CAST_TO_VARCHAR2(UTL_ENCODE.BASE64_ENCODE(UTL_RAW.CAST_TO_RAW(lv_OrgUserName)));
DBMS_OUTPUT.PUT_LINE (lv_encodedUserName);
lv_UserName := UTL_RAW.CAST_TO_VARCHAR2(UTL_ENCODE.BASE64_DECODE(UTL_RAW.CAST_TO_RAW (lv_encodedUserName)));
DBMS_OUTPUT.PUT_LINE (lv_UserName);
END;
How can I overcome this?
First and foremost WE8ISO8859P1 (Western European 8-bit ISO 8859 Part 1, or - ISO8859 Part 1) does not support cyryllic characters:
see this link: https://en.wikipedia.org/wiki/ISO/IEC_8859-1
Therefore if you try to store a string like алекс into VARCHAR2 variable/column, you will always get a???? as an outcome.
Probably during the database installation someone has not considered cyryllic characters and has choosen a bad codepage.
A better option is ISO/IEC 8859-5 (part 5), see this link: https://en.wikipedia.org/wiki/ISO/IEC_8859-5
One option is to change this encoding - but this is not easy and it is beyound of this question.
What you can do is to strictly use NVARCHAR2 datatype instead of VARCHAR2 datatype in all places of your application that must support cyrillic characters.
There are still some pitfalls though you need to be aware of:
You cannot use DBMS_OUTPUT package to debug your code, because this package support only VARCHAR2 datatype, it doesn't support NVARCHAR
you must use N'some string' literals (with N prefix) in all literals --> 'алекс' is of VARCHAR2 datatype and it is always automatically converted to 'a????' in your encoding, while n'алекс' is of NVARCHAR2 datatype and such conversion doesn't occur.
The below code is tested on version 12c, I am using EE8MSWIN1250 code page (it also desn't support cyrillic characters):
select * from nls_database_parameters
where parameter like '%CHARACTERSET%';
PARAMETER VALUE
----------------------- ------------
NLS_NCHAR_CHARACTERSET AL16UTF16
NLS_CHARACTERSET EE8MSWIN1250
please give it a try:
CREATE OR REPLACE PACKAGE my_base64 AS
FUNCTION BASE64_ENCODE( str nvarchar2 ) RETURN varchar2;
FUNCTION BASE64_DECODE( str varchar2 ) RETURN nvarchar2;
END;
/
CREATE OR REPLACE PACKAGE BODY my_base64 AS
FUNCTION BASE64_ENCODE( str nvarchar2 ) RETURN varchar2
IS
lv_encodedUserName VARCHAR2(2000);
BEGIN
lv_encodedUserName := UTL_RAW.CAST_TO_VARCHAR2(UTL_ENCODE.BASE64_ENCODE(UTL_RAW.CAST_TO_RAW(str)));
RETURN lv_encodedUserName;
END;
FUNCTION BASE64_DECODE( str varchar2 ) RETURN nvarchar2
IS
lv_UserName nVARCHAR2(2000);
BEGIN
lv_UserName := UTL_RAW.CAST_TO_nVARCHAR2(UTL_ENCODE.BASE64_DECODE(UTL_RAW.CAST_TO_RAW (str)));
RETURN lv_UserName;
END;
END;
/
A few examples:
select 'aлекс' As A, n'aлекс' As B from dual;
A B
----- -----
a???? aлекс
select my_base64.BASE64_ENCODE( n'аaaлекс' ) As aleks from dual;
ALEKS
--------------------------------------------------------------------------------
BDAAYQBhBDsENQQ6BEE=
select my_base64.BASE64_DECODE( 'BDAAYQBhBDsENQQ6BEE=' ) as aleks from dual;
ALEKS
--------------------------------------------------------------------------------
аaaлекс
select my_base64.BASE64_DECODE( my_base64.BASE64_ENCODE( n'аaaлекс' ) ) as Aleks from dual;
ALEKS
--------------------------------------------------------------------------------
аaaлекс

How to convert BLOB to CLOB?

I'm using Oracle 11g and I'm trying to find out the length of a text. I normally use select length(myvar) from table, but I can't do that.
The table which I want to query has a BLOB column that saves characters or photos. I want to know the number of characters that my BLOB column has.
I tried to convert my BLOB into a char using UTL_RAW.CAST_TO_VARCHAR2(myblob) from table, but this functions isn't working correctly or maybe I'm making a mistake.
For example:
My BLOB have the word Section, when I see this in the databse in the hexadecimal form I see S.e.c.t.i.o.n.. I don't know why it have those points in between each letter.
Then I used the this query:
select UTL_RAW.CAST_TO_VARCHAR2(myblob)
from table
The result of this query is 'S' so it's not the complete word that my BLOB has, and when I make this query:
select length(UTL_RAW.CAST_TO_VARCHAR2(myblob))
from table
the result is 18, but the word Sections doesn't have 18 characters.
I was trying to convert the BLOB into a VARCHAR, although I think my best choise would be a CLOB because the length of the text that it can save is more than the limit that VARCHAR has. I tried to do that by making this query (I'm not sure if this is correct but is what I found in the internet):
select UTL_RAW.CAST_TO_VARCHAR2(DBMS_LOB.SUBSTR(myblob, 32767, 1))
from table
This query also returns 'S'
For anyone coming to this thread and wants to know how to convert a blob to a clob. Here is an example.
create function clobfromblob(p_blob blob) return clob is
l_clob clob;
l_dest_offsset integer := 1;
l_src_offsset integer := 1;
l_lang_context integer := dbms_lob.default_lang_ctx;
l_warning integer;
begin
if p_blob is null then
return null;
end if;
dbms_lob.createTemporary(lob_loc => l_clob
,cache => false);
dbms_lob.converttoclob(dest_lob => l_clob
,src_blob => p_blob
,amount => dbms_lob.lobmaxsize
,dest_offset => l_dest_offsset
,src_offset => l_src_offsset
,blob_csid => dbms_lob.default_csid
,lang_context => l_lang_context
,warning => l_warning);
return l_clob;
end;
To convert blob to clob, try this:
SELECT TO_CLOB(UTL_RAW.CAST_TO_VARCHAR2(DBMS_LOB.SUBSTR(MYBLOB,2000)))
FROM MYTABLE;
SELECT DBMS_LOB.GetLength( myblob ) length_in_bytes
FROM table
will return the length of the BLOB in bytes. It sounds like the character data in your BLOB is probably encoded using the UTF-16 character set so the number of bytes is probably twice the number of characters (depending on the version of Unicode that is being used and the specific data being stored, some characters might require 4 bytes of storage but it is relatively unlikely that you're dealing with any of those characters).
You can use the DBMS_LOB.ConvertToClob procedure to convert a BLOB to a CLOB (though since this is a procedure, you'll need to call it in a PL/SQL block). As part of that conversion, you'll almost certainly need to specify the character set that the data is encoded in-- my assumption is that your application is using the UTF-16 character set but that's just an assumption.
Only converting to CLOB:
select TO_CLOB(UTL_RAW.CAST_TO_VARCHAR2(YOURCLOB)) from DUAL;
Inspired by Craig without limitation on size.

Making a sha1-hash of a row in Oracle

I'm having a problem with making a sha1-hash of a row in a select on an Oracle database. I've done it in MSSQL as follows:
SELECT *,HASHBYTES('SHA1',CAST(ID as varchar(10)+
TextEntry1+TextEntry2+CAST(Timestamp as varchar(10)) as Hash
FROM dbo.ExampleTable
WHERE ID = [foo]
However, I can't seem to find a similar function to use when working with Oracle.
As far as my googling has brought me, I'm guessing dbms_crypto.hash_sh1 has something to do with it, but I haven't been able to wrap my brain around it yet...
Any pointers would be greatly appreciated.
The package DBMS_CRYPTO is the correct package to generate hashes. It is not granted to PUBLIC by default, you will have to grant it specifically (GRANT EXECUTE ON SYS.DBMS_CRYPTO TO user1).
The result of this function is of datatype RAW. You can store it in a RAW column or convert it to VARCHAR2 using the RAWTOHEX or UTL_ENCODE.BASE64_ENCODE functions.
The HASH function is overloaded to accept three datatypes as input: RAW, CLOB and BLOB. Due to the rules of implicit conversion, if you use a VARCHAR2 as input, Oracle will try to convert it to RAW and will most likely fail since this conversion only works with hexadecimal strings.
If you use VARCHAR2 then, you need to convert the input to a binary datatype or a CLOB, for instance :
DECLARE
x RAW(20);
BEGIN
SELECT sys.dbms_crypto.hash(utl_raw.cast_to_raw(col1||col2||to_char(col3)),
sys.dbms_crypto.hash_sh1)
INTO x
FROM t;
END;
you will find additional information in the documentation of DBMS_CRYPTO.hash
The DBMS_crypto package does not support varchar2. It works with raw type so if you need a varchar2 you have to convert it. Here is a sample function showing how to do this :
declare
p_string varchar2(2000) := 'Hello world !';
lv_hash_value_md5 raw (100);
lv_hash_value_sh1 raw (100);
lv_varchar_key_md5 varchar2 (32);
lv_varchar_key_sh1 varchar2 (40);
begin
lv_hash_value_md5 :=
dbms_crypto.hash (src => utl_raw.cast_to_raw (p_string),
typ => dbms_crypto.hash_md5);
-- convert into varchar2
select lower (to_char (rawtohex (lv_hash_value_md5)))
into lv_varchar_key_md5
from dual;
lv_hash_value_sh1 :=
dbms_crypto.hash (src => utl_raw.cast_to_raw (p_string),
typ => dbms_crypto.hash_sh1);
-- convert into varchar2
select lower (to_char (rawtohex (lv_hash_value_sh1)))
into lv_varchar_key_sh1
from dual;
--
dbms_output.put_line('String to encrypt : '||p_string);
dbms_output.put_line('MD5 encryption : '||lv_varchar_key_md5);
dbms_output.put_line('SHA1 encryption : '||lv_varchar_key_sh1);
end;
Just to put it here, if someone will search for.
In Oracle 12 you can use standard_hash(<your_value>, <algorythm>) function.
With no parameter <algorythm> defined, it will generate SHA-1 hash (output datatype raw(20))
You can define this function in your favorite package, I defined in utils_pkg.
FUNCTION SHA1(STRING_TO_ENCRIPT VARCHAR2) RETURN VARCHAR2 AS
BEGIN
RETURN LOWER(TO_CHAR(RAWTOHEX(SYS.DBMS_CRYPTO.HASH(UTL_RAW.CAST_TO_RAW(STRING_TO_ENCRIPT), SYS.DBMS_CRYPTO.HASH_SH1))));
END SHA1;
Now to call it
SELECT UTILS_PKG.SHA1('My Text') AS SHA1 FROM DUAL;
The response is
SHA1
--------------------------------------------
5411d08baddc1ad09fa3329f9920814c33ea10c0
You can select a column from some table:
SELECT UTILS_PKG.SHA1(myTextColumn) FROM myTable;
Enjoy!
Oracle 19c:
select LOWER(standard_hash('1234')) from dual;
which is equivalent to
select LOWER(standard_hash('1234','SHA1')) from dual;
will return an SHA1 hash.
For alternative algorithms see: https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/STANDARD_HASH.html

Resources