We have one requirement to mask a particular table column using a Oracle function which gives persistent masked output string.
We tried Oracle Hash Function but it does not give String type return value.
We tried Oracle Random function (dbms_random.string) but it does not give Persistent output string.
I read on internet that this is called deterministic masking. But we do not want to use Oracle Enterprise Manager; however we require a direct Oracle function.
Please suggest.
This problem is easily solved in 12c with the function STANDARD_HASH.
The solution in previous versions is only slightly more complicated. Build a simple wrapper around DBMS_CRYPTO that acts just like STANDARD_HASH:
--Imitation of the 12c function with the same name.
--Remember to drop this function when you upgrade!
create or replace function standard_hash(
p_string varchar2,
p_method varchar2 default 'SHA1'
) return varchar2 is
v_method number;
v_invalid_identifier exception;
pragma exception_init(v_invalid_identifier, -904);
begin
--Intentionally case-sensitive, just like the 12c version.
if p_method = 'SHA1' then
v_method := dbms_crypto.hash_sh1;
--These algorithms are only available in 12c and above.
$IF NOT DBMS_DB_VERSION.VER_LE_11 $THEN
elsif p_method = 'SHA256' then
v_method := dbms_crypto.hash_sh256;
elsif p_method = 'SHA384' then
v_method := dbms_crypto.hash_sh384;
elsif p_method = 'SHA512' then
v_method := dbms_crypto.hash_sh512;
$END
elsif p_method = 'MD5' then
v_method := dbms_crypto.hash_md5;
else
raise v_invalid_identifier;
end if;
return rawToHex(dbms_crypto.hash(utl_raw.cast_to_raw(p_string), v_method));
end;
/
You may need to logon with SYS and grant your user access to DBMS_CRYPTO to make the function work:
grant execute on sys.dbms_crypto to <your_schema>;
Create a public synonym, grant it to everyone, and it works exactly the same way.
create public synonym standard_hash for <schema with function>.standard_hash;
grant execute on standard_hash to public;
select standard_hash('Some text', 'MD5') from dual;
9DB5682A4D778CA2CB79580BDB67083F
select standard_hash('Some text', 'md5') from dual;
ORA-00904: : invalid identifier
Here is a simple example of using the function:
update some_table
set column1 = standard_hash(column1),
column2 = standard_hash(column2);
But updating large amounts of data can be slow. It may be faster to create a new table, drop the old one, rename the new one, etc. And the hash value may be larger than the column size, it may be necessary to alter table some_table modify column1 varchar2(40 byte);
It amazes me how many products and tools there are to do such a simple thing.
If you looking something like mask the production data to move it into non-prod for integration testing. Below the "user defined" function would be helpful to you. This function will work only 10G and above.
create or replace function scrubbing(word in varchar2)
return varchar2
as
each_var char(2);
final_val varchar2(100);
complete_data varchar2(4000);
each_word varchar2(1000);
cursor val is select substr(replace(word,' ','#'),-level,1) from dual connect by level<=length(word);
begin
open val;
--final_val:= '';
loop
fetch val into each_var;
exit when val%NOTFOUND;
--dbms_output.put_line(each_var);
final_val := trim(final_val)||trim(each_var);
--dbms_output.put_line(final_val);
select regexp_substr(final_val,'[A-Za-z]+') into each_word from dual;
select replace(translate(final_val,each_word,dbms_random.string('L',length(word))),'#',' ') into complete_data from dual;
end loop;
return complete_data;
end;
In Oracle 12C dbms_redact.add_policy is available. It can be used to get the masked value in the select query itself.
You can use dbms_crpyto package of oracle , first you need to convert varchar2 type to raw then mask the data according to the hash value.
Related
Example in T-SQL (SQL Server - taken from here):
CREATE PROC proc_authors
#au_lname VARCHAR(40)
AS
SELECT
au_id, au_fname, au_lname, city, state
FROM authors
WHERE au_lname = #au_lname
go
Is it possible in Oracle to create a stored procedure that returns an inline table (without declaring a type - like the above)? If not, what would be the closest alternative? i.e. declare inline type, then use it. The idea is to minimize number of DB permissions that are granted.
Please include sample code as part of your answer.
Reasoning behind using stored procedure vs function - we have legacy software that can only execute stored procedures, or raw queries. It appears that only stored procedures in there have support for parameterized execution, which is what we are after.
try this with ref cursor
PROCEDURE proc_get_tada(ip_user IN VARCHAR2,
op_error_code OUT NUMBER,
op_cursor OUT SYS_REFCURSOR,) AS
BEGIN
OPEN op_cursor FOR
SELECT * FROM your_table yt where yt.user = ip_user;
EXCEPTION
WHEN OTHERS THEN
op_error_code := -1;
END proc_get_tada;
you will get collection of all data from you table you can iterate in java or calling program.
Maybe you are searching for something like this:
create table author
(
au_id number,
au_name varchar2(100)
);
insert into author (au_id, au_name) values(1, 'ME');
create or replace function getAuthor(auName varchar2)
return author%rowtype
is
retval author%rowtype;
begin
select * into retval from author where au_name=auName;
return retval;
end;
declare
auth author%rowtype;
begin
auth := getAuthor('ME');
dbms_output.put_line(auth.au_id);
end;
This is my first (edited) stackoverflow question, so please bear with me.
In Oracle 11g, I have a need to describe/interrogate the underlying columns of a reference cursor returned from a procedure call on another database over a dblink, in which the actual SQL is not always "explicit", but sometimes dynamically generated.
For example:
declare
v_ref_cur sys_refcursor;
v_cur_num number;
v_col_count number;
v_col_table dbms_sql.desc_tab3;
begin
myProc#myDblink(v_ref_cur, 'myvalue');
v_cur_num := dbms_sql.to_cursor_number(v_ref_cur);
dbms_sql.describe_columns3(v_cur_num, v_col_count, v_col_table);
...
end
If myProc() on the other database has an "explicit" SQL statement like:
open cursor for select * from foo where bar = myParam;
The cursor conversion and description (still) work just fine - I can determine the column names, types, lengths, etc returned by the procedure.
BUT, if myProc() on the other database involves dynamic SQL, like:
v_sql := 'select * from foo where bar = ''' || myParam || '''';
open cursor for v_sql;
I get an ORA-01001 invalid cursor error when attempting to call dbms_sql.to_cursor_number().
Is there a way to convert/describe a reference cursor derived from dynamic SQL as called from a remote procedure? If so, how? If not, why not?
Thanks for any/all assistance!
Using DBMS_SQL across a database link raises many different errors, at least some of which are Oracle bugs. Those problems can be avoided by putting all of the logic in a function compiled on the remote server. Then call that function remotely.
--Create and test a database link
create database link myself connect to <schema> identified by "<password>"
using '<connect string or service name>';
select * from dual#myself;
--myProc
create procedure myProc(p_cursor in out sys_refcursor, p_value varchar2) is
begin
--open p_cursor for select * from dual where dummy = p_value;
open p_cursor for 'select * from dual where dummy = '''||p_value||'''';
end;
/
--Simple function that counts and displays the columns. Expected value is 1.
create or replace function count_columns return number is
v_ref_cur sys_refcursor;
v_cur_num number;
v_col_count number;
v_col_table dbms_sql.desc_tab3;
begin
--ORA-01001: invalid cursor
--myProc#myself(v_ref_cur, 'myvalue');
myProc(v_ref_cur, 'myvalue');
--ORA-03113: end-of-file on communication channel
--v_cur_num := dbms_sql.to_cursor_number#myself(v_ref_cur);
v_cur_num := dbms_sql.to_cursor_number(v_ref_cur);
--Compilation error: PLS-00306:
-- wrong number or types of arguments in call to 'DESCRIBE_COLUMNS3'
--dbms_sql.describe_columns3#myself(v_cur_num, v_col_count, v_col_table);
dbms_sql.describe_columns3(v_cur_num, v_col_count, v_col_table);
return v_col_count;
end;
/
begin
dbms_output.put_line('Number of columns: '||count_columns#myself());
end;
/
Number of columns: 1
I have a function that looks like this.
Function GetNewBatch ( CourseName Varchar2 ) Return RefCursor
As
Results RefCursor;
CourseId Number;
Begin
CourseId := Courselist.GetId( CourseName );
Open Results For
Select q.user_abn UserAbn,
q.completed_t DateCompleted,
CourseName,
q.batch_n BatchId
From GAK.GAKHR02_ACK q
Where q.crse_i = CourseId
And q.batch_n is null
And rownum < 1000;
GAK.SEQ1_GAKHR03.NextVal;
Return Results;
End;
I want to increment the sequence after the select, but SQL Developer is giving me the error:
"Error(194,5): PLS-00313: 'NEXTVAL' not declared in this scope."
How can I do this?
It's because you're not assigning the nextval to a variable. Assume that .nextval is a function of some description. It's not, Oracle actually describe it as a psuedocolumn; something used in the same context as a column but not written to the disk.
How you increment a sequence depends on the version of Oracle you're using. Prior to 11G you could do the following in a PL/SQL block:
declare
i number;
begin
select my_sequence.nextval
into i
from dual;
insert into my_table(id)
values(my_sequence.nextval);
update my_table
set id = my_sequence.nextval;
end;
In 11G this changed slightly so you could also assign the "return" value from .nextval to a variable:
declare
l_next_val number;
begin
l_next_val := my_sequence.nextval;
end;
The reason for PLS-00313 is that Oracle is assuming that SEQ1_GAKHR03 is a package or some other object and .nextval is a sub-type of this object.
It's very unusual to increment a sequence without actually using the value. Is this intended behaviour?
Is it possible to have conditional compilation in Oracle, where the condition is the existence of a database object (specifically, a table or view or synonym)? I'd like to be able to do something like this:
sp_some_procedure is
$IF /*check if A exists.*/ then
/* read from and write to A as well as other A-related non-DML stuff...*/
$ELSE /*A doesn't exist yet, so avoid compiler errors*/
dbms_output.put_line('Reminder: ask DBA to create A!')
$ENDIF
end;
Yes it is. Here a sample where the first stored procedure wants to select from XALL_TABLES, but if this table doesn't exist, select from dual. Finally, because I haven't got an XALL_TABLES object, the first stored procedure selects from dual. The second one, does the same thing on the ALL_TABLES object. Because the ALL_TABLES exists, the second stored procedure selects from all_tables but not from DUAL.
This kind of construction is useful where the package have to be deployed on all your database and use tables that are not deployed everywhere ... (ok, perhaps there is a conceptual problem, but it happens).
--conditionals compilation instructions accept only static condition (just with constants)
--passing sql bind variable doesn't work
--To pass a value to a conditional compilation instruction, I bypasses the use of input parameters of the script
--these 4 next lines affect a value to the first and the second input parameter of the script
--If your originally script use input script parameter, use the next free parameter ...
column param_1 new_value 1 noprint
select nvl(max(1), 0) param_1 from all_views where owner = 'SYS' and view_name = 'XALL_TABLES';
column param_2 new_value 2 noprint
select nvl(max(1), 0) param_2 from all_views where owner = 'SYS' and view_name = 'ALL_TABLES';
CREATE or replace PACKAGE my_pkg AS
function test_xall_tables return varchar2;
function test_all_tables return varchar2;
END my_pkg;
/
CREATE or replace PACKAGE BODY my_pkg AS
function test_xall_tables return varchar2 is
vch varchar2(50);
begin
$IF (&1 = 0) $THEN
select 'VIEW XALL_TABLES D''ONT EXISTS' into vch from dual;
$ELSE
select max('VIEW XALL_TABLES EXISTS') into vch from XALL_TABLES;
$END
return vch;
end test_xall_tables;
function test_all_tables return varchar2 is
vch varchar2(50);
begin
$IF (&2 = 0) $THEN
select 'VIEW ALL_TABLES D''ONT EXISTS' into vch from dual;
$ELSE
select max('VIEW ALL_TABLES EXISTS') into vch from ALL_TABLES;
$END
return vch;
end test_all_tables;
END my_pkg;
/
the test :
select my_pkg.test_xall_tables from dual;
give
VIEW XALL_TABLES D'ONT EXISTS
select my_pkg.test_all_tables from dual;
give
VIEW ALL_TABLES EXISTS
I would use 'EXECUTE IMMEDIATE' and a EXCEPTION clause.
Use dynamic SQL to create package constants to track which objects exist, and then use those constants in conditional compilation.
--E.g., say there are two possible tables, but only one of them exists.
--create table table1(a number);
create table table2(a number);
--Create a package with boolean constants to track the objects.
--(Another way to do this is to use ALTER SESSION SET PLSQL_CCFLAGS)
declare
table1_exists_string varchar2(10) := 'true';
table2_exists_string varchar2(10) := 'true';
temp number;
begin
begin
execute immediate 'select max(1) from table1 where rownum <= 1' into temp;
exception when others then
table1_exists_string := 'false';
end;
begin
execute immediate 'select max(1) from table2 where rownum <= 1' into temp;
exception when others then
table2_exists_string := 'false';
end;
execute immediate '
create or replace package objects is
table1_exists constant boolean := '||table1_exists_string||';
table2_exists constant boolean := '||table2_exists_string||';
end;
';
end;
/
--Look at the results in the source:
select * from user_source where name = 'OBJECTS';
--Create the object that refers to the tables.
create or replace function compile_test return varchar2 is
v_test number;
begin
$if objects.table1_exists $then
select max(1) into v_test from table1;
return 'table1 exists';
$elsif objects.table2_exists $then
select max(1) into v_test from table2;
return 'table 2 exists';
$else
return 'neither table exists';
$end
end;
/
--Check the dependencies - only TABLE2 is dependent.
select * from user_dependencies where name = 'COMPILE_TEST';
--Returns 'table 2 exists'.
select compile_test from dual;
Mixing dynamic SQL, dynamic PL/SQL, and conditional compilation is usually a very evil idea. But it will allow you to put all of your ugly dynamic SQL in one installation package, and maintain real dependency tracking.
This may work well in a semi-dynamic environment; for example a program that is installed with different sets of objects but does not frequently change between them.
(Also, if the whole point of this is just to replace scary error messages with friendly warnings, in my opinion that is a very bad idea. If your system is going to fail, the failure should be obvious so it can be immediately fixed. Most people ignore anything that starts with "Reminder...".)
No - that is not possible... but if you create a stored procedure referencing a non-existent DB object and try to compile it the compilation will show errors... the stored procedure will be there but "invalid"... and the compilation errors are accessible for the DBA whenever he looks at it... so I would just go ahead and create all needed stored procedures, if any compilation errors arise ask the DBA (sometimes the object exists but the stored procedure need permissions to access it...)... after the reason for the error(s) is fixed you can just recompile the stored procedure (via ALTER PROCEDURE MySchema.MyProcName COMPILE;) and all is fine...
IF you don't want code to be there you can just DROP the strored procedure and/or replace is via CREATE OR REPLACE... with dbms_output.put_line('Reminder: ask DBA to create A!') in the body.
The only other alternative is as kevin points out EXECUTE IMMEDIATE with proper EXCEPTION handling...
What I would do is check the existence via all_objects, something like:
declare
l_check_sql varchar2(4000);
l_cnt number;
begin
l_check_sql := q'{
select count(1)
from all_objects
where object_name = 'MY_OBJ'
and owner = 'MY_OWNER'
}';
execute immediate l_check_sql into l_cnt;
if (l_cnt > 0) then
-- do something referring to MY_OBJ
else
-- don't refer to MY_OBJ
end if;
end;
I'm having a problem with making a sha1-hash of a row in a select on an Oracle database. I've done it in MSSQL as follows:
SELECT *,HASHBYTES('SHA1',CAST(ID as varchar(10)+
TextEntry1+TextEntry2+CAST(Timestamp as varchar(10)) as Hash
FROM dbo.ExampleTable
WHERE ID = [foo]
However, I can't seem to find a similar function to use when working with Oracle.
As far as my googling has brought me, I'm guessing dbms_crypto.hash_sh1 has something to do with it, but I haven't been able to wrap my brain around it yet...
Any pointers would be greatly appreciated.
The package DBMS_CRYPTO is the correct package to generate hashes. It is not granted to PUBLIC by default, you will have to grant it specifically (GRANT EXECUTE ON SYS.DBMS_CRYPTO TO user1).
The result of this function is of datatype RAW. You can store it in a RAW column or convert it to VARCHAR2 using the RAWTOHEX or UTL_ENCODE.BASE64_ENCODE functions.
The HASH function is overloaded to accept three datatypes as input: RAW, CLOB and BLOB. Due to the rules of implicit conversion, if you use a VARCHAR2 as input, Oracle will try to convert it to RAW and will most likely fail since this conversion only works with hexadecimal strings.
If you use VARCHAR2 then, you need to convert the input to a binary datatype or a CLOB, for instance :
DECLARE
x RAW(20);
BEGIN
SELECT sys.dbms_crypto.hash(utl_raw.cast_to_raw(col1||col2||to_char(col3)),
sys.dbms_crypto.hash_sh1)
INTO x
FROM t;
END;
you will find additional information in the documentation of DBMS_CRYPTO.hash
The DBMS_crypto package does not support varchar2. It works with raw type so if you need a varchar2 you have to convert it. Here is a sample function showing how to do this :
declare
p_string varchar2(2000) := 'Hello world !';
lv_hash_value_md5 raw (100);
lv_hash_value_sh1 raw (100);
lv_varchar_key_md5 varchar2 (32);
lv_varchar_key_sh1 varchar2 (40);
begin
lv_hash_value_md5 :=
dbms_crypto.hash (src => utl_raw.cast_to_raw (p_string),
typ => dbms_crypto.hash_md5);
-- convert into varchar2
select lower (to_char (rawtohex (lv_hash_value_md5)))
into lv_varchar_key_md5
from dual;
lv_hash_value_sh1 :=
dbms_crypto.hash (src => utl_raw.cast_to_raw (p_string),
typ => dbms_crypto.hash_sh1);
-- convert into varchar2
select lower (to_char (rawtohex (lv_hash_value_sh1)))
into lv_varchar_key_sh1
from dual;
--
dbms_output.put_line('String to encrypt : '||p_string);
dbms_output.put_line('MD5 encryption : '||lv_varchar_key_md5);
dbms_output.put_line('SHA1 encryption : '||lv_varchar_key_sh1);
end;
Just to put it here, if someone will search for.
In Oracle 12 you can use standard_hash(<your_value>, <algorythm>) function.
With no parameter <algorythm> defined, it will generate SHA-1 hash (output datatype raw(20))
You can define this function in your favorite package, I defined in utils_pkg.
FUNCTION SHA1(STRING_TO_ENCRIPT VARCHAR2) RETURN VARCHAR2 AS
BEGIN
RETURN LOWER(TO_CHAR(RAWTOHEX(SYS.DBMS_CRYPTO.HASH(UTL_RAW.CAST_TO_RAW(STRING_TO_ENCRIPT), SYS.DBMS_CRYPTO.HASH_SH1))));
END SHA1;
Now to call it
SELECT UTILS_PKG.SHA1('My Text') AS SHA1 FROM DUAL;
The response is
SHA1
--------------------------------------------
5411d08baddc1ad09fa3329f9920814c33ea10c0
You can select a column from some table:
SELECT UTILS_PKG.SHA1(myTextColumn) FROM myTable;
Enjoy!
Oracle 19c:
select LOWER(standard_hash('1234')) from dual;
which is equivalent to
select LOWER(standard_hash('1234','SHA1')) from dual;
will return an SHA1 hash.
For alternative algorithms see: https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/STANDARD_HASH.html