Can a sequence guarantee the given order? - oracle

Using this Oracle sequence definition:
CREATE SEQUENCE MY_SPECIAL_COUNTER
START WITH 100
INCREMENT BY -1
MAXVALUE 100
MINVALUE 0
NOCYCLE
NOCACHE
ORDER
;
Is it guaranteed, that this sequence will ALWAYS return
each number once from 100 to 0
in the given order from 100 to 0 and
no single number will be obmitted and
independent of multiple concurrent session in the very moment of the request
the correct number?

Yes, the sequence will return each number once, starting with 100, progressively down to 0.
Yes, it will return them in order from 100 down to 0.
Correct, it will not skip any numbers.
Multiple sessions? That depends.
Consider the following:
create table junk ( id number );
CREATE SEQUENCE MY_SPECIAL_COUNTER
START WITH 100
INCREMENT BY -1
MAXVALUE 100
MINVALUE 0
NOCYCLE
NOCACHE
ORDER
;
-- in session 1 do the following:
insert into junk
select my_special_counter.nextval from dual;
insert into junk
select my_special_counter.nextval from dual;
SQL> select * from junk;
ID
----------
100
99
SQL>
-- in session 2 do the following:
insert into junk
select my_special_counter.nextval from dual;
SQL> select * from junk;
ID
----------
98
SQL> commit;
-- in session 1 do the following:
rollback;
-- in session 3 do the following:
SQL> select * from junk;
ID
----------
98
SQL>
As you can see once you introduce multiple sessions, all kinds of fun things can happen. Numbers can be "lost" or "skipped" .. note that it is NOT the sequence doing it, but rather the session or whoever pulled the sequence value then subsequently drops it (ie job abends, and rollsback, logic error, etc.)
Also, if session 1 pulls sequence first, but session 2 commits first, others will "think" session 2 inserted a number "out of order" .. so to understand what you're after, we really need to understand your requirements in detail to advise further.
However, that should help you understand sequences properly. :)
Good luck!

Related

Using EXECUTE IMMEDIATE based on entries of table

I (using Oracle 12c, PL/SQL) need to update an existing table TABLE1 based on information stored in a table MAP. In a simplified version, MAP looks like this:
COLUMN_NAME
MODIFY
COLUMN1
N
COLUMN2
Y
COLUMN3
N
...
...
COLUMNn
Y
COLUMN1 to COLUMNn are column names in TABLE1 (but there are more columns, not just these). Now I need to update a column in TABLE1 if MODIFY in table MAP contains a 'Y' for that columns' name. There are other row conditions, so what I would need would be UPDATE statements of the form
UPDATE TABLE1
SET COLUMNi = value_i
WHERE OTHER_COLUMN = 'xyz_i';
where COLUMNi runs through all the columns of TABLE1 which are marked with MODIFY = 'Y' in MAP. value_i and xyz_i also depend on information stored in MAP (not displayed in the example).
The table MAP is not static but changes, so I do not know in advance which columns to update. What I did so far is to generate the UPDATE-statements I need in a query from MAP, i.e.
SELECT <Text of UPDATE-STATEMENT using row information from MAP> AS SQL_STMT
FROM MAP
WHERE MODIFY = 'Y';
Now I would like to execute these statements (possibly hundreds of rows). Of course I could just copy the contents of the query into code and execute, but is there a way to do this automatically, e.g. using EXECUTE IMMEDIATE? It could be something like
BEGIN
EXECUTE IMMEDIATE SQL_STMT USING 'xyz_i';
END;
only that SQL_STMT should run through all the rows of the previous query (and 'xyz_i' varies with the row as well). Any hints how to achieve this or how one should approach the task in general?
EDIT: As response to the comments, a bit more background how this problem emerges. I receive an empty n x m Matrix (empty except row and column names, think of them as first row and first column) quarterly and need to populate the empty fields from another process.
The structure of the initial matrix changes, i.e. there may be new/deleted columns/rows and existing columns/rows may change their position in the matrix. What I need to do is to take the old version of the matrix, where I already have filled the empty spaces, and translate this into the new version. Then, the populating process merely looks if entries have changed and if so, alters them.
The situation from the question arises after I have translated the old version into the new one, before doing the delta. The new matrix, populated with the old information, is TABLE1. The delta process, over which I have no control, gives me column names and information to be entered into the cells of the matrix (this is table MAP). So I need to find the column in the matrix labeled by the delta process and then to change values in rows (which ones is specified via other information provided by the delta process)
Dynamic SQL it is; here's an example, see if it helps.
This is a table whose contents should be modified:
SQL> select * from test order by id;
ID NAME SALARY
---------- ---------- ----------
1 Little 100
2 200
3 Foot 0
4 0
This is the map table:
SQL> select * from map;
COLUMN CB_MODIFY VALUE WHERE_CLAUSE
------ ---------- ----- -------------
NAME Y Scott where id <= 3
SALARY N 1000 where 1 = 1
Procedure loops through all columns that are set to be modified, composes the dynamic update statement and executes it:
SQL> declare
2 l_str varchar2(1000);
3 begin
4 for cur_r in (select m.column_name, m.value, m.where_clause
5 from map m
6 where m.cb_modify = 'Y'
7 )
8 loop
9 l_str := 'update test set ' ||
10 cur_r.column_name || ' = ' || chr(39) || cur_r.value || chr(39) || ' ' ||
11 cur_r.where_clause;
12 execute immediate l_str;
13 end loop;
14 end;
15 /
PL/SQL procedure successfully completed.
Result:
SQL> select * from test order by id;
ID NAME SALARY
---------- ---------- ----------
1 Scott 100
2 Scott 200
3 Scott 0
4 0
SQL>

Constant-time index for string column on Oracle database

I have an orders table. The table belongs to a multi-tenant application, so there are orders from several merchants in the same table. The table stores hundreds of millions of records. There are two relevant columns for this question:
MerchantID, an integer storing the merchant's unique ID
TransactionID, a string identifying the transaction
I want to know whether there is an efficient index to do the following:
Enforce a unique constraint on Transaction ID for each Merchant ID. The constraint should be enforced in constant time.
Do constant time queries involving exact matches on both columns (for instance, SELECT * FROM <table> WHERE TransactionID = 'ff089f89feaac87b98a' AND MerchantID = 24)
Further info:
I am using Oracle 11g. Maybe this Oracle article is relevant to my question?
I cannot change the column's data type.
constant time means an index performing in O(1) time complexity. Like a hashmap.
Hash clusters can provide O(1) access time, but not O(1) constraint enforcement time. However, in practice the constant access time of a hash cluster is worse than the O(log N) access time of a regular b-tree index. Also, clusters are more difficult to configure and do not scale well for some operations.
Create Hash Cluster
drop table orders_cluster;
drop cluster cluster1;
create cluster cluster1
(
MerchantID number,
TransactionID varchar2(20)
)
single table hashkeys 10000; --This number is important, choose wisely!
create table orders_cluster
(
id number,
MerchantID number,
TransactionID varchar2(20)
) cluster cluster1(merchantid, transactionid);
--Add 1 million rows. 20 seconds.
begin
for i in 1 .. 10 loop
insert into orders_cluster
select rownum + i * 100000, mod(level, 100)+ i * 100000, level
from dual connect by level <= 100000;
commit;
end loop;
end;
/
create unique index orders_cluster_idx on orders_cluster(merchantid, transactionid);
begin
dbms_stats.gather_table_stats(user, 'ORDERS_CLUSTER');
end;
/
Create Regular Table (For Comparison)
drop table orders_table;
create table orders_table
(
id number,
MerchantID number,
TransactionID varchar2(20)
) nologging;
--Add 1 million rows. 2 seconds.
begin
for i in 1 .. 10 loop
insert into orders_table
select rownum + i * 100000, mod(level, 100)+ i * 100000, level
from dual connect by level <= 100000;
commit;
end loop;
end;
/
create unique index orders_table_idx on orders_table(merchantid, transactionid);
begin
dbms_stats.gather_table_stats(user, 'ORDERS_TABLE');
end;
/
Trace Example
SQL*Plus Autotrace is a quick way to find the explain plan and track I/O activity per statement. The number of I/O requests is labeled as "consistent gets" and is a decent way of measuring the amount of work done. This code demonstrates how the numbers were generated for other sections. The queries often need to be run more than once to warm things up.
SQL> set autotrace on;
SQL> select * from orders_cluster where merchantid = 100001 and transactionid = '2';
no rows selected
Execution Plan
----------------------------------------------------------
Plan hash value: 621801084
------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 16 | 1 (0)| 00:00:01 |
|* 1 | TABLE ACCESS HASH| ORDERS_CLUSTER | 1 | 16 | 1 (0)| 00:00:01 |
------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("MERCHANTID"=100001 AND "TRANSACTIONID"='2')
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
31 consistent gets
0 physical reads
0 redo size
485 bytes sent via SQL*Net to client
540 bytes received via SQL*Net from client
1 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
0 rows processed
SQL>
Find Optimal Hashkeys, Trade-Offs
For optimal read performance all the hash collisions should fit in one block (all Oracle I/O is done per block, usually 8K). Getting the ideal storage right is tricky and requires knowing the hash algorithm, storage size (not the same as the block size), and number of hash keys (the buckets). Oracle has a default algorithm and size so it is possible to focus on only one attribute, the number of hash keys.
More hash keys leads to fewer collisions. This is good for TABLE ACCESS HASH performance as there is only one block to read. Below are the number of consistent gets for different hashkey sizes. For comparison an index access is also included. With enough hashkeys the number of blocks decreases to the optimal number, 1.
Method Consistent Gets (for transactionid = 1, 20, 300, 4000, and 50000)
Index 4, 3, 3, 3, 3
Hashkeys 100 1, 31, 31, 31, 31
Hashkeys 1000 1, 3, 4, 4, 4
Hashkeys 10000 1, 1, 1, 1, 1
More hash keys also lead to more buckets, more wasted space, and a slower TABLE ACCESS FULL operation.
Table type Space in MB
HeapTable 24MB
Hashkeys 100 26MB
hashkeys 1000 30MB
hashkeys 10000 81MB
To reproduce my results, use a sample query like select * from orders_cluster where merchantid = 100001 and transactionid = '1'; and change the last value to 1, 20, 300, 4000, and 50000.
Performance Comparison
Consistent gets are predictable and easy to measure, but at the end of the day only the wall clock time matters. Surprisingly, the index access with 4 times more
consistent gets is still faster than the optimal hash cluster scenario.
--3.5 seconds for b-tree access.
declare
v_count number;
begin
for i in 1 .. 100000 loop
select count(*)
into v_count
from orders_table
where merchantid = 100000 and transactionid = '1';
end loop;
end;
/
--3.8 seconds for hash cluster access.
declare
v_count number;
begin
for i in 1 .. 100000 loop
select count(*)
into v_count
from orders_cluster
where merchantid = 100000 and transactionid = '1';
end loop;
end;
/
I also tried the test with variable predicates but the results were similar.
Does it Scale?
No, hash clusters do not scale. Despite the O(1) time complexity of TABLE ACCESS HASH, and the O(log n) time complexity of INDEX UNIQUE SCAN, hash clusters never seem to outperform b-tree indexes.
I tried the above sample code with 10 million rows. The hash cluster was painfully slow to load, and still under-performed the index on SELECT performance. I tried to scale it up to 100 million rows but the insert was going to take 11 days.
The good news is that b*trees scale well. Adding 100 million rows to the above example only require 3 levels in the index. I looked at all DBA_INDEXES for a large database environment (hundreds of databases and a petabyte of data) - the worst index had only 7 levels. And that was a pathological index on VARCHAR2(4000) columns. In most cases your b-tree indexes will stay shallow regardless of the table size.
In this case, O(log n) beats O(1).
But WHY?
Poor hash cluster performance is perhaps a victim of Oracle's attempt to simplify things and hide the kind of details necessary to make a hash cluster work well. Clusters are difficult to setup and use properly and would rarely provide a significant benefit anyway. Oracle has not put a lot of effort into them in the past few decades.
The commenters are correct that a simple b-tree index is best. But it's not obvious why that should be true and it's good to think about the algorithms used in the database.

Split amount into multiple rows if amount>=$10M or <=$-10B

I have a table in oracle database which may contain amounts >=$10M or <=$-10B.
99999999.99 chunks and also include remainder.
If the value is less than or equal to $-10B, I need to break into one or more 999999999.99 chunks and also include remainder.
Your question is somewhat unreadable, but unless you did not provide examples here is something for start, which may help you or someone with similar problem.
Let's say you have this data and you want to divide amounts into chunks not greater than 999:
id amount
-- ------
1 1500
2 800
3 2500
This query:
select id, amount,
case when level=floor(amount/999)+1 then mod(amount, 999) else 999 end chunk
from data
connect by level<=floor(amount/999)+1
and prior id = id and prior dbms_random.value is not null
...divides amounts, last row contains remainder. Output is:
ID AMOUNT CHUNK
------ ---------- ----------
1 1500 999
1 1500 501
2 800 800
3 2500 999
3 2500 999
3 2500 502
SQLFiddle demo
Edit: full query according to additional explanations:
select id, amount,
case
when amount>=0 and level=floor(amount/9999999.99)+1 then mod(amount, 9999999.99)
when amount>=0 then 9999999.99
when level=floor(-amount/999999999.99)+1 then -mod(-amount, 999999999.99)
else -999999999.99
end chunk
from data
connect by ((amount>=0 and level<=floor(amount/9999999.99)+1)
or (amount<0 and level<=floor(-amount/999999999.99)+1))
and prior id = id and prior dbms_random.value is not null
SQLFiddle
Please adjust numbers for positive and negative borders (9999999.99 and 999999999.99) according to your needs.
There are more possible solutions (recursive CTE query, PLSQL procedure, maybe others), this hierarchical query is one of them.

Using existing Database Sequence and creating custom generator

I have created a sequence in my Database as follows:
CREATE SEQUENCE "SCOTT"."ATA_SEQ_USERID"
MINVALUE 1 MAXVALUE 9999999999999999999999999999
INCREMENT BY 1 START WITH 1000 CACHE 20 NOORDER NOCYCLE ;
Now, I want to append the first two letters of the name of the user to the number created by this sequence and generate a user id everytime a new user registers, using Hibernate. How can I do that?
You just need to use:
SUBSTR function
concatenation operator || .
For example,
SQL> CREATE SEQUENCE s;
Sequence created.
SQL>
SQL> SELECT substr(ename, 1, 2)||s.nextval custom_seq FROM emp;
CUSTOM_SEQ
------------------------------------------
SM1
AL2
WA3
JO4
MA5
BL6
CL7
SC8
KI9
TU10
AD11
JA12
FO13
MI14
14 rows selected.
SQL>
Finally found the way to do it.
String sql = "select MY_SEQ_ID.nextval from dual";
SQLQuery query = session.getCurrentSession().createSQLQuery(sql);
List idList=query.list();
BigDecimal number=(BigDecimal) idList.get(0);
System.out.println(number);
where MY_SEQ_ID is my sequence id.

Find next id from varchar in Oracle

I have a row that is a varchar(50) that has a unique constraint and i would like to get the next unique number for an new insert but with a given prefix.
My rows could look like this:
ID (varchar)
00010001
00010002
00010003
00080001
So if I would like to get the next unqiue number from the prefix "0001" it would be "00010004" but if I would want it for the prefix "0008" it would be "00080002".
There will be more then 1 millon entries in this table. Is there a way with Oracle 11 to perform this kind of operation that is fairly fast?
I know that this setup is totaly insane but this is what I have to work with. I cant create any new tables etc.
You can search for the max value of the specified prefix and increment it:
SQL> WITH DATA AS (
2 SELECT '00010001' id FROM DUAL UNION ALL
3 SELECT '00010002' id FROM DUAL UNION ALL
4 SELECT '00010003' id FROM DUAL UNION ALL
5 SELECT '00080001' id FROM DUAL
6 )
7 SELECT :prefix || to_char(MAX(to_number(substr(id, 5)))+1, 'fm0000') nextval
8 FROM DATA
9 WHERE ID LIKE :prefix || '%';
NEXTVAL
---------
00010004
I'm sure you're aware that this is an inefficient method to generate a primary key. Furthermore it won't play nicely in a multi-user environment and thus won't scale. Concurrent inserts will wait then fail since there is a UNIQUE constraint on the column.
If the prefix is always the same length, you can reduce the workload somewhat: you could create a specialized index that would find the max value in a minimum number of steps:
CREATE INDEX ix_fetch_max ON your_table (substr(id, 1, 4),
substr(id, 5) DESC);
Then the following query could use the index and will stop at the first row retrieved:
SELECT id
FROM (SELECT substr(id, 1, 4) || substr(id, 5) id
FROM your_table
WHERE substr(id, 1, 4) = :prefix
ORDER BY substr(id, 5) DESC)
WHERE rownum = 1
If you need to do simultaneous inserts with the same prefix, I suggest you use DBMS_LOCK to request a lock on the specified newID. If the call fails because someone is already inserting this value, try with newID+1. Although this involves more work than traditional sequence, at least your inserts won't wait on each others (potentially leading to deadlocks).
This is a very unsatisfactory situation for you. As other posters have pointed out - if you don't use sequences then you will almost certainly have concurrency issues. I mentioned in a comment the possibility that you live with big gaps. This is the simplest solution but you will run out of numbers after 9999 inserts.
Perhaps an alternative would be to create a separate sequence for each prefix. This would only really be practical if the number of prefixes is fairly low but it could be done.
ps - your requirement that > 1000000 records should be possible may, in fact, mean you have no choice but to redesign the database.
SELECT to_char(to_number(max(id)) + 1, '00000000')
FROM mytable
WHERE id LIKE '0001%'
SQLFiddle demo here http://sqlfiddle.com/#!4/4f543/5/0

Resources