I've come across a very weird Oracle sequence behavior. I have the following Sequence:
CREATE SEQUENCE SEQ1 INCREMENT BY 10 START WITH 100 MAXVALUE 200 CYCLE NOCACHE;
Here's an excerpt from "OCA/OCP Oracle Database 11g All-in-One Exam Guide":
CYCLE Controls the behavior on reaching MAXVALUE or MINVALUE. The
default behavior is to give an error, but if CYCLE is specified the
sequence will return to its starting point and repeat.
From this I infer that after reaching the MAXVALUE of 200, I'll get 100, as the starting point is 100. But surprisingly I get one. Why is that?
Let's look at the following excerpt from document:
Specify CYCLE to indicate that the sequence continues to generate
values after reaching either its maximum or minimum value. After an
ascending sequence reaches its maximum value, it generates its minimum
value. After a descending sequence reaches its minimum, it generates
its maximum value.
It means that START WITH value is not enough in your case, so both MINVALUE and MAXVALUE should be settled. Without given MINVALUE, cycle will start from number 1.
When your sequence cycles, it starts again at the MINVALUE of the sequence. That defaults to 1 if you don't specify a value. If you wanted the sequence to start again at 100, you'd need to specify a MINVALUE of 100.
I was reading the oracle document on this, and it was written like 'it will start with 100' in their case 1000
Creating a Sequence: Example
The following statement creates the sequence customers_seq in the sample schema oe. This sequence could be used to provide customer ID numbers when rows are added to the customers table.
CREATE SEQUENCE customers_seq
START WITH 1000
INCREMENT BY 1
NOCACHE
NOCYCLE;
The first reference to customers_seq.nextval returns 1000. The second returns 1001. Each subsequent reference will return a value 1 greater than the previous reference.
Related
I have tried to find some real use cases fot sequences in cycle mode.
we can use use sequence for generating unique IDs for primary keys but it is not needed because we can use IDENTITY as well.
can you give me some good practical scenarios for using CYCLE sequence incrementing by 1 and then by a higher number like 10 or 100?
thanks
As you said, we usually use sequences for generating unique values - a good choice for primary keys. As primary keys can't be duplicated, there's no much sense in cycling the sequence.
(As of the identity columns: yes, that's an option in recent Oracle database versions, but they didn't exist until 12c so we used and still use sequences in lower versions).
Personally, I've never used MAXVALUE; most of my sequences are simple, using default options, such as
create sequence seq;
However, if you do set MAXVALUE and don't pay attention to number of values you use from it, once you reach its maximum you'll get
ORA-08004: sequence SEQ.NEXTVAL exceeds MAXVALUE and cannot be instantiated
One solution to that problem is to remove maxvalue (or set it to a higher value); another one is to use CYCLE so that - once you reach the maximum - sequence keeps working. Though, you have to use the CACHE parameter along with it (and its value must be lower than one cycle):
SQL> create sequence seq maxvalue 3 cycle;
create sequence seq maxvalue 3 cycle
*
ERROR at line 1:
ORA-04013: number to CACHE must be less than one cycle
Cycle is "3", so you can't set CACHE to value higher than that. This works:
SQL> create sequence seq maxvalue 3 cycle cache 2;
Sequence created.
When to use it?
In cases where its value is part of a composite primary key, where the second (actually, other) column(s) make sure that cycled sequence values won't violate primary key.
Another option is a staging table; for example, you daily get up to 1 million rows that represent payments. If primary key is set to number(6), you can't let sequence unconstrained (without the maxvalue) because you won't be able to insert value higher than 999.999 into that column. But, if you use the CYCLE parameter, everything will work OK - there won't be duplicates and values will fit the column.
I created a counter that goes up from 0 to 9999 until it resets again. I use the output of this counter as a value to make unique entries. However, the application needs to find its last created number each time the application is restarted. Therfore I am looking for a method which avoids any sort of object storage and relies solely on random number generation.
Something like:
int randomTimeBasedGenerator() {
Random r = new Random(System.currentTimeMillis())
int num = r.nextInt() % 9999
return num
}
But what guarantee do I have that this method generates unique numbers? And, if not, how long would it remain unique? Are there any study papers I can look into for this sort of scenario?
Random number generation would be an elegant solution for my situation, if I can at least guarantee it won't repeat within a couple of weeks or months. But random number generation would be useless in my case if no such guarantee exists.
You have no guarantee that the return value of a random number generator remains unique. Random number generators generate unique sequences of numbers, not unique numbers. Random numbers will always repeat themselves, sooner or later.
As suggested by #Thilo, UUIDs are unique numbers. But an even better approach in your case might be to set up a lightweight database (sqlite will do) and add a record to a table with incremental id's. It is not possible to keep track of a process without storing values somewhere.
I understood the basic rationale for a reverse key index that it will reduce index contention. Now if I have 3 numbers in the index: 12345, 27999, 30632, i can see that if i reverse these numbers, the next number in the sequence won't always hit the same leaf block.
But if the numbers were like :12345,12346,12347, then the next numbers 12348,12349 (incremented by 1) would hit the same leaf block even if the index is reversed:
54321,64321,74321,84321,94321.
So how is the reverse index helping me? It was supposed to help particularly while using sequences
If we're talking about a sequence-generated value, you can't look at 5 values and draw too many conclusions. You need to think about the data that has already been inserted and the data that will be inserted in the future.
Assuming that your sequence started at 12345, the first 5 values would be inserted sequentially. But then the sixth value will be 12350. Reverse that and you get 05321 which would go to the far left of the index. Then you'd generate 12351. Reverse that to get 15321 and that's again toward the left-hand side of the index between the first value you generated (54321) and the most recent value (05321). As the sequence generates new values, they'll go further to the right until everything resets every 10 numbers and you're inserting into the far left-hand side of the index again.
If I start using a HiLo generator to assign ID's for a table, and then decide to increase or decrease the capacity (i.e. the maximum 'lo' value), will this cause collisions with the already-assigned ID's?
I'm just wondering if I need to put a big red flag around the number saying 'Don't ever change this!'
Note - not NHibernate specific, I'm just curious about the HiLo algorithm in general.
HiLo algorithms in general basically map two integers to one integer ID. It guarantees that the pair of numbers will be unique per database. Typically, the next step is to guarantee that a unique pair of numbers maps to a unique integer ID.
A nice explanation of how HiLo conceptually works is given in this previous SO answer
Changing the max_lo will preserve the property that your pair of numbers will be unique. However, will it make sure that the mapped ID is unique and collision-free?
Let's look at Hibernate's implementation of HiLo. The algorithm they appear to use (as from what I've gathered) is: (and I might be off on a technicality)
h = high sequence (starting at 0)
l_size = size of low block
l = low sequence (starting at 1)
ID = h*l_size + l
So, if your low block is, say, 100, your reserved ID blocks would go 1-100, 101-200, 201-300, 301-400...
Your High sequence is now 3. Now what would happen if you all of a sudden changed your l_size to 10? Your next block, your High is incremented, and you'd get 4*10+1 = 41
Oops. This new value definitely falls within the "reserved block" of 1-100. Someone with a high sequence of 0 would think, "Well, I have the range 1-100 reserved just for me, so I'll just put down one at 41, because I know it's safe."
There is definitely a very, very high chance of collision when lowering your l_max.
What about the opposite case, raising it?
Back to our example, let's raise our l_size to 500, turning the next key into 4*500+1 = 2001, reserving the range 2001-2501.
It looks like collision will be avoided, in this particular implementation of HiLo, when raising your l_max.
Of course, you should do some own tests on your own to make sure that this is the actual implementation, or close to it. One way would be to set l_max to 100 and find the first few keys, then set it to 500 and find the next. If there is a huge jump like mentioned here, you might be safe.
However, I am not by any means suggesting that it is best practice to raise your l_max on an existing database.
Use your own discretion; the HiLo algorithm isn't exactly one made with varying l_max in mind, and your results may in the end be unpredictable depending on your exact implementation. Maybe someone who has had experience with raising their l_max and finding troubles can prove this count correct.
So in conclusion, even though, in theory, Hibernate's HiLo implementation will most likely avoid collisions when l_max is raised, it probably still isn't good practice. You should code as if l_max were not going to change over time.
But if you're feeling lucky...
See the Linear Chunk table allocator -- this is logically a more simple & correct approach to the same problem.
What's the Hi/Lo algorithm?
By allocating ranges from the number space & representing the NEXT directly, rather than complicating the logic with high words or multiplied numbers, you can directly see what keys are going to be generated.
Essentially, "Linear Chunk allocator" uses addition rather than multiplication. If the NEXT is 1000 & we've configured range-size of 20, NEXT will advance to 1020 and we'll hold keys 1000-1019 for allocation.
Range-sized can be tuned or reconfigured at any time, without loss of integrity. There is a direct relationship between the NEXT field of the allocator, the generated keys & MAX(ID) existing in the table.
(By comparison, "Hi-Lo" uses multiplication. If the next is 50 & the multiplier is 20, then you're allocating keys around 1000-1019. There are no direct correlation between NEXT, generated keys & MAX(ID) in the table, it is difficult to adjust NEXT safely and the multiplier can't be changed without disturbing current allocation point.)
With "Linear Chunk", you can configure how large each range/ chunk is -- size of 1 is equivalent to traditional table-based "single allocator" & hits the database to generate each key, size of 10 is 10x faster as it allocates a range of 10 at once, size of 50 or 100 is faster still..
A size of 65536 generates ugly-looking keys, wastes vast numbers of keys on server restart, and is equivalent to Scott Ambler's original HI-LO algorithm.
In short, Hi-Lo is an erroneously complex & flawed approach to what should have been conceptually trivially simple -- allocating ranges along a number line.
I tried to unearth behviour of HiLo algorith through a simple helloWrold-ish hibernate application.
I tried a hibernate example with
<generator class="hilo">
<param name="table">HILO_TABLE</param>
<param name="column">TEST_HILO</param>
<param name="max_lo">40</param>
</generator>
Table named "HILO_TABLE" created with single column "TEST_HILO"
Initially I set value of TEST_HILO column to to 8.
update HILO_TABLE set TEST_HILO=8;
I observed that pattern to create ID is
hivalue * lowvalue + hivalue
hivalue is column value in DB (i.e. select TEST_HILO from HILO_TABLE )
lowvalue is from config xml (40 )
so in this case IDs started from 8*40 + 8 = 328
In my hibernate example i added 200 rows in one session. so rows were created with IDs 328 to 527
And in DB hivalue was incremented till 13.
The increment logic seems to be :-
new hivalue in DB = inital value in DB + (rows_inserted/lowvalue + 1 )
= 8 + 200/40 = 8 + 5 =13
Now if I run same hibernate program to insert rows, the IDs should start from
13*40 + 13 = 533
When ran the program it was confirmed.
Just by experience I'd say: yes, decreasing will cause collisions. When you have a lower max low, you get lower numbers, independent of the high value in the database (which is handled the same way, eg. increment with each session factory instance in case of NH).
There is a chance that increasing will not cause collisions. But you either need to try or ask someone who knows better then I do to be sure.
Old question, I know, but worth answering with a 'yes, you can'
You can increase or decrease your nex_hi at any point as long as you recompute your hibernate_unique_key table based on the current Id numbers of your tables.
In our case, we have a Id per entity hibernate_unique_key table with two columns:
next_hi
EntityName.
The next_hi for any given Id is calculated as
SELECT MAX(Id) FROM TableName/(#max_lo + 1) + 1
The script below runs through every table with an Id column and updates our nex_hi values
DECLARE #scripts TABLE(Script VARCHAR(MAX))
DECLARE #max_lo VARCHAR(MAX) = '100';
INSERT INTO #scripts
SELECT '
INSERT INTO hibernate_unique_key (next_hi, EntityName)
SELECT
(SELECT ISNULL(Max(Id), 0) FROM ' + name + ')/(' + #max_lo + ' + 1) + 1, ''' + name + '''
'
FROM sys.tables WHERE type_desc = 'USER_TABLE'
AND COL_LENGTH(name, 'Id') IS NOT NULL
AND NOT EXISTS (select next_hi from hibernate_unique_key k where name = k.EntityName)
DECLARE curs CURSOR FOR SELECT * FROM #scripts
DECLARE #script VARCHAR(MAX)
OPEN curs
FETCH NEXT FROM curs INTO #script
WHILE ##FETCH_STATUS = 0
BEGIN
--PRINT #script
EXEC(#script)
FETCH NEXT FROM curs INTO #script
END
CLOSE curs
DEALLOCATE curs
I'm working on an application where I need to generate unique, non-sequential IDs. One of the constraints I have is that they must consist of 3 digits followed by 2 letters (only about 600k IDs). Given my relatively small pool of IDs I was considering simply generating all possible IDs, shuffling them and putting them into a database. Since, internally, I'll have a simple, sequential, ID to use, it'll be easy to pluck them out one at a time & be sure I don't have any repeats.
This doesn't feel like a very satisfying solution. Does anyone out there have a more interesting method of generating unique IDs from a limited pool than this 'lottery' method?
This can be done a lot of different ways, depending on what you are trying to optimize (speed, memory usage, etc.).
ID pattern = ddd c1c[0]
Option 1 (essentially like hashing, similar to Zak's):
1 Generate a random number between 0 and number of possibilities (676k).
2- Convert number to combination
ddd = random / (26^2)
c[0] = random % (26)
c[1] = (random / 26) % 26
3- Query DB for existence of ID and increment until a free one is found.
Option 2 (Linear feedback shift register, see wikipedia):
1- Seed with a random number in range (0,676k). (See below why you can't seed with '0')
2- Generate subsequent random numbers by applying the following to the current ID number
num = (num >> 1) ^ (-(num & 1u) & 0x90000u);
3- Skip IDs larger than range (ie 0xA50A0+)
4- Convert number into ID format (as above)
*You will need to save the last number generated that was used for an ID, but you won't need to query the DB to see if it is used. This solution will enumerate all possible IDs except [000 AA] due to the way the LFSR works.
[edit] Since your range is actually larger than you need, you can get back [000 AA] by subtracting 1 before you convert to the ID and have your valid range be (0,0xA50A0]
Use a finite group. Basically, take a 32 or 64-bit integer, and find a large number that is coprime to the maximum value for your integer; call this number M. Then, for all integers n, n * M will result in a unique number that has lots of digits.
This has the advantage that you don't need to pre-fill the database, or run a separate select query -- you can do this all from within one insert statement, by having your n just be an auto-increment, and have a separate ID column that defaults to the n * M.
You could generate a random ID conforming to that standard, do a DB select to see if it exists already, then insert it into a DB to note it has been "used". For the first 25% of the life of that scheme (or about 150k entries), it should be relatively fast to generate new random ID's. After that though, it will take longer and longer, and you might as well pre-fill the table to look for free IDs.
Depending on what you define as sequential, you could just pick a certain starting point on the letters, such as 'aa', and just loop through the three digits, so it would be:
001aa
002aa
003aa
Once you get to zz then increment the number part.
You could use modular arithmetic to generate ids. Pick a number that is coprime with 676,000 and for a seed. id is the standard incrementing id of the table. Then the following pseudocode is what you need:
uidNo = (id * seed) % 676000
digits = uidNo / 676
char1 = uidNo % 26
char2 = (uidNo / 26) % 26
uidCode = str(digits) + chr(char1+65) + chr(char2+65)
If a user has more than one consecutively issued id, they could guess the algorithm and the seed and generate all the ids in order. This may mean the algorithm is not secure enough for your use case.