What number would require 22 bytes for Oracle to store? - oracle

In Oracle 19c, using both DUMP and block dumps I have been able to store NUMBER values as large as to require 21 bytes (when negative). Any higher rounds off significant digits and allows numbers up to 125 digits but never uses more than 21 bytes. So if 21 is the greatest # of bytes that Oracle will ever use to store a number, why does dba_tab_cols and the official docs all say the maximum size is 22 bytes? What number would actually require 22 bytes?
If someone can do it, please provide DUMP output to show a 22-byte number, and what number it was.
Update: I inserted a single row with two NUMBER columns of -9+E38 each. Per block dump, they supposedly are stored this way:
But block dumps are interpreted for human consumption. So, I did an octal dump (od) on the datafile itself. The following octal dump shows that the size of the field is being stored first (0x15 = 21 bytes) adjacent to the first value by (0x2c), followed by 20 more value bytes ending with 0x6602. The next number then repeats the same thing. So, I guess one could say it does require 22 bytes to store a maximally precise number, if we count the size byte. But this is also true of all other datatypes, so it doesn't make sense that they would count this toward the "22" for numbers and not for other datatypes.

It's unclear why 22 bytes are allowed, and documented. Possibly some ancient history and the size has survived.
The 10gR2 documentation and Oracle support document 1031902.6 give different formulas for working out the length required to store a number; both are based on the precision (but not scale) and sign of the value, and both give a maximum value - for precision of 38 - of 21 bytes, not 22.
So it seems there can be no number which requires or can utilise all 22 bytes.
The Oracle Call Interface Programmer's Guide also says that the maximum internal storage for NUMBER is 21 bytes. The explanation of the NUMBER datatype in that document also explains the storage, probably better than the support document:
Oracle Database stores values of the NUMBER data type in a variable-length format. The first byte is the exponent and is followed by 1 to 20 mantissa bytes.
and as it's an OCI reference it also mentions "The output variable should be a 21-byte array to accommodate the largest possible number", but earlier says not to use it.
But it does also talk about the VARNUM datatype for which it says "Reserve 22 bytes to receive the longest possible VARNUM", which may be related to why 22 bytes are available, or may be a coincidence.

A partial answer of there does not appear to be one (but trying all values would take too long).
Starting from the reverse premise of can I force a 22-byte value into a number:
CREATE FUNCTION createNumber(hex VARCHAR2)
RETURN NUMBER DETERMINISTIC
IS
n NUMBER;
BEGIN
DBMS_OUTPUT.PUT_LINE( hex );
DBMS_STATS.CONVERT_RAW_VALUE( HEXTORAW( hex ), n );
RETURN n;
END;
/
Then:
CREATE TABLE table_name (value number);
INSERT INTO table_name (value)
SELECT createNumber(
LPAD(LEVEL - 1, 44, LEVEL - 1)
)
FROM DUAL
CONNECT BY LEVEL <= 10;
INSERT INTO table_name (value)
SELECT createNumber(
LPAD(CHR(96+LEVEL), 44, CHR(96+LEVEL))
)
FROM DUAL
CONNECT BY LEVEL <= 6;
Should insert 22-byte values into the table. However:
SELECT value, DUMP(value, 16) FROM table_name;
Outputs:
VALUE
DUMP(VALUE,16)
-~
Typ=2 Len=21: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
-84848484848484848484848484848484848484840000000000000000000000000000000000000000000000000000
Typ=2 Len=21: 11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11
-6767676767676767676767676767676767676767000000000000000000
Typ=2 Len=21: 22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22,22
-505050505050505050505050.505050505050505
Typ=2 Len=21: 33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,33
-.00000000003333333333333333333333333333333333333333
Typ=2 Len=21: 44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44,44
-.000000000000000000000000000000000000000000001616161616161616161616161616161616161616
Typ=2 Len=21: 55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55,55
null
Typ=2 Len=21: 66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66,66
null
Typ=2 Len=21: 77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77,77
null
Typ=2 Len=21: 88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88,88
null
Typ=2 Len=21: 99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99,99
null
Typ=2 Len=21: aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa
null
Typ=2 Len=21: bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb,bb
null
Typ=2 Len=21: cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc,cc
null
Typ=2 Len=21: dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd,dd
null
Typ=2 Len=21: ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee,ee
null
Typ=2 Len=21: ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff,ff
Which are all 21 bytes.
fiddle

Related

Execute immediate use in select * from in oracle

I am trying to get maximum length of column data in oracle by executing a dynamic query as part of Select statement , except seems i don't think we can use execute immediate in an select clause. Could i get some help with syntax or understanding as to better way to do this.
SELECT
owner OWNER,
table_name,
column_name,
'select max(length('||column_name||')) from '||table_name||';' max_data_length
FROM
dba_tab_columns
WHERE
AND ( data_type = 'NUMBER'
OR data_type = 'INTEGER' )
the 4th column in above query spits out a sql string rather than computing the value and returning it.
Here is some food for thought. Note that I am only looking for numeric columns that don't already have precision specified in the catalog. (If you prefer, you can audit all numeric columns and compare the declared precision against the actual precision used by your data.)
I am also looking only in specific schemas. Instead, you may give a list of schemas to be ignored; I hope you are not seriously considering making any changes to SYS, for example, even if it does (and it does!) have numeric columns without specified precision.
The catalog doesn't store INTEGER in the data type; instead, it stores that as NUMBER(38) So I am not searching for data type INTEGER in DBA_TAB_COLUMNS. But this raises an interesting question - perhaps you should search for all columns where DATA_PRECISION is null (as in my code below), but also for DATA_PRECISION = 38.
In the code below I use DBMS_OUTPUT to display the findings directly to the screen. You will probably want to do something smarter with this; either create a table function, or create a table and insert the findings in it, or perhaps even issue DDL already (note that those also require dynamic SQL).
This still leaves you to deal with scale. Perhaps you can get around that with a specification like NUMBER(prec, *) - not sure if that will meet your needs. But the idea is similar; you will just need to write code carefully, like I did for precision (accounting for the decimal point and the minus sign, for example).
Long story short, here is what I ran on my system, and the output it produced.
declare
prec number;
begin
for rec in (
select owner, table_name, column_name
from all_tab_columns
where owner in ('SCOTT', 'HR')
and data_type = 'NUMBER'
and data_precision is null
)
loop
execute immediate
'select max(length(translate(to_char(' || rec.column_name ||
'), ''0-.'', ''0'')))
from ' || rec.owner || '.' || rec.table_name
into prec;
dbms_output.put_line('owner: ' || lpad(rec.owner, 12, ' ') ||
' table name: ' || lpad(rec.table_name, 12, ' ') ||
' column_name: ' || lpad(rec.column_name, 12, ' ') ||
' precision: ' || prec);
end loop;
end;
/
PL/SQL procedure successfully completed.
owner: HR table name: REGIONS column_name: REGION_ID precision: 1
owner: HR table name: COUNTRIES column_name: REGION_ID precision: 1
owner: SCOTT table name: SALGRADE column_name: GRADE precision: 1
owner: SCOTT table name: SALGRADE column_name: LOSAL precision: 4
owner: SCOTT table name: SALGRADE column_name: HISAL precision: 4
PL/SQL procedure successfully completed.
EDIT
Here are several additional points (mostly, corrections) based on extended conversations with Sayan Malakshinov in comments to my answer and to his.
Most importantly, even if we can figure out max precision of numeric columns, that doesn't seem directly related to the ultimate goal of this whole thing, which is to determine the correct Postgre data types for the existing Oracle columns. For example in Postgre, unlike Oracle, it is important to distinguish between integer and non-integer. Unless scale is explicitly 0 in Oracle, we don't know that the column is "integers only"; we could find that out, through a similar dynamic SQL approach, but we would be checking for non-integer values, not precision.
Various corrections: My query is careless with regard to quoted identifiers (schema name, table name, column name). See the proper use of double-quotes in the dynamic query in Sayan's answer; my dynamic query should be modified to use double-quotes in the same way his does.
In my approach I pass numbers through TO_CHAR and then remove minus sign and decimal period. Of course, one's system may use comma, or other symbols, for decimal separator; the safer approach is to remove everything that is not a digit. That can be done with
translate(col_name, '0123456789' || col_name, '0123456789')
The query also doesn't handle very large or very small numbers, which can be stored in the Oracle database, but can only be represented in scientific notation when passed through TO_CHAR().
In any case, since "max precision" doesn't seem directly related to the ultimate goal of mapping to correct data types in Postgre, I am not changing the code - leaving it in the original form.
Thanks to Sayan for pointing out all these issues.
One more thing - *_TAB_COLUMNS contains information about view columns too; very likely those should be ignored for the task at hand. Very easy to do, as long as we realize it needs to be done.
Reading carefully that AWS article and since both previous answers (including mine) use rough estimate (length+to_char without format model and vsize operate decimal length, not bytes), I decided to write another full answer.
Look at this simple example:
with
function f_bin(x number) return varchar2 as
bi binary_integer;
e_overflow exception;
pragma exception_init(e_overflow, -1426);
begin
bi:=x;
return case when bi=x then 'ok' else 'error' end;
exception when e_overflow then return 'error';
end;
function f_check(x number, f varchar2) return varchar2 as
begin
return case when to_number(to_char(abs(x),f),f) = abs(x) then 'ok' else 'error' end;
exception when VALUE_ERROR then return 'error';
end;
a(a1) as (
select * from table(sys.odcinumberlist(
1,
0.1,
-0.1,
-7,
power(2,15)-1,
power(2,16)-1,
power(2,31)-1,
power(2,32)-1
))
)
select
a1,
f_check(a1,'fm0XXX') byte2,
f_check(a1,'fm0XXXXXXX') byte4,
f_bin(a1) ff_signed_binary_int,
to_char(abs(a1),'fm0XXXXXXXXXXXXXXX') f_byte8,
f_check(a1,'fm0XXXXXXXXXXXXXXX') byte8,
vsize(a1) vs,
dump(a1) dmp
from a;
Result:
A1 BYTE2 BYTE4 FF_SIGNED_ F_BYTE8 BYTE8 VS DMP
---------- ---------- ---------- ---------- ---------------- ---------- ---------- ----------------------------------------
1 ok ok ok 0000000000000001 ok 2 Typ=2 Len=2: 193,2
.1 error error error 0000000000000000 error 2 Typ=2 Len=2: 192,11
-.1 error error error 0000000000000000 error 3 Typ=2 Len=3: 63,91,102
-7 ok ok ok 0000000000000007 ok 3 Typ=2 Len=3: 62,94,102
32767 ok ok ok 0000000000007FFF ok 4 Typ=2 Len=4: 195,4,28,68
65535 ok ok ok 000000000000FFFF ok 4 Typ=2 Len=4: 195,7,56,36
2147483647 error ok ok 000000007FFFFFFF ok 6 Typ=2 Len=6: 197,22,48,49,37,48
4294967295 error ok error 00000000FFFFFFFF ok 6 Typ=2 Len=6: 197,43,95,97,73,96
Here I used PL/SQL functions for readability and to make it more clear.
Function f_bin casts an input number parameter to PL/SQL binary_integer (signed int4) and compares the result with input parameter, ie it checks if it loses accuracy. Defined exception shows that it can raise an exception 1426 "numeric overflow".
Function f_check does double conversion to_number(to_char(...)) of the input value and checks if it's still equal to the input value. Here I use hexadecimal format mask (XX = 1 byte), so it checks if an input number can fit an in this format mask. Hexadecimal format model works with non-negative numbers, so we need to use abs() here.
F_BYTE8 shows formatted value that uses a function from the column BYTE8, so you can easily see the loss of accuracy here.
All the above were just for readability, but we can make the same using just pure SQL:
with
a(a1) as (
select * from table(sys.odcinumberlist(
1,
0.1,
-0.1,
-7,
power(2,15)-1,
power(2,16)-1,
power(2,31)-1,
power(2,32)-1
))
)
select
a1,
case when abs(a1) = to_number(to_char(abs(a1),'fmXXXXXXXXXXXXXXX') default null on conversion error,'fmXXXXXXXXXXXXXXX')
then ceil(length(to_char(abs(a1),'fmXXXXXXXXXXXXXXX'))/2)
else -1
end xx,
vsize(a1) vs,
dump(a1) dmp
from a;
Result:
A1 XX VS DMP
---------- ---------- ---------- ----------------------------------------
1 1 2 Typ=2 Len=2: 193,2
.1 -1 2 Typ=2 Len=2: 192,11
-.1 -1 3 Typ=2 Len=3: 63,91,102
-7 1 3 Typ=2 Len=3: 62,94,102
32767 2 4 Typ=2 Len=4: 195,4,28,68
65535 2 4 Typ=2 Len=4: 195,7,56,36
2147483647 4 6 Typ=2 Len=6: 197,22,48,49,37,48
4294967295 4 6 Typ=2 Len=6: 197,43,95,97,73,96
As you can see, here I return -1 in case of conversion errors to byte8 and number of non-zero bytes otherwize.
Obviusly it can be simplified even more: you can just check range limits and that x=trunc(x) or mod(x,1)=0.
Looks like that is what you need:
VSIZE returns the number of bytes in the internal representation of expr. If expr is null, then this function returns null.
https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/VSIZE.html
In Oracle INTEGER is just a number(*,0): http://orasql.org/2012/11/10/differences-between-integerint-in-sql-and-plsql/
select
owner,table_name,column_name,
data_type,data_length,data_precision,data_scale,avg_col_len
,x.vs
from (select/*+ no_merge */ c.*
from dba_tab_columns c
where data_type='NUMBER'
and owner not in (select username from dba_users where ORACLE_MAINTAINED='Y')
) c
,xmltable(
'/ROWSET/ROW/VSIZE'
passing dbms_xmlgen.getxmltype('select nvl(max(vsize("'||c.column_name||'")),0) as VSIZE from "'||c.owner||'"."'||c.table_name||'"')
columns vs int path '.'
) x
;
Update: if you read oracle internal number format (exponent+bcd-mantissa and look at a result of dump(x) function, you can see that Oracle stores numbers as a decimal value(4 bits per 1 decimal digit, 2 digits in 1 byte), so for such small ranges you can just take their maximum BCD mantissa+1 exponent as a rough estimation.

Need specific number format in Oracle S9(15)V9(2)

I have a requirement to produce amount fields in zoned decimal format with this specific syntax below.
I don’t know if I need to create a function to handle this or if I can tweak the Oracle number format model. I’m thinking it might require some conditional formatting within a function due to the different requirement for number of digits between positive and negative. I will be performing this formatting on a couple of dozen data elements in the procedure so that might be another reason to use a function. Thoughts?
Requirement:
Amount should be represented by 17 characters (positive number) or 16 characters plus a “}” appended to the end (negative number).
Ex. 0.00 should show as 00000000000000000.
Ex. -935,560.00 should show as 00000000093556000}
Using Oracle 12c.
If I understood you correctly, the input is already formatted and its datatype is VARCHAR2. If that's so, then this might do the job:
SQL> with test (col) as
2 (select '0.00' from dual union all
3 select '25.34' from dual union all
4 select '-935,560.00' from dual
5 )
6 select col,
7 lpad(translate(col, 'x,.-', 'x'),
8 case when substr(col, 1, 1) = '-' then 16
9 else 17
10 end, '0') ||
11 case when substr(col, 1, 1) = '-' then '}'
12 else null
13 end result
14 from test;
COL RESULT
----------- --------------------
0.00 00000000000000000
25.34 00000000000002534
-935,560.00 0000000093556000}
SQL>
What does it do?
lines #1 - 5 - sample data
line #7 - translate removes minus sign, commas and dots
lines #7 - 10 - lpad pads the number (without characters from the previous step) with zeros up to the length of 16 (for negative values) or 17 (for positive values) characters
lines #11 - 13 - if it is a negative value, concatenate } to the end of the result string

PLSQL: ORA-01438: value larger than specified precision allowed for this column

Please check my inserting in oracle.
SQL> desc post;
Name Null? Type
----------------------------------------- -------- --------------------
POST_BY NOT NULL NUMBER(15)
POST_NO NOT NULL NUMBER(8)
TEXT VARCHAR2(500)
LANTITUDE NUMBER(3,4)
LONGITUDE NUMBER(3,4)
NO_LIKE NUMBER(6)
POST_DATE DATE
SQL> INSERT INTO post
2 values(
3 1,
4 1,
5 'Say somthing from user ',
6 1,
7 1,
8 0,
9 sysdate
10 );
Result:
1,*
ERROR at line 6:
ORA-01438: value larger than specified precision allowed for this column
I believe the error is actually being caused by the way you declared the NUMBER type for certain of your columns:
NUMBER(3,4)
This is defining a number with a precision of 3 significant figures, with 4 of them occurring after the decimal place and -1 of them occurring before the decimal place. Read this last sentence again carefully until you see why it doesn't work for the value 1. (It would work OK if you tried to insert the value 0.002 though... up to four decimal places, and the first has to be zero.)
If you want to give 4 decimal places of precision to your latitude and longitude values, then use the following defintion:
NUMBER(7, 4)
This means 3 digits before the decimal places and 4 digits after the decimal place.
This error is coming because you are using number(3,4) as its datatype.Number(3,4) means there are 3 significant digits and 4 digits after decimal.Change it.I think then it will work.

TO_char returning slash value after converting a number to String

I am having a database column amount [Data type Number(32,12)].When i use to_char on the amount field i get a slash value appended in the output.
When i directly used the value stored in the amount field ,i am getting the correct value
select TO_Char(0.000000000099,'FM99999999999999999999999999999990.099999999999') from dual;
Output:-
0.000000000099
It looks like you have corrupted data in your table. Which leads to a few questions including how did it get there, and what can you do about it?
Corrupt numeric (or date) values often come from OCI programs, but there are some bug reports that suggest imp has been known to cause corruption. The internal representation is documented in support note 1007641.6, but I find something like this explanation easier to work with when recreating problems, and using a PL/SQL block is possible in place of an OCI program.
The two numbers you're having problems with should be represented internally like this:
select dump(0.000000000099, 16) as d1,
dump(0.000000001680, 16) as d2
from dual;
D1 D2
------------------ ---------------------
Typ=2 Len=2: bb,64 Typ=2 Len=3: bc,11,51
I haven't figured out exactly what values you have in your table, but I can show a similar result:
create table t42 (amount number(32,12)) nologging;
declare
n number;
begin
dbms_stats.convert_raw_value('bb65', n);
insert into t42 (amount) values (n);
dbms_stats.convert_raw_value('bc100000', n);
insert into t42 (amount) values (n);
end;
/
Dumping the values shows they look a bit odd:
column d1 format a25
column d2 format a25
select amount, dump(amount) d1, dump(amount, 16) d2
from t42;
AMOUNT D1 D2
--------------------------- ------------------------- -------------------------
0.00000000010 Typ=2 Len=2: 187,101 Typ=2 Len=2: bb,65
0.000000001499 Typ=2 Len=3: 188,16,0 Typ=2 Len=3: bc,10,0
Running your formatting against that gives similar results:
select amount as actual__________amount,
TO_CHAR(amount,'FM99999999999999999999999999999990.099999999999')
as amount__________Changed
from t42
order by amount;
ACTUAL__________AMOUNT AMOUNT__________CHANGED
--------------------------- ----------------------------------------------
0.00000000010 ##############################################
0.000000001499 0.00000000150/
If you can add the dump() output for your own data to the question then I can see if I can recreate exactly the values you're seeing.
Anecdotally, it might be possible to 'correct' this by updating the data, e.g.:
update t42 set amount = amount * 1;
select amount, dump(amount) d1, dump(amount, 16) d2
from t42;
AMOUNT D1 D2
--------------------------- ------------------------- -------------------------
0.0000000001 Typ=2 Len=2: 188,2 Typ=2 Len=2: bc,2
0.000000001499 Typ=2 Len=3: 188,15,100 Typ=2 Len=3: bc,f,64
select amount as actual__________amount,
TO_CHAR(amount,'FM99999999999999999999999999999990.099999999999')
as amount__________Changed
from t42
order by amount;
ACTUAL__________AMOUNT AMOUNT__________CHANGED
--------------------------- ----------------------------------------------
0.0000000001 0.0000000001
0.000000001499 0.000000001499
However, you have to ask what the actual correct value is, which probably comes back to how/why/when it was corrupted. I would be very wary of touching this data if it is at all important, and would really have to second #DazzaL's advice to get Oracle Support involved to sort it out.

Oracle BINARY_FLOAT: 2 integers give same value?

I have a table full of 10-digit integers and thought to speed up queries/math in Oracle by storing them as BINARY_FLOAT. That's more CPU-friendly than NUMBER and won't take as much space (I think), which means more data in memory.
However, it appears that BINARY_FLOAT yields the same bytes (and hence value) for two different numbers...which obviously won't work.
Example:
SQL> select dump(to_binary_float(25185387)) from dual;
DUMP(TO_BINARY_FLOAT(2518538
----------------------------
Typ=100 Len=4: 203,192,38,54
SQL> select dump(to_binary_float(25185388)) from dual;
DUMP(TO_BINARY_FLOAT(2518538
----------------------------
Typ=100 Len=4: 203,192,38,54
SQL> CREATE TABLE blah ( somenum BINARY_FLOAT );
Table created.
SQL> insert into blah (somenum) values (25185387);
1 row created.
SQL> insert into blah (somenum) values (25185388);
1 row created.
SQL> select somenum from blah;
SOMENUM
----------
2.519E+007
2.519E+007
SQL> select to_number(somenum) from blah;
TO_NUMBER(SOMENUM)
------------------
25185388
25185388
SQL> select dump(somenum) from blah;
DUMP(SOMENUM)
------------------------------------------------------------------------------------------------------------------------
Typ=100 Len=4: 203,192,38,54
Typ=100 Len=4: 203,192,38,54
I expected that if I got into floating point, I might have some problem, but these are integers. I've tried various incantations - 25185387f, 25185387.0, 25185387*1.0, to_number(25185387), etc.
As I read the docs, BINARY_FLOAT should store to 1.79e308, so it can't be a rounding problem.
I'm using Oracle 11.2.0.3 on a 64-bit platform.
Ideas? Thanks.
Since the implementation of the oracle is BINARY_FLOAT standard ieee 754. BINARY_FLOAT is same as singe.
single have only 23 bits for significant bits.
25185387 = 11000000001001100011010 11 (length = 25)
25185388 = 11000000001001100011011 00 (length = 25)
hence the importance of these oracle rounds, discarding the least significant bits
25185387 ~ 11000000001001100011011 * 2^2
25185388 ~ 11000000001001100011011 * 2^2
so get the same value

Resources