Could you please let me know what is the Data length for the 2nd column_id of "CLOB" data type in the Employee table? I see some blogs where it says maximum data length is : (4GB -1)* (database block size).
I'm new to this data designing.
Table : Employee
**Column_Name ----- Data_Type ------- Nullable ---- Column_Id**
Emp_ID NUMBER No 1
Emp_details CLOB NO 2
Please help me.
To get CLOB size for a given column in a given row, use DBMS_LOB.GETLENGTH function:
select dbms_lob.getlength(emp_details) from employee from emp_id=1;
To get CLOB size for a given column in a given table that is allocated in the tablespace, you need to identify both segments implementing the LOB.
You can compare both size with following query:
select v1.col_size, v2.seg_size from
(select sum(dbms_lob.getlength(emp_details)) as col_size from employee) v1,
(select sum(bytes) as seg_size from user_segments where segment_name in
(
(select segment_name from user_lobs where table_name='EMPLOYEE' and column_name='EMP_DETAILS')
union
(select index_name from user_lobs where table_name='EMPLOYEE' and column_name='EMP_DETAILS')
)
) v2
;
LOBs are not stored in the table, but outside of it in a dedicated structure called LOB segment, using an LOB index. As #pifor explains, you can inspect those structures in the dictionary view user_lobs.
The LOB segment uses blocks of usually 8192 bytes (check the tablespace in user_lobs), so the minimum size allocated for a single LOB is 8K. For 10.000 bytes, you need two 8K blocks and so on.
Please note that if your database is set to Unicode (as most modern Oracle databases are), the size of a CLOB is roughly 2x as expectet, because they are stored in a 16 bit encoding.
This gets a bit better if you compress the LOBs, but your Oracle license needs to cover "Advanced Compression".
For very small LOBs (less than ca 4000 bytes), you can avoid the 8K overhead and store them in the table where all the other columns are (enable storage in row).
Related
I am always use TRUNCATE TABLE to free up space and always the space returned to free space
when I check the free space in oracle tablespace I found the free space increased always
I am using TOAD program to
TRUNCATE TABLE
But this time different I truncated 2 tables but its not free the table space and when I checked
the table size after truncated its still same size used before around 400 MB
why its not free the space , and how to free the space table already truncated and no data in it.
also I moved the table to another tablespace after truncate but its moved with same size
ALTER TABLE CAS_NOSHOW MOVE TABLESPACE TRNG;
how to free up space please your help .
I am using oracle 10g database
If you're looking to recover space above and below the high water mark you can shrink the table.
To do this you need to have row movement enabled.
create table t as
select rownum x, lpad('x', 500, 'x') xx from dual connect by level <= 10000;
/
-- ALTER TABLE t SHRINK SPACE CHECK;
select bytes from user_segments
where segment_name = 'T';
/
delete t where x <= 9900;
/
select bytes from user_segments
where segment_name = 'T';
/
alter table t enable row movement;
/
alter table t shrink space CASCADE;
/
select bytes from user_segments
where segment_name = 'T';
I have a table that stores the changes to a transaction. All the changes are captured into a table. One of the column that comes as part of the transaction can have many comma separated values. The number of occurrences cannot be predicted. Also this field is not mandatory and can have null values as well.
The total number of transactions that i have in the table is around 100M. Out of those the number of records for which the value is populated is 1M. Out of the 1M transactions the number of records for which the length of the record exceeds 4000 is ~37K.
I mentioned the length as 4000 since in my oracle table the column which would save this has been defined as varchar2(4000).
I check at places and found that if I have to save something of unknown length then i should define the table column datatype as clob. But clob is expensive for me since only a very small amount of data has length > 4000. If I snowflake my star schema and create another table to store the values then even though, I have transactions for which the length is much smaller than 4000 would be saved as part of the clob column. This would be expensive both in terms of storage and performance.
Can someone suggest me an approach to solve this problem.
Thanks
S
You could create a master - detail table to store the comma separated values, then you could have rows rather than save all comma separated values in a single column. This could be managed with a foregn key using a pseudo key between master and detail table.
Here's one option.
Create two columns, e.g.
create table storage
(id number primary key,
long_text_1 varchar2(4000),
long_text_2 varchar2(4000)
);
Store values like
insert into storage (id, long_text_1, long_text_2)
values (seq.nextval,
substr(input_value, 1, 4000),
substr(input_value, 4001, 4000)
);
When retrieving them from the table, concatenate them:
select id,
long_text_1 || long_text_2 as long_text
from storage
where ...
You might benefit from using inline SecurFile CLOBs. With inline CLOBs, up to about 4000 bytes of data can be stored in rows like a regular VARCHAR2 and only the larger values will be stored in a separate CLOB segment. With SecureFiles, Oracle can significantly improve CLOB performance. (For example, import and export of SecureFiles is much faster than the old-fashioned BasicFile LOB format.)
Depending on your version, parameters, and table DDL, your database may already store CLOBs as inline SecureFiles. Ensure that your COMPATIBLE setting is 11.2 or higher, and that DB_SECUREFILE is one of "permitted", "always", or "preferred":
select name, value from v$parameter where name in ('compatible', 'db_securefile') order by 1;
Use a query like this to ensure that your tables were setup correctly, and nobody overrode the system settings:
select dbms_metadata.get_ddl('TABLE', 'YOUR_TABLE_NAME') from dual;
You should see something like this in the results:
... LOB ("CLOB_NAME") STORE AS SECUREFILE (... ENABLE STORAGE IN ROW ...) ...
One of the main problems with CLOBs is that they are stored in a separate segment, and a LOB index must be traversed to map each row in the table to a value in another segment. The below demo creates two tables to show that LOB segments do not need to be used when the the data is small and stored inline.
--drop table clob_test_inline;
--drop table clob_test_not_in;
create table clob_test_inline(a number, b clob) lob(b) store as securefile (enable storage in row);
create table clob_test_not_in(a number, b clob) lob(b) store as (disable storage in row);
insert into clob_test_inline select level, lpad('A', 900, 'A') from dual connect by level <= 10000;
insert into clob_test_not_in select level, lpad('A', 900, 'A') from dual connect by level <= 10000;
commit;
The inline table segment is large, because it holds all the data. The out of line table segment is small, because all of its data is held elsewhere.
select segment_name, bytes/1024/1024 mb_inline
from dba_segments
where segment_name like 'CLOB_TEST%'
order by 1;
SEGMENT_NAME MB_INLINE
---------------- ---------
CLOB_TEST_INLINE 27
CLOB_TEST_NOT_IN 0.625
Looking at the LOB segments, the sizes are reversed. The inline table doesn't store anything in the LOB segment.
select table_name, bytes/1024/1024 mb_out_of_line
from dba_segments
join dba_lobs
on dba_segments.owner = dba_lobs.owner
and dba_segments.segment_name = dba_lobs.segment_name
where dba_lobs.table_name like 'CLOB_TEST%'
order by 1;
TABLE_NAME MB_OUT_OF_LINE
------------ --------------
CLOB_TEST_INLINE 0.125
CLOB_TEST_NOT_IN 88.1875
Despite the above, I can't promise that CLOBs will still work for you. All I can say is that it's worth testing the data using CLOBs. You'll still need to look out for a few things. CLOBs store text slightly differently (UCS2 instead of UTF8), which may take up more space depending on your character sets. So check the segment sizes. But also beware that segment sizes can lie when they are small - there's a lot of auto-allocated overhead for sample data, so you'll want to use realistic sizes when testing.
Finally, as Raul pointed out, storing non-atomic values in a field is usually a terrible mistake. That said, there are rare times when data warehouses need to break the rules for performance, and data needs to be stored as compactly as possible. Before you store the data this way, ensure that you will never need to join based on those values, or query for individual values. If you think dealing with 100M rows is tough, just wait until you try to split 100M values and then join them to another table.
I have very limited experience when using Oracle, and am after a rather simple query I imagine. I have a table which contains 1 million rows, Im trying to proof that compressing the data uses less space, however im not sure how to do this, based on this table creation below could someone please show me what i need to write to see the blocks used before/after?
CREATE TABLE OrderTableCompressed(OrderID, StaffID, CustomerID, TotalOrderValue)
as (select level, ceil(dbms_random.value(0, 1000)),
ceil(dbms_random.value(0,10000)),
round(dbms_random.value(0,10000),2)
from dual
connect by level <= 1000000);
ALTER TABLE OrderTableCompressed ADD CONSTRAINT OrderID_PKC PRIMARY KEY (OrderID);
--QUERY HERE THAT SHOWS BLOCKS USED/TIME TAKEN
SELECT COUNT(ORDERID) FROM OrderTableCompressed;
ALTER TABLE OrderTableCompressed COMPRESS;
--QUERY HERE THAT SHOWS BLOCKS USED/TIME TAKEN WHEN COMPRESSED
SELECT COUNT(ORDERID) FROM OrderTableCompressed;
I know how the compression works etc... its just applying the code to proove my theory. thanks for any help
--QUERY HERE THAT SHOWS BLOCKS USED
SELECT blocks, bytes/1024/1024 as MB
FROM user_segments
where segment_name = 'ORDERTABLECOMPRESSED';
Now compress the table: (Note the move. Without it you just change the attribute of the table and subsequent direct path inserts will create compressed blocks)
ALTER TABLE OrderTableCompressed MOVE COMPRESS;
Verify blocks:
--QUERY HERE THAT SHOWS BLOCKS USED TAKEN WHEN COMPRESSED
SELECT blocks, bytes/1024/1024 as MB
FROM user_segments
where segment_name = 'ORDERTABLECOMPRESSED';
Does anybody know how SecureFile stores LOBs into chunks? The documentation says the CHUNK parameter is only advisory (http://docs.oracle.com/cd/E11882_01/appdev.112/e18294/adlob_smart.htm#CIHIDGJA).
I did some initial tests, it looks like Oracle uses minimally one block per LOB (EDIT: if it is stored in the lobsegment). Is that right?
CREATE TABLE sftest (mylob CLOB)
LOB (mylob) STORE AS SECUREFILE sftest_mylob (DISABLE STORAGE IN ROW);
INSERT INTO sftest SELECT object_name FROM all_objects;
11,825 rows inserted
SELECT blocks FROM user_segments WHERE segment_name='SFTEST_MYLOB';
14336
Actually, minimum one block per LOB is incorrect. It can be much smaller, depending on the size of your LOB. From the Oracle docs
LOB values are stored inline when any of the following conditions
apply:
When the size of the LOB stored in the given row is small, approximately 4000 bytes or less, and you either explicitly specify
ENABLE STORAGE IN ROW or the LOB storage clause when you create the
table, or when you do not specify this parameter (which is the
default).
When the LOB value is NULL (regardless of the LOB storage properties
for the column).
Using the default LOB storage properties (inline storage) can allow
for better database performance; it avoids the overhead of creating
and managing out-of-line storage for smaller LOB values. If LOB values
stored in your database are frequently small in size, then using
inline storage is recommended.
Yes, the minimal number of blocks of a SecureFile LOB is 1 (if it is not stored in row).
We are using SecureFile LOBs for a number of years in production, and I found out how to inspect the block number of an individual LOB. First, you need to get the number of extents the LOB has by calling DBMS_LOBUTIL.GETINODE(mylob).EXTENTS, then you can find out for each extent, how many blocks are stored it it with DBMS_LOBUTIL.GETLOBMAP(mylob, myextent).NBLKS, e.g.
SELECT DBMS_LOBUTIL.GETINODE(mylob).LENGTH AS len,
DBMS_LOBUTIL.GETINODE(mylob).EXTENTS AS extents,
DBMS_LOBUTIL.GETLOBMAP(mylob, 0).NBLKS AS nblks
FROM sftest;
LEN EXTENTS NBLKS
34 1 1
24 1 1
42 1 1
I have a Tablespace with total Size 600 MB(this is my test machine), and free space of 110 MB. I was trying to compress a table that's size is 120 MB. When execute the commands:
alter table DOWNLOADSESSIONLOG move compress;
I'm getting error saying:
(Error): ORA-01659: unable to allocate MINEXTENTS beyond 16 in tablespace WAP.
I was searching and all are saying to increase the Tablespace, but I wanted to know how much space I need to give, because I don't want to gave some 200 /300 MB extra space. I wanted to test this in my Test Machine and wanted to implement on Live system that have 60 GB tablespace with one 47 GB table. I wanted to compress 47 GB table on live system, before that I wanted to test it in test environment. Is there any calculation how much space we need to give; otherwise on live system I need to give Lots of space unnecessarily and its difficult.
Appreciate somebody can give some ideas.
I would create a new throw-away tablespace in your test environment that has the same characteristics as your target tablespace in the production environment and then move/compress your table into that. This will give you the best estimate of how much additional space will be necessary. You can move the table back to the original tablespace and drop the new tablespace once you have this number.
Remember that you'll need at least (original size) + (compressed size) available in the tablespace for the duration of the move.
The key thing about compression is that it works by removing duplicate values in each block. So your test table needs to have a representative spread of data.
Two extreme tables ...
SQL> create table totally_similar
2 ( txt varchar2(1000) )
3 /
Table created.
SQL> insert into totally_similar
2 select rpad('a', 1000, 'a')
3 from dual connect by level <= 1000
4 /
1000 rows created.
SQL> create table totally_different
2 ( txt varchar2(1000) )
3 /
Table created.
SQL>
Before we compress let's just check the table sizes...
SQL> insert into totally_different
2 select dbms_random.string('A',1000)
3 from dual connect by level <= 1000
4 /
1000 rows created.
SQL> select segment_name
2 , sum(bytes)
3 , sum(blocks)
4 from user_segments
5 where segment_name in ('TOTALLY_SIMILAR', 'TOTALLY_DIFFERENT')
6 group by segment_name
7 /
SEGMENT_NAME SUM(BYTES) SUM(BLOCKS)
-------------------- ---------- -----------
TOTALLY_SIMILAR 2097152 256
TOTALLY_DIFFERENT 2097152 256
SQL>
If we compress them we get two radically different results ...
SQL> alter table totally_similar move compress
2 /
Table altered.
SQL> alter table totally_different move compress
2 /
Table altered.
SQL> select segment_name
2 , sum(bytes)
3 , sum(blocks)
4 from user_segments
5 where segment_name in ('TOTALLY_SIMILAR', 'TOTALLY_DIFFERENT')
6 group by segment_name
7 /
SEGMENT_NAME SUM(BYTES) SUM(BLOCKS)
-------------------- ---------- -----------
TOTALLY_SIMILAR 65536 8
TOTALLY_DIFFERENT 2097152 256
SQL>
Note that TOTALLY_SIMILAR is eight blocks big, even though every single row was the same. So you need to understand the distribution of your data before you can calculate the compression ratio. The Oracle documentation has this to say:
The compression factor that can be
achieved depends on the cardinality of
a specific column or column pairs
(representing the likelihood of column
value repetitions) and on the average
row length of those columns. Oracle
table compression not only compresses
duplicate values of a single column
but tries to use multi-column value
pairs whenever possible.
Its advice when it comes to estimation of the return is that a sample table of 1000 blocks of the target table should give you a good enough prediction (although more blocks give a more accurate forecast). It is hard to tell without knowing your blocksize, but it seems likely that your TEST table is much bigger than it needs to be. The important thing is whether the data in the test table is representative of your target table. So, did you create it using an export or a sample from the target table, e.g.
create table test_table as select * from big_table sample block (1)
/
You will need to adjust the percentage in the SAMPLE() clause to ensure you get at least 1000 blocks.
edit
In most cases compression should actually speed up data retrieval but YMMV. The cost of compression is paid when inserting or updating the data. How much that tax is and whether you can do anything to avoid it rather depends on the profile of your table.