I'm using integration services (SSIS), at the moment I'm getting the data from an excel source, the string Description comes with a length greater than 15 chars: the problem is that I can't find a way to truncate this data in order to save it in the database (the column database is varchar(15) and I can't change it).
I was trying to use a derived column in order to truncate the data with no success.
Add a derived column transformation and use the SUBSTRING function to get only the first 15 characters of the string. Read about the Substring function in SSIS here SUBSTRING SSIS Expression
Your expression in the derived column would look something like SUBSTRING(Description, 0, 15)
Related
I'd like to add a large string (above 76k characters) to CLOB column in Oracle database.
I need to run a script from liquibase framework.
How can achive this?
Simple insert
INSERT INTO table_clob (clob_column) VALUES (to_Clob('string above 72000 chars...'));
with and without to_clob() method is returning exception like:
ORA-01704: string literal too long
Cannot load data from file as described here with procedure: https://oracle-base.com/articles/8i/import-clob
as I don't have priviliges to any directory
Searched google but didn't enounter any solution for my requirement.
Any advice?
UPDATE:
After hours of searching finally found a workaround here: https://stackoverflow.com/a/49817056/1622703
It is not sufficient as I need to cut the text for 3 chunks manually (with around 30k chars), but it works.
Now just need to figure our how to do it dynamically in case that the string will have vary lenghts of chars (above 10k chars for example).
A hard-coded string enclosed in single quotes is known as a string literal. An example is 'Hello world'. Another example is the very long string you are trying to insert in the table. By contrast, 'abc' || 'def' is a string expression but it is not a string literal. Similarly, to_char(sysdate, 'yyyy-mm-dd') is a string expression, but not a literal. "Literal" means constant, hard-coded text.
The issue you are facing has nothing to do with insert, or to_clob(), or the data type of columns in your table, etc. It only has to do with the string literal itself.
In Oracle, a string literal can be at most 4000 bytes long (or 32767 bytes if the database is set up with extended MAX_STRING_SIZE). PERIOD! There is no way around it.
So, the question is, how can you ever get a string as long as the one you have into a table with a CLOB column. The answer depends on how you are receiving the string in the first place. The best option would be if it came in chunked already - as a collection of strings, with a tag (an id) to keep track of which fragment belongs to which CLOB and an ordinal number (to show if it's the first chunk, the second, etc.) Then you could re-assemble them using TO_CLOB() on the first chunk, plus the concatenation operator.
If your process is to type 72000 characters at the keyboard, you will have to type 4000 of them at a time, enclose in single quotes, and use the concatenation operator (essentially doing by hand what I described above). You would also have to use TO_CLOB() on the first fragment (otherwise the concatenation will fail).
Another option is for the string to come as a value, from some application that supports long strings (something compatible with Oracle's CLOB) and that can hand over such values to the Oracle database without the need to write out the hard-coded string in full.
So, the ball is in your court. The first question is, Where is the long string coming from in the first place?
I need to get part of a blob field which has some json data. one part of the blob is like this CustomData:{HDFC;1;0;sent} . I need separate values after CustomData like I need to get HDFC, 1, 0, sent separately.
This is what I have tried in two separate queries which works:
This gives me index of CustomData within payment_data blob field for example it returns 11000
select dbms_lob.instr(payment_data, utl_raw.cast_to_raw('CustomData'))
from table_x;
I am specifying 3rd parameter as what first query returned + length of test CustomData: to get {HDFC;1;0;sent}
select UTL_RAW.CAST_TO_VARCHAR2(dbms_lob.substr(payment_data,1000,11011))
from table_x;
Problem is I need to take dynamic offset in 2nd query and not run 1st query as individual. Specifying dynamic offset is not working with dbms_lob.substr() function. Any suggestions how can I combine these two queries into one?
Once I get {HDFC;1;0;sent}, I also need to get these delimited values separately, so combining these three into one would even be better if someone can help with it. I can use regexp_substr to get delimited text once I get first two combined.
If you want extract text data from blob first u need convert it to clob using dbms_lob.converttoclob.
If you have Oracle 12c or higher you may use JSON SQL functions, for example, JSON_TABLE.
If your Oracle version between 10 and 11 you may use regex functions or instr + substr if your version less than 10.
I'm new to Oracle, I'm using oracle 11g. I'm storing postal codes of UK. Values are like these.
N22 5HF
SW1 4JD
N14 8IT
N22 1JT
E1 5DP
e1 8DS
E3 8TU
I should be able to easily compare first four characters of each postal code.
What is the best data type to store these data ?
As a slight variation on Lalit's answer, since you want the outward code rather than a fixed substring of the first four characters (which could incude a space and the start of the inward code), you can create a virtual column based on the first word of the value:
postcode varchar2(8),
outward_code generated always as
(substr(postcode, 1, instr(postcode, ' ', 1, 1) - 1))
And optionally, but probably if you're using this to search, an index on the virtual column.
This assumes the post codes are formatted properly in the first place. It won't work if you don't always have the space between the outward and inward codes. And to answer your original question, the actual post code should be a varchar2(8) column to hold alphanumeric valus up to the maximum size and with the standard format.
SQL Fiddle demo.
I should be able to easily compare first four characters of each postal code.
Then keep these first four characters in a separate column. And index this column. You could keep the other characters in different column. Now, if the codes are a mixture of alphanumeric characters, then you are left with VARCHAR2 data type.
Your query predicate would like -
WHERE post_code_col = substr('N22 5HF', 1, 4)
Thus the indexed column post_code_col would be efficient in performance.
On 11g, you have the option to create a virtual column. However, indexing it would be equivalent to a function-based index. So I woukd prefer the first way as I suggested above.
It is better to normalize the table during the design phase, else the issues would start creeping in later.
In my opinion you should use varchar2 data type because this field will not going to be in mathematical calculations (they should not be int or decimal) and these fields are not big enough (so this should not be text)
I need an urgent help from you guys, the thing i have a column which represent the full name of a user , now i want to split it into first and last name.
The format of the Full name is "World, hello", now the first name here is hello and last name is world.
I am using Derived Column(SSIS) and using Right Function for First Name and substring function for last name, but the result of these seems to be blank, this where even i am blank. :)
It's working for me. In general, you should provide more detail in your questions on places such as this to help others recreate and troubleshoot your issue. You did not specify whether we needed to address NULLs in this field nor do I know how you'd want to interpret it so there is room for improvement on this answer.
I started with a simple OLE DB Source and hard coded a query of "SELECT 'World, Hello' AS Name".
I created 2 Derived Column Tasks. The first one adds a column to Data Flow called FirstCommaPosition. The formula I used is FINDSTRING(Name,",", 1) If NAME is NULLable, then we will need to test for nullability prior to calling the FINDSTRING function. You'll then need to determine how you will want to store the split data in the case of NULLs. I would assume both first and last are should be NULLed but I don't know that.
There are two reasons for doing this in separate steps. The first is performance. As counter-intuitive as it sounds, doing less in a derived column results in better performance because the SSIS engine can better parallelize the operations. The other is more simple - I will need to use this value to make the first and last name split so it will be easier and less maintenance to reference a column than to copy paste a formula.
The second Derived Column is going to actually perform the split.
My FirstNameUnicode column uses this formula (FirstCommaPosition > 0) ? RTRIM(LTRIM(RIGHT(Name,FirstCommaPosition))) : "" That says "If we found a comma in the preceding step, then slice out everything from the comma's position to the end of the string and apply trim operations. If we didn't find a comma, then just return a blank string. The default string type for expressions will be the Unicode (DT_WSTR) so if that is not your need, you will need to cast the resultant into the correct string codepage (DT_STR)
My LastNameUnicode column uses this formula (FirstCommaPosition > 0) ? SUBSTRING(Name,1,FirstCommaPosition -1) : "" Similar logic as above except now I use the SUBSTRING operation instead of RIGHT. Users of the 2012 release of SSIS and beyond, rejoice fo you can use the LEFT function instead of SUBSTRING. Also note that you will need to back off 1 position to remove the comma.
I have the following setup.
'Apps/Reports' <---------> 'DB2 Connect' <------------> 'Legacy DB2 on AS400'
`Hibernate` `native calls`
When data is retrieved from by the application, it will be padded with extra spaces if the length is less that the column length. Of note when running a query, if the WHERE cause parameter is not padded, its automatically padded with extra spaces such that the query will retrieve the same records for a padded and non-padded parameter.
Is there a way (preferably on IBM DB2 Connect or connection string parameter) to remove extra whitespaces from a resultset?
You could implement an hibernate UserType that automatically trim the strings. There are good exmaples on
https://forum.hibernate.org/viewtopic.php?t=928294
http://java.dzone.com/articles/annotating-custom-types
http://santescas.blogspot.de/2014/02/creando-un-usertype-de-hibernate-que-se.html
Are the columns in the iSeries defined as CHAR()? I'm assuming so, because this is how CHAR() works -- it's a fixed field length, not a variable field length (that's what VARCHAR is for).