In Datastage, we have source system as Oracle and target system as Netezza. In Oracle the column datatype is varchar whereas in Netezza it is nvarchar. Most of the characters are Latin and Dutch.
We are getting character in our table row which is exactly opposite to the one mentioned in bracket (`) means it's heading towards right and slanting on left(mostly dutch). We feel it is Dutch character which represent apostrophe. The table consists of million records and many values in table have this special character. We want to process the value as it is but we are getting garbage value. Can anyone help us in which conversion function we should try?
I tried iso-8859-1 and iso-8859-15
Related
I need to update value in one table, which is having special character.
Below is the Update Query I have Executed:
UPDATE TABLE_X
SET DISPLAY_NAME = 'AC¦', NATIVE_IDENTITY='AC¦'
WHERE ID='idNumber'
Special Character "¦" is not getting updated in Oracle.
I have already tried below approaches:
Checked the character set being used in Oracle using below query
select * from nls_database_parameters where parameter='NLS_CHARACTERSET';
It is having "US7ASCII" Character set.
I have tried to see if any of the character set will help using below query
SELECT CONVERT('¦ ', 'ASCII') FROM DUAL;
I have tried below different encoding:
WE8MSWIN1252
AL32UTF8
BINARY - this one is giving error "ORA-01482: unsupported character set"
Before Changing the character set in DB i wanted to try out 'CONVERT' function from Oracle, but above mentioned character set is either returning "Block Symbol" or "QuestionMark � " Symbol.
Any idea how can I incorporate this special symbol in DB?
Assuming that the character in question is not part of the US7ASCII character set, which it does not appear to be unless you want to replace it with the ASCII vertical bar character |, you can't validly store the character in a VARCHAR2 column in the database.
You can change the database character set to a character set that supports all the characters you want to represent
You can change the data type of the column to NVARCHAR2 assuming your national character set is UTF-16 which it would normally be.
You can store a binary representation of the character in some character set you know in a RAW column and convert back from the binary representation in your application logic.
I would prefer changing the database character set but that is potentially a significant change.
I have some issues related to special characters in some tables. For example, some words with character ü were inserted in database as NŒ. Is there a way to find this unicode problems and convert it in a table?
I also checked NLS_CHARACTERSET and LS_NCHAR_CHARACTERSET v$nls_parameters table and it looks fine (AL32UTF8 and UTF8).
I do find a problem with enconding of characters into Oracle
They are two inverted interrogation characters coming from imported data.
How can i search for two inverted characters into Oracle so i can see how many lines have this problem ?
I assume by "inverted interrogation character" you mean character ¿.
There are two possibilities:
Character ¿ is actually stored in your database, because your database character set (check with SELECT * FROM V$NLS_PARAMETERS WHERE PARAMETER LIKE '%CHARACTERSET') is not capable to support special characters you tried to import.
You can find affected rows with SELECT * FROM TABLE_NAME WHERE REGEXP_LIKE(COL_NAME, '¿');
Your client (e.g. SQL*Plus) is not able to display the special character and substitute those by placeholder ¿. In this case set your NLS_LANG value properly, see this answer for more details.
Unfortunately you did not tell us how you imported your data nor any of your characters sets. Thus I cannot provide you a guideline to set NLS_LANG properly.
I have a problem inserting special characters (á é í ú or ñ) in a char(1) field.
CREATE TABLE sgc2."tabtest2"(field1 CHAR(1), field2 VARCHAR(1));
INSERT INTO sgc2."tabtest2" values('á', 'á');
ERROR:
Value "á" is too long.. SQLCODE=-433, SQLSTATE=22001, DRIVER=4.13.111
Apparently to insert these characters take two byte, and as the field only accepts one can not end with the insertion.
Is there any way to configure the database, to support these special characters taking only 1 byte?
Apparently your database was created with the Unicode codeset, where special characters are represented by multiple bytes. If you only need to represent a limited range of accented characters you can choose one of the supported codesets, specified by ISO-8859, for the corresponding language -- details in the manual. You will have to re-create the database using an appropriate CODESET option, as you cannot change the codeset of an existing database.
However, you should consider changing your tables instead, as Unicode gives you more flexibility. A Unicode database can also be a requirement for certain DB2 features, for example BLU Acceleration.
Recently I came across a unicode character (\u2019) in a database table column while parsing using Python.
Question: What are the reasons that can result in unicode characters showing up in the database table? Is it data entry issue?
Appreciate any input.
When you set up your Oracle Database you choose a character set which will be used in the SQL char datatypes (char, varchar2 etc).
Suppose you chose your character set and you have a table with a column of VARCHAR2 type. Suddenly you need to store some string with non-ASCII symbols not supported by your database (chosen character set). You may convert this string into ASCII string by calling ASCIISTR function for example and store it in your VARCHAR2 column (but it's not a good idea because many SQL built-in functions don't understand '\u2019' (they think it's just 6 symbols)). That's how Unicode may appear in your table column (ASCIISTR converts non-ascii symbols into unicode representation such as '\u2019').
Another option is special Oracle nchar datatypes which were designed to store UNICODE without altering global database settings.
Here is the link with Oracle documentation: https://docs.oracle.com/cd/B19306_01/server.102/b14225/ch6unicode.htm