Oracle SQL Developer client encoding - windows

I read many of the related Stack Overflow's topics and I spent a whole day with googleing the following problem but I haven't found anything that would help, however the problem not seems to be complicated.
I have an Oracle database.
Let's see the following PL/SQL script:
CREATE TABLE Dummy(
id number(19,0),
tclob clob,
tnclob nclob,
PRIMARY KEY (id));
INSERT INTO dummy (id, tclob, tnclob) VALUES (1, 'ñ$ߤ*>;''<’', 'ñ$ߤ*>;''<’');
SELECT tclob, tnclob FROM dummy;
My problem is that 'ñ' and '’' characters are stored as a question mark.
I also tried to load the previously inserted values through JAVA, but I get the question marks instead of the special characters.
I created a small Java method which uses OraclePreparedStatement to save test data, and I use setNString() method to attach the nclob data to the query. In this case all characters are displayed fine in Java and also in SqlDeveloper.
So a possible solution is to use JAVA to save my data into the db. I have a thousands of lines SQL script which inserts data and I don't necessarily want to write the whole thing again in java.
So the question is: why the SqlDeveloper breaks the special characters?
My settings:
SELECT DECODE(parameter, 'NLS_CHARACTERSET', 'CHARACTER SET',
'NLS_LANGUAGE', 'LANGUAGE',
'NLS_TERRITORY', 'TERRITORY') name,
value from v$nls_parameters
WHERE parameter IN ( 'NLS_CHARACTERSET', 'NLS_LANGUAGE', 'NLS_TERRITORY')
Result:
+---------------+--------------+
| NAME | VALUE |
+---------------+--------------+
| LANGUAGE | HUNGARIAN |
| TERRITORY | HUNGARY |
| CHARACTER SET | EE8ISO8859P2 |
+---------------+--------------+
I changed SqlDeveloper/Preferences/Environment/Encoding to UTF-8.
I also changed HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_OraDb11g_home1 value to HUNGARIAN_HUNGARY.UTF8
Update: I tried to insert the data with the following syntaxes:
INSERT INTO dummy (id, tclob, tnclob) VALUES (1, N'ñ$ߤ*>;''<’', N'ñ$ߤ*>;''<’');
INSERT INTO dummy (id, tclob, tnclob) VALUES (1, 'ñ$ߤ*>;''<’', to_nclob('ñ$ߤ*>;''<’'));
Nothing helped.
So what can I do?

On the PC that PLSQL is installed, set the value of NLS_LANG registery entry equal to the PC's operation system locale (code page), equivalent value.
How to detect operating system language locale?
How to map OS locale to NLS_LANG value?
When using PLSQL the initial parameter of client-language that is required to create an Oracle session is read from NLS_LANG registry entry.
Due to Oracle documents, invalid data usually occurs in a database because the NLS_LANG parameter is not set properly on the client.
The NLS_LANG value should reflect the client operating system code page.
For example, in an English Windows environment, the code page is WE8MSWIN1252. When the NLS_LANG parameter is set properly, the database can automatically convert incoming data from the client operating system to its encoding.
When using JAVA method, the client-language parameter is set by the value from the Control Panel, under Regional and Language Options, so the things will be OK.

You can try to change NLS_LANG value on your Win PC with regedit tool.
Path is: \HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE, when NLS_LANG contain value of your Oracle client's current encoding.
There is Oracle's list of available encodings:
Commonly Used Values for NLS_LANG

Related

Gibberish filed values when Loading data with Oracle SQL Loader (sqlldr)

Using
sqlldr
to load data in an oracle table doesn't result in any errors but the value of the fields are gibberish. not all the values of course, only the values which was in persian(arabic) format.
already read many questions here but could not resolve the issue, most relatable topic to this problem were :[Unreadable character in generated sqlplus file
although the values in my case isn't question mark, but something like ,ÑÇå Âåä or äÇ ãÔÎÕ
Also played with the NLS_LANG environment variable but it was to no avail.
Created different oracle databases with different character set's, this also was to no avail.
I'm new to oracle, so it is highly likely that I'm making a rookie mistake while creating database and setting character set or something else, to be honest i have no idea. but tried many responses from users and here i am.
I Uploaded the table schema and Ctl extension File to replicate the problem, the link to the related files are resided in this link: [https://www.dropbox.com/sh/ejxvast0ruioksk/AABXhjujqzhRpuMVjl7V-zxUa?dl=0][1]
In total you have three character sets or encodings.
What is the encoding of your file? Check the save options of the editor or the application which created the file.
The character set of your command line window cmd.exe, called "codepage". You can interrogate (or change) with command chcp
The character set of your database.
1) and 2) must be the same. Use command chcp to set them equal (or change settings in your editor)
3) can be different but the character set must support persian/arabic characters, so most likely AL32UTF8 which is the default nowadays.
Use the NLS_LANG value to tell the database which character set is used for 1) and 2), example
C:\>chcp 1256
Aktive Codepage: 1256.
C:\>set NLS_LANG=.AR8MSWIN1256
C:\>sqlldr ...
You can get a list of codepages vs. Oracle character set with this query:
SELECT VALUE AS ORACLE_CHARSET, UTL_I18N.MAP_CHARSET(VALUE) AS IANA_NAME
FROM V$NLS_VALID_VALUES
WHERE PARAMETER = 'CHARACTERSET';
And here a list of Code Page Identifiers
See also OdbcConnection returning Chinese Characters as "?" to get more details.

Compilation converts utf-8 characters to question mark in PLSQL developer

I am using PLSQL developer to work with oracle db. When I compile a view which stores code like this:
select 'xidmət' as service from dual
the 'ə' character in the string becomes '?' character.
I think it's some oracle or plsql developer configuration problem, but I don't know what.What do you think the problem is?
First, check what your character set is using this:
select value from nls_database_parameters where parameter='NLS_CHARACTERSET';
Then set your NLS_LANG environment variable to AMERICAN_AMERICA.CHARSET where CHARSET is the value found with that select. If you are in Windows, you will have to go to Control Panel, System, Advanced, Environment Variables, and set NLS_LANG under System variables.
Oracle does at least a 'one-pass' conversion between client and database, but the problem is that there are so many layers between client and database, including your client software, that it is usually better to match your client NLS_LANG with the database setting.
It also depends how that character was inserted. It might have been inserted using a different client tool using a different NLS_LANG, so you might have to update your extended ASCII characters (or foreign characters) before you get a consistent view from your select.
Need to configure your systems NLS_LANG parameter, according to your language preferences. Here's a link:
http://www.nazmulhuda.info/setting-nls_lang-environment-variable-for-windows-and-unix-for-oracle-database
For example, we have letters like "ąčęėįšųū". So in order to see them in pl/sql, we set NLS_LANG with value "LITHUANIAN_LITHUANIA.BLT8MSWIN1257".
Hope it helps. Good luck.

Implementing Japanese localization

I've been tasked with determining if our web platform can be 'localized' to Japanese, and how to do so. The platform is PL/SQL based in an Oracle 10g database. We have localized it for French Canadian and Brazilian Portuguese in the past, but I'm wondering what issues I may run into with Japanese (Kanji, I believe). Am I correct that Japanese is a double-byte char set while the others we've used are single-byte? How will this impact code and/or database table structure and access?
The various sentences/phrases/statements are stored in a database table and are looked up as needed based on the user's id and language setting. The table field that stores the 'text' is defined as a CLOB. It's often read into a VARCHAR2 variable.
I tried to copy/past some Japanese characters into the table via a direct paste to the field in a TOAD schema browser. That resulted in '??' being displayed.
Is there anything I have to do in order to be able to store Japanese characters in that table? Or access/display them from that table?
Check your database character set by
SELECT *
FROM V$NLS_PARAMETERS
WHERE PARAMETER IN ('NLS_CHARACTERSET', 'NLS_NCHAR_CHARACTERSET');
If the character set support Japanese (e.g. AL32UTF8) it should be no big deal to localize your application also to Japanese. Changing the character set on an existing database is also possible but requires some effort, see Character Set Migration
Check also this answer for topics related to database character set vs. client character set, i.e. NLS_LANG setting.

How do I change the NLS_CHARACTERSET setting on Windows 7 for Oracle 11g ODBC client

I'm trying to retrieve data from an Oracle database into MS Access or Excel on to a Windows 7 PC. The data includes characters such as degree and diameter symbols. These are not displayed correctly on the PC. I can see that the data is stored correctly using the dump function in my query, and that it is the translation to the client character set that not bringing them in.
By looking at this query, and also querying the database parameters with SELECT * FROM NLS_database_PARAMETERS, I can see that the NLS_CHARACTERSET = US7ASCII. I think this is cause of the problem as the diameter symbol etc is not included in this character set.
But, I've checked the registry to see what NLS_LANG is set to, and it is ENGLISH_UNITED KINGDOM.WE8MSWIN1252. I know that I'm looking in the correct registry key because when I change the language/territory (to GERMAN_GERMANY for example) it does change for my queries.
I've also checked for an environment variable that is overriding this setting and couldn't find one. I trying creating an NLS_LANG environment variable with the same settings as above but that didn't make any difference.
My Windows code page is set to 1252. The database NLS_NCHAR_CHARACTERSET setting is AL16UTF16.
Any ideas of what to do next?
If NLS_CHARACTERSET is set to US7ASCII then you will never get any special characters from it. With NLS_NCHAR_CHARACTERSET=AL16UTF16 you should get any character, provided the data type of your column is NVARCHAR2 or NCHAR (not VARCHAR2)
Set your local NLS_LANG to ENGLISH_UNITED KINGDOM.AL32UTF8 then it should work.

Defining a Character Set for a column For oracle database tables

I am running following query in SQL*Plus
CREATE TABLE tbl_audit_trail (
id NUMBER(11) NOT NULL,
old_value varchar2(255) NOT NULL,
new_value varchar2(255) NOT NULL,
action varchar2(20) CHARACTER SET latin1 NOT NULL,
model varchar2(255) CHARACTER SET latin1 NOT NULL,
field varchar2(64) CHARACTER SET latin1 NOT NULL,
stamp timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
user_id NUMBER(11) NOT NULL,
model_id varchar2(65) CHARACTER SET latin1 NOT NULL,
PRIMARY KEY (id),
KEY idx_action (action)
);
I am getting following error:
action varchar2(20) CHARACTER SET latin1 NOT NULL,
*
ERROR at line 5:
ORA-00907: missing right parenthesis
Can you suggest what am I missing?
The simple answer is that, unlike MySQL, character sets can't be defined at column (or table) level. Latin1 is not a valid Oracle character set either.
Character sets are consistent across the database and will have been specified when you created the database. You can find your character by querying NLS_DATABASE_PARAMETERS,
select value
from nls_database_parameters
where parameter = 'NLS_CHARACTERSET'
The full list of possible character sets is available for 11g r2 and for 9i or you can query V$NLS_VALID_VALUES.
It is possible to use the ALTER SESSION statement to set the NLS_LANGUAGE or the NLS_TERRITORY, but unfortunately you can't do this for the character set. I believe this is because altering the language changes how Oracle would display the stored data whereas changing the character set would change how Oracle stores the data.
When displaying the data, you can of course specify the required character set in whichever client you're using.
Character set migration is not a trivial task and should not be done lightly.
On a slight side note why are you trying to use Latin 1? It would be more normal to set up a new database in something like UTF-8 (otherwise known as AL32UTF8 - don't use UTF8) or UTF-16 so that you can store multi-byte data effectively. Even if you don't need it now it's wise to attempt - no guarantees in life - to future proof your database with no need to migrate in the future.
If you're looking to specify differing character sets for different columns in a database then the better option would be to determine if this requirement is really necessary and to try to remove it. If it is definitely necessary1 then your best bet might be to use a character set that is a superset of all potential character sets. Then, have some sort of check constraint that limits the column to specific hex values. I would not recommend doing this at all, the potential for mistakes to creep in is massive and it's extremely complex. Furthermore, different character sets render different hex values differently. This, in turn, means that you need to enforce that a column is rendered in a specific character, which is impossible as it falls outside the scope of the database.
1. I'd be interested to know the situation
According to provided DDL statement it's some need to use 2 character sets. The implementation of this functionality in Oracle is different from MySQL and done with n* data types like nvarchar2, nchar... Latin1 is similar to some Western European character set that might be default. So you able to define for example "Latin1" (WE**) and some Unicode (UTF8..).
The NVARCHAR2 datatype was introduced by Oracle for databases that want to use Unicode for some columns while keeping another character set for the rest of the database (which uses VARCHAR2). The NVARCHAR2 is a Unicode-only datatype.
The reason you want to use NVARCHAR2 might be that your DB uses a non-Unicode character and you still want to be able to store Unicode data for some columns.
Columns in your example would be able to store the same data, however the byte storage will be different.

Resources