Special characters (german umlauts) are displayed as ASCII - oracle

In our database german umlauts are displayed as ASCII.
Here some examples
Ü is displayed as "š"
ä is displayed as "„"
Ä is displayed as "Ž"
ö is displayed as "”"
Ö is displayed as "™"
Here some Oracle enviroment variables:
SELECT distinct client_charset FROM v$session_connect_info
WHERE sid = sys_context('USERENV','SID');
retuns Unknown
select * from v$NLS_PARAMETERS;
NLS_LANGUAGE GERMAN
NLS_CHARACTERSET WE8MSWIN1252
I have also noticed, that there is no NLS_LANG setted up.
Can someone give me any clue how to check if this is a problem of env variable or character-encoding itself? Or how to fix it so that the umlauts would be displayed correctly?
Thanks in advance!

The first sentence is already wrong, it must be "my client application displays german umlauts as ..." Characters like Ž are not ASCII either. The Oracle environment variables have nothing to do how your client displays any characters.
There could be two reasons:
Characters like š „ Ž ” ™ are really stored in your database because they are inserted wrong, and you just think it should be Ü ä Ä ö Ö
Your client uses the wrong character set.
NLS_LANG is used to tell the database "my client uses character set XYZ" - no more, no less! Well, some clients inherit the NLS_LANG value for own character set setting but that is not always the case.
Actual answer:
You have wrong data in your database! When you inserted the data then your client was using CP437 (Oracle Character Set US8PC437) but you did not tell this the database.
Most likely data was inserted by sqlplus which inherits the codepage from cmd.exe which is by default 437 on English Windows machines. NLS_LANG character set was set to WE8MSWIN1252. When character set of client and database is the same then data is transferred byte-by-byte without any conversion - there is the mismatch!
Solutions:
Before you run your sqlplus script Set NLS_LANG accordingly, e.g.
SET NLS_LANG=.US8PC437
sqlplus ...
Language and Territory are optional and thus can be skipped.
Your database uses character set WE8MSWIN1252, then it would be beneficial to use it also by your client. By this you can insert and display any character which is supported by the database (no less but also no more)
chcp 1252
SET NLS_LANG=.WE8MSWIN1252
sqlplus ...
Correction of wrong data:
In order to correct the wrong data try this:
UPDATE table_name SET column_name = CONVERT(column_name, 'WE8MSWIN1252', 'US8PC437');
Note, this should be a one-time correction only. Do not put such update as part of your application code!
Update
For correction of PL/SQL code, try this one (not tested):
Create a SQL script like this:
SET HEADING OFF
SET FEEDBACK OFF
SET NEWPAGE NONE
SET VERIFY OFF
SET TRIMSPOOL ON
SET DEFINE OFF
SET ECHO OFF
SET TERMOUT OFF
SET PAGESIZE 0
SET LINESIZE 32767
SET LONG 2000000000
SET LONGCHUNKSIZE 32767
spool PACKAGE_NAME.sql;
SELECT DBMS_METADATA.GET_DDL('PACKAGE', 'PACKAGE_NAME', USER) FROM DUAL t;
spool OFF;
EXIT
And the run it like this
c:\> chcp 437
c:\> SET NLS_LANG=.WE8MSWIN1252
c:\> sqlplus -s ... #<above script>
c:\> SET NLS_LANG=.US8PC437
c:\> sqlplus ... #PACKAGE_NAME.sql

Related

Export SQLPluse Query from Oracle to CSV with UTF-8 encoding

I am using the below SQLPluse Script to export data out of Oracle into CSV files. The issue is I need the files to be in UTF-8 and its currently ANSI. How do i go about getting the files to be UTF-8?
SET colsep ';'
SET TERMOUT OFF
SET NEWPAGE 0
SET SPACE 0
SET LINESIZE 800
SET PAGESIZE 0
SET ECHO OFF
SET FEEDBACK OFF
SET HEADING OFF
SET NUMW 50
column timecol new_value timestamp
column spool_extension new_value suffix
SELECT '.csv' spool_extension
FROM dual;
/
SPOOL C:\job_files\production\Data_Upload\TEST_1&&suffix
##"C:\job_files\production\Data_Upload\TEST_1";
/
Exit;
Not entirely clear from your question what you mean, but I assume the issue is the data your query returns contains UTF8 data but your output does not show the UTF8 characters correctly.
The NLS_LANG environment variable controls how Oracle tools/programs including SQL*Plus handle such data.
You would normally set the variable to something that matches the database, e.g
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
Then run SQL*Plus.
You can query NLS_DATABASE_PARAMETERS to see what language settings are in the database and experiment with the NLS_LANG variable as above.
NLS_LANG value format is LANGUAGE_COUNTRY.CHARSET.

ORACLE - NLS_LANG, CHARSET, SPECIAL CHARS

I have the following problem on WINDOWS (Italian):
my NLS_LANG parameter is: ITALIAN_ITALY.UTF8
i want to execute the following query:
INSERT INTO SCHEMA.MY_TABLE("NAME") VALUES('ò');
Doing it by using command line (pure sqlplus) stores invalid data inside DB.
Doing it by using SQLDEVELOPER stores correct data.
I cannot find any way to set this stuff correctly, what should I do? Using SQLPLUS from command line is required.
Any help is appreciated.
When you use sqlplus then it inherits the character set from command line window. You can interrogate and modify character set (aka encoding) with chcp, I assume it is CP850 - which is not UTF8.
Run chcp 65001 before you start sqlplus, then it should work. See also Converting German special characters to English equivalent one in Oracle SQL / PL-SQL or to read more details OdbcConnection returning Chinese Characters as "?"

Oracle german character are converted to junk character?

I have created simple table which has only one field TCHAR
SQL> desc test;
Name Null? Type
----------------------------------------- -------- --------------
TCHAR CHAR(20)
I have created one .sql file and run the sql script from sqlplus using
#filepath..\germantest.sql then output was getting replaced with some junk words
Output:
SQL> select * from test;
TCHAR
--------------------
Ä
Wir bestß
Nauch à â Ý
But when I try to insert same data run directly on sqlplus instead of file then
Input
insert into test values('Ärger Ökonom');
insert into test values('Ä');
insert into test values('Wir bestß');
insert into test values('Nauch à â Ý');
Output
SQL> select * from test;
TCHAR
--------------------
Ärger Ökonom
Ä
Wir bestß
Nauch à â Y
The output shows here is correct.
What I tried till now.
I checked file format and it must be utf-8
I checked NLS_charset and it is AL32UTF8.
What did I expect?
When I run same file from sqldeveloper, then I am getting correct output.
I want to run same script from .sql file and want to retain german characters in database as it is in original file.
Your codepage in the Windows console must match the NLS_LANG setting for SQL*Plus.
In a nutshell you need:
c:\> chcp 65001
c:\> SET NLS_LANG=GERMAN_GERMANY.AL32UTF8
You can easily verify if your code page is correct if you run type germantest.sql in the console. If that doesn't display the file contents correctly you have the wrong codepage:
Now the codepage has to match the NLS_LANG setting:
The correct value for NLS_LANG is also important when running a SQL script because it determines the encoding in which the file is read by SQL*Plus.
Of course cmd.exe needs to be configured to use a font that can actually display those characters.

Invalid Character in a SQL request with Vim

I'm facing a weird issue since yesterday. I'm coding a PHP webservice with request a Oracle DB.
I write a request in vim, say:
SELECT
RBK_ISBREF AS ISBREF,
RBK_KEY AS BL_KEY,
RBK_USERID AS USER_ID,
FROM RULBLACKLIST
WHERE
RBK_KEY = 'identiteClient'
AND RBK_VALUE = 'foo'
It returns ORA-00911 Invalid Character.
The weird part is, my request is OK in SQL developper (and returns values) and if i change my IDE from Vim to whatever it returns the correct values…
Is is a format/encoding missconfiguration in Vim ?
In my .vimrc i got these:
set fileformat=unix
set fileformats=unix,dos
" Indentation
"-----------------------
"-----------------------
set autoindent
set smartindent
set backspace=indent,eol,start
" Convert tabs to spaces
set tabstop =4
set shiftwidth =4
set softtabstop =4
set expandtab
Any tips?
Check the file's encoding; Vim might write in a format that the Oracle tool doesn't understand.
:setlocal fileencoding? bomb? fileformat?
Especially a byte order mark might be the culprit. Alternatively, you can also do a binary compare of otherwise identical good and bad files.

How to store ® in oracle table?

I need to insert value into an Oracle table
'RR® 5'?
I created a table column varchar2 data type.
But when I am displaying the data it is showing:
'RR?? 5'
Any idea, please.
First of all you have to check if your database runs on character encoding which supports your special characters.
Execute this query to check.
SELECT *
FROM V$NLS_PARAMETERS
WHERE PARAMETER IN ('NLS_CHARACTERSET', 'NLS_NCHAR_CHARACTERSET');
Character sets like AL32UTF8, UTF8, WE8ISO8859P1 are able to store special characters ®.
In order to check whether your characters are supported you can run this command
SELECT
ASCIISTR(CAST(UNISTR('\00C2') AS VARCHAR2(4))) AS VARCHAR_SUPPORT,
ASCIISTR(CAST(UNISTR('\00C2') AS NVARCHAR2(4))) AS NVARCHAR_SUPPORT
FROM dual;
VARCHAR_SUPPORT NVARCHAR_SUPPORT
---------------- ------------------------
\00C2 \00C2
1 row selected.
If your result value (00C2) is equal to your input value then your characters are supported.
Suppose your database supports the characters (otherwise the game ends here)
Solution 1
Run this command from any SQL editor or from SQL*plus
INSERT INTO THE_TABLE VALUES (UNISTR('RR\00C2\00AE 5'));
Depending on your actual requirements this can be a very tedious work.
Solution 2
Use SQL*plus command. For this you have to set NLS_LANG environment variable according to code page of cmd
C:\>chcp
Active code page: 850
C:\>set NLS_LANG=.WE8PC850
C:\>sqlplus user/pwd#DB
SQL> INSERT INTO THE_TABLE VALUES ('®');
When you work on Linux/Unix the chcp equivalent is locale charmap or echo $LANG. You can also change current code page, e.g. chcp 65001 for UTF-8
Solution 3a
Create a SQL script containing the same INSERT command as above, and save it with local encoding (typically Windows CP1252, many times misnamed as ANSI)
c:\>set NLS_LANG=.WE8MSWIN1252
c:\>sqlplus user/pwd#DB #Special_Char_CP1252.sql
Solution 3b
Create a SQL script and save it with UTF-8 which is supported by most modern editors. Note: SQL scripts must not contain a BOM (Byte Order Mark), otherwise SQL*plus will raise an error. I found this tiny command line tool to add and remove BOM from arbitary text files: UTF BOM Utils
c:\>set NLS_LANG=.AL32UTF8
c:\>sqlplus user/pwd#DB #Special_Char_UTF8.sql
Note for solutions above: you can skip the set NLS_LANG command in case your registry is set accordingly. Check with this command
C:\>REM for x64 Oracle Client
c:\>reg query HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\ /s /v NLS_LANG
HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_OraClient11g_home1
NLS_LANG REG_SZ AMERICAN_SWITZERLAND.AL32UTF8
End of search: 1 match(es) found.
C:\>REM for x86 Oracle Client
c:\>reg query HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ORACLE\ /s /v NLS_LANG
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ORACLE\KEY_OraClient11g_home1
NLS_LANG REG_SZ AMERICAN_SWITZERLAND.AL32UTF8
End of search: 1 match(es) found.
C:\>
Solution 4
Run your INSERT statements from a more sophisticated editor like TOAD or SQL Developer. Ensure that NLS_LANG value in Registry, resp. your system environment variable matches the file save options of the editor.
Some external resources which may help you:
Oracle documentation NLS_LANG Settings in MS-DOS Mode and Batch Mode
List of Code pages: Code page
Find a Unicode Codepoint: Codepoints

Resources