I am executing in shell script a simple command to extract the content of query in a file.
sqlplus -s /#P2AX << EXIT > /temporary/test.csv
alter session set NLS_DATE_FORMAT = 'yyyy-mm-dd HH24:mi:ss';
select *
from japhonie.ad_contacts
where con_id = 3720;
EXIT
The problem is that my base contains japanese characters. The current encoding in the db is SJIS (classic encoding for japanese language). So my output file /temporary/test.csv is automatically in SJIS too. But I would like to have it in UTF-8. Is there any command I could add to my shell script to extract directly the content of my query in UTF-8? or do I have to do it after extraction?
If you have extracted the file without mojibake, I think it is better to convert /temporary/test.csv to UTF-8.
In Linux environment, the command below will help you.
iconv -f Shift-JIS -t UTF8 /temporary/test.csv > /temporary/test.csv.utf8
Related
In Lua 5.4, I tried to print sone strings in Latin1 encoding with io.write(), but some characters (à,é...) are not well printed,
How could I perform this ?
Here is a screenshot of failed print with win-125x.lua
I guess you are running Lua on Windows.
Because you are converting Latin1 characters to UTF8, you should set the Windows console codepage to UTF8 before running your Lua script, with the following command :
chcp 65001
An other option is to save your script with UTF8 encoding without the need to convert strings from cp1252 to UTF8 and use the chcp command before running your script.
Remember that standard Lua has no concept of string encoding and that Windows support for UTF8 characters in the console is incomplete. Hence this kind of problems.
Check that related question too : Problem with accents while copying files in LUA
If you have the table utf8 you can do...
> do io.write(utf8.char(8364):rep(3)..'\n'):flush() end
€€€
To get the code you can do...
> do io.write(utf8.codepoint('€')..'\n'):flush() end
8364
But i am not sure if that works on windows.
...i am on linux.
I have the following problem on WINDOWS (Italian):
my NLS_LANG parameter is: ITALIAN_ITALY.UTF8
i want to execute the following query:
INSERT INTO SCHEMA.MY_TABLE("NAME") VALUES('ò');
Doing it by using command line (pure sqlplus) stores invalid data inside DB.
Doing it by using SQLDEVELOPER stores correct data.
I cannot find any way to set this stuff correctly, what should I do? Using SQLPLUS from command line is required.
Any help is appreciated.
When you use sqlplus then it inherits the character set from command line window. You can interrogate and modify character set (aka encoding) with chcp, I assume it is CP850 - which is not UTF8.
Run chcp 65001 before you start sqlplus, then it should work. See also Converting German special characters to English equivalent one in Oracle SQL / PL-SQL or to read more details OdbcConnection returning Chinese Characters as "?"
I have created simple table which has only one field TCHAR
SQL> desc test;
Name Null? Type
----------------------------------------- -------- --------------
TCHAR CHAR(20)
I have created one .sql file and run the sql script from sqlplus using
#filepath..\germantest.sql then output was getting replaced with some junk words
Output:
SQL> select * from test;
TCHAR
--------------------
Ä
Wir bestß
Nauch à â Ý
But when I try to insert same data run directly on sqlplus instead of file then
Input
insert into test values('Ärger Ökonom');
insert into test values('Ä');
insert into test values('Wir bestß');
insert into test values('Nauch à â Ý');
Output
SQL> select * from test;
TCHAR
--------------------
Ärger Ökonom
Ä
Wir bestß
Nauch à â Y
The output shows here is correct.
What I tried till now.
I checked file format and it must be utf-8
I checked NLS_charset and it is AL32UTF8.
What did I expect?
When I run same file from sqldeveloper, then I am getting correct output.
I want to run same script from .sql file and want to retain german characters in database as it is in original file.
Your codepage in the Windows console must match the NLS_LANG setting for SQL*Plus.
In a nutshell you need:
c:\> chcp 65001
c:\> SET NLS_LANG=GERMAN_GERMANY.AL32UTF8
You can easily verify if your code page is correct if you run type germantest.sql in the console. If that doesn't display the file contents correctly you have the wrong codepage:
Now the codepage has to match the NLS_LANG setting:
The correct value for NLS_LANG is also important when running a SQL script because it determines the encoding in which the file is read by SQL*Plus.
Of course cmd.exe needs to be configured to use a font that can actually display those characters.
I have created a script (.sh file) to convert a CSV file from ANSI encoding to UTF-8.
The command I used is:
iconv -f "windows-1252" -t "UTF-8" $csvname -o $newcsvname
I got this from another Stack Overflow post.
but the iconv command doesn't seem to be working.
Snapshot of input file contents in Notepad++
Snapshot of firstcsv file below
Snapshot of second csv file below,
EDIT: I tried reducing the problematic input CSV file contents to a few lines (similar to the first file), and now it gets converted fine. Is there something wrong with the file contents itself then? How do I check that?
You can use python chardet Character Encoding Detector to ensure existing character encoding format.
iconv -f {character encoding} -t utf-8 {FileName} > {Output FileName}
This should work. Also check if any junk characters are exist in file or not, that may create error in conversion.
I got this SQL statement that I wish to run on SQLite:
INSERT INTO tEntity (name) VALUES ('Roger Café');
Note the é character. Using the SQLite browser, I can insert this statement with the proper encoding.
However, if I save the above statement as a file (my.sql) and then run it on the Windows command line, I am having an encoding problem. The é in Café is garbled up.
C:\somewhere> sqlite3.exe my.db
sqlite> .read my.sql
I'm using Notepad++ to create the file in ANSI encoding. I have tried to use UTF-8 encoding but sqlite3.exe gives me a syntax error while reading the SQL file.
Is there any solution to fix this?
The encoding UTF-8 in notepad++ has BOM, which sqlite3.exe does not know about. Try using UTF-8 without BOM.