I'm trying to import a product CSV which has Bulgarian product names/descriptions (using the standard import under import/export->import)
The only way I've been able to import any so far is by wrapping them in quotes or by putting roman characters in front of the Bulgarian.
e.g. 'Ламинирани ПДЧ' or xxx Ламинирани ПДЧ
without adding these characters it outputs the error: Required attribute 'name' has an empty value in rows: 1
It seems like the Bulgarian is being stripped out completely? My file is encoded as UTF-8 and I've also set the default charset as UTF-8 in the htaccess file.
Is it possible to import the Bulgarian without quotes/roman characters?
I haven't personally tried it, but I have seen this answer that seems to work.
Try saving your csv file through Libre Office Calc with UTF-8 and try import again.
Related
I'm using Laravel Excel for importing products from CSV files.
CSV settings:
ISO-8859-1 format
Comma as delimiter.
" as enclosure
A few of the products have a name ending with a backslash.
Ex.: Example product model DN123\
Looking at the raw CSV file in a text editor, it will look like this:
... "Example product model DN123\", "Shoes", "Men" ...
When importing it's perceived as the enclousure (") after the backslash is escaped. How can I prevent this? When I open the CSV in the mac-program Numbers everything is fine.
It's not well documented, but the escape_character can be changed in custom CSV settings. I changed it to " and it worked.
I'm trying to import a CSV file (UTF-8 encoding) in Ruby (2.0.0) in to my database (MSSQL 2008R2, COLLATION French_CI_AS), but the special characters (French accents on vowels) are not stored properly : éèçôü becomes éèçôü (or other similar jibberish).
I use this piece of code to read the file :
CSV.foreach(file, col_sep: ';', encoding: "utf-8") do |row|
# ...
end
I tried various encoding in the CSV options (utf-8, iso-8859-1, windows-1252), but none would store the special characters correctly.
Before you ask, my database collation supports those characters, since we have successfully imported data containing those using PHP importers. If I dump the data using puts or a file logger, everything is correct.
Is something wrong with my code, or do I need to specify something else (like the ruby class file encoding for example) ?
Thanks
EDIT : The data saving is done by a PHP REST API that works fine with accented characters. It stores data as it is received.
In Ruby, I parse my data, store it in an object and then send the JSON-encoded object in the body of my PUT request. But if I use an SQL query directly from Ruby, the problem remains :
query = <<-SQL
UPDATE MyTable SET MyTable_title = '#{row_data['title']}' WHERE MyTable_id = '#{row_data['id']}'
SQL
res = db.execute query
I was thinking that this had something to do with the encoding type on your CSV file, so started digging around on that. I did find that windows-1252 encoding will insert control characters.
You can read more about it here: Converting special charactes such as ü and à back to their original, latin alphbet counterparts in C#
I'm doing the following to create a label that I use as part of attribution for a photo:
CCLabelTTF *imageSourceLabel = [CCLabelTTF labelWithString:[_organism imageSource] fontName:[[UIFont systemFontOfSize:12] fontName] fontSize:12];
Several of the image sources include Turkish letters. For example, in this URL:
http://commons.wikimedia.org/wiki/File:Şahlûr-33.jpg
This displays improperly in my iPad app; the Turkish letters are missing.
How do I create a label that will work with text like that in the URL above?
Edit:
Nevermind... the problem is with exporting from Excel. See the comments on the answer below. This link provides some additional information: Excel to CSV with UTF8 encoding
Additional Edit:
Actually, it's still a problem, even after I export correctly and verify that I have the proper UTF-8 (or is it 16?) letters in the CSV file. For example, this string:
Dûrzan cîrano / CC BY-SA 3.0
Is displayed like this:
and this string:
Christian Mehlführer / CC-BY 2.5
is displayed like this:
It's definitely being processed improperly upon import, as CCLOG generates the following:
Photo Credit: Dûrzan cîrano / CC BY-SA 3.0
More Info:
Upon import, I'm storing the following value as a string in an array:
"Christian Mehlf\U00c3\U00bchrer / CC-BY 2.5"
Wikipedia says the UTF-8 value for ü, in hex, is C3 BC. It looks like the c3bc is in there, but masked as \U00c3\U00bc.
Is there any way to convert this properly? Or is something fundamentally broken at the CSV import level?
The solution is below.
There were several problems:
Excel on the Mac doesn't export UTF-8 properly. The solution I used was to paste the data into Google Spreadsheet and export from there. More info here: Excel to CSV with UTF8 encoding
I realized that once I had the proper data in the CSV file, that I was importing it with the improper settings. I'm using parseCSV and needed to set _encoding in the -init method to NSUTF8StringEncoding instead of the default, NSISOLatin1StringEncoding.
if you try this:
[CCLabelTTF labelWithString:[[_organism imageSource] stringByUnescapingHTML] fontName:[[UIFont systemFontOfSize:12] fontName] fontSize:12];
it will likely work better. I suspect your url string is escaped HTML.
I have a string contain some special char like "\u2012" i.e. FIGURE DASH. When i am trying to print this on console I am getting a '?' mark instead of its symbol. I have an editor where in I can insert the symbol using alt+numpad like alt+2012. In editor it I could see the symbol save it in a xml file and get the value using nodevalue, I get a '?' mark.
To summerize I am facing problem to read extended latin a charset. What i need is When i insert such symbols and read it, i should get something like &#xXXXX;.
Please help!
TIA :)
Simply I have a String inpath = "À";, I want to get its unicode value..like &#xXXXX;
The default console encoding in Windows is some MS-DOS code page and they don't support the character. You can try running chcp 65001 before running the program but you might also need to change the console font as well.
You don't need to do anything you wouldn't do with any other character, as long as you use UTF-8. You aren't doing that in many places. You need to explicitly write in your code to save and read the file in UTF-8, and not rely on the platform default encoding.
I am trying to import Japanese strings inside my "Basic MSI" project, it use to work before without any issues but now when I try to import some Japanese strings from a text file then it throws following error (I have changed some of the personal data from the error message.)
ISDEV : error -7185: The Japanese: 日本語 translation for string identifier IDS_XXXX_1111 includes characters that are not available on code page 932.
I think there are some of the characters inside the IDS_XXXX_1111 are not part of code page 932. How to detect those characters using some tool?
Also documentation mentions about changing some encoding settings to UTF-8 in InstallShield 2011, if you are aware then please guide me.
Thanks in advance
Rahul
My favorite way to detect such characters is with python. For example, reading a file like the InstallShield string tables in python 2.x:
import codecs
strings = codecs.open("strings.txt", "r", "UTF-16"):
for line in strings.readlines():
line = line.strip()
try:
line.encode("cp932")
except UnicodeError:
print "Can't encode: " + line.encode("cp932", "replace")
Your alternatives are to pinpoint the characters that cannot be represented on the relevant code page and replace them with ones that can, or to go to the Releases view and select yes for the Build UTF-8 Database setting.