I found out (though my other question) that my IME outputs Hangul Compatibility Jamo (U+3130 – U+318F) instead of regular Hangul Jamo(U+1100 – U+11FF).
So I tried asking a question in superuser about other IMEs, no replies yet.
Should I just convert it myself? What exactly does that entail? Is it too complicated? Any ideas on how to? Any help would be appreciated.
Language: Delphi
OS: WinXP
IME: Korean Input System (IME 2002)
There is no reason you could not write an interesting experimental editor control with its own built in Unicode Compose feature. However, before you did that, you might look for a way to change the configuration of the IME. This seems to be a really interesting corner-case you have to work with. I was already surprised about your other question - that Windows has the ability to handle Raw Input from keyboards.
I found that source code for something that says it is the Korean IME is available for Windows CE. You might learn something by studying it, even though it is for Windows CE rather than XP.
http://msdn.microsoft.com/en-us/library/ee491900.aspx
Related
I want to test my software works on Windows regardless of language. Is there a difference between a localised install Windows 7 and an install with the language changed by a language pack? Is it enough just change language packs to confirm my software works or do I need to install a localised version? In particular I want to be sure that the code page used by the API changes.
It's equivalent, you are perfectly safe using a language pack. I'm a bit concerned about your comment regarding code pages though--Windows 7 natively runs in Unicode (Windows 95/98/ME did not and relied on code pages, but this has changed from Windows XP) and it seems unthinkable nowadays not to develop in Unicode, or are you doing something very specific that requires you to use non-Unicode encodings?
If that clarifies things, on Windows XP and beyond, all the ANSI versions of the APIs (ending with a A) simply wrap the Unicode APIs (ending with a W), doing conversion to Unicode and back to ANSI on the way out.
A customer asked for a text conversion feature from traditional Chinese to simplified Chinese.
I did a little research and apparently it's the kind of thing that can be automated quite readily and Mac OS X even has a system wide service for doing this.. there seems to be something built into OS X to perform those changes, but I'm no expert at internationalization..
Does anybody know how to perform this miracle?
Best regards,
Frank
I think the functionality you requested exists inside an app that Apple shipped with OS X--I don't know of any built-in functionality.
If you are trying to implement conversion, it looks like wikimedia has some code that might do what you want, but you'll have to convert it from PHP:
http://svn.wikimedia.org/doc/classZhConverter.html
Also, it may not be just a matter of converting the characters, although that's most of it... there are also phrases that are different in mainland Chinese (simplified characters) and other places (traditional characters).
i'm running Boa Constructor, an old wxpython gui builder.
today, the program decided to play tricks and switched all its GUI from right to left.
far worse, when i'm editing text inside the program, each letter is reveresed.
What part of windows does font mirroring?
How to disable letter mirroring?
why a program i used for several years suddenly thinks i need spanish internationalization? nothing in my computer is set for spanish.
how can I program this behavior myself (i don't really care, just curious. any programming language solution is ok here)
i'm asking several questions together because it seems somehow related.
my settings:
windows xp, (English interface setup!)
python 2.5.4
wxpython 2.8.11
All other Windows programs are left to right,
regional settings are for hebrew, israel
I don't know what you did, but you might want to check out the following instructions for enabling RTL: http://www.tavultesoft.com/kb/?id=41 and then see if you can reverse it.
I know wxPython can do internationalization. See http://wiki.wxpython.org/Internationalization and http://wiki.wxpython.org/RecipesI18n for information on that. Basically it involves .po and .mo files, which I doubt you can accidentally create. That's a weird issue.
As far as I know, Unix-like systems use UTF-8 for encoding filenames, while Windows system use their own Windows single-byte encodings.
I am working with archives with japanese filenames in them quite oftenly. When I open such archive created in Windows, japanese letters are dead, because filename encoding is incorrect.
Same thing happens, when I create archive in my Linux and then someone opens it under Windows.
So, I have thought that this should be quite common problem, and, because filenames are recoverable there must already exist correcting .sh script for linux and .bat script for Windows.
But after googling for quite a long time I still have not found anything.
Is there such scripts at all? If not, what difficultuies may have stopped people from creating them?
Update
I would be happy with a solution that works for most Linux systems and most Windows systems.
Windows uses the two byte encoding UTF-16. Your problem is most likely that you are using single byte ANSI versions of whatever archive tool you are using.
Until you give more details of the code and tools you are using it's hard to give specific advice. However, the are no limitations on using the full range of Unicode characters in modern Windows file systems.
Thank you for your input. Case indeed looks quite complex for simple bash script, I'll need to use programming language.
I don't see anything like "close question" button, so I'll use this answer to do so.
Take a look to convmv tool available for Unix systems
I would like all my toolkit to use UTF-8 but find that some tools on Windows seem to use CP1252 (which appears to be Windows-specific). Does this create output which is incompatible and if so at which codepoints? If so, can I do anything about it?
(I don't completely understand the issues so I'd be grateful for basic education on these encodings).
Tools hard-coding for code page 1252 on Windows is very unlikely. Much more likely is that it happens to be the default code page on your machine. 1252 is used in Western Europe and the Americas. It is configured in Control Panel, Regional and Language options. They've been using different names for it, on Win7 it is in the Administrative tab, Change System Locale.
Yes, many tools use the default code page unless they have a good reason to chose another encoding. The BOM is such a good reason. Notable examples are Notepad (unless you change the Encoding in the File + Open dialog to something else than Ansi) and C/C++ compilers. There typically isn't anything special you need to do to use the default code page. Guessing the correct code page for a text file when you don't have a BOM is impossible to do accurately. Google "bush hid the facts" for a very amusing war story.
Six years old and still relevant: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
Now, about your question: Yes, there are still tools out there that choke on UTF-8 files. But more and more tools are "getting it". If you're developing your own stuff, you might want to look into Python 3 where all strings are Unicode. The philosophy is to convert all your inputs into Unicode (if necessary) as early as possible, and reconvert them to a target encoding as late as possible. There are toolkits out there that will do a good job of guessing the encoding of a particular file (for example, Mark Pilgrim's chardet, a port of Mozilla's encoding detector). This is nice if you're working with files that don't specify an encoding.
CP1252 and UTF-8 are the same for all characters < 128. They differ above that. So if you stick to English and stay away from diacritical marks these will be the same.
Most of the Windows tools will use whatever is set as the current user's current codepage, which will default to 1252 for US Windows. You can change that to another codepage pretty easily. But UTF-8 is NOT one of the available codepage options for Windows. (I wish it was).
Some utilities under Windows will understand the UTF-8 byte-order mark at the start of a file. Unfortunately I don't know how to determine if this will work except to try it.
UTF-8 is supported on Windows but not as a current codepage. You can use UTF-8 for converting to/from it but you cannot set is as current codepage.
First do not try to waste time by setting the codepage - this approach will remind you of Sisyphus myth - you can't really solve the problem using codepages, you have to use Unicode.
The only real solution for you is to build your application as Unicode so it will use UTF-16 and to convert to/from UTF-8 on in/out operations. This is done quite simple because fopen supports reading or writing UTF-8.
Regarding the usage of other Windows tools with UTF-8 file, you should not be aware because if the tool is able to work with ASCII it will work with UTF-8 (even so it may not be able to distinguish between Unicode chars but at least it will be able to load/parse the files).
BTW, You forgot to specify what programming language are you using and what Windows tools are you considering for usage.
Also, if you ware interested about more internationalization stuff please visit my blog.i18n.ro