wcslen() works differently in Xcode and VC++ - xcode

Question

I found that wcslen() in VC++2010 returns correct count of letters; meanwhile Xcode does not.
For example, the code below returns correct 11 in VC++ 2010, but returns incorrect 17 in Xcode 4.2.
const wchar_t *p = L"123abc가1나1다";
size_t plen = wcslen(p);
I guess Xcode app stores wchar_t string as UTF-8 in memory. This is another strange thing.
How can I get 11 just like VC++ in Xcode too?

Answer 1

I ran this program on a Mac Mini running MacOS X 10.7.2 (Xcode 4.2):
#include <stdio.h>
#include <wchar.h>
int main(void)
{
const wchar_t p[] = L"123abc가1나1다";
size_t plen = wcslen(p);
if (fwide(stdout, 1) <= 0)
{
fprintf(stderr, "Failed to make stdout wide-oriented\n");
return -1;
}
wprintf(L"String <<%ls>>\n", p);
putwc(L'\n', stdout);
wprintf(L"Length = %zu\n", plen);
for (size_t i = 0; i < sizeof(p)/sizeof(*p); i++)
wprintf(L"Character %zu = 0x%X\n", i, p[i]);
return 0;
}
When I do a hex dump of the source file, I see:
0x0000: 23 69 6E 63 6C 75 64 65 20 3C 73 74 64 69 6F 2E #include <stdio.
0x0010: 68 3E 0A 23 69 6E 63 6C 75 64 65 20 3C 77 63 68 h>.#include <wch
0x0020: 61 72 2E 68 3E 0A 0A 69 6E 74 20 6D 61 69 6E 28 ar.h>..int main(
0x0030: 76 6F 69 64 29 0A 7B 0A 20 20 20 20 63 6F 6E 73 void).{. cons
0x0040: 74 20 77 63 68 61 72 5F 74 20 70 5B 5D 20 3D 20 t wchar_t p[] =
0x0050: 4C 22 31 32 33 61 62 63 EA B0 80 31 EB 82 98 31 L"123abc...1...1
0x0060: EB 8B A4 22 3B 0A 20 20 20 20 73 69 7A 65 5F 74 ...";. size_t
0x0070: 20 70 6C 65 6E 20 3D 20 77 63 73 6C 65 6E 28 70 plen = wcslen(p
0x0080: 29 3B 0A 20 20 20 20 69 66 20 28 66 77 69 64 65 );. if (fwide
0x0090: 28 73 74 64 6F 75 74 2C 20 31 29 20 3C 3D 20 30 (stdout, 1) <= 0
0x00A0: 29 0A 20 20 20 20 7B 0A 20 20 20 20 20 20 20 20 ). {.
0x00B0: 66 70 72 69 6E 74 66 28 73 74 64 65 72 72 2C 20 fprintf(stderr,
0x00C0: 22 46 61 69 6C 65 64 20 74 6F 20 6D 61 6B 65 20 "Failed to make
0x00D0: 73 74 64 6F 75 74 20 77 69 64 65 2D 6F 72 69 65 stdout wide-orie
0x00E0: 6E 74 65 64 5C 6E 22 29 3B 0A 20 20 20 20 20 20 nted\n");.
0x00F0: 20 20 72 65 74 75 72 6E 20 2D 31 3B 0A 20 20 20 return -1;.
0x0100: 20 7D 0A 20 20 20 20 77 70 72 69 6E 74 66 28 4C }. wprintf(L
0x0110: 22 53 74 72 69 6E 67 20 3C 3C 25 6C 73 3E 3E 5C "String <<%ls>>\
0x0120: 6E 22 2C 20 70 29 3B 0A 20 20 20 20 70 75 74 77 n", p);. putw
0x0130: 63 28 4C 27 5C 6E 27 2C 20 73 74 64 6F 75 74 29 c(L'\n', stdout)
0x0140: 3B 0A 20 20 20 20 77 70 72 69 6E 74 66 28 4C 22 ;. wprintf(L"
0x0150: 4C 65 6E 67 74 68 20 3D 20 25 7A 75 5C 6E 22 2C Length = %zu\n",
0x0160: 20 70 6C 65 6E 29 3B 0A 20 20 20 20 66 6F 72 20 plen);. for
0x0170: 28 73 69 7A 65 5F 74 20 69 20 3D 20 30 3B 20 69 (size_t i = 0; i
0x0180: 20 3C 20 73 69 7A 65 6F 66 28 70 29 2F 73 69 7A < sizeof(p)/siz
0x0190: 65 6F 66 28 2A 70 29 3B 20 69 2B 2B 29 0A 20 20 eof(*p); i++).
0x01A0: 20 20 20 20 20 20 77 70 72 69 6E 74 66 28 4C 22 wprintf(L"
0x01B0: 43 68 61 72 61 63 74 65 72 20 25 7A 75 20 3D 20 Character %zu =
0x01C0: 30 78 25 58 5C 6E 22 2C 20 69 2C 20 70 5B 69 5D 0x%X\n", i, p[i]
0x01D0: 29 3B 0A 20 20 20 20 72 65 74 75 72 6E 20 30 3B );. return 0;
0x01E0: 0A 7D 0A .}.
0x01E3:
The output when compiled with GCC is:
String <<123abc
Length = 11
Character 0 = 0x31
Character 1 = 0x32
Character 2 = 0x33
Character 3 = 0x61
Character 4 = 0x62
Character 5 = 0x63
Character 6 = 0xAC00
Character 7 = 0x31
Character 8 = 0xB098
Character 9 = 0x31
Character 10 = 0xB2E4
Character 11 = 0x0
Note that the string is truncated at the zero byte - I think that is probably a bug in the system, but it seems a little unlikely that I'd manage to find one on my first attempt at using wprintf(), so it is more likely I'm doing something wrong.
You're right, in the multi-byte UTF-8 source code, the string occupies 17 bytes (8 one-byte basic Latin-1 characters, and 3 characters each encoded using 3 bytes). So, the raw strlen() on the source string would return 17 bytes.
GCC version is:
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Just for giggles, I tried clang, and I get a different result. Compiled using:
clang -o row row.c -Wall -std=c99
using:
Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
Target: x86_64-apple-darwin11.3.0
Thread model: posix
The output when compiled with clang is:
String <<123abc가1나1다>>
Length = 17
Character 0 = 0x31
Character 1 = 0x32
Character 2 = 0x33
Character 3 = 0x61
Character 4 = 0x62
Character 5 = 0x63
Character 6 = 0xEA
Character 7 = 0xB0
Character 8 = 0x80
Character 9 = 0x31
Character 10 = 0xEB
Character 11 = 0x82
Character 12 = 0x98
Character 13 = 0x31
Character 14 = 0xEB
Character 15 = 0x8B
Character 16 = 0xA4
Character 17 = 0x0
So, now the string appears correctly, but the length is given as 17 instead of 11. Superficially, you can take your choice of bugs - string looks OK (in a terminal - /Applications/Utilities/Terminal - acclimatized to UTF8) but length is wrong, or length is right but string does not appear correctly.
I note that sizeof(wchar_t) in both gcc and clang is 4.
The left hand does not understand what the right hand is doing. I think there's a case for claiming both are broken, in different ways.