GetCharABCWidthsFloat works for most of UNICODE, except CJKV characters - winapi

I am attempting to render a series of UNICODE characters onto a spritesheet. This all works quite well for most characters, including Cyrillic ones.
When using GetCharABCWidthsFloat with certain CJKV characters however, the ABCFLOAT::abcfB parameter provides a value lower than expected. It does not account for underhangs or overhangs, which is the exact purpose of the ABCs:
The B spacing is the width of the drawn portion of the character glyph.
Source: ABCFLOAT | Microsoft Docs
As you can see, all characters do not overlap left-to-right, except the last few characters:
I get around this by creating a customizable padding option, to handle such cases, but this bloats the rest of the glyphs and thus requires a larger surface:
Font being used is Arial. For the character 美, ABC returns (2, 10, 2), which sums to a advance of 14 pixels, when in fact, 17 pixels are needed.
I use TextOut to actually render the glyphs, but I do wonder if there is someone out there who's experienced this and came up with a universal solution.
Using functions like GetTextExtentPoint32W or DrawTextEx to get the rectangle does not allow precise per-character placement, which is the whole point of the ABC. And some unmentioned functions only work with TrueType fonts.
I question if certain characters shift to a different font under certain conditions, causing the results to be inaccurate. If that is the case, is there a way to determine if a character is not available for a font, knowing what Windows does automatically so I can reproduce the behaviour? That is, is there some sort of way to determine when a character should fall back on another font, and a way to determine what that font should be?
I have been on this problem for quite some time, so anyone with experience with these APIs would be greatly welcomed!

From the documentation on GetCharABCWidthsFloat:
The ABC widths of the default character are used for characters outside the range of the currently selected font.
Arial contains a lot of characters, including Cyrillic, but it does not contain CJKV ideographs. Other text-related calls may give you the false impression that it does have those characters (through a default/fallback font mechanism).
Before using (maybe before getting) the ABCFLOAT, you should first check that the characters you want metrics for are within the range of the currently selected font.

Related

NSTextView line height changes when using CJK characters

I use NSTextView to render NSAttributedStrings that may contain non-Latin characters, and it seems that lines containing any CJK character are always 6 pixels taller than lines without those. Even setting the NSParagraphStyle's minimumLineHeight property to a much higher value (e.g. 32 pixels, when using the standard system font size) retains this problem (Lines with CJK characters a rendered as 38 pixels).
Moreover, NSAttributedString's boundingRectWithSize seems to report the wrong ("correct") size (without the extra 6 pixels).
What am I missing?
Setting layoutManager.usesFontLeading to NO solved this problem.

MacOS CALayer Character Spacing

I have a problem which I hope you can help me solving.
I'm creating a program using Xamarin.Mac (C# for Mac) and I need to draw a DNA sequence (ATGC and so on). However, I need to know the exact position of each character so I can draw several other objects which should be aligned with the characters in DNA sequence.
Screenshot of the Windows version of my app which illustrates the behavior I'm looking for:
Currently I'm looking to use the CALayer drawing method, which appears to be fast enough to render 12 lines of 70 characters in less than 50 ms. CALayers are not fast enough to render 1000 CATextLayers with one (A/T/G/C) character each, so (I think) I need to render them as lines with specific spacing. This means that I need to have exactly 10 (example) pixels between the center of each character.
However, I cannot find a way to do this.
The NSAttributedString Kerning seems be added to an unknown existing tracking (or spacing) of the font, and thus may be used with monospace fonts but still results in an unknown actual spacing.
I CAN get around the issue by trial and error until the letter spacing appears to match the desired spacing, but I'm not very confident in robustness of this method across different devices (screen resolutions). This requires that I use a monospace font, which is okay, but not optimal.
Is it possible to have specific character spacing using a single CATextLayer and what are my options if not? is it possible to have 1000 characters drawn individually without a huge performance impact?
Thank you.

Font Width Ratios: GetTextExtent32 versus Word and PDF

GetTextExtent32 returns different character width ratios (e.g., width of '9' versus space) than Word or Acrobat use when displaying the same font (e.g., 10-point Arial).
This matters because I'm trying to prepare clipboard strings that will get pasted into apps that don't support much formatting (no tabs or tables), but I still need to align certain columns of info. I'm trying to overcome this challenge by dynamically calculating the number of spaces I need to insert (remember, no tabs allowed!).
For example, calling GetTextExtent32 with a selected font of Arial 10-point gives a logical unit width of 7 for the digit '9', and a logical unit width of 4 for a space. This ratio proves correct when using something like DrawText.
However, when I export strings to Word or Acrobat, it turns out that 2 spaces in this example font exactly equals the width of one 9 (whether looking at a single 9, or nine contiguous 9s). I don't know much about fonts, but it doesn't appear to be any kind of juxtaposition issue; GetCharABCWidths shows 0 for both the a and c widths.
Does anyone know why Word and Acrobat are not showing the same proportions/measurements for a given font as Windows itself? Is there are a way to calculate this?

In Win7 some fonts don't work like they did in Win2K/XP

My question is about how font handling needs to be changed in order to work correctly under Windows 7. I'm sure that I've made an assumption about something that was valid before, but is no longer valid. But I don't even know where to begin looking! I'm praying someone can help! Here are the details as I understand them (I've also posted this question on a Microsoft Windows Developers forum, but they're not answering):
Yes, I'm behind the times (heck, I still write WIN32 code in plain C!) I have a 10 yr old DLL I wrote that mimics an even older DOS screen I/O library within the client area of a window. Needless to say, it only allows the use of fixed-width fonts. When some of the programs using the DLL have been moved to Windows 7, there is a strange flickering that appears when a fixed-width TRUE TYPE font is used (bitmap fonts still work perfectly.) We've tracked the problem down to the fact that a single character written with ExtTextOut is wider than it should be. I've checked the measurements three different ways (by using GetTextExtentPoint32 on a 132 character string and dividing by 132, by calling GetTextMetrics and even by using GetCharABCWidths for all 256 characters) and they all agree that the font is the same width. But ExtTextOut is rendering the background rectangle one or two pixels wider than the font width. Either than, or it is beginning the background rendering a pixel or two to the left of the position given in the parameters [I call it like this: ExtTextOut( hdc, r.left, r.top, ETO_OPAQUE, &r, &ch, 1, NULL ).] And remember, this EXACT code worked perfectly under Windows 2000, Windows XP and, with bitmap fonts on Windows 7 -- but it no longer works correctly with fixed-width true type fonts under Windows 7.
For anyone who isn't grasping what I need to do: try to imagine writing one character per square on a piece of graph paper. Every square uses the same font, but may have a different foreground and/or background color. I use TA_TOP|TA_LEFT text alignment, because it is the simplest and any consistently applied alignment should work for a fixed-width font.
What I'm seeing is that ExtTextOut is emitting a larger background rectangle than I've specified in the RECT * parameter. Since the rectangle I'm providing is created from the reported size of the font, this should NEVER happen -- and it never happened on Windows XP and earlier, and doesn't happen with bitmap (i.e. .FON) fonts under Windows 7, either. But it ALWAYS happens with fixed-width TrueType fonts under Windows 7. This is with the EXACT SAME EXECUTABLE running on Windows 2000, Windows XP and Windows 7 (32 & 64.) While I would love to simply say Windows 7 has a bug, I'm more inclined to believe that some fundamental assumption that I've made about font handling under Windows is no longer true (after 20 years of writing software for Windows.)
But I have no idea how or where to discover what that might be! Please, PLEASE help me!
--- ammendment ---
For anyone interested, I've managed to work around what I am considering a bug -- until I find documentation to the contrary. My workaround consists of two changes to my library:
Use the size returned from GetTextExtentPoint32() of an 'X' instead
of data from TEXTMETRICS.
Include the ETO_CLIPPING flag in all ExtTextOut() calls.
Previously, I was using tmHeight+tmExternalLeading for the number of pixels between the tops of consecutive rows of text, as is documented. I discovered that the size.cy value coming back from the GetTextExtentPoint32() wasn't the same and seemed more accurate. The worst example I found was the OCRB true type font. Here's what I saw in the debugger for the OCRB font I'd created (using the system font selection dialog):
ocrbtm.tmHeight = 11
ocrbtm.tmExternalLeading = 7
ocrbsize.cy = 11
So, for some reason that I've yet to discover, Windows is ignoring the external leading value defined for the OCRB font. Using the size value instead of the TM results in nice, neat, close packed text, which is just what I wanted.
The ETO_CLIPPING flag should not be necessary for me because I am setting the rectangle to exactly the dimensions of a single character and using ETO_OPAQUE to fill in the background (and overwrite the previous cell content.) But without the clipping flag, a single character is wider than either the size, text metric, or ABC width would indicate -- at least, that is true based on all of the documentation I've found so far.
I believe that HEIGHT issue has existed for a long time, but the rest was unnecessary until we ran our software under Windows 7. I'm appending this to my question to see if anyone can explain what I obviously don't understand.
-- ammendment 2 --
1: All documentation I can find says that tmHeight+tmExternalLeading should produce single spaced lines of text. Period. But that is not always true and I cannot find documentation indicating how Windows determines the different values that are sometimes returned by GetTextExtentPoint32().
2: under Win7 (maybe Vista) ExtTextOut started filling in a little more background than it should (by adding a couple extra pixels to the right), but only when a true type font is selected. It does this even if the rectangle is double the expected size of the character (in BOTH dimensions.) DPI/Scaling might be a factor, but since my system is set to 100%, it would seem that Windows is having trouble with a 1:1 scaling factor and that would seem to be a bug. The fact that it only affects true type and not bitmap (.FON) fonts also seems to rule out scaling (unless there is a bug in the scaling system), since Windows should attempt to scale all of the text, not just some of it. Also, there's a greyed (but checked) setting "Use Windows XP style DPI scaling" in the "Custom DPI Setting" dialog. Lastly, this entire issue may be a result of my running under the Windows Classic theme instead of one of the Aero or other Win7 native themes.
-- ammendment 3 --
Simply calling SetProcessDPIAware() has no effect on the issue I'm having. Since my problem exists at the 100% DPI setting (scale 1:1), if my problem is DPI-related, then I must have discovered a bug in the DPI virtualization because this is how Microsoft describes the feature:
This feature works by providing "virtualized" system metrics and UI elements to the application, as if it were running at 96 DPI. The application then renders to a 96-DPI off-screen surface, and the Desktop Windows Manager scales the resulting application window to match the DPI setting.
All of my settings show that I'm at 100% scaling, and looking in the custom settings box clearly shows that means 96 DPI. So, if the DPI virtualization from 96 DPI to 96 DPI is not working for my fixed-width true type fonts, then Windows has a problem, right? Or is there some function I need to call (or stop calling?) in order allow the DPI virtualizer to work correctly?
I'm still not convinced that the supposed scaling issue actually has as much to do with the font SIZE as I originally thought. That's because the problem is manifesting in the background rectangle being filled by ExtTextOut() instead of the text character being emitted. The background rectangle gets enlarged a bit when the font is true type. I've also now verified that this problem occurs whether using the Windows Classic theme or the standard Windows Aero theme. Now to build a simplified example so others can experiment with it.
-- ammendment 4 --
I've created a minimal demo program that shows what I'm seeing (and what I'm doing.) The Visual Studio 2010 project/source may be downloaded from http://www.svalli.com/files/fwtt.7z -- I intentionally didn't include executables because I don't want to risk spreading malware. The program has you choose a fixed-width font and then writes two 5x5 character grids to the client area, one created using the GetTextExtentPoint32 size and one using the TEXTMETRIC size as documented by Microsoft. The grids are in a black&white checkerboard pattern with a yellow on red character written last into the center to show the overlap effect (you may need a zoom utility to see it clearly.) The program also draws a string that starts with 5 X's just below the grid, starting at the same left offset, to be used as a comparison for my method of placing individual characters (I match the string.) The menu allows toggling clipping on/off in ExtTextOut and selection of other fonts. There is also a command line option dpiaware (case-sensitive) that causes the program to call SetProcessDPIAware() when it starts up, so that the effect of that call may also be evaluated.
From creating this I've learned that ExtTextOut is filling the correct background rectangle, but the character being rendered with an opaque background may be wider than it should be and may not even begin where ExtTextOut was told to begin drawing! I said "should be" because the character spacing I'm ending up with matches what I get when I have ExtTextOut render a whole string. The overlap may apparently be on either or both sides of the given rectangle, for example, OCRB adds an extra pixel to both the left and right sides of the character cell while the other true type fonts I've checked add two pixels to the right edge.
I really want to do this the "right" way, but I cannot find any documentation that shows what I'm doing wrong or am missing. Well, I am probably missing something for DPI Aware at scales other than 100%, but otherwise, I'm just baffled.
-- ammendment 5 --
Slightly less baffled... the problem is caused by ClearType. Turning off ClearType made all of the fonts work again. Turning ON ClearType under XP causes the same problem. Apparently ClearType can silently (until someone tells me how to detect it) stretch characters horizontally by a couple pixels in order to make space for the shaded pixels it adds to smooth things out.
Is clipping the only way around this problem?
-- ammendment 6 --
Partial answer to my clipping question above: When creating a new font I now do the following (in pseudo code):
CreateFontIndirect
SelectFont
GetTextMetrics
if( (tmPitchAndFamily & TMPF_TRUETYPE) && Win6.x or above )
if( SystemParametersInfo( SPI_GETCLEARTYPE ) )
lfQuality = NONANTIALIASED_QUALITY
DeleteObject( font )
CreateFontIndirect
Without enabling clipping this almost always works with the font sizes I'm using, though I've found a few that still render an extra pixel to the right (or left) of the character cell. Luckily, those appear to be free fonts found on the internet, so their overall quality might be below the standards of professional font foundries.
If anyone can find a better answer, I'd really, REALLY love to hear it! Until then, I think this is as good as it will get. Thanks for reading this far!
Make sure your code is high DPI aware, and then tell the OS that your process is DPI aware.
If you don't tell the OS that you're DPI aware, some of the measurement functions will lie and give you numbers based on the assumption that the display DPI is actually 96 dpi regardless of what it really is. Meanwhile, the drawing functions will try to scale in the other direction. For simple high-level drawing, this approach generally works (though it often leads to fuzzy text). For small measurements and precise placement of individual characters, this often results in round off problems that lead to things like inconsistent font sizes. This behavior was introduced in Windows Vista.
You can see it all the time in Visual Studio 2010+ as the syntax highlighter colors the text and words shift by a couple pixels here and there as you type. Really frickin' annoying.
Regarding the amendment:
tmExternalLeading is simply a recommendation from the font designer as to how much extra space to put between lines of text. MSDN documentation typically says, "the amount of extra leading (space) that the application adds between rows." Well, you're the application, so the "Right Thing To Do" is to add it between rows when you're drawing text yourself, but it really is up to you. (I suspect higher level functions like DrawText will use it.
It is perfectly correct for GetTextExtentPoint32 (and friends) to return a size.cy equal to tmHeight and to ignore tmExternalLeading. As the programmer, it's ultimately your choice how much leading to actually use.
You can see that this with some simply drawing code. Select a font with a non-zero tmExternalLeading (Arial works for me). Draw some text using TextOut and a unique background color. Then measure the text with GetTextExtentPoint32 and draw some lines based on the values you get back. You'll see that the background color rectangle excludes the external leading. External leading is just that: external. It's not in the bounds of the character cell.
// Draw the sample text with an opaque background.
assert(::GetMapMode(ps.hdc) == MM_TEXT);
assert(::GetBkMode(ps.hdc) == OPAQUE);
assert(::GetTextAlign(ps.hdc) == TA_TOP);
COLORREF rgbOld = ::SetBkColor(ps.hdc, RGB(0xC0, 0xFF, 0xC0));
::TextOutW(ps.hdc, x, y, pszText, cchText);
::SetBkColor(ps.hdc, rgbOld);
// This vertical line at the right side of the text shows that opaque
// background is exactly the height returned by GetTextExtentPoint32.
SIZE size = {0};
if (::GetTextExtentPoint32W(ps.hdc, pszText, cchText, &size)) {
::MoveToEx(ps.hdc, x + size.cx, y, NULL);
::LineTo(ps.hdc, x + size.cx, y + size.cy);
}
// These horizontal lines show the normal line spacing, taking into
// account tmExternalLeading.
assert(tm.tmExternalLeading > 0); // ensure it's an interesting case
::MoveToEx(ps.hdc, x, y, NULL);
::LineTo(ps.hdc, x + size.cx, y); // top of this line
const int yNext = y + tm.tmHeight + tm.tmExternalLeading;
::MoveToEx(ps.hdc, x, yNext, NULL);
::LineTo(ps.hdc, x + size.cx, yNext); // top of next line
The gap between the bottom of the colored rectangle and the top of the next line represents the external leading, which is always outside the character cell.
OCR-B is designed for reliable optical character recognition in banking equipment. Having a large external leading (relative to the height of the actual text) may be appropriate for some OCR applications. For this particular font, it's probably not an aesthetic choice.

Which characters to choose when "drawing" a box on Windows console?

I'm trying to port a curses program to Windows. Now one of the problems is that the default ACS_XXXX characters become double-width on Windows console, thus breaking the alignment.
I tried looking for other characters to do the job, like '-' or '|' in basic ASCII, but none of them looks good because the line is not continuous. And finding characters to "draw" corners seems more difficult.
Are there any commonly used characters in such a situation?
I got it to work using the MingLiu font. That is, to draw boxes around Chinese characters with ASCII characters without any alignment issues.
There are border characters in the system font. This includes joints, corners, and both double and single edges. They appear in the higher positions.
Check out http://www.asciitable.com/ for details. They range from 179 to 218 (decimal) in the extended ascii table.
There are a few box drawing characters that were available in the old DOS days - you should be able to use those.
However, keep in mind that the Windows console may require some jumping through hoops to output this as Unicode, which might be a problem unless you accept that your code editor is unlikely to display the character correctly. Michael Kaplan summarizes the problem quite nicely, with information about how to get around this.

Resources