How To Deal With MaxLineLengthExceeded With Indy TIdTCPClient - indy

I'm new to indy, using whatever version comes with CBuilder XE4. Here's very simple code that works fine until what I'm reading exceeds the 16K limit....
String Ttcp_mgr::send(String data)
{
tcpClient->Socket->WriteLn(data);
return tcpClient->Socket->ReadLn();
}
The server is not using indy, there is no length header, going both ways is json terminated by \r\n. Blocking reads are fine, there's nothing for my app to do until it get's it's response, and it will be coming very quickly anyway. But the amount of data that is returned could be a few bytes or 100K, in a few cases. Generally the length will be < 500 bytes.
I've looked at IOHandler but I have no idea how to apply it to what I'm doing, not even sure it's what I need. As you can probably tell I'm not using the component on a form which probably makes no difference.

TIdIOHandler::ReadLn() has an optional AMaxLineLength input parameter. If you do not specify a value for it, the TIdIOHandler::MaxLineLength property is used, which is set to 16K by default. The TIdIOHandler::MaxLineAction property specifies what happens if ReadLn() actually reaches the max line length.
If MaxLineAction is maException (the default), an EIdReadLnMaxLineLengthExceeded exception is raised.
If MaxLineAction is maSplit, the TIdIOHandler::ReadLnSplit property is set to true and ReadLn() returns what it can. You would have to call ReadLn() again to read more data for the current line. This can end up chopping the data incorrectly if it is using a multi-byte encoding for non-ASCII characters, like UTF-8 (which is JSON's default encoding), so I do not recommend this approach.
In your case, you should either:
set the TIdIOHandler::MaxLineLength property to MaxInt:
// TIdTCPClient::OnConnected event handler...
void __fastcall Ttcp_mgr::tcpCllientConnected(TObject *Sender)
{
tcpClient->IOHandler->MaxLineLength = MaxInt;
}
pass MaxInt as a parameter to TIdIOHandler::ReadLn().
String Ttcp_mgr::send(String data)
{
tcpClient->Socket->WriteLn(data);
return tcpClient->Socket->ReadLn(EOL, IdTimeoutDefault, MaxInt);
}

To all Delphi users: setting IOHandler.MaxLineLength to "MaxInt" must be done after starting the connection, otherwise you will get a memory error.
IdPOP31.Connect;
IdPOP31.IOHandler.MaxLineLength := MaxInt;
Everything else works as above, and solves the problem of parsing too long emails for Delphi.

Related

What is the maximum length of a window title passed to SetWindowText?

The SetWindowText function's documentation does not set a limit on the length of the string that may be used as a window title.
In the documentation for WM_SETTEXT (the message sent by calling SetWindowText), it is noted that the return value of this message's processing may be:
FALSE (for an edit control), LB_ERRSPACE (for a list box), or CB_ERRSPACE (for a combo box) if insufficient space is available to set the text in the edit control.
However, it says nothing about the case when a window's title is being set. Is a strict limit set, or is it up to the programmer to use common sense to provide their own title length limit?
I have posted this because I am developing a graphics engine which allows the user to supply their own title for the main window. The idea is that I would define a constant such as
const static int MAX_APP_TITLE_LENGTH = /* ??? */;
within my application class, and check the length of the user-provided title string against this.
If the title string is too long, I can throw a warning message and truncate it, rather than passing it straight into SetWindowText with unintended consequences.
EDIT: After some discussion in the comments, it seems that Windows will not complain even if a string of length 100,000 is used as a window title, so this issue is not worth worrying about (beyond the basic sanitization of input, of course)!
There is technically no limit to the title size, but the lpClassName field has a strict limit of 256 chars (i didnt want you to think you could have an infinite class name and your code crash.)
SOURCE: https://learn.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-wndclassexw

What kind of encoding does posFlag requires?

How can I encode the position of the form /pathto/file.go:40:32 which is returned by token.Position.String() to a posFlag param required by ParseQueryPos which looks like /pathto/file.go:#550.
Why?
I'm using the Oracle tool to do some static analysis. I need to run Oracle.Query which requires a param of type *QueryPos. The only way to get *QueryPos is using ParseQueryPos.
The source to tools/pos.go called by ParseQueryPos says
// parsePosFlag parses a string of the form "file:pos" or
// file:start,end" where pos, start, end match #%d and represent byte
// offsets, and returns its components.
If you really had to convert from line:column strings, you'd look at the file contents and count up bytes (including newlines) leading to that line:column. But since you're working with a token.Position, it looks like you can get what you need from token.Position.Offset.

How to display UTF-8 in a HITextView (MLTE) control?

I have a UTF-8 string which I want to display in an HITextView (MLTE) control. Theoretically, HITextView requires either "Text" or UTF-16, so I'm converting:
UniChar uniput[STRSIZE];
ByteCount converted,unilen;
err = ConvertFromTextToUnicode(C2UInfo, len, output,
kUnicodeUseFallbacksMask, 0, NULL, 0, NULL,
sizeof(UniChar)*STRSIZE,
&converted, &unilen, uniput);
err=TXNSetData(MessageObject, kTXNUnicodeTextData, uniput, unilen, kTXNEndOffset,
kTXNEndOffset);
I have defined the converter C2UInfo as follows:
UnicodeMapping uMapping;
uMapping.unicodeEncoding = CreateTextEncoding(kTextEncodingUnicodeV2_0,
kUnicodeCanonicalDecompVariant,
kUnicode16BitFormat);
uMapping.otherEncoding = GetTextEncodingBase(kUnicodeUTF8Format);
uMapping.mappingVersion = kUnicodeUseLatestMapping;
err = CreateTextToUnicodeInfo(&uMapping, &C2UInfo);
It works fine for plain old ASCII characters, but multi-byte UTF-8 is being mapped to the wrong characters. For example, æ (LATIN SMALL LETTER AE) is being mapped to 疆 (CJK UNIFIED IDEOGRAPH-7586).
I've tried checking and unchecking "Output Text in Unicode" in Interface Builder, and I've tried varying some of the conversion constants, with no effect.
This is being built with Xcode 3.2.6 using the MacOSX10.5.sdk and tested under 10.6.
The “Text” that ConvertFromTextToUnicode expects is probably the same “Text” that is one of your two options for MLTE. If you had the sort of “Text” that ConvertFromTextToUnicode converts from, you could just pass it to MLTE directly.
(For the record, “Text” is almost certainly either MacRoman or whatever is dictated by the user's locale-determined current script.)
Instead, you should use a Text Encoding Converter. Create one, use it, finish using it, and dispose of it when you're done.
There are two other ways.
One is to create a CFString from the UTF-8, then Get its characters. You would do this instead of using a TEC. It's functionally equivalent and possibly a little bit easier. On the other hand, you don't get to reuse the converter, for whatever that's worth.
The other, since you have an HITextView, would be to create a CFString from the UTF-8 and just use that. Like Cocoa objects, HIToolbox objects have an inheritance hierarchy; since an HITextView is a kind of HIView, HIViewSetText should just work. (And if not, try HIViewSetValue.)
The last method also gets you that much closer to your eventual move away from MLTE/HITextView, since it's essentially what you'll do with an NSTextView. (HITextView and MLTE are deprecated.)

Why are LoadResource/LockResource sometimes returning concatenated resource data?

I'm working with a Visual Studio C++ project that contains a number of HTML resources. They are loaded by a method that looks like this:
LPCTSTR loadHTML(HMODULE hModule, LPCTSTR sResourceName)
{
HRSRC hResource = FindResource(hModule, sResourceName, RT_HTML);
if(!hResource)
return 0;
HGLOBAL hResourceData = LoadResource(hModule, hResource);
if(!hResourceData)
return 0;
return reinterpret_cast<LPCTSTR>(LockResource(hResourceData));
}
Most of the time, this works fine. Some times, though, it returns a resource concatenated with another resource. When this happens, it is a persistent problem in that build. I can "fix" it by adding a few blank lines to the resource in question and then rebuilding the project. It happens periodically, even when the resources haven't changed.
I am keen to get to the bottom of why it is happening. Has anyone else come across it? Could there be something peculiar about my resources that is causing the problem? Is my code wrong?
Sadly, I'm reluctant to post an example resource here; they're pretty long and this is proprietary software.
Whats peculiar about your resources is you are expecting them to be zero terminated. iirc resource sections are aligned on 16 byte boundries, which means that whenever a "blob" is a multiple of 16 bytes long there won't be any separating byte's between the resource and the next.
Either ensure that the resources are saved with a terminating zero character, or use SizeofResource to determine where the resource ends.
How do you determine the end of a resource? Do your resource files end in a (double for unicode) NULL? I don't think there is any guarantee that a resource is NULL terminated in the PE file and you seem to be treating it as a string.

How can I programmatically determine the current default codepage of Windows?

I have to convert the encoding of a string output of a VB6 application to a specific encoding.
The problem is, I don't know the encoding of the string, because of that:
According to the VB6 documentation when accessing certain API functions the internal Unicode strings are converted to ANSI strings using the default codepage of Windows.
Because of that, the encoding of the string output can be different on different systems, but I have to know it to perform the conversion.
How can I read the default codepage using the Win32 API or - if there's no other way - by reading the registry?
It could be even more succinct by using GetACP - the Win32 API call for returning the default code page! (Default code page is often called "ANSI")
int nCodePage = GetACP();
Also many API calls (such as MultiByteToWideChar) accept the constant value CP_ACP (zero) which always means "use the system code page". So you may not actually need to know the current code page, depending on what you want to do with it.
GetSystemDefaultLCID() gives you the system locale.
If the LCID is not enough and you truly need the codepage, use this code:
TCHAR szCodePage[10];
int cch= GetLocaleInfo(
GetSystemDefaultLCID(), // or any LCID you may be interested in
LOCALE_IDEFAULTANSICODEPAGE,
szCodePage,
countof(szCodePage));
nCodePage= cch>0 ? _ttoi(szCodePage) : 0;
That worked for me, thanks, but can be written more succinctly as:
UINT nCodePage = CP_ACP;
const int cch = ::GetLocaleInfo(LOCALE_SYSTEM_DEFAULT,
LOCALE_RETURN_NUMBER|LOCALE_IDEFAULTANSICODEPAGE,
(LPTSTR)&nCodePage, sizeof(nCodePage) / sizeof(_TCHAR) );

Resources