I'm working with a Visual Studio C++ project that contains a number of HTML resources. They are loaded by a method that looks like this:
LPCTSTR loadHTML(HMODULE hModule, LPCTSTR sResourceName)
{
HRSRC hResource = FindResource(hModule, sResourceName, RT_HTML);
if(!hResource)
return 0;
HGLOBAL hResourceData = LoadResource(hModule, hResource);
if(!hResourceData)
return 0;
return reinterpret_cast<LPCTSTR>(LockResource(hResourceData));
}
Most of the time, this works fine. Some times, though, it returns a resource concatenated with another resource. When this happens, it is a persistent problem in that build. I can "fix" it by adding a few blank lines to the resource in question and then rebuilding the project. It happens periodically, even when the resources haven't changed.
I am keen to get to the bottom of why it is happening. Has anyone else come across it? Could there be something peculiar about my resources that is causing the problem? Is my code wrong?
Sadly, I'm reluctant to post an example resource here; they're pretty long and this is proprietary software.
Whats peculiar about your resources is you are expecting them to be zero terminated. iirc resource sections are aligned on 16 byte boundries, which means that whenever a "blob" is a multiple of 16 bytes long there won't be any separating byte's between the resource and the next.
Either ensure that the resources are saved with a terminating zero character, or use SizeofResource to determine where the resource ends.
How do you determine the end of a resource? Do your resource files end in a (double for unicode) NULL? I don't think there is any guarantee that a resource is NULL terminated in the PE file and you seem to be treating it as a string.
Related
I'm new to indy, using whatever version comes with CBuilder XE4. Here's very simple code that works fine until what I'm reading exceeds the 16K limit....
String Ttcp_mgr::send(String data)
{
tcpClient->Socket->WriteLn(data);
return tcpClient->Socket->ReadLn();
}
The server is not using indy, there is no length header, going both ways is json terminated by \r\n. Blocking reads are fine, there's nothing for my app to do until it get's it's response, and it will be coming very quickly anyway. But the amount of data that is returned could be a few bytes or 100K, in a few cases. Generally the length will be < 500 bytes.
I've looked at IOHandler but I have no idea how to apply it to what I'm doing, not even sure it's what I need. As you can probably tell I'm not using the component on a form which probably makes no difference.
TIdIOHandler::ReadLn() has an optional AMaxLineLength input parameter. If you do not specify a value for it, the TIdIOHandler::MaxLineLength property is used, which is set to 16K by default. The TIdIOHandler::MaxLineAction property specifies what happens if ReadLn() actually reaches the max line length.
If MaxLineAction is maException (the default), an EIdReadLnMaxLineLengthExceeded exception is raised.
If MaxLineAction is maSplit, the TIdIOHandler::ReadLnSplit property is set to true and ReadLn() returns what it can. You would have to call ReadLn() again to read more data for the current line. This can end up chopping the data incorrectly if it is using a multi-byte encoding for non-ASCII characters, like UTF-8 (which is JSON's default encoding), so I do not recommend this approach.
In your case, you should either:
set the TIdIOHandler::MaxLineLength property to MaxInt:
// TIdTCPClient::OnConnected event handler...
void __fastcall Ttcp_mgr::tcpCllientConnected(TObject *Sender)
{
tcpClient->IOHandler->MaxLineLength = MaxInt;
}
pass MaxInt as a parameter to TIdIOHandler::ReadLn().
String Ttcp_mgr::send(String data)
{
tcpClient->Socket->WriteLn(data);
return tcpClient->Socket->ReadLn(EOL, IdTimeoutDefault, MaxInt);
}
To all Delphi users: setting IOHandler.MaxLineLength to "MaxInt" must be done after starting the connection, otherwise you will get a memory error.
IdPOP31.Connect;
IdPOP31.IOHandler.MaxLineLength := MaxInt;
Everything else works as above, and solves the problem of parsing too long emails for Delphi.
I'm using the following function to delete a file to the recycle bin: (C++, MFC, Unicode)
bool DeleteFileToPaperbasket (CString filename)
{
TCHAR Buffer[2048+4];
_tcsncpy_s (Buffer, 2048+4, filename, 2048);
Buffer[_tcslen(Buffer)+1]=0; //Double-Null-Termination
SHFILEOPSTRUCT s;
s.hwnd = NULL;
s.wFunc = FO_DELETE;
s.pFrom = Buffer;
s.pTo = NULL;
s.fFlags = FOF_ALLOWUNDO | FOF_SILENT | FOF_NOERRORUI;
s.fAnyOperationsAborted = false;
s.hNameMappings = NULL;
s.lpszProgressTitle = NULL;
int rc = SHFileOperation(&s);
return (rc==0);
}
This works nicely for most files. But if path+filename exceed 255 characters (and still much shorter that 2048 characters), SHFileOperation returns 124. Which is DE_INVALIDFILES.
But what's wrong? I checked everything a million times. The path is double-null terminated, I'm not using \\?\ and it works for short filenames.
I'm totally out of ideas...
I think backwards comparability is biting you in the --- in several ways, and I'd need to actually see the paths your using and implement some error checking code to help. But here are some hints.
You would not get a DE_INVALIDFILES 0x7C "The path in the source or destination or both was invalid." for a max path violation, you'd get a DE_PATHTOODEEP 0x79 "The source or destination path exceeded or would exceed MAX_PATH."
These error codes(return value) do, can, and have changed over time, to be sure what your specific error code means, you need to check it with GetLastError function(msdn)
Also, taken from the SHFileOperation function documentation: "If you do not check fAnyOperationsAborted as well as the return value, you cannot know that the function accomplished the full task you asked of it and you might proceed under incorrect assumptions."
You should not be using this API for extremely long path names, it has been replaced in vista+ by IFileOperation interface
The explanation for why it may work in explorer and not thru this LEGACY api is - Taken from the msdn page on Naming Files, Paths, and Namespaces
The shell and the file system have different requirements. It is
possible to create a path with the Windows API that the shell user
interface is not able to interpret properly.
Hope this was helpful
The recycle bin doesn't support files whose paths exceed MAX_PATH in length. You can verify this for yourself by trying to recycle such a file in Explorer - you will get an error message about the path being too long.
I have the following line in a long loop
page = Nokogiri::HTML(open(topic[:url].first)).xpath('//ul[#class = "pages"]//li').first
Sometimes my Ruby application crashes raising the "End of file reached " exception in this line.
How can I resolve this problem? Just a begin;raise;end block?
Is a script that performs a forum backup, so is important that doesn't skip any thread.
Thanks in advance.
In addition to #Phrogz's excellent advice (in particular about at_css with the simpler expression), I would pull the raw xml [content] separately:
page = if (content = open(topic[:url].first)).strip.length > 0
Nokogiri::HTML(content).xpath('//ul[#class = "pages"]//li').first
end
I would suggest that you should first to fix the underlying issue so that you do not get this error.
Does the same URL always cause the problem? (Output it in your log files.) If so, perhaps you need to URI encode the URL.
Is it random, and therefor likely related to a connection hiccup or server problem? If so, you should rescue the specific error and then retry one or more times to get the crucial data.
Secondarily, you should know that the CSS syntax for that query is far simpler:
page = Nokogiri.HTML(...).at_css('ul.pages li')
Not only is this less than half the bytes, it allows for cases like <ul class="foo pages"> that the XPath would miss.
Using at_css (or at_xpath) is the same as .css(...).first, but is faster and simpler.
I've written a duplicate finder in Java, but I need to include hard link support for it. Unfortunately, there seems to be no way to dig out a file's MFT entry in Java.
Although there is a method called fileKey() in the BasicFileAttributeView class, it won't work on the NTFS file system (I haven't tested it on ext yet).
I also found the method isSameFile() (in java.nio.file.Path). Does anyone know how this method works? It seems to be doing the right thing, but it returns a Boolean value, so it is worthless for me (I wish to put the results into a map and group them by their MFT entries).
I can always compare the creation times, modification times, etc. for each file, but this is just giving up.
Is there any way to accomplish what I am trying to do in either C++ or Java? I care more about making it work on NTFS than ext.
You would need to use the FILE_ID_FULL_DIRECTORY_INFORMATION structure along with the NtQueryDirectoryFile function (or the FILE_INTERNAL_INFORMATION structure along with the NtQueryInformationFile, if you already have a handle) inside ntdll.dll (available since Windows XP, if not earlier) to get the 8-byte file IDs and check if they are the same.
This will tell you if they are the same file, but not if they are the same stream of the same file.
I'm not sure how to detect if two files are the same stream from user-mode -- there is a structure named FILE_STREAM_INFORMATION which can return all the streams associated with a file, but it doesn't tell you which stream you have currently opened.
Detecting hard links is usually accomplished by calling FindFirstFileNameW. But there is a lower level way.
To get the NTFS equivalent to inodes, try the FSCTL_GET_OBJECT_ID ioctl code.
There's a unique (until the file is deleted) identifier in the BY_HANDLE_FILE_INFORMATION structure as well.
If the volume has an enabled USN Change Journal, you can issue the FSCTL_READ_FILE_USN_DATA ioctl code. Check the FileReferenceNumber member in the USN_RECORD structure
In Java you can use sun.nio.ch.FileKey which is a non-transparent enclosure for NTFS Inode. All the hard links share the same Inode.
Therefore, if you need to collect hard links, you can create FileKey from each suspect and compare them (e.g. by putting pairs of FileKey -> File into a Multimap)
I find fileKey is always null. Here is some code that can actually read the NTFS inode number. There remain many aspects I'm not happy with, not least, it relies on reflection.
import sun.nio.ch.FileKey;
import java.io.*;
import java.lang.reflect.Field;
import java.nio.file.Path;
class NTFS {
static long inodeFromPath(Path path) throws IOException, NoSuchFieldException, IllegalAccessException {
try (FileInputStream fi = new FileInputStream(path.toFile())) {
FileDescriptor fd = fi.getFD();
FileKey fk = FileKey.create(fd);
Field privateField = FileKey.class.getDeclaredField("nFileIndexHigh");
privateField.setAccessible(true);
long high = (long) privateField.get(fk);
privateField = FileKey.class.getDeclaredField("nFileIndexLow");
privateField.setAccessible(true);
long low = (long) privateField.get(fk);
long power = (long) 1 << 32;
long inode = high * power + low;
return inode;
}
}
}
i'm writing an app that stores passwords on the keychain, and later obtains them with SecKeychainFindGenericPassword(). this works file 90% of the time, but every once in a while, the call to SecKeychainFindGenericPassword() will fail with errSecAuthFailed (-25293). when it does, just trying again, or restarting the app fixes it.
does anyone have an idea what could be causing this? general Google search on this error points to keychain corruption or the keychain being locked - neither of which is the case here, since subsequent calls succeed again...
This link suggests the passphrase you entered is not correct. See Here
Is it possible that sometimes you are sending a null object as a passphrase just by chance?
Alternatively, you could try out EMKeychain. I have a more up-to-date version on GitHub here: http://github.com/ctshryock/EMKeychain
You haven't shared the code around your problem, So I'll just guess your problem is not with dysfunctional keychain, but rather some coding error.
Here is a common pitfall: Since KeyChain APIs are 'C', and they only accept C-style null terminated string buffers, you'll usually need to convert your CFString/NSString objects to C buffers before handing them to the API.
Many use things like:
const char *usernameCStr = [username UTF8String];
For an NSString, or its CFString companion...
const char *CFStringGetCStringPtr(CFStringRef theString, CFStringEncoding encoding); /* May return NULL at any time; be prepared for NULL */
Dismissing the fact these APIs may return NULL. Either because the internal buffer of the CF/NSString is non-contiguous, or not in the encoding you asked for, or otherwise not-c-compatible.
Such issue can behave in runtime exactly like what you describe.
In such cases you should catch the problem and use different API to copy the CF/NS string into a C-buffer:
Boolean CFStringGetCString(CFStringRef theString, char *buffer, CFIndex bufferSize, CFStringEncoding encoding);
or
- (BOOL)getCString:(char *)buffer maxLength:(NSUInteger)maxBufferCount encoding:(NSStringEncoding)encoding;
I'm not sure this was the problem (i don't see how it could have been) but i recently changed my code to properly pass the strlen() of the cStrings, rather then the length of the NSStrings into the call. Technically this is more correct (since the string length might differ from the cString, if UTF-8 dual-byte characters are involved.
BUT, none of the usernames/passwords i tested with contained non-ASCII characters, so i don;t see how this problem could have actually affected the errors I was seeing. My new code is as follows, and i have not seen the error with it:
UInt32 length;
void *data;
const char *account = [[BC_HOST stringByAppendingFormat:#":%#", login] cStringUsingEncoding:NSUTF8StringEncoding];
NSLog(#"Getting password from keychain.");
OSStatus s = SecKeychainFindGenericPassword (nil,
strlen(BC_APPNAME), BC_APPNAME,
strlen(account), account,
&length, &data, &keychainItem);
if (s != 0) NSLog(#"Error %d obtaining password from keychain.", s);
if (s == 0)
{
password = [[NSString alloc] initWithBytes:data length:length encoding:NSUTF8StringEncoding];
}
I was having the same issue and in my case this turned out to be the cause: Cannot access keychain item after SMJobBless update
One possible reason that this issue can occur is if the executable making the call does not have access to the keychain item. In Keychain Access you can see a list of the apps that have permission to access the item under the Access Control tab for the item in question.
If your app is running from a different location, you will get this error. For example, I have a Privileged Helper Tool that on my dev machine I typically run through Xcode as root. The path to this executable is the path at which Xcode creates it which is a path in ~/Library/Developer/Xcode/DerivedData//myexecutable. When I run it as a user would, it is being run from /Library/PrivilegedHelperTools/myexecutable. So if the password was initially created by one version of the app and I try to read it using the other path, I will see the errSecAuthFailed error.
This isn't the only reason. Someone else mentioned the upgrade in place problem that SMJobBless has. That can also result in the same error code, but for sure I see it for both reasons - although I programmatically solved the upgrade in place issue by ordering the helper tool to move itself to a different location before I do the upgrade.