I am trying to use combination of functions CreateFileMapping , MapViewOfFile, FlushViewOfFile.
the total buffer size is more than the mapped view.
example buffer is 50KB. and mapped view is 2KB. in such scenario,
i want to write the total buffer to a physical file, using the above function.
First part i am able to write to file. but the remaining part how to write to file. I mean, how to move to next page and write the next part of data.
#define MEM_UNIT_SIZE 100
-first module...Memory map creator
GetTempPath (256, szTmpFile);
GetTempFileName (szTmpFile, pName, 0, szMMFile);
hFile = CreateFile (szMMFile, GENERIC_WRITE | GENERIC_READ, FILE_SHARE_WRITE,
NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY, NULL);
HANDLE hFileMMF = CreateFileMapping( hFile ,NULL,PAGE_READWRITE,0,
(MEM_UNIT_SIZE),pName)
-second module... Memory writer
long lBinarySize = 1000;
long lPageSize = MEM_UNIT_SIZE;
HANDLE hFileMMF = OpenFileMapping(FILE_MAP_WRITE,FALSE,pMemName);
LPVOID pViewMMFFile = MapViewOfFile(hFileMMF,FILE_MAP_WRITE,0,0, lPageSize );
CMutex mutex (FALSE, _T("Writer"));
mutex.Lock();
try
{
ASSERT(FALSE);
CopyMemory(pViewMMFFile,pBinary,lPageSize); // write
FlushViewOfFile(pViewMMFFile,lPageSize);
// first 100 bytes flushed to file.
//how to move to next location and write next 900 bytes..<---??
}
catch(CException e)
{
...
}
please share if you have any suggestion.
thanks in advance,
haranadh
Repeat your call to MapViewOfFile with a different range.
as described in the following link,
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366761(v=VS.85).aspx
can you please check "allocation granularity", I think you should use this parameter to set the values for "dwFileOffsetLow" or "dwFileOffsetHigh".
Related
I know lots of similar questions on this topic have been asked before but so far I have been unable to find a solution that actually works. I want to start a console program from my program and capture its output. My implementation should be in a way that is compatible with WaitForMultipleObjects(), i.e. I want to get notified whenever there is new data to read in the pipe.
My implementation is based on this example from MSDN. However, I had to modify it a little because I need overlapped I/O in order to be able to wait for ReadFile() to finish. So I'm using named pipes created using Dave Hart's MyCreatePipeEx() function from here.
This is my actual code. I have removed error checks for readability reasons.
HANDLE hReadEvent;
HANDLE hStdIn_Rd, hStdIn_Wr;
HANDLE hStdOut_Rd, hStdOut_Wr;
SECURITY_ATTRIBUTES saAttr;
PROCESS_INFORMATION piProcInfo;
STARTUPINFO siStartInfo;
OVERLAPPED ovl;
HANDLE hEvt[2];
DWORD mask, gotbytes;
BYTE buf[4097];
saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
saAttr.bInheritHandle = TRUE;
saAttr.lpSecurityDescriptor = NULL;
MyCreatePipeEx(&hStdOut_Rd, &hStdOut_Wr, &saAttr, 0, FILE_FLAG_OVERLAPPED, FILE_FLAG_OVERLAPPED);
MyCreatePipeEx(&hStdIn_Rd, &hStdIn_Wr, &saAttr, 0, FILE_FLAG_OVERLAPPED, FILE_FLAG_OVERLAPPED);
SetHandleInformation(hStdOut_Rd, HANDLE_FLAG_INHERIT, 0);
SetHandleInformation(hStdIn_Wr, HANDLE_FLAG_INHERIT, 0);
memset(&piProcInfo, 0, sizeof(PROCESS_INFORMATION));
memset(&siStartInfo, 0, sizeof(STARTUPINFO));
siStartInfo.cb = sizeof(STARTUPINFO);
siStartInfo.hStdError = hStdOut_Wr;
siStartInfo.hStdOutput = hStdOut_Wr;
siStartInfo.hStdInput = hStdIn_Rd;
siStartInfo.dwFlags |= STARTF_USESTDHANDLES;
CreateProcess(NULL, "test.exe", NULL, NULL, TRUE, 0, NULL, NULL, &siStartInfo, &piProcInfo);
hReadEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
for(;;) {
int i = 0;
hEvt[i++] = piProcInfo.hProcess;
memset(&ovl, 0, sizeof(OVERLAPPED));
ovl.hEvent = hReadEvent;
if(!ReadFile(hStdOut_Rd, buf, 4096, &gotbytes, &ovl)) {
if(GetLastError() == ERROR_IO_PENDING) hEvt[i++] = hReadEvent;
} else {
buf[gotbytes] = 0;
printf("%s", buf);
}
mask = WaitForMultipleObjects(i, hEvt, FALSE, INFINITE);
if(mask == WAIT_OBJECT_0 + 1) {
if(GetOverlappedResult(hStdOut_Rd, &ovl, &gotbytes, FALSE)) {
buf[gotbytes] = 0;
printf("%s", buf);
}
} else if(mask == WAIT_OBJECT_0) {
break;
}
}
The problem with this code is the following: As you can see, I'm reading in chunks of 4kb using ReadFile() because I obviously don't know how much data the external program test.exe will output. Doing it this way was suggested here:
To read a variable amount of data from the client process just issue
read requests of whatever size you find convenient and be prepared to
handle read events that are shorter than you requested. Don't
interpret a short but non-zero length as EOF. Keep issuing read
requests until you get a zero length read or an error.
However, this doesn't work. The event object passed to ReadFile() as part of the OVERLAPPED structure will only trigger once there are 4kb in the buffer. If the external program just prints "Hello", the event won't trigger at all. There need to be 4kb in the buffer for hReadEvent to actually trigger.
So I thought I should read byte by byte instead and modified my program to use ReadFile() like this:
if(!ReadFile(hStdOut_Rd, buf, 1, &gotbytes, &ovl)) {
However, this doesn't work either. If I do it like this, the read event is not triggered at all which is really confusing me. When using 4096 bytes, the event does indeed trigger as soon as there are 4096 bytes in the pipe, but when using 1 byte it doesn't work at all.
So how am I supposed to solve this? I'm pretty much out of ideas here. Is there no way to have the ReadFile() event trigger whenever there is some new data in the pipe? Can't be that difficult, can it?
Just for the record, while there are some problems with my code (see discussion in comments below the OP), the general problem is that it's not really possible to capture the output of arbitrary external programs because they will typically use block buffering when their output is redirected to a pipe, which means that output will only arrive at the capturing program once that buffer is flushed so real time capturing is not really possible.
Some workarounds have been suggested though:
1) (Windows) Here is a workaround that uses GetConsoleScreenBuffer() to capture the output from arbitrary console programs but it currently only supports one console page of output.
2) (Linux) On Linux, it's apparently possible to use pseudo-terminals to force the external program to use unbuffered output. Here is an example.
I'm reading a volume (logical drive) with ReadFile. I'm using DeviceIoControl with FSCTL_ALLOW_EXTENDED_DASD_IO code, because I want to have access to all (including the last) bytes and had an issue trying to read last 512 bytes (ReadFile successed, but reported 0 bytes read) and saw advice to use it. Unfortunately, ReadFile fails being called after that DeviceIoControl called.
In code it looks like this (all success checks are omitted for the brevity):
HANDLE fd;
DWORD junk;
int lenToBeRead = 0x1000;
DWORD nread;
char* alignedBuf = new char[lenToBeRead];
fd = CreateFile("path to volume", FILE_READ_DATA,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL, OPEN_EXISTING, 0, NULL)) //success
DeviceIoControl(fd, FSCTL_ALLOW_EXTENDED_DASD_IO,
NULL, 0, NULL, 0, &junk, (LPOVERLAPPED) NULL) //success
ReadFile(fd, alignedBuf, (DWORD) lenToBeRead, &nread, NULL)
// fails with 0x57 code, ERROR_INVALID_PARAMETER
All work with fd handle is synchronous.
EDIT. I solved the problem. I was trying to read last bytes. So my volume was of length L = 0x...200 and I had my handle on position pos = L - 0x200. What I had done before I did the FSCTL_ALLOW_EXTENDED_DASD_IO thing - I cut lenToBeRead to fit in remaining space (so, if it was 0x1000, it would change to 0x200), because I had found that ReadFile did not guarantee read all the bytes to the EOF in case of lenToBeRead is greater than amount of bytes remained from current handle position. This did not help, ReadFilewas still returning with success and 0 bytes read. I deleted that fix and then used FSCTL_ALLOW_EXTENDED_DASD_IO, which deliver me then ReadFile failing with ERROR_INVALID_PARAMETER on lenToBeRead = 0x1000. I totally forgot about the first fix and remembered now and now it works.
I found the solution and add it to the question body.
What one has to keep in mind when working with ReadFile is to control arguments (length) to not cross the boundary of the file.
I had tried it as a fix before doing the FSCTL_ALLOW_EXTENDED_DASD_IO thing and it did not help. But combination of the FSCTL_ALLOW_EXTENDED_DASD_IO thing and the boundary check gave me wanted result - I could read that last bytes.
How can I read a file from 100th byte, using ReadFile function and C++
I used this code but it reads the first 100 bytes of the file
I want to read the second 100 bytes
hndl = CreateFileW(L"1.txt", GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
SetFilePointer(hndl, 100, NULL, FILE_BEGIN);
ReadFile(hndl, pbytReadBuffer, 100, NULL, &ol);
The ReadFile API offers two distinct ways to set the starting offset when doing synchronous I/O:
By using the implicitly stored file pointer (which can be manipulated through the SetFilePointer API call).
Or by passing an explicit offset through the OVERLAPPED structure.
Your code fails because you are setting the implicitly stored file pointer, but then (presumably) pass a zero-initialized OVERLAPPED structure, which ignores the file pointer (see Synchronization and File Position for more information).
Either of the following solutions will work. First, by using the implicitly stored file pointer. This is useful, when you want to read chunks of a file in consecutive calls:
hndl = CreateFileW(L"1.txt", GENERIC_READ, 0, nullptr,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr);
// Move the file pointer to offset 100
SetFilePointer(hndl, 100, NULL, FILE_BEGIN);
// Read contents from the current offset
DWORD dwBytesRead{0};
ReadFile(hndl, pbytReadBuffer, 100, &dwBytesRead, nullptr);
Alternatively, you can pass an OVERLAPPED structure to pass the offset. This ignores the implicitly stored file pointer. It is slightly more efficient, because it doesn't need two calls to the file I/O API.
hndl = CreateFileW(L"1.txt", GENERIC_READ, 0, nullptr,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr);
OVERLAPPED ol{};
// Set the offset from the start of the file
ol.Offset = 100;
ReadFile(hndl, pbytReadBuffer, 100, nullptr, &ol);
Note that error handling is elided in these samples for brevity. In real code, you always must check for errors.
you need code like this
HANDLE hFile = CreateFile(L"c:\\windows\\notepad.exe", FILE_GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, 0, 0);
if (hFile != INVALID_HANDLE_VALUE)
{
OVERLAPPED ov = {};
ov.Offset = 100;
UCHAR buf[100];
ULONG cb;
if (ReadFile(hFile, buf, sizeof(buf), &cb, &ov))
{
}
else
{
GetLastError();
}
CloseHandle(hFile);
}
you not need call SetFilePointer - much better set offset direct in OVERLAPPED structure
EDIT
for every read/write operation we need to specify the starting byte offset. we can do this directed by setting values in Offset and OffsetHigh from OVERLAPPED.
or indirect only if file opened if synchronous mode - I/O Manager can use the current file position in FILE_OBJECT - so we can not direct set offset - it will be get from FILE_OBJECT.CurrentByteOffset. FILE_OBJECT.CurrentByteOffset we can set with SetFilePointer also every read/write operation update this offset - move forward to count of bytes which readed/ written. of course this is correct only if file used in synchronous mode when all operations with file is sequential
if we use direct offset in OVERLAPPED he and used - so FILE_OBJECT.CurrentByteOffset is ignored - this mean that previous call to SetFilePointer - also lost all effect - will be used offset from OVERLAPPED and after read operation FILE_OBJECT.CurrentByteOffset will be updated based on offset + bytes readed
Under Linux, my program would do something like this:
Process 1 opens a file (e.g. by mapping it into memory). Let's call this file#1
Process 2 unlinks the file, and creates a new file with the same name. Let's call this file#2.
Process 1 continues to work with file#1. When it is closed, it is deleted (since it has no links). Process 1 continues to work with the content in file#1, and does not see content from file#2.
When both processes have exited, file#2 remains on disk.
I want to achieve the same semantics in Windows. After reading this question, I think FILE_SHARE_DELETE does basically this. Is opening the file with FILE_SHARE_DELETE enough, or do I need to consider something more?
The above execution flow is just an example, I know there are other ways of solving that exact problem in Windows, but I want to understand how to make such code portable between Windows and Linux.
Edit: to clarify: The use cases would be to reuse a filename for different unrelated files, but let existing processes keep their data (think transactional update of a config file for example), and to make a file anonymous (unnamed), but continue to use it like an anonymous memory map. Again I know both are possible on Windows through other means, but I am trying to find a way that is portable across platforms.
You can achieve this by using a combination of CreateFile, CreateFileMapping and MapViewOfFile calls. MapViewOfFile will give you a memory-mapped buffer of the file backed by the file on disk.
Following code when executed from different processes, will write the process id of last closing process in the file at c:\temp\temp.txt
int main()
{
TCHAR szMsg[256];
HANDLE hMapFile;
LPCTSTR pBuf;
HANDLE hFile = CreateFileW(
L"C:\\Temp\\temp.txt",
GENERIC_WRITE|GENERIC_READ,
FILE_SHARE_DELETE|FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,NULL);
hMapFile = CreateFileMapping(
hFile, // Handle of file opened with rw access
NULL, // default security
PAGE_READWRITE, // read/write access
0, // maximum object size (high-order DWORD)
BUF_SIZE, // maximum object size (low-order DWORD)
szName); // name of mapping object
if (hMapFile == NULL)
{
printf( "Could not create file mapping object (%d).\n", GetLastError());
return 1;
}
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to map object
FILE_MAP_ALL_ACCESS, // read/write permission
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
printf("Could not map view of file (%d).\n", GetLastError());
CloseHandle(hMapFile);
return 1;
}
wsprintfW(szMsg, L"This is process id %d", GetCurrentProcessId());
CopyMemory((PVOID)pBuf, szMsg, (wcslen(szMsg) * sizeof(TCHAR)));
MessageBoxW(NULL, szMsg, L"Check", MB_OK);
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
CloseHandle(hFile);
return 0;
}
Make sure you open the file with GENERIC_READ|GENERIC_WRITE access and allow FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE access to subsequent opens.
Also note the use of CREATE_ALWAYS in CreateFile which will delete the old file and open a new one every-time CreateFile is called. This is the 'unlink' effect you talk about.
Code inspired from Creating Named Shared Memory at msdn.
I am a new member and joined this site after referring to it loads of times when i was stuck with some programming problems. I am trying to code a media player (Win32 SDK VC++ 6.0) for my college project and I am stuck. I have searched on various forums and msdn and finally landed on the function GetShortPathName which enables me to play through folders and files which have a whitespace in their names. I will paste the code here so it will be much more clearer as to what i am trying to do.
case IDM_FILE_OPEN :
ZeroMemory(&ofn, sizeof(ofn));
ofn.lStructSize = sizeof(ofn);
ofn.hwndOwner = hwnd;
ofn.lpstrFilter = "Media Files (All Supported Types)\0*.avi;*.mpg;*.mpeg;*.asf;*.wmv;*.mp2;*.mp3\0"
"Movie File (*.avi;*.mpg;*.mpeg)\0*.avi;*.mpg;*.mpeg\0"
"Windows Media File (*.asf;*.wmv)\0*.asf;*.wmv\0"
"Audio File (*.mp2;*.mp3)\0*.mp2;*.mp3\0"
"All Files(*.*)\0*.*\0";
ofn.lpstrFile = szFileName;
ofn.nMaxFile = MAX_PATH;
ofn.Flags = OFN_EXPLORER | OFN_FILEMUSTEXIST | OFN_HIDEREADONLY | OFN_ALLOWMULTISELECT | OFN_CREATEPROMPT;
ofn.lpstrDefExt = "mp3";
if(GetOpenFileName(&ofn))
{
length = GetShortPathName(szFileName, NULL, 0);
buffer = (TCHAR *) malloc (sizeof(length));
length = GetShortPathName(szFileName, buffer, length);
for(i = 0 ; i < MAX_PATH ; i++)
{
if(buffer[i] == '\\')
buffer[i] = '/';
}
SendMessage(hList,LB_ADDSTRING,0,(LPARAM)buffer);
mciSendString("open buffer alias myFile", NULL, 0, NULL);
mciSendString("play buffer", NULL, 0, NULL);
}
return 0;
using the GetShortPathName function i get the path as : D:/Mp3z/DEEPBL~1/03SLEE~1.mp3
Putting this path directly in Play button case
mciSendString("open D:/Mp3jh/DEEPBL~1/03SLEE~1.mp3 alias myFile", NULL, 0, NULL);
mciSendString("play myFile", NULL, 0, NULL);
the file opens and plays fine. But as soon as i try to open and play it through the open file dialog box, nothing happens. Any input appreciated.
It looks like the problem is that you're passing the name of the buffer variable to the mciSendString function as a string, rather than passing the contents of the buffer.
You need to concatenate the arguments you want to pass (open and alias myFile) with the contents of buffer.
The code can also be much simplified by replacing malloc with an automatic array. You don't need to malloc it because you don't need it outside of the block scope. (And you shouldn't be using malloc in C++ code anyway; use new[] instead.)
Here's a modified snippet of the code shown in your question:
(Warning: changes made using only my eyes as a compiler! Handle with care.)
if(GetOpenFileName(&ofn))
{
// Get the short path name, and place it in the buffer array.
// We know that a short path won't be any longer than MAX_PATH, so we can
// simply allocate a statically-sized array without futzing with new[].
//
// Note: In production code, you should probably check the return value
// of the GetShortPathName function to make sure it succeeded.
TCHAR buffer[MAX_PATH];
GetShortPathName(szFileName, buffer, MAX_PATH);
// Add the short path name to your ListBox control.
//
// Note: In C++ code, you should probably use C++-style casts like
// reinterpret_cast, rather than C-style casts!
SendMessage(hList, LB_ADDSTRING, 0, reinterpret_cast<LPARAM>(buffer));
// Build the argument string to pass to the mciSendString function.
//
// Note: In production code, you probably want to use the more secure
// alternatives to the string concatenation functions.
// See the documentation for more details.
// And, as before, you should probably check return values for error codes.
TCHAR arguments[MAX_PATH * 2]; // this will definitely be large enough
lstrcat(arguments, TEXT("open"));
lstrcat(arguments, buffer);
lstrcat(arguments, TEXT("alias myFile"));
// Or, better yet, use a string formatting function, like StringCbPrintf:
// StringCbPrintf(arguments, MAX_PATH * 2, TEXT("open %s alias myFile"),
// buffer);
// Call the mciSendString function with the argument string we just built.
mciSendString(arguments, NULL, 0, NULL);
mciSendString("play myFile", NULL, 0, NULL);
}
Do note that, as the above code shows, working with C-style strings (character arrays) is a real pain in the ass. C++ provides a better alternative, in the form of the std::string class. You should strongly consider using that instead. To call Windows API functions, you'll still need a C-style string, but you can get one of those by using the c_str method of the std::string class.