I am trying to learn more about DriverKit and memory management, and I read this question:
How to allocate memory in a DriverKit system extension and map it to another process?
And I would like to understand how to use IOMemoryDescriptor::CreateMapping.
I wrote a little app to test this where I do (very simplified code):
uint8_t * buffer = new uint8_t[256];
for (int i = 0 ; i < 256 ; i++)
buffer[i] = 0xC6;
clientData in, out;
in.nbytes = 256;
in.pbuffer = buffer;
size_t sout = sizeof(out);
IOConnectCallStructMethod(connection, selector,&in,sizeof(in),&out,&sout);
// out.pbuffer now has new values in it
In my Kext user client class, I was doing (I am simplifying):
IOReturn UserClient::MyExtFunction(clientData * in, clientData * out, IOByteCount inSize, IOByteCount * outSize)
{
MyFunction(in->nBytes, in->pbuffer);//this will change the content of pbuffer
*out = *in;
}
IOReturn UserClient::MyFunction(SInt32 nBytesToRead,void* pUserBuffer,SInt32* nBytesRead)
{
PrepareBuffer(nBytesToRead,&pBuffer);
...
(call function that will fill pBuffer)
}
IOReturn UserClient::PrepareBuffer(UInt32 nBytes,void** pBuffer);
{
IOMemoryDescriptor * desc = IOMemoryDescriptor::withAddressRange((mach_vm_address_t)*pBuffer,nBytes,direction, owner task);
desc->prepare();
IOMemoryMap * map = desc->map();
*pBuffer = (void*)map->getVirtualAddress();
return kIOReturnSuccess;
}
This is what I don't know how to reproduce in a DExt and where I think I really don't understand the basic of CreateMapping.
Or is what I used to do not possible?
In my driver, this is where I don't know how to use CreateMapping and IOMemoryMap so this buffer can be mapped to a memory location and updated with different values.
I can create an IOBufferMemoryDescriptor but how do I tie it to the buffer from my application? I also don't understand the various options for CreateMapping.
Please note that in another test app I have successfully used IOConnectMapMemory64()/CopyClientMemoryForType() but I would like to learn specifically about CreateMapping().
(I hope it is alright I edited this question a lot... still new to StackOverflow)
Or is what I used to do not possible?
In short, no.
You're attempting to map arbitrary user process memory, which the client application did not explicitly mark as available to the driver using IOKit. This doesn't fit with Apple's ideas about safety, security, and sandboxing, so this sort of thing isn't available in DriverKit.
Obviously, kexts are all-powerful, so this was possible before, and indeed, I've used the technique myself in shipping drivers and then ran into trouble when porting said kexts to DriverKit.
The only ways to gain direct access to the client process's memory, as far as I'm aware, are:
By passing buffers >= 4097 bytes as struct input or output arguments to IOConnectCall…Method()s so they arrive as IOMemoryDescriptors in the driver. Note that these can be retained in the driver longer term, but at least for input structs, updates on the user space side won't be reflected on the driver side as a copy-on-write mapping is used. So they should be used purely for sending data in the intended direction.
By the user process mapping an existing IOMemoryDescriptor into its space using IOConnectMapMemory64()/CopyClientMemoryForType().
This does mean you can't use indirect data structures like the one you are using. You'll have to use "packed" structures, or indices into long-lasting shared buffers.
By "packed" structures I mean buffers containing a header struct such as your clientData which is followed in contiguous memory by further data, such as your buffer, referencing it by offset into this contiguous memory. The whole contiguous memory block can be passed as an input struct.
I have filed feedback with Apple requesting a more powerful mechanism for exchanging data between user clients and dexts; I have no idea if it will be implemented, but if such a facility would be useful, I recommend you do the same. (explaining what you'd like to use it for with examples) The more of us report it, the more likely it'll happen. (IOMemoryDescriptor::CreateSubMemoryDescriptor() was added after I filed a request for it; I won't claim I was the first to do so, or that Apple wasn't planning to add it prior to my suggestion, but they are actively improving the DriverKit APIs.)
Original answer before question was edited to be much more specific:
(Retained because it explains in general terms how buffer arguments to external methods are handled, which is likely helpful for future readers.)
Your question is a little vague, but let me see if I can work out what you did in your kext, vs what you're doing in your dext:
You're calling IOConnectCallStructMethod(connection, selector, buffer, 256, NULL, NULL); in your app. This means buffer is passed as a "struct input" argument to your external method.
Because your buffer is 256 bytes long, which is less than or equal to sizeof(io_struct_inband_t), the contents of the buffer is sent to the kernel in-band - in other words, it's copied at the time of the IOConnectCallStructMethod() call.
This means that in your kext's external method dispatch function, the struct input is passed via the structureInput/structureInputSize fields in the incoming IOExternalMethodArguments struct. structureInput is a pointer in the kernel context and can be dereferenced directly. The pointer is only valid during execution of your method dispatch, and can't be used once the method has returned synchronously.
If you need to use the buffer for device I/O, you may need to wrap it in an IOMemoryDescriptor. One way to do this is indeed via IOMemoryDescriptor::CreateMapping().
If the buffer was 4097 bytes or larger, it would be passed via the structureInputDescriptor IOMemoryDescriptor, which can either be passed along to device I/O directly, or memory-mapped for dereferencing in the kernel. This memory descriptor directly references the user process's memory.
DriverKit extensions are considerably more limited in what they can do, but external method arguments arrive in almost exactly the same way.
Small structs arrive via the IOUserClientMethodArguments' structureInput field, which points to an OSData object. You can access the content via the getBytesNoCopy()/getLength() methods.
If you need this data in an IOMemoryDescriptor for onward I/O, the only way I know of is to create an IOBufferMemoryDescriptor using either IOUSBHostDevice::CreateIOBuffer() or IOBufferMemoryDescriptor::Create and then copying the data from the OSData object into the buffer.
Large buffers are again already referenced via an IOMemoryDescriptor. You can pass this on to I/O functions, or map it into the driver's address space using CreateMapping()
namespace
{
/*
**********************************************************************************
** create a memory descriptor and map its address
**********************************************************************************
*/
IOReturn arcmsr_userclient_create_memory_descriptor_and_map_address(const void* address, size_t length, IOMemoryDescriptor** memory_descriptor)
{
IOBufferMemoryDescriptor *buffer_memory_descriptor = nullptr;
uint64_t buffer_address;
uint64_t len;
#if ARCMSR_DEBUG_IO_USER_CLIENT
arcmsr_debug_print("ArcMSRUserClient: *******************************************************\n");
arcmsr_debug_print("ArcMSRUserClient: ** IOUserClient IOMemoryDescriptor create_with_bytes \n");
arcmsr_debug_print("ArcMSRUserClient: *******************************************************\n");
#endif
if (!address || !memory_descriptor)
{
return kIOReturnBadArgument;
}
if (IOBufferMemoryDescriptor::Create(kIOMemoryDirectionInOut, length, 0, &buffer_memory_descriptor) != kIOReturnSuccess)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnError;
}
if (buffer_memory_descriptor->Map(0, 0, 0, 0, &buffer_address, &len) != kIOReturnSuccess)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnError;
}
if (length != len)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnNoMemory;
}
memcpy(reinterpret_cast<void*>(buffer_address), address, length);
*memory_descriptor = buffer_memory_descriptor;
return kIOReturnSuccess;
}
} /* namespace */
I am just starting with ALEA and I am curious how you can access other types and references inside a given gpu parallel.for. when i do the following i get a runtime error that states "Cannot get field random. Possible reasons: 1) Static field is not supported.2) The field type is not supported. 3) In closure class, the field doesn't have [GpuParam] attribute."
This error makes sense but I am not sure what the correct implementation would be
[GpuManaged]
public void InitPoints()
{
var gp = Gpu.Default;
gp.For(1, (10), (i) =>
{
int pointStart = random.Next(totalPoints) + 1;
Pt point = new Pt(pointStart, ptAt[i]);
point.Process();
});
}
You try to call the System.Random.Next. This is .NET library code and cannot be compiled to GPU. There is no MSIL behind that function that could be accessed and compiled to run on the GPU. Also the System.Random.Next is a random number generator implemented for serial applications. You should use the parallel random number generators provided in cuRand, which are also exposed in Alea GPU.
I just upgraded my V8 version to 3.20.16 (from some very old version). I can no longer use
Handle<Object> obj /* = ... */;
Persistent<Object> p = Persistent<Object>::New( obj );
to create a persistent handle to an object. The compiler suggests using static T* v8::Persistent<T>::New(v8::Isolate*, T*) [with T = v8::Object] instead. However, if I change my code to
Handle<Object> obj /* = ... */;
Persistent<Object> p = Persistent<Object>::New( Isolate::GetCurrent(), *obj );
the compiler complains that this function is private. How do I create a Persistent<Object> handle from a normal Handle<Object> now?
I've googled and the only thing I found was that the documentations seem to contradict each other:
https://developers.google.com/v8/embed#handles says that persistent handles are now created using the Persistence constructor
http://bespin.cz/~ondras/html/classv8_1_1Persistent.html indicates that Persistence<T>::New is still the way to go
thanks for any help in advance
There is a constructor that accepts normal Handle<T> you don't need to dereference it.
Persistent<Object>::New(Isolate::GetCurrent(), obj)
should work.
Lets say I have this class in foobar-shared.lib:
class FooBar {
std::string m_helloWorld;
}
And I have a call in foobar-from.exe using SendCopyData like so:
extern HWND hMainWnd; // foobar-from.exe
{
FooBar fooBar;
HWND hWnd = FindAppWindow(); // foobar-to.exe
COPYDATASTRUCT cds;
cds.dwData = ('f'|('o'<<8)|('o'<<16));
cds.cbData = sizeof(FooBar);
cds.lpData = (LPVOID)fooBar;
SendCopyData(hWnd, (WPARAM)hMainWnd, (LPARAM)&cds);
}
When from a foobar-to.exe, I handle OnCopyData:
BOOL CMainFrame::OnCopyData(CWnd* pWnd, COPYDATASTRUCT* pCopyDataStruct) {
if (pCopyDataStruct->dwData==('f'|('o'<<8)|('o'<<16))) {
FooBar fooBar = *(FooBar *)pCopyDataStruct->lpData;
}
}
This worked fine when FooBar was a struct, but now that it's a class I get this error:
First-chance exception at 0x0064ef81 in foobar-to.exe: 0xC0000005:
Access violation reading location 0x0231dd7c.
I assumed originally that this was because my fooBar instance is on the stack, so I tried moving it to the heap but got a slightly different error (I can post the result here if necessary).
According to MSDN, "The data being passed must not contain pointers or other references to objects not accessible to the application receiving the data." so I suspect that this only possible with struct data. Am I correct?
you are both correct and incorrect.
your problem here is that you don't know the implementation details of std::string. unfortunately, it seems this (standard) class uses a dynamicaly allocated buffer to store its character data. that's why WM_COPYDATA doesn't work with it.
but if your class does not contain a pointer to any external data, as suggested in the documentation, then it would be perfectly valid to copy it using WM_COPYDATA. unfortunately, this greatly limits the possible types for members of your class.
(think WM_COPYDATA is like sending data over a network: you should take care of serializing your class before sending it out in the wild...)
Has anyone managed to do this? I tried making a managed wrapper class for IPropertyStore but am getting AccessViolationExceptions on the methods (i.e. IPropertyStore::GetValue) that take a pointer to PROPVARIANT (rendered as a MarshalAs(UnmanagedType.Struct) out parameter in my managed version) Probably my understanding of COM and interop is inadequate --- I'm not sure if the problems are in my PROPVARIANT struct declaration (which currently just uses StructLayout.Sequential, declares a sequence of bytes, and manually manipulates the bytes to get values of the various types in the union etc.), COM issues with what process owns what, or something else. I've tried various other versions of the PROPVARIANT such as using StructLayout.Explicit for the unions, nothing's worked. Retrieving PROPERTYKEYs with IPropertyStore::GetAt --- which is declared natively as taking a pointer to PROPERTYKEY and as having an out parameter of my own StructLayout.Sequential PROPERTYKEY in my wrapper --- works just fine, by the way.
You should check out http://code.msdn.microsoft.com/WindowsAPICodePack . It has support for consuming the Windows Property System, and a bunch of other windows shell capabilities. I think it's exactly what you are looking for.
Well, here's the version from MS.Internal.Interop (a trove of knowledge):
[StructLayout(LayoutKind.Sequential), FriendAccessAllowed]
internal struct PROPVARIANT
{
internal VARTYPE vt;
internal ushort wReserved1;
internal ushort wReserved2;
internal ushort wReserved3;
internal PropVariantUnion union;
}
[FriendAccessAllowed]
internal enum VARTYPE : short
{
VT_BSTR = 8,
VT_FILETIME = 0x40,
VT_LPSTR = 30,
// etc...
}
[StructLayout(LayoutKind.Explicit), FriendAccessAllowed]
internal struct PropVariantUnion
{
[FieldOffset(0)]
internal BLOB blob;
[FieldOffset(0)]
internal short boolVal;
// etc... see MS.Internal.Interop for full definition
}
These definitions will help you make sure your structures are at least correct. As for your other problems, I don't have an answer.