I have a register which needs to be accessed from more then one driver.
It is a global read-only register resides in FPGA space
The register address is exported via device tree.
The first call to "request_mem_region" is ok, but any consecutive call fails.
Is there a way to share a register between drivers ?
Linux Kernel release is 4.14 , using petalinux
Thanks,
Ran
You need to remap the memory region with something like ioremap() after you have requested it.
Then, as Tsyvarev and others mentioned, create and export a function in your "parent" driver that returns the mapped memory.
Here is some rough code:
void * mapped_mem;
void * map_addr(unsigned int phy_addr, char * name) {
struct resource * resource;
void * mapped_mem;
resource = request_mem_region(phy_addr, page_size * 4, name);
// check for errors
mapped_mem= ioremap_nocache(phy_addr, page_size * 4);
// check for errors
return mappedMem;
//handle errors
}
void * get_mapped_addr(void) {
return mapped_mem
}
EXPORT_SYMBOL( get_mapped_addr);
Now, mapped_mem should actually be tracked as part of your devices private info, but I figure thats beyond the scope of the question. Also, make sure to check for all possible errors. Make sure that request_mem_region() returns something >0 and not Null.
Related
I am trying to learn more about DriverKit and memory management, and I read this question:
How to allocate memory in a DriverKit system extension and map it to another process?
And I would like to understand how to use IOMemoryDescriptor::CreateMapping.
I wrote a little app to test this where I do (very simplified code):
uint8_t * buffer = new uint8_t[256];
for (int i = 0 ; i < 256 ; i++)
buffer[i] = 0xC6;
clientData in, out;
in.nbytes = 256;
in.pbuffer = buffer;
size_t sout = sizeof(out);
IOConnectCallStructMethod(connection, selector,&in,sizeof(in),&out,&sout);
// out.pbuffer now has new values in it
In my Kext user client class, I was doing (I am simplifying):
IOReturn UserClient::MyExtFunction(clientData * in, clientData * out, IOByteCount inSize, IOByteCount * outSize)
{
MyFunction(in->nBytes, in->pbuffer);//this will change the content of pbuffer
*out = *in;
}
IOReturn UserClient::MyFunction(SInt32 nBytesToRead,void* pUserBuffer,SInt32* nBytesRead)
{
PrepareBuffer(nBytesToRead,&pBuffer);
...
(call function that will fill pBuffer)
}
IOReturn UserClient::PrepareBuffer(UInt32 nBytes,void** pBuffer);
{
IOMemoryDescriptor * desc = IOMemoryDescriptor::withAddressRange((mach_vm_address_t)*pBuffer,nBytes,direction, owner task);
desc->prepare();
IOMemoryMap * map = desc->map();
*pBuffer = (void*)map->getVirtualAddress();
return kIOReturnSuccess;
}
This is what I don't know how to reproduce in a DExt and where I think I really don't understand the basic of CreateMapping.
Or is what I used to do not possible?
In my driver, this is where I don't know how to use CreateMapping and IOMemoryMap so this buffer can be mapped to a memory location and updated with different values.
I can create an IOBufferMemoryDescriptor but how do I tie it to the buffer from my application? I also don't understand the various options for CreateMapping.
Please note that in another test app I have successfully used IOConnectMapMemory64()/CopyClientMemoryForType() but I would like to learn specifically about CreateMapping().
(I hope it is alright I edited this question a lot... still new to StackOverflow)
Or is what I used to do not possible?
In short, no.
You're attempting to map arbitrary user process memory, which the client application did not explicitly mark as available to the driver using IOKit. This doesn't fit with Apple's ideas about safety, security, and sandboxing, so this sort of thing isn't available in DriverKit.
Obviously, kexts are all-powerful, so this was possible before, and indeed, I've used the technique myself in shipping drivers and then ran into trouble when porting said kexts to DriverKit.
The only ways to gain direct access to the client process's memory, as far as I'm aware, are:
By passing buffers >= 4097 bytes as struct input or output arguments to IOConnectCall…Method()s so they arrive as IOMemoryDescriptors in the driver. Note that these can be retained in the driver longer term, but at least for input structs, updates on the user space side won't be reflected on the driver side as a copy-on-write mapping is used. So they should be used purely for sending data in the intended direction.
By the user process mapping an existing IOMemoryDescriptor into its space using IOConnectMapMemory64()/CopyClientMemoryForType().
This does mean you can't use indirect data structures like the one you are using. You'll have to use "packed" structures, or indices into long-lasting shared buffers.
By "packed" structures I mean buffers containing a header struct such as your clientData which is followed in contiguous memory by further data, such as your buffer, referencing it by offset into this contiguous memory. The whole contiguous memory block can be passed as an input struct.
I have filed feedback with Apple requesting a more powerful mechanism for exchanging data between user clients and dexts; I have no idea if it will be implemented, but if such a facility would be useful, I recommend you do the same. (explaining what you'd like to use it for with examples) The more of us report it, the more likely it'll happen. (IOMemoryDescriptor::CreateSubMemoryDescriptor() was added after I filed a request for it; I won't claim I was the first to do so, or that Apple wasn't planning to add it prior to my suggestion, but they are actively improving the DriverKit APIs.)
Original answer before question was edited to be much more specific:
(Retained because it explains in general terms how buffer arguments to external methods are handled, which is likely helpful for future readers.)
Your question is a little vague, but let me see if I can work out what you did in your kext, vs what you're doing in your dext:
You're calling IOConnectCallStructMethod(connection, selector, buffer, 256, NULL, NULL); in your app. This means buffer is passed as a "struct input" argument to your external method.
Because your buffer is 256 bytes long, which is less than or equal to sizeof(io_struct_inband_t), the contents of the buffer is sent to the kernel in-band - in other words, it's copied at the time of the IOConnectCallStructMethod() call.
This means that in your kext's external method dispatch function, the struct input is passed via the structureInput/structureInputSize fields in the incoming IOExternalMethodArguments struct. structureInput is a pointer in the kernel context and can be dereferenced directly. The pointer is only valid during execution of your method dispatch, and can't be used once the method has returned synchronously.
If you need to use the buffer for device I/O, you may need to wrap it in an IOMemoryDescriptor. One way to do this is indeed via IOMemoryDescriptor::CreateMapping().
If the buffer was 4097 bytes or larger, it would be passed via the structureInputDescriptor IOMemoryDescriptor, which can either be passed along to device I/O directly, or memory-mapped for dereferencing in the kernel. This memory descriptor directly references the user process's memory.
DriverKit extensions are considerably more limited in what they can do, but external method arguments arrive in almost exactly the same way.
Small structs arrive via the IOUserClientMethodArguments' structureInput field, which points to an OSData object. You can access the content via the getBytesNoCopy()/getLength() methods.
If you need this data in an IOMemoryDescriptor for onward I/O, the only way I know of is to create an IOBufferMemoryDescriptor using either IOUSBHostDevice::CreateIOBuffer() or IOBufferMemoryDescriptor::Create and then copying the data from the OSData object into the buffer.
Large buffers are again already referenced via an IOMemoryDescriptor. You can pass this on to I/O functions, or map it into the driver's address space using CreateMapping()
namespace
{
/*
**********************************************************************************
** create a memory descriptor and map its address
**********************************************************************************
*/
IOReturn arcmsr_userclient_create_memory_descriptor_and_map_address(const void* address, size_t length, IOMemoryDescriptor** memory_descriptor)
{
IOBufferMemoryDescriptor *buffer_memory_descriptor = nullptr;
uint64_t buffer_address;
uint64_t len;
#if ARCMSR_DEBUG_IO_USER_CLIENT
arcmsr_debug_print("ArcMSRUserClient: *******************************************************\n");
arcmsr_debug_print("ArcMSRUserClient: ** IOUserClient IOMemoryDescriptor create_with_bytes \n");
arcmsr_debug_print("ArcMSRUserClient: *******************************************************\n");
#endif
if (!address || !memory_descriptor)
{
return kIOReturnBadArgument;
}
if (IOBufferMemoryDescriptor::Create(kIOMemoryDirectionInOut, length, 0, &buffer_memory_descriptor) != kIOReturnSuccess)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnError;
}
if (buffer_memory_descriptor->Map(0, 0, 0, 0, &buffer_address, &len) != kIOReturnSuccess)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnError;
}
if (length != len)
{
if (buffer_memory_descriptor)
{
OSSafeReleaseNULL(buffer_memory_descriptor);
}
return kIOReturnNoMemory;
}
memcpy(reinterpret_cast<void*>(buffer_address), address, length);
*memory_descriptor = buffer_memory_descriptor;
return kIOReturnSuccess;
}
} /* namespace */
i was trying to learn sysfs and was trying to write a simple sysfs directory. The code is as below
static struct kobject *example_kobject;
static int __init mymodule_init (void)
{
pr_debug("Module initialized successfully \n");
example_kobject = kobject_create_and_add("kobject_example",
kernel_kobj);
if(!example_kobject)
return -ENOMEM;
return 0;
}
static void __exit mymodule_exit (void)
{
pr_debug ("Module un initialized successfully \n");
// kobject_put(example_kobject); <-- Forgot to delete
}
module_init(mymodule_init);
module_exit(mymodule_exit);
As shown in the mymodule_exit, i had by mistake forgot to uncomment the code and then inserted and rmmod the module.
Now when i try to insert the module again, the initialization is failing as the entry is already present.
I know, it does not make sense to allow userspace to remove the entry that the kernel made. But, i was still wondering if there is any other way to remove the particular /sys/kernel/kobject_example entry other than rebooting the box.
Firstly, merely doing kobject_put() is not good enough, you must use kobject_del() instead. kobject_put() does not do complete clean-up. In your case, since kobject_example is a file (not dir), mere 'put' will still leave the entry in the parent dir (kset).
If you must, there is a away to remove such an entry without reboot, and that is by writing another module to do that. Here is what the module should be doing:
/* Find the kobj from the path and parent kset */
kobj = kset_find_obj(kernel_kobj->kset, "kobject_example");
...
/* check kobj is not null etc. */
...
/* Remove the sysfs entry */
kobject_del(kobj);
This will delete the sysfs entry. Reboot is easy, but this is nifty when your system does not have an option to go out of service.
I am working on a Linux kernel module that needs to modify network packets and append an extra header. I already implemented the modification part, recomputed the check-sums and it worked nice. But I don't know how to safely append an extra header. If my input packet is something like:
ip-header / tcp-header / data
I would like to have an output packet like:
ip-header / tcp-header / my-header / data
For what I read, I think I need something like the following code. I wrote my specific questions on the code as comments. My general concern is if the code I am writing here is memory-safe or what should I do to have a memory-safe way to append the new header. Also, if I am doing something wrong or there is a better way to do it I will also appreciate the comment. I have tried to find examples but no luck so far. Here is the code:
static unsigned int my_iptables_target(struct sk_buff *skb, const struct xt_action_param *par) {
const struct xt_mytarget_info *info = par->targinfo;
/* Some code ... */
if (!skb_make_writable(skb, skb->len)) {
//Drop the packet
return NF_DROP;
}
struct newheader* myheader;
// Check if there is enough space and do something about it
if (skb_headroom(skb) < sizeof(struct newheader)) {
// So there is no enugh space.
/* I don't know well what to put here. I read that a function called pskb_expand_head might
* do the job. I do not understand very well how it works, or why it might fail (return value
* different from zero). Does this code work:
*/
if (pskb_expand_head(skb, sizeof(struct newheader) - skb_headroom(skb), 0, GPF_ATOMIC) != 0) {
// What does it mean if the code reaches this point?
return NF_DROP;
}
}
// At this point, there should be enough space
skb_push(skb, sizeof(struct newheader));
/* I also think that skb_push() creates space at the beggining, to open space between the header and
* the body I guess I must move the network/transport headers up. Perhaps something like this:
*/
memcpy(skb->data, skb->data + sizeof(struct newheader), size_of_all_headers - sizeof(struct newheader));
// Then set myheader address and fill data.
myheader = skb->data + size_of_all_headers;
//Then just set the new header, and recompute checksums.
return XT_CONTINUE;
}
I assumed that the variable size_of_all_headers contains the size in bytes of the network and transport headers. I also think that memcpy copies bytes in increasing order, so that call shouldn't be a problem. So does the above code works? It is all memory-safe? Are there better ways to do it? Are there examples (or can you provide one) that does something like this?
I used a code similar to the one in the question and so far it has worked very well for all the test I have done. To answer some of the specific questions, I used something like:
if (skb_headroom(skb) < sizeof(struct newheader)) {
printk("I got here!\n");
if (pskb_expand_head(skb, sizeof(struct newheader) - skb_headroom(skb), 0, GPF_ATOMIC) != 0) {
printk("And also here\n");
return NF_DROP;
}
}
But none of the print statements ever executed. I suppose that happens because the OS reserves enough space in memory such that there can be no problems given the limits of the IP header. But I think it is better to leave that if statement to grow the packet if necessary.
The other difference of the code that I tested and worked is that instead of moving all the other headers up to create a space for my header, I chose to move the body of the packet down.
I am trying to implement device mapper target by referring to the already existing ones dm-linear, dm-snapshot, dm-cache etc. In my implementation, I need to perform a read/modify/write operation on a certain sector range. Since the device mapper directly talks to the block layer, I am not sure what data structures/functions to use to read the sectors in the memory, modify the buffer and write it back to another sector range.
At the application level, we have syscalls and below we have vfs_read/vfs_write. Is there anything similar for device mapper layer?
I have been stuck here for very long. Any help will be appreciated.
NOTE: My answer is related to kernel version < 3.14, because since 3.14 API is slightly changed.
In kernel you read/write certain sectors with struct bio. This struct is used for all block level I/O. Comprehensive documentation can be found in kernel and on lwn. These are the several most significant members of this structure:
bio->bi_sector - first sector of block I/O request
bio->bi_size - size of I/O request
bio->bi_bdev - device to read/write
bio->bi_end_io - callback that kernel will call on the end of request
What you do in device mapper target is map incoming bio. When you creating your device mapper target you supply at least 2 callbacks: ctr, and map. For example, the simplest device-mapper target dm-zero declares it's callbacks as following:
static struct target_type zero_target = {
.name = "zero",
.version = {1, 1, 0},
.module = THIS_MODULE,
.ctr = zero_ctr,
.map = zero_map,
};
map is a key callback - it's a heart of every device-mapper target. map receive incoming bio and it can do anything with it. For example, dm-linear just shift sector of every incoming bio by predefined offset. See the code:
static sector_t linear_map_sector(struct dm_target *ti, sector_t bi_sector)
{
struct linear_c *lc = ti->private;
return lc->start + dm_target_offset(ti, bi_sector);
}
static void linear_map_bio(struct dm_target *ti, struct bio *bio)
{
struct linear_c *lc = ti->private;
bio->bi_bdev = lc->dev->bdev;
if (bio_sectors(bio))
bio->bi_sector = linear_map_sector(ti, bio->bi_sector);
}
static int linear_map(struct dm_target *ti, struct bio *bio)
{
linear_map_bio(ti, bio);
return DM_MAPIO_REMAPPED;
}
Because map receives pointer to bio it can change value under that pointer and that's it.
That's how you map I/O requests. If you want to create your own requests then you must allocate bio, fill it's sector, device, size, end callback and add buffers to read into/write from. Basically, it's just a few steps:
Call to bio_alloc to allocate bio.
Set bio->bi_bdev, bio->bi_sector, bio->bi_size, bio->bi_end_io
Add pages via bio_add_page.
Call submit_bio.
Handle results and errors in bio->bi_end_io callback
Example can be found in dm-crypt target in crypt_alloc_buffer function.
I am using SOLR 4.4.0 - I found (possible) issue related to internal caching mechanism.
JVM: -Xmx=15g but 12g was never free.
I created heap dump and analyze it using MemoryAnyzer - I found 2 x 6Gb used as cache data.
In second time I do the same for -Xmx12g - I found 1 x 3.5Gb
It was always the same cache.
I check in source code and I found:
/** Expert: The cache used internally by sorting and range query classes. */
public static FieldCache DEFAULT = new FieldCacheImpl();
see http://grepcode.com/file/repo1.maven.org/maven2/org.apache.lucene/lucene-core/4.4.0/org/apache/lucene/search/FieldCache.java#FieldCache.0DEFAULT
This is very bad news because it is public static field and it is used in about 160 places in source code.
MemoryAnalyzer say:
One instance of "org.apache.lucene.search.FieldCacheImpl" loaded by
"org.apache.catalina.loader.WebappClassLoader # 0x58c3a9848" occupies
4,103,248,240 (80.37%) bytes. The memory is accumulated in one
instance of "java.util.HashMap$Entry[]" loaded by "".
Keywords java.util.HashMap$Entry[]
org.apache.catalina.loader.WebappClassLoader # 0x58c3a9848
org.apache.lucene.search.FieldCacheImpl
I do not know how to manage this kind of caches - any advice?
And finally I got OutOfMemoryError + 12Gb of memory is blocked.
I implemented kind of workaround:
I created this kind of class:
public class InternalApplicationCacheManager implements InternalApplicationCacheManagerMBean {
public synchronized int getInternalCacheSize() {
return FieldCache.DEFAULT.getCacheEntries().length;
}
public synchronized void purgeInternalCaches() {
FieldCache.DEFAULT.purgeAllCaches();
}
}
and registered it in JMX via org.apache.lucene.search.FieldCacheImpl
...
private synchronized void init() {
...
initBeans();
}
private void initBeans() {
try {
InternalApplicationCacheManager cacheManagerMBean = new InternalApplicationCacheManager();
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName name = new ObjectName("org.apache.lucene.search.jmx:type=InternalApplicationCacheManager");
mbs.registerMBean(cacheManagerMBean, name);
} catch (InstanceAlreadyExistsException e) {
...
}
}
...
This solution provide you invalidate internal caches - which solve partially this issue.
Unfortunately there are other places (mostly caches) where some data is stored and not removed as fast as I expect.
If you use FieldCacheRangeFilter you may wanna try range filters which work without field cache. If sorting is an issue, you may try using less sort fields or ones with a data type using less memory.
The field cache for each reader/atomic reader is thrown away when the reader is garbage collected. so a re-initialization of the reader should clear the cache which also means that the first operation using the cache will be a lot slower.
Fact is: FieldCache based range filter and sorting relies on the cache. There is no getting around when you really need those. You only can adapt your usage to minimize the memory consumption.