PCI-ISA bridge communication from linux kernel driver - linux-kernel

I have a custom linux kernel driver that communicates to an old ISA card (from an old single processor pc with a true ISA bus). I am trying to port this driver into a new system equipped with a PCI-ISA bridge.
The old driver was writing to I/O ISA ports with:
request_region(0x0280, 8, "foo"); //0x0280 is a jumper-configured address in ISA card hardware.
//Then lots of:
outw_p(val, 0x0280);
val = inw_p(0x0282);
... (ports in use 0x0280, 0x0282 and 0x0284)
I've tried the same code but the address mapping seems to not work anymore. Region request does not give errors, but I keep getting always 65535 from all inw_p reads (while in the old system with the same code the card was answering with meaningful data).
I can't find anywhere what to edit in this code to make it work with the bridge.
I've tried opening the bridge as a PCI device and getting its I/O port address with:
dev = pci_get_device(vid, id, NULL); //Called with hardcoded bridge ids from lspci
result = pci_enable_device(dev); //dev not null, no errors
result = pci_request_regions(dev, "foo"); //No errors
value = pci_resource_start(dev, bar); //value is always 0 with any bar value
Device is working as I can get its vendor id by using pci_read_config_word, but I get always 0 from any BAR value, and also lspci -vvvv gives me no address/region section:
04:08.0 ISA bridge: Integrated Technology Express, Inc. IT8888F/G PCI to ISA Bridge with SMB [Golden Gate] (rev 03)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Also, no configuration options seems to be available in BIOS for this bridge.
On the internet i found only a few infos about PCI - ISA bridges in general, so I'm asking: what is the procedure necessary to successfully communicate with an ISA card behind a PCI - ISA bridge from a custom linux kernel driver?

Ok, after a lot of trial and error I was able to find the problem. I will share my experience in hope for someone to find it useful (my pc has a HD620-H81 motherboard with IT8888F/G PCI-ISA bridge).
PCI-ISA bridge
My PCI-ISA bridge (and probably many others) is configured by default in "Subtractive Decoding" pci mode, which means "if no other pci device claim an address, then the bridge claims it".
You can check for subtractive decoding mode by running lspci -t and lscpi -vv, and you should see that every PCIe-PCI/PCI-PCI bridge in the tree leading to your PCI-ISA bridge is configured in subtractive decode mode (the ISA bridge itself won't appear as subtractive because from a PCI perspective the ISA bridge is just a PCI device and not a bridge).
That means that you don't have a BAR assigned to the bridge, nor you need to interact with the pci bridge device directly in any way. You can just access directly the absolute address you need and the bridge will take care of managing it correctly (a lot of information is already available on the internet for subtractive decoding anyway if you're interested in details).
To sum up: no action is needed regarding the bridge itself or pci devices, just make sure in the bios that no other device, such as serial or parallel ports, gets assigned the port/address range that you are interested into (if it does, either disable it or change its address).
I/O Ports
I found that my ISA card was located at a port which is 0x0800 ports above the one where it should be (and that was the main problem for me). I'm not sure about why, maybe my bridge adds a fixed offset (if you know the reason maybe comment it below!).
What I did to find out the correct address was to run a function that iterates all port addresses from 0x0100 (which skips the first zone where is better not to write random data into) to 0xFFFF and runs a card check routine at each single port address until it finds the correct answer I expected from the card.
int cardFound = 0;
int i;
for(i = 0x0100; i < 0xFFFF && !cardFound; i++)
{
if (request_region(i, SIZE) == NULL)
continue;
//Do card detection with ioport_map, iowrite16/ioread16, etc.
cardFound = do_custom_card_detection_procedure();
release_region(i, SIZE);
}
if (cardFound)
printk(KERN_INFO "Card found at port %d\n", i);
The downside of this approach is that you may do really bad things by randomly writing data at ports, so be careful and if you really want to try check /proc/ioport for bad ones before doing it (I messed up rendering for a quarter of the screen and I had to do a complete power reset by removing power cord and CMOS battery to bring it back to normal :D).

Related

What happens when we press a key on Windows?

First of all, I would say to you that I write this question from nothing because I have attempt to find good documentation but nothing stand out...
What happens when we squeeze a key?
I think this is complex but I hope you can help me.
What I search to know : all (but especially the program start on the host machine and how the key electric signal is encoded and send...)
The eXtensible Host Controller (xHC) has a Periodic Transfer Ring. Windows programs this ring to trigger a transfer every time an interval in milliseconds has passed. The right interval is specified in the USB descriptor returned by the USB device. When the transfer occurs, the xHC puts a Transfer Event TRB on the event ring and triggers an MSI-X interrupt which bypasses the IOAPIC as some kind of inter-processor interrupt. If Windows detects some change in the keys pressed, it will send a message to the application which currently has focus (calling the window's procedure) with the key pressed in one of the argument.
I don't know about electrical signals but I know the eXtensible Host Controller is the USB controller responsible to interact with USB on modern Windows systems. Since Windows nowadays requires an x64 processor, the xHC must be present on your motherboard. The xHC is a PCI-Express device which is compliant with the PCI-Express specification.
To find an xHC, you:
Find the RSDP ACPI table in RAM;
This table will be found by the UEFI firmware which acts as some kind of small operating-system (OS) during boot of the computer. Then, the OS developers will write a small UEFI application named bootx64.efi that they will place on a FAT32 partition on the hard-disk. They will place this app in the /boot/efi directory. The UEFI firmware will directly launch that application on boot of the computer which allows to have an OS which doesn't require user input to be launched (similarly to how it used to work with the legacy BIOS fetching the first sector of the hard-disk and executing the instructions found there).
The UEFI application is compiled in practice with either EDK2 or gnu-efi. These compilers are aware of the UEFI environment and specification. They thus compile the code to system calls that are present during boot and available for the UEFI application written by the OS developers. The System Tables (often the ACPI tables) are given as an argument to the "main" function (often called UefiMain) called by the UEFI firmware in the UEFI application. The code of the application can thus simply use these arguments to find the RSDP table and pass it to the OS.
Find the MCFG ACPI table using the RSDP;
The chain of table is RSDP -> XSDT -> MCFG. Once the OS found the MCFG, this table specifies the base address of the PCI configuration space. To interact with PCI devices you use memory mapped IO (MMIO). You write to some position in RAM and it will instead write to the registers of the PCI devices. The MCFG thus specifies the base address at which you will start finding MMIO registers for the different PCI devices that are plugged into the computer.
Iterate on the PCI devices and look at their IDs until you find an xHC.
To iterate on the PCI devices, the PCI convention specifies a formula which is the following:
UINT64 physical_address = base_address + ((bus - first_bus) << 20 | device << 15 | function << 12);
The base_address is for a specific segment group. Each segment group can have 256 buses (suitable for large servers or large computers with lots of components). There can be up to 65536 segment groups and each can have up to 256 PCI buses. Each PCI bus can have up to 32 devices plugged onto it and each device can have up to 8 functions. Each function can also be a PCI bridge. This is quite straightforward to understand because the terminology is clear. The bus here is an actual serial bus that the PCI devices (like a network card, a graphics card, an xHC, an AHCI, etc.) use to communicate with RAM. The function is a functionality of the PCI device like controlling USB devices, hard-disks, HDMI screens (for graphics cards), etc. The PCI bridge bridges a PCI bus to another PCI bus. It means you can have almost an infinite amount of devices with the PCI specification because the bridges allow to extend the tree of devices by adding other PCI host controllers.
Meanwhile, the bus is simply a number between 0 and 255. The first bus is specified in the MCFG ACPI table for a specific segment group. The device is a number between 0 and 31 and the function is a number between 0 and 7. This formula returns a physical address which points to a conventional configuration space (it is the same for all functions) which has specific registers. These registers are used to determine what is the type of device and to load a proper driver for it. Each function of each device thus gets a configuration space.
For the xHC, there will be only one function and the IDs returned by its configuration space will be 0x0C for the class ID and 0x03 for the subclass ID (https://wiki.osdev.org/EXtensible_Host_Controller_Interface).
Once you found an xHC, it gets rather complex. You need to initialize it and get the USB devices which are plugged in the computer at the current moment. You need to take several steps to get the xHC operational. For this part, I'll leave you to read the xHCI specification which (on chapter 4) specifies exactly the steps which need to be taken (https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf).
For the keyboard portion I'll leave you to read one of my answer on the stackexchange for computer science: https://cs.stackexchange.com/questions/141870/when-are-a-controllers-registers-loaded-and-ready-to-inform-an-i-o-operation/141918#141918.
Some good links:
https://wiki.osdev.org/Universal_Serial_Bus
https://wiki.osdev.org/PCI

How does PCIe Endpoint device memory is mapped into the systems memory map (MMIO)?

How does Linux Kernel or BIOS map the PCIe endpoint device memory into systems MMIO space ? Is there any API to achieve it ?
Lets assume that when writing a Linux device driver for a PCIe endpoint device, How can we map PCIe device memory into MMIO space ? Or Is it true that the device is already mapped into MMIO by BIOS during enumeration and what I would need to do it just remap the device MMIO into the kernel virtual address space using ioremap() ?
Platform : Linux on x86
There are two parts to this answer
Role of the BIOS
The BIOS (typically UEFI based) will do some sort of Depth-First Search (DFS) and enumerate all the children as PCIe is a self-enumerating bus. Since it has the view of the world (device, buses, processors) it will write an address to the BAR registers (could be BAR0 and or multiple of them). This will be the address the system will use and it will actually route these requests from the Host Agent (HA on x86/Intel platforms) to the Root Port to a PCIe switch all the way to the end point.
Each of these elements track what address ranges belong to themselves or one of their child devices (example a Switch may be the child of a Root Port)
Role of the Device Driver
The OS/Kernel will provide a toolkit of helper routines that the driver authors will use to access the device registers. Typically a driver may follow the folling routines
This is some sample driver pseudo-code, just to help illustrate the idea
1. pci_resource_flags(pdev, 0) & IORESOURCE_MEM
Check if a resource region is valid, here check for BAR 0
2. pci_request_regions(pdev, "region")
Take ownership of the resource/region
3. drv->registers = pci_iomap(pdev, 0, SIZE_YOU_WANT_TO_MAP)
This will give you kernel virtual address to device register mapping
Note : In case the BIOS does not enumerate, through Linux one can rescan the PCIe tree to see if a device can be seen or not.

Programmatically determine NUMA node or PCI bus, device, function number of Direct3D9Ex device

I'm looking for a way to programmatically determine which NUMA node a particular Direct3D9Ex display adapter is connected to so that I can allocate memory for host to device transfers on that node.
I can use IDirect3D9Ex::GetAdapterIdentifier() to obtain a DeviceName of the form "\.\DISPLAY1".
I can use EnumDisplayDevices() to enumerate the display devices and match this DeviceName to a DeviceKey of the form "\Registry\Machine\System\CurrentControlSet\Control\Video{62E184AC-2614-4AA8-844C-47454B92C142}\0000".
I can use SetupDiEnumDeviceInfo() to enumerate devices and SetupDiGetDeviceProperty() to query properties including DEVPKEY_NAME (e.g. "NVIDIA GeForce GTX TITAN X"), DEVPKEY_Device_LocationInfo (e.g. "PCI bus 2, device 0, function 0") and DEVPKEY_Device_Numa_Node (e.g. 0) but I can't find anything to link one of these devices to either the DeviceName or DeviceKey in EnumDisplayDevices().
I can do equivalent thing is CUDA because cudaDeviceGetPCIBusId() provides the PCI bus, device and function information but I also need to do it in Direct3D9Ex (when CUDA is unavailable).
The other thing I've noticed is that in the registry under "HKLM\SYSTEM\CurrentControlSet\Enum\PCI\VEN_10DE&DEV_17C2&SUBSYS_113210DE&REV_A1" there is a key for each of my display devices and each one contains another key called "Device Parameters" with a value called "VideoID" of the form "{62E184AC-2614-4AA8-844C-47454B92C142}". Unfortunately applications are not supposed to read this part of the registry directly and I don't know how to obtain the same information through the SETUPAPI.

PCIe driver error for enabling device and allocating memory

I'm using PCIe bus on Freescale MPC8308 (as root complex) and the endpoint device is an ASIC with just one 256 MB memory region and just one BAR register. The device configuration space registers are readily accessible through "pciutils" package. At first I tried to access memory region by using mmap() but it didn't work. So at the next level, I prepared a device driver for the PCIe endpoint device which is a kernel module that I load into kernel after Linux booting.
In my driver the endpoint device is identified from device ID table but when I want to enable the device by pci_enable_device(), I see this error:
driver-pci 0000:00:00.0: device not available because of BAR 0 [0x000000-0xfffffff] collisions
Also when I want to allocate memory region for PCIe device by using pci_request_region(), it is not possible.
Here is the part of driver code which is not working:
pci_enable_result = pci_enable_device (pdev);
if (pci_enable_result)
{
printk(KERN_INFO "PCI enable encountered a problem \n");
return pci_enable_result;
}
else
{
printk(KERN_INFO "PCI enable was succesfull \n");
}
And here is the result in "dmesg" :
driver-pci 0000:00:00.0: device not available because of BAR 0 [0x000000-0xfffffff] collisions
PCI enable encountered a problem
driver-pci: probe of 0000:00:00.0 failed with error -22
It is worth noting that in the driver I can read and write configuration registers correctly by using functions like pci_read_config_dword() and pci_write_config_dword().
What's the problem do you think? is it possible that the problem appears because the kernel initializes the device prior to kernel module? what should I do to prevent this to occur?
BAR registers access are generally for small region. Your BAR0 size seems to be too large. Try with less memory (less than 1MB), it should works.

Looking for an explanation of kernel driver I/O interface capability

I am looking at ways of interfacing to specific hardware I/O addresses from various Windows versions from 32-bit XP up 64-bit Win7 and beyond. There seem to be various solutions published with varying degrees of capability under different Windows versions and I am trying to understand the possibilities for creating my own kernel driver. The most basic kernal I/O R/W capability seems to be the direct I/O operations such as READ_PORT_UCHAR and WRITE_PORT_UCHAR (and their word and long derivatives). I have also seen the technique below which I dont understand, appearing to be some memory mapping capability of which I have no experience and can find little readable documentation. Could someone comment on the suitability / compatibility of READ_PORT_UCHAR / WRITE_PORT_UCHAR versus this mapping technique that I reproduce below please?
Thanks in advance.
case IOCTL_PHYMEM_MAP:
if (dwInBufLen==sizeof(PHYMEM_MEM) && dwOutBufLen==sizeof(PVOID))
{
PHYSICAL_ADDRESS phyAddr;
PVOID pvk, pvu;
phyAddr.QuadPart=(ULONGLONG)pMem->pvAddr;
//get mapped kernel address
pvk=MmMapIoSpace(phyAddr, pMem->dwSize, MmNonCached);
if (pvk)
{
//allocate mdl for the mapped kernel address
PMDL pMdl=IoAllocateMdl(pvk, pMem->dwSize, FALSE, FALSE, NULL);
if (pMdl)
{
PMAPINFO pMapInfo;
//build mdl and map to user space
MmBuildMdlForNonPagedPool(pMdl);
pvu=MmMapLockedPages(pMdl, UserMode);
//insert mapped infomation to list
pMapInfo=(PMAPINFO)ExAllocatePool(\
NonPagedPool, sizeof(MAPINFO));
pMapInfo->pMdl=pMdl;
pMapInfo->pvk=pvk;
pMapInfo->pvu=pvu;
pMapInfo->memSize=pMem->dwSize;
PushEntryList(&lstMapInfo, &pMapInfo->link);
DebugPrint("Map physical 0x%x to virtual 0x%x, size %u", \
pMem->pvAddr, pvu, pMem->dwSize);
RtlCopyMemory(pSysBuf, &pvu, sizeof(PVOID));
irp->IoStatus.Information=sizeof(PVOID);
}
else
{
//allocate mdl error, unmap the mapped physical memory
MmUnmapIoSpace(pvk, pMem->dwSize);
irp->IoStatus.Status=STATUS_INSUFFICIENT_RESOURCES;
}
}
else
irp->IoStatus.Status=STATUS_INSUFFICIENT_RESOURCES;
}
else
irp->IoStatus.Status=STATUS_INVALID_PARAMETER;
break;
What are these I/O ports that you're trying to access? It's generally a Really Bad Idea to go partying on ports that you don't own because you have no way of synchronizing access to those ports with the driver that owns them, the O/S, or the BIOS (it's possible to take an SMI and have the BIOS start talking to ports that it thinks it owns).
The code snippet provided is also a horribly bad idea and should be burned. Basically, all it's doing is mapping a kernel virtual address to a device register (MmMapIoSpace) and then doing the work to then map that device register into user mode (MmMapLockedPages). There are two obvious problems with it:
1) You don't know the caching attributes of the memory, so randomly specifying MmNonCached can hang the system
2) Same as with I/O ports, you can't just arbitrarily access a device's registers. You can't properly synchronize yourself with the driver that owns them, so you're doomed to eventually borking your system.
-scott

Resources