How to tell linux retrain and scan PCIe bus? - linux-kernel

We have an embedded board that has an iMX8M-Plus Processor and Linux v5.4.161. This board has one PCIe bus and that one is connected to an FPGA. When we power up the board, the FPGA is not yet configured, so it acts as if it was not on the PCIe bus.
Once the Linux is fully booted, we configure the FPGA and only after that it starts acting as a PCIe endpoint (device).
At this point, when I run lspci -> it returns nothing.
When I first execute echo "1" > /sys/bus/pci/rescan as suggested here and here and then lspci, I still get nothing.
But if I reboot the linux without reseting the FPGA, it starts being visible in the lspci list. Rebooting the linux is not an option for us. Somehow I need to tell the linux that whatever it's doing at the boot time, please do it again at runtime. But I couldn't find a solution for this so far.
According to the Texas Instrument support forum, they said if the PCIe link is not trained at the boot time, rescan command never works.
At the boot time, while linux loads a pci driver, it tries to establish a PCIe link, I can see that with an oscilloscope, PERST pin is asserted and PCIE_CLK generated for a while and then stops if it can not detect any device. But the rescan command never does that.
Also in the system there is no pcie device to executeecho 1 > $pcidevice/remove in order to make rescan functional. Or there is no device or bus to set power off and on back like echo 0 > /sys/bus/pci/slots/.../power
I also learned that there was a method in old linux times (v2.6) called adding a Fake PCIe Device which physically doesn't exist to solve this problem. For that I took the fakephp.c driver from an old linux repo and ported it to ours. After solving a couple of deprecated function problems, it is compiled for Linux Kernel v5.4. modprobe fakephp worked and driver loaded but somehow I didn't get this fake device in my device list. Here it is mentioned that the fakephp driver was removed from mainstream linux since PCI core has similar functionality, but he never mentioned how.
Short of the story is that, I am stuck here, I need my FPGA to be visible in the lspci list without restarting the linux.

I recommend configuring the FPGA in u-boot to get away from these kinds of problems. Connect up SPI pins to FPGA's config pins & run it in Slave configuration mode.

Related

Embedded Linux device blocking RS485 bus during startup

I'm having trouble with an industrial Linux computer I'm working with to achieve communication over an RS485 bus with multiple connected devices. What I've encountered is that the IO pins used by the RS485 USART driver are set to different levels at startup instead of going to the RS485 idle/tri-state. As a result, the other devices on the bus are blocked for more than 30 seconds while the device boots up, triggering all sorts of external problems. The course of events can be viewed in the attached image, where I've measured the output voltages with an oscilloscope during startup.
My guess is that the actual driver is not started until the voltage levels reach their tri-state levels (e.g. ~2.2V for this device). After that everything works as expected.
I've tried to find any config-files to set the default IO level of the pins at boot (thinking this may be set by the bootloader) to no avail.
Also, I've tried to apply a startup-script to run "early enough" to set DATA- high, but the device in question does not provide any interface to control these pins as regular GPIOs as far as I can tell.
Any help, tips or insights would be much appreciated!
EDIT: I am not an experienced Linux developer, so please highlight if I've left out any important details.
Some specs:
ARM920T rev 0 (v41) CPU
Proprietary distribution of Linux 2.6
Uses Busybox
Atmel USART drivers
Extract from boot log:
Linux version 2.6.28.10 (root#) (gcc version 4.1.2) #94 PREEMPT Tue Oct 29 10:22:19 CET 2013
CPU: ARM920T [41129200] revision 0 (ARMv4T), cr=c0003177
/...
.../
RS485 mode for port /dev/ttyS3 enabled
/...
... (I'm guessing the ~30 secs elapse here)
.../
atmel_usart.3: ttyS3 at MMIO 0xfffcc000 (irq = 9) is a ATMEL_SERIAL
atmel_serial.3: Putting the RS485 RTS pin down
/...
...
.../
Full boot log: https://drive.google.com/file/d/0B2XYl1mNCa8jNUZ5V0Nic1hkU0U/view
Similar issue:
Possibly a similar issue is discussed here: UART initialisation: Prevent UART to pull RTS high
But I'm not sure how to proceed with the suggested solution.
This is little more than wild speculation, but it might be worth adding a start-up script that echoes a NULL character to the device (e.g. /dev/ttyS1 or whatever) as early as possible during start-up. This might be enough to kick the driver into initialising the hardware.
You could also try to locate the driver in the Linux source to look at how it starts up.
Probably you have access to the source code, so you can investigate who and when mess with the that GPIO. Just grep the kernel source for the atmel gpio controller port addresses to figure out what happen. If you are lucky may be there will be kernel command line option that you can pass from the bootloader to set the line to what you need in advance.
this answer may work if you can find required things mentioned below
of your board!
Once I also had a same issue on PWM. There I found that my bootloader was responsible for the same, I changed in bootloader configuration and it started working fine.
Check your BSP provided by board vendor or third party (If you have the source), If your bootloader is U-boot you can find it inside U-boot-(source)/include/configs/(your-board).h there you can find configuration for RS484. As per your datasheet for the board you can check for other things which are muxed on the same pin and disable those if not required for boot time and enable RS485.
enabling/disabling can be done by changing the values 0, 1 or 2 as per your configuration and also you can simply disable anything just by commenting // out the line.

Detect spi device from another driver

I have a Freescale imx.6q (arm) based board.
Hardware is configured with devicetree.
It had a change major incompatible change to timings and voltage for an onboard fpga, but these changes are invisible to the kernel.
The EE's tell us we shouldn't load the old fpga firmware for fear of damaging it. I would like to support both hardware from the same code (It is already causing confusion)
The solution I have thought of is this:
There are several new spi temperature sensors on the board. If I can read from one of those devices, I can infer that I need the new firmware.
How can I (in one driver) grab an spi device and then release it?
I suspect that I might be able to do something like this with device tree,
But I don't want to make the device unavailable.
Any ideas or examples of something like this being done?
After reading question i think your concern is how to add software support for more than one hardware.
If that is the case i think we can write two drivers supporting both hardware's with different configuration such as irq, voltage, register set etc.
So i will enable both drivers in Makefile and config file.
So at the time of boot when probe of drivers gets called we can check the hardware id by using spi_read command from driver.
If hardware id matches then driver probe gets successful and driver can be used to interact with hardware.
If spi_read fails then driver probe itself will fail.
I think this will do the trick.
EDIT (answer the question)
To detect use an SPI device from another driver use a reference to the device in the devicetree structure.
Short answer: add a reference to the spi device in your devices dts entry.
Slightly longer answer:
When adding spi to another device driver, you are effectively adding a subdevice, which may want its own driver. I have an FPGA which loads its firmware over (something close enough to be considered) SPI. I started with the idea of just treating the spi device as part of the larger driver, but the more work went into it, the more obvious it was that it is its own device, with a purpose and function that is distinct from the rest of the driver. I separated that code into its own driver.
Now instead of a reference to an SPI device, my driver just has a reference to an FPGA Manager device.
See line 98, 370 of https://github.com/d4ddi0/linux/blob/v4.12evi/arch/arm/boot/dts/imx6q-evi.dts
and
make sure the spi driver is loaded before your driver completes loading
My original answer to my question (for historical purposes):
What I ended up doing was using different devicetree files. The difference is know at initial install time (based on the serial number). The bootloader knows which dts filename to load.
There are multiple FPGA firmware versions and the right one is chosen based on the description in the dts.
This way, I can still update the driver and/or dts without breakage.
This works well in practice even though it does not detect anything at runtime.
One problem still exists, if I take an SD card from a new revision, and put it into an old one, the incorrect firmware will be loaded. To really solve this last problem, we've talked about adding an EEPROM to uniquely identify the hardware revision on future boards.

USB bandwidth / host controller issues - Linux

I have 12 USB 2.0 devices plugged into an Intel NUC D54250WYK running Ubuntu 14.04.
Running lshw -short shows two different USB buses and two host controllers (xHCI and eHCI).
All of the USB devices appear on the same bus and use xHCI regardless of the ports they are plugged into. As a result I'm seeing the following errors in dmsg:
Not enough host controller resources for new device state.
Not enough bandwidth for altsetting 0.
Is there a way to force devices to a specific bus?
I've also read that Linux can have problems with xHCI. Is there a way to force eHCI without recompiling the kernal? Intel does not provide that option in BIOS.
Last I checked on this, you're in a bit of a bind. It seems xHCI is compiled into the kernel, not as a module, and if you compile in eHCI/aHCI/oHCI and not xHCI, USB as a whole breaks, possibly due to some built-in support for on-board USB controlled BlueTooth and WIFI devices on certain mobos. DO NOT UPDATE YOUR BIOS yet... see if the option to disable xHCI still exists on yours.
At this time, it seems your best option is to disable xHCI in your BIOS. This will likely disable all USB3 controllers, but allow USB2 controllers to work without this issue impeding you.
With respect to the Intel device you described, I don't see many USB ports on it, so I assume you're using hubs. From the tech specs for your device, it looks like you'll have to get access to the internal header to get at the USB2 ports.
Good news for anyone else facing this issue. Intel released a new bios (v40) that adds back the option to disables xHCI. In my case I updated the bios, disabled xHCI, and everything works as expected.
Beware of platforms that have XHCI ONLY (Apollolake, Denverton).
You will brick your HW if you disable XHCI there.

View linux kernel drivers built into the kernel, and how do they get binded/mounted/started

I'm having a bit of a hard time fully understanding how the kernel starts in linux. I'm a wince developer and our company decided to run with linux instead now.
We outsourced all of the board bringup and the package I recieved is quit a bit different for the prototype board we have compared to the nitrogen6x we have been using.
Before i start listing the differences for the distro we created, the kernels are identical. The distro we have been using is a busybox system. The one we recieved from the vendor is sysvinit. I removed mdev from busybox and we are only using udev.
when I use the kernel on our busybox build the touch screen drivers breaks, or doesn' run, or does something totally magical. I'm not quit sure... there is a /dev/input/event0 device which when run on the sysvinit side is a touch device.. Is the kernel not the mechanism that binds the built-in drivers to a device node? I thought udev was for more dynamic events in the system.
On the other hand I can't really tell whats been loaded on my device. Is there a way to list running drivers that were built into the kernel? my touch pad is up? This is a fairly simple process of looking at the registry on wince to see which devices were loaded.
I guess what I'm really trying to discover, isn't so much how to add a driver to the kernel, its how the whole thing gets is plumbed together. I've found plenty of documents on createing kernel modules, but i haven't found a good resource on how to pull everything together from scratch so you can actually use said modules. Going back to the example of the touchscreen driver, its built into the kernel, how does that get plugged into /dev/input/event0??
I'm kind of having a difficult time finding good resources mostly because searching google for varations of linux/drivers/device nodes/ piles in tons of random crap from everywhere.
What you probably want to use now is evtest. It will allow you to know what are the input devices that are present and ready to use on your system.
To get more information on the input subsystem and more generic information on how the kernel is working, I can direct you to our training materials. The materials are free to download, use and redistribute.
The general answer is, there is no single, easy place to look to discover what drivers have been loaded by the kernel if they are compiled in. Of course, lsmod will display any drivers that were dynamically loaded after kernel boot.
The kernel does not create device nodes. That is, to quote your question, the kernel does not "bind" the driver to the device node. The association between kernel driver and device node is contained in the major and minor numbers registered when the driver is initialized. You can have a device node on your file system for which there is no corresponding driver (common especially in older devices where device nodes were statically created on the file system) and you can also have a driver installed for which there is no device node.
Modern Linux distros have dynamically created device nodes created on a mount point called /dev and this is usually a tmpfs file system, meaning it is volatile - it gets destroyed on every boot and recreated dynamically on each new boot.
udev is the magic that creates most device nodes based on events that it receives from the kernel when a new device is discovered (this can be after boot on device plugin, like a USB disk) or on startup when udev reads the queued events and acts on them. As you noted, busybox has a limited udev implementation called mdev.
Study udev and you will get a much better understanding of the process. Hope this helps a little.

USB linux gadget zero driver communicate with Windows host

I need to set up USB communication between a Windows 7 host and a Linux device for data transfer. I was able to compile the Linux kernel on the device to include the Gadget Zero driver in the kernel (not as a loadable module - Linux version 3.0.15). My project has some requirements, which also explains why I chose Gadget Zero:
1) On the Windows 7 host, a kernel mode driver must be used to communicate over the USB connection for sending and receiving bulk data. (speed is not important, not a lot of data at once).
2) On the linux device, no requirements on USB side except send and receive data easily over USB link. The data received will eventually be "unmarshalled" to call functions in another kernel module (and those responses packaged and sent back to the host).
3) Multiple linux devices will be connected to the host, so need easy way to enumerate connected devices and communicate with them.
So due to the requirements, I decided against the Gadget Serial. I'm having serious issues sending and receiving data over the virtual COM port in kernel mode (KMDF) in Win 7 host. WinUSB does not seem to want to open my attached device (I'm using KMDF windows USB driver from template in VS2012) Also, the gadget serial driver on the linux side, I cannot find the functions where the data is received and sent. Plus, any received data on the linux device seems to be echoed back to the host for some reason. (and to test this, I wrote a simple user-mode app in Windows, which is a no-no for my project).
Gadget Zero, it appears much simpler on the linux side. I can plug the USB cable to the Win7 host, and I can get the device to appear in the device manager. However, again I am having problems with getting communication going over the link. Gadget Zero has 2 bulk endpoints, so this shouldn't be an issue. Surely, someone has made data communication possible between a Windows host and a linux device using Gadget Zero? With Gadget Zero, it should be easy to enumerate the connected linux devices and communicate with them.
The trick is to keep the Windows side communication in kernel mode. Can someone point me in the right direction perhaps with Gadget Zero, Windows 7 KMDF, and some sample source code? I have a hard time believing no one has done this before because my internet searches don't turn up much. (and mostly user-mode solutions with Gadget Serial).
Thanks!
So you're writing a Win32 driver in which you want to communicate with your linuxed usb? I haven't written much win32 kernel code, but I believe I've seen a huge section in the doc, saying something like "This is how you make usb drivers"... That'd be it. In other words, when in kernel mode you have access to the full kernel usb layer. You don't need an existing driver or whatnot.
On the linux side you can use the serial gadget, in a different run mode. Only the default run mode, registers it self as VCP. There exist a more basic mode:
modprobe g_serial use_acm=0
Give it your own vendor id and you'll be able to attach your very own custom win32 driver. The 'multiple linux devices' will be handled by Windows. (Multiple instances of your driver, will be initiated.)
The echo you're seeing btw, is most likely a terminal feature. (The terminal mode on uarts will echo.) You have to disable it, when connecting. And now that you're at it, you also have to disable the xon/xoff, esc chars etc. (Standard legacy rubbish.)
And another thing. I'm not sure the gadget zero actually sends the data onto the line. It's meant for testing the gadget framework. (I could be mistaken though.)
Anyway, you've prolly solved this issue years ago. I'd be nice to know what you came up with.

Resources