NVIDIA has recently announced they are open-sourcing (a variant of) their GPU Linux kernel driver. They are not, however, open-sourcing the user-mode driver libraries (e.g. libcuda.so).
It's a gradual process and not all GPUs are supported initially, but regardless of these details: Is there some way that developers of user-space code can leverage this open-sourcing? Or is it only interesting/useful for kernel developers?
What I would personally love to be able to do is avoid having to make libcuda calls to get the current context. If that piece of information were somehow readable now from userspace, that could be neat. Of course that's just wishful thinking on my part - I don't know how to check what the driver directly "exposes" - if anything.
Related
I've seen that userspace version of ebpf (runtime, assembler, dissasembler) are being developped (uBPF, rbpf).
Why is having an userspace version of eBPF interesting ?
Do those alternatives focus on the same goal than the eBPF program types (network, observability and security) ?
One of the main advantages of eBPF is that it runs code in the kernel. Observability, in-kernel data aggregation, early packet processing: it all happens in kernel space. So the question sounds legit: Why were uBPF or rbpf created?
I think they were created mostly as prototypes. uBPF was introduced very early in eBPF history, and was probably a proof-of-concept implementation of an eBPF interpreter and x86_64 JIT in user space. I wrote rbpf, which is strongly based on uBPF, and the main objective for me was to get more familiar with two things: eBPF and Rust. Very little afterthought :).
I've always been curious to see what people could do with it. Truth to tell, there are not so many users. The biggest users for rbpf are probably the people from Solana, who implement some blockchain tool with smart contracts run in the eBPF machine. One other use case I've had in the past was to debug some eBPF bytecode, because it is easy to have breakpoints in an user space interpreter (by contrast, runtime debugging for regular kernel eBPF is quite limited at this time).
uBPF had more success and was used as a basis for other projects like DPDK as a filtering library or Oko, an extension to Open vSwitch (both about high-performance network processing). [Edit August 2021] More recently, it was chosen to serve as an eBPF runtime for the implementation of eBPF for Windows (references: announcement, my analysis).
As you can see, the interest of having these user space eBPF machines entirely depends on what you do with it. They're available to run eBPF programs, they don't have a specific focus by themselves, but they can help if you have a use case. In that regard, one of the particularities for uBPF and rbpf is their licenses (Apache, MIT): They are not under GPL, which means that you could reuse them with a larger number of projects, including proprietary ones. This is not the case with code from the kernel.
One big limitation for those user space eBPF machines is also that they tend to be quite out-of-date with regards to what happens in the kernel, where things evolve fast. They don't have a solid verifier, so you cannot assert security or safety of the programs. They hardly support eBPF maps if at all, they do not support function calls, or BTF, or even the latest eBPF instructions for that matter. Some of it could be added, but this would require some engineering efforts and time. [Edit August 2021] uBPF is getting a lot of activity now that Microsoft contributes to it for its eBPF implementation. They also use a user-space verifier, PREVAIL.
There was a discussion here that is now closed: https://stackoverflow.com/questions/25871579/what-is-the-difference-between-rtos-and-embedded-linux#:~:text=Many%20RTOS%20are%20not%20full,essentially%20the%20scheduling%20kernel%20only.&text=Critically%20Linux%20is%20not%20real%2Dtime%20capable
The accepted answer could be summarised as "Critically Linux is not real-time capable".
However, I think this is misleading. When it comes to an OS like linux, you have to distinguish between applications, the kernel, and device drivers.
According to my understanding (having written many Unix device drivers and worked on the Unix kernel decades ago), the device drivers and the kernel are real-time capable.
Device drivers respond to hardware events (interrupts) and the kernel, in turn, can perform actions based on these events.
So, yes, while it's true that certain real-time features are not available to applications, if you have the capability of writing a device driver, you can circumvent this limitation by moving your application logic into a specialised device driver or kernel module.
(Of course it has to make logical sense to do that...)
Have I missed something? Does anyone disagree with this?
Background: There are vulnerable kernel mode drivers for Windows systems, which can be loaded into the system for various purposes. Loaded kernel mode drivers leave traces in the system. Anti-cheat software for video games, for example, look for vulnerable driver traces in various parts of the system because they are used for cheating. The logic used by anti-cheat software could perhaps be (or were already) used by anti-rootkit tools or rootkits themselves.
I am wondering where traces are left after drivers are loaded and then unloaded. From my research, I found these two places in Windows NT kernel, where unloaded drivers leave traces:
PiDDBCacheTable
MmUnloadedDrivers
(Just to let you know, those are undocumented data structures)
Where else could they leave traces? Is it possible for me to learn it without reverse-engineering the Windows kernel by myself?
To my knowledge, ExpCovUnloadedDrivers is another function to detect unloaded drivers. But as far as I know, only drivers that use code coverage get added to this list.
I am just trying to understand the differences to patching into the kernel and writing a driver.
It is my understanding that a kernel mode driver can do anything the kernel can do, and is similar in some ways to a linux module.
Why then, were AV makers so upset when Microsoft stopped them from patching into the Windows kernel?
What kind of stuff can you do through kernel patching that you can't do through a driver?
In this context patching the kernel means modifying its (undocumented?) internal structures in order to achieve some functionality, typically hooking various functions (e.g. opening a file). You are not supposed to go messing around with internal kernel structures that do not belong to you. In the past Microsoft did not provide official hooks for some things, so security companies reverse engineered the internals and hooked the kernel directly. Recently Microsoft has provided official hooks for some things, so the need to hook the kernel directly is not as strong.
It's true that a kernel-mode driver can do anything the kernel can do - after all, they both run in ring 0. The key question here is: how difficult is it? Patching things relies on internal details that may change between different kernel releases. For example, the system call number of NtTerminateProcess will change between versions, so a driver which hooks the SSDT will break between versions (although the system call number can be obtained through other means). Reading or modifying fields of internal structures such as EPROCESS or ETHREAD is risky as well, because again, these structures change between versions. None of this is impossible for a driver to do, but it's hard.
If an official interface is provided for hooking, Microsoft can guarantee compatibility between versions as well as being able to control who can do what (e.g. only signed drivers can use the object manager callbacks). However, Microsoft can't do this for everything, because some things are just implementation details that drivers shouldn't know about.
I am very curious in messing up with HW. But my top level "messing" so far was linked or inline assembler in C program. If my understanding of CPU and ring mode is right, I cannot directly from user mode app access some low level CPU features, like disabling interrupts, or changing protected mode segments, so I must use system calls to do everything I want.
But, if I am right, drivers can run in ring mode 0. I actually don´t know much about drivers, but this is what I ask for. I just want to know, is learning how to write your own drivers and than call them the way I should go, to do what I wrote?
I know I could write whole new OS (at least to some point), but what I exactly want to do is acessing some low level features of HW from standart windows application. So, is driver the way to go?
Short answer: yes.
Long answer: Managing access to low-level hardware features is exactly the job of the OS kernel and if you only want access to a single feature there's no need to start your own OS from scratch. Most modern OSes, such as WIndows, Linux, or the BSDs, allow you to add code to the kernel through kernel modules.
When writing a kernel module (or device driver), you write code that is going to be executed inside the OS kernel and will thus be running in CPU ring 0. Great power comes with great responsibility, which in this case means that you should really know what you're doing as there will be no pre-configured OS interface to prevent you from doing the wrong things. You should therefore study the manuals of your hardware (e.g., Intel's x86 software developer's manuals, device specs, ...) as well as standard operating systems development literature (where you're also going to find plenty on the web -- OSDev, OSDever, OSR, Linux Device Drivers).
If you want to play with HW write some programs for 16-bit real-mode (or even with your own transition to protected-mode). There you have to deal with ASM, BIOS interrupts, segments, video memory and a lot of other low-level stuff.