I'm new to programming Linux kernel modules, and many getting started guides on the topic include little information about how to build a kernel module which will run on many versions and CPU platforms of Linux. Most of the guides I've seen simply state things like, "Linux doesn't ensure any ABI/API compatibility between versions." However, other OSes do provide these guarantees for major versions, and the guides are mostly targeting 2.7 (which is a bit old now).
I was wondering if there is any kind of ABI/API compatibility now, or if there are any standard ways to deal with versioning other than isolating the kernel-dependent bits of my code into files with a ton of preprocessor directives. (Also, are there any standard preprocessor symbols I should be using in the second case?)
There isn't a stable ABI for the kernel and most likely never will be because it'd make Linux suck. The reasons for not having one are all pretty much documented in that link.
The best way to deal with this is to get your driver merged upstream where it'll be maintained by other kernel developers.
As to being cross-platform, that pretty much comes free with the Linux kernel as long as you only use the standard, platform-independent functions provided in the API.
Linux, the ying and the yang. Tangrs answer is good; it answers your question. However, there is the linux compat projects. See the backports wiki. Basically, there are libraries that provide shim functionality for newer Linux ABI's which you can use to link your code. The KERNEL_VERSION macro that Eugene notes is inspected in a compat.h, and appropriate compat-2.6.38.h, etc are included where each version has either macros and/or library functions to provide a forward API.
This lets the Linux Wifi group write code for the bleeding edge kernel, while still making it possible to compile on older kernel versions.
I guess this answers the question,
if there are any standard ways to deal with versioning?
The compat library is not a panacea, but at least it is there and under development.
Open source - There are many mutations. They all have a different plan.
Related
We are trying to optimize HPC applications using OpenMP on a new hardware platform. These applications need precise placement/pinning of their cores or performance falls in half. Currently, we provide the user a custom GOMP_CPU_AFFINITY map for each platform, but this is cumbersome, because it's different on each hardware version, and even platforms with different firmware versions sometimes change their CoreID physical mappings - all things impossible for the user to detect on the fly.
It would be a great help if HPC applications could simply set GOMP_PROC_BIND to "close" and OpenMP would do the right thing for the given platform - but to make this possible, the hardware vendor would need to define what "close" means for each machine. We'd like to do this, but we can't tell how/where OpenMP gets CoreID lists to use for things like close, spread, etc. (For various external requirements, the CoreID spatial pattern on this machine would appear utterly random to a software writer.)
Any advice as to where/how OpenMP defines the CoreID lists for OMP_PROC_BIND so we could configure them? We are comfortable with the idea that we might need a custom version of OpenMP (with altered source code) for this platform if needed.
Thanks, everyone. :)
Jeff
Expanding on what #VictorEijkhout said...
You seem have invented an envirable that I can't find anywhere with Google (GOMP_PROC_BIND), with the OpenMP standard envirable (OMP_PROC_BIND). If GOMP_PROC_BIND exists the name suggests that it is a GNU feature. Note too that one of the two Google hits for GOMP_PROC_BIND says "Code that reads the setting is buggy. Setting is invalid and ignored at runtime." So, if you are setting that it is unsurprising that it has no effect!
I will therefore answer for the more general case of OMP_PROC_BIND.
The binding of OpenMP threads to logicalCPUs clearly has to be done at runtime, since, beyond its ISA, the compiler has no knowledge of the hardware on which the compiled code will run. Therefore you need to be looking at the runtime library code.
I have not looked at GNU's libgomp, but, where it can, LLVM's libomp uses the hwloc library to explore the machine hardware. Since hwloc also includes other useful tools for machine exploration (such as lstopo) it is likely that your effort is best invested in ensuring good hwloc support on your machine, at which point there will be no need to delve inside the OpenMP runtime.
If I build a DLL with Rust language, does it require libgcc*.dll to be present on run time?
On one hand:
I've seen a post somewhere on the Internet, claiming that yes it does;
rustc.exe has libgcc_s_dw2-1.dll in its directory, and cargo.exe won't run without the dll when downloaded from the http://crates.io website;
On the other hand:
I've seen articles about building toy OS kernels in Rust, so they most certainly don't require libgcc dynamic library to be present.
So, I'm confused. What's the definite answer?
Rust provides two main toolchains for Windows: x86_64-pc-windows-gnu and x86_64-pc-windows-msvc.
The -gnu toolchain includes an msys environment and uses GCC's ld.exe to link object files. This toolchain requires libgcc*.dll to be present at runtime. The main advantage of this toolchain is that it allows you to link against other msys provided libraries which can make it easier to link with certain C\C++ libraries that are difficult to under the normal Windows environment.
The -msvc toolchain uses the standard, native Windows development tools (either a Windows SDK install or a Visual Studio install). This toolchain does not use libgcc*.dll at either compile or runtime. Since this toolchain uses the normal windows linker, you are free to link against any normal Windows native libraries.
If you need to target 32-bit Windows, i686- variants of both of these toolchains are available.
NOTE: below answer summarizes situation as of Sep'2014; I'm not aware if it's still current, or if things have changed to better or worse since then. But I strongly suspect things have changed, given that 2 years have already passed since then. It would be cool if somebody tried to ask steveklabnik about it again, then update below info, or write a new, fresher answer!
Quick & raw transcript of a Rust IRC chat with steveklabnik, who gave me a kind of answer:
Hi; I have a question: if I build a DLL with Rust, does it require libgcc*.dll to be present on run time? (on Windows)
I believe that if you use the standard library, then it does require it;
IIRC we depend on one symbol from it;
but I am unsure.
How can I avoid using the standard library, or those parts of it that do? (and/or do you know which symbol exactly?)
It involves #[no_std] at your crate root; I think the unsafe guide has more.
Running nm -D | grep gcc shows me __gc_personality_v0, and then there is this: What is __gxx_personality_v0 for?,
so it looks like our stack unwinding implementation depends on that.
I seem to recall I've seen some RFCs to the effect of splitting standard library, too; are there parts I can use without pulling libgcc in?
Yes, libcore doesn't require any of that.
You give up libstd.
Also, quoting parts of the unsafe guide:
The core library (libcore) has very few dependencies and is much more portable than the standard library (libstd) itself. Additionally, the core library has most of the necessary functionality for writing idiomatic and effective Rust code. (...)
Further libraries, such as liballoc, add functionality to libcore which make other platform-specific assumptions, but continue to be more portable than the standard library itself.
And fragment of the current docs for unwind module:
Currently Rust uses unwind runtime provided by libgcc.
(The transcript was edited slightly for readability. Still, I'll happily delete this answer if anyone provides something better formatted and more thorough!)
I am asking here because I have no idea where to find any information about this problem. If you could recommend me a book or an article about it, I would be pleased.
Where can I find any information about correlation between Linux kernel and GLIBC's version? I know that, the kernel itself contains implementation of libc's functions, but I do not know, how they are delivered to it.
For example:
Recently I had to build the kernel for an old PowerPC processor. It came with libc's dynamic library files in version 2.3.6 out-of-the-box. In /lib/ path there are files with names like librt.so-2.3.6.
What is the simplest way to update this lib to a newer version?
Is it possible to configure kernel's build system to make it generating uImage file with a newer GLIBC version or an alternative one (ex. EGLIBC)?
There is little correlation, the same kernel should work with a wide range of glibc versions, and viceversa. The library finds out what the kernel handles, and uses that. For the gory details of what has changed in glibc (this is what you interact with, including support for new kernel features), you should look at the upstream changelog. For new features in the Linux kernel, perhaps the best source are the periodical "What's new in..." articles the kernel section of LWN
If a particular piece of software is made to be run on one platform and the programmer/company/whatever wants to port it to the other, what exactly is done? I mean, do they just rewrite linux or windows-specific references to the equivalent in the other? Or is an entire rewrite necessary?
Just trying to understand what makes it so cost-prohibitive that so many major vendors don't port their software to Linux (specifically thinking about Adobe)
Thanks
this is the point of a cross-platform toolkit like qt or gtk, they provide a platform-agnostic API, which delegates to whichever platform the program is compiled for.
some companies don't use such a toolkit, and write their own (for whatever reason - could well be optimisation-related), meaning they can't just recompile their code for another os.
There are also libraries available that ease, at least on a specific area, the port of Windows API calls to Linux. See the Windows to Linux porting library.
In my experience, there are three main reasons why it's cost-prohibitive to take a large existing program on one platform and port it to another:
it has (not necessarily purposely) extensively used some library or API (often GUI, but there are also plenty of other things) that turns out not to exist on the other platform
it has unknowingly become riddled with dependency on nonstandard features or oddities of the compiler or other tools
it was written by somebody who didn't know that you had to use some oddball feature to get things to work on the other platform (like a Linux library that isn't sprinkled with the right __declspec directives you need for a good Windows DLL).
It's much easier to write a cross-platform app if you consider that a design goal from the start, and I have three specific recommendations:
Use Boost—oodles of handy things you might ordinarily get from platform-specific APIs and libraries, but by using Boost you get it cross-platform.
Do all your GUI programming using a cross-platform library. My favorite these days is Qt, but there are other worthy ones as well.
Build and test every day on both platforms, never provide an opportunity for the code to develop a dependency on only one platform and discover it only too late.
There are many reasons why it may be very difficult to port an application to a different platform, most often it is because some interfaces the application uses to communicate with the system are not available, and one either has to implement them on their own, port a library your application depends on, or rewrite the application, so that it uses alternative functions. Most languages today are very portable across hardware architectures and operating systems, but the problem is with libraries, system calls and potentially other interfaces the OS (or platform) provides. To be more specific:
Compilers may differ in their configuration and the standard functions they provide. On Windows the most popular compiler for C/C++ is Visual Studio, while on unix it is gcc and llvm (in combination with the standard library glibc or BSD libc). They expect different flags, different forms of declaration, produce different file format of executables and shared libraries. Even though C and C++ have standard libraries, they are implemented differently across platforms. There are some systems whose aim is to make compilation portable, such as Autotools, CMake and SCons.
On top of standard libraries there are additional functions OS provides. On Windows they are covered by win32 API, on unix systems these are part of the POSIX standard, with various GNU, BSD and Linux specific extensions, and there are also plain system calls (the lowest-level interface between applications and the operating system). POSIX functions are available on Windows via systems such as cygwin and mingw, win32 API function are available on unix via Wine. The problem is that these implementations are not complete, and often there are minor (but important) differences.
Communication with the desktop system (in order to make a GUI interface) is done differently. On Linux this might be the X Window System (together with freedesktop libraries) or Wayland, while Windows has its own systems. There are GUI libraries which try to provide an abstract interface for these, such as Qt, GTK, wxWidgets, EFL.
Other services the OS provides, such as network communication may be implemented differently. On Windows many applications use .NET libraries, for which there is only limited support on unix systems. Some unix applications rely on Linux-specific features such as systemd, /proc, KMS, cgroups, namespaces. This limits portability even among unix systems (Linux, BSD systems, Mac OS X, ...). Even .NET libraries are not very compatible across different versions, and they might not be available on an older version of Windows or on embedded systems. Android and iOS have different interfaces entirely.
Web applications are usually the most portable, but HTML5 is a live standard, and many interfaces may not be available yet in some browsers/web engines. This requires the use of polyfills, but it is usually much less painful than the situation with "native" applications.
Because of all of these limitations, porting can be a pretty hard work and sometimes it is easier to create a new application from scratch, either specifically for the other platform, using cross-platform abstraction libraries/platforms (such as Qt or Java), or as a web application (potentially bundled in something like Electron). It is a good idea to use these from the beginning, but many programmers choose not to because the applications tend to look and behave differently from "native" applications on the platform, and they might also be slower and more restricted in the way they interact with the OS.
Porting a piece of software that has not been made platform-independant upfront can be an enormous task. Often, the code is deeply ingrained with non-portable APIs, whether 3rd party or just OS libraries. If the 3rd party vendor does not provide the API for the platform you are porting on, you are pretty much forced into a full rewrite of that functionality, or finding another 3rd party that is portable. This only can be awfully costly.
Finally, porting software also means supporting it on another platform, which means hiring some specialists, and training support to answer more complex queries.
In the end, such a process can be very costly, for very little additional sales. Sadly, the decision is easy: concentrate on new functionality on your current platform that you know your customers are going to pay for.
If the software was written for a single OS, a major rework is likely. The first step is to move absolutely all platform-specific code into a single area of the code base; this area should have little or no app-specific stuff. Then rewrite this isolated portion of the code for the new target OS.
Of course, this glosses over some extremely major implications. For instance, if your first version targeted the Win32 API, then any GUI code will be heavily tied to Windows, and to maintain any hope of preserving your sanity, you will need to move all that code to a cross-platform GUI framework like Qt or GTK.
Under Mono, you can write a C# Winforms program that works on both platforms. But to make that possible, the Mono team had to write their own Winforms library that essentially duplicates all of the functions of Winforms. So there is still no free lunch.
Most software is portable to some extent. In the case of a C app - there will be a lot of #ifdefs in the area, apart from path changes, etc.
Rarely windows/linux version of the same software don't share a common codebase - this would actually mean that they only share a common name. It's always harder to maintain more codebases, but I think that the actual problem with porting applications has little to do with the technical side and a lot with business side. Linux has much fewer users that Windows/OSX, most of them expect everything to be free as in beer or simply hate commercial software on some religious grounds.
When you come to think about it - most open source software is multiplatform, no matter what language was used to implement it. This speaks for itself...
P.S. Disclaimer - I'm an avid supporter of Free and Open source software, I don't want to insult anybody - I just share my perspective on the topic.
I am on a Mac with Snow Leopard (10.6.3). I hear that the assembly language I work with has to be valid with the chipset that you use. I am completely new to this I have a basic background in C and Objective-C programming and an almost strong background in PHP. I have always wanted to see what assembly is all about.
The tutorial I'll be looking at is by VTC [link].
What I want to know is: are the tutorials that I'm about to do compatible with the assembly version on the Mac that I have?
I am completely new to this language although I do recall studying some of it way, way back in the day. I do have Xcode and what I'm wondering is what kind of document would I open in Xcode to work with assembly and does the Mac have a built in hex editor (when it comes time to needing it)?
The assembly language you use is not dependent on your OS but rather your CPU's instruction set. Judging by your Mac version, I'd say you are using an Intel processor - so you would want to learn x86 or amd64 assembly.
A good way to pick up assembly is to get yourself an embedded device to play with.
TI has some nice, inexpensive devkits to play with. I've poked around with the Chronos kit ($50) which has digital watch with a programmable MSP430 microcontroller with a wireless link to your computer. It's pretty sweet.
Update: I forgot to mention the Arduino. It's a pretty nifty open platform with tons of interesting peripherals and projects online.
An assembly language is instruction architecture specific. Chips are an instantiation of an instruction architecture.
In my opinion, you are best served by getting TextWrangler and directly compiling with gcc.
The file extension you are looking for is .s.
Assembly, for any processor, will be more or less the same in concept. However, the complexity varies between processors. From what I see in your site, you'd be doing x86 assembler, (x86 being the instruction set all consumer-line Intel processors use, which recent Macs and all PCs use) which can turn out to be fairly complex, but not overwhelming if you learn by steps.
XCode works with plain text files, I believe. Hex Fiend for your hex editing needs, if you come across them.
Do keep in mind, Assembly is extremely low-level. No ifs, whiles, or in fact any control loop save for "do operation and GOTO if results in (not) zero/equal" (unless your assembler provides them as syntactic sugar, which kind of beats the purpose, in my opinion). PHP knowledge will be at most tangentially useful. You C knowledge should serve you well, though.
The linked tutorials look like they use NASM, which is included with Macs. However, system calls are usually different on different platforms (they're very different between Mac and Linux), and without seeing the tutorials, it's hard to know whether they'll target different platforms (I'd guess not, though). A better bet might be to install SPIM and to learn MIPS assembly, which is more straightforward than x86 anyways.