While reading up about IO in Ruby, moreover refreshing my own albeit limited knowledge about I/O in generally POSIX-friendly libc environments, I found a question here at Stack Overflow: poll() in Ruby? such that raises the question that was the object of research.
The responses had mentioned the availability of a select method in Ruby. However, it also raised a concern about using select under certain conditions on some operating systems - including Linux - e.g when there may be 1024 or more file descriptors open in the Ruby process.
In some of the responses to the question, poll() in Ruby?, it was suggested that if select was called in such an environment, that it could result in memory corruption in the application. While the concern might not be represented as being of such severity, in other documentation, and there may be a way to portably avoid calling select in such circumstances - as later reading has indicated - perhaps the question remains as to how to address this portably, for Ruby's select.
Reading more about it, I noticed that the "BUGS" section of the select(2) manual page, on Linux, provides what may represent an expansive discussion of the issue. The text mentions a constant, FD_SETSIZE as apparently representing the exclusive upper limit on the number of file descriptors that can be open at the time when select is called, such that select might be expected to perform normally then (roughly paraphrased).
Quoting the select(2) manual page:
POSIX allows an implementation to define an upper limit,
advertised via the constant FD_SETSIZE, on the range of file
descriptors that can be specified in a file descriptor set. The
Linux kernel imposes no fixed limit, but the glibc implementation
makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024,
and the FD_*() macros operating according to that limit. To
monitor file descriptors greater than 1023, use poll(2) or
epoll(7) instead.
The implementation of the fd_set arguments as value-result
arguments is a design error that is avoided in poll(2) and
epoll(7).
According to POSIX, select() should check all specified file
descriptors in the three file descriptor sets, up to the limit
nfds-1. However, the current implementation ignores any file
descriptor in these sets that is greater than the maximum file
descriptor number that the process currently has open. According
to POSIX, any such file descriptor that is specified in one of
the sets should result in the error EBADF.
Towards making use of this in Ruby, albeit in what may be a guess of an approach: What might be the best way to determine FD_SETSIZE for the Ruby environment?
If it was available as a constant, this assumes that the value of that constant could be used in a conditional test before calling 'select' on any open file descriptor. The Ruby program might then raise an exception internally, before calling select on any file descriptor equal to or greater than the value of FD_SETSIZE for the instance, at least for generally POSIX-friendly operating systems?
If there's no cleaner way to approach this, maybe it could be worked into the distribution tooling for a project, such as to determine that constant's value for the target architecture then to store it along with any other application constants? I'm guessing a cpp could be used for this - whether from GCC, LLVM, or any other toolchain - perhaps in some ways similar to sb-grovel.
Maybe there's some other way to approach this, and portably so? Perhaps there's already a constant for it, somewhere in Ruby?
Maybe there's already some checking about it, in the Ruby source code? I suppose it's time to look for that GitHub repository now....
Ruby does not export FD_SETSIZE in any way, so the only way to check the size is to use a compiler.
Instead of building your own extension, the least hassle-free way may be to use RubyInline, which makes the check very simple:
gem install RubyInline
Then:
require 'inline'
class FdTest
inline do |builder|
builder.c "
long fd_setsize(void) {
return FD_SETSIZE;
}"
end
end
puts FdTest.new.fd_setsize
=> 1024
This is even semi-portable to Windows, provided you are running under WSL, Cygwin, MinGW, or something similar. Might even work under Visual Studio, provided it is installed with C-support.
Building it as an extension might be another solution to ensure better compatibility, which you can then ship with precompiled binaries for your required platforms.
It all depends on how much trouble you are willing to go through in order to extract this information on all possible platforms, since there really does not exist a fully platform independent solution to something like this.
Related
I have an existing Linux device driver that exposes a basic char device to userland. (I am not its original author, but I'm trying to modify it.)
Currently it provides a maze of ioctl functions to do various things (though also wrapped in a handy library so most user code doesn't need to deal with the details of it).
One of the things that it does is to provide a sub-stream interface, where given a bunch of device-specific identifying information (including a string and some numeric ids) it can read or write (but not both at once) some data (up to a small number of MB) in a strictly sequential manner. Currently it does this with explicit ioctls.
I'm wondering if there is a way to leverage the existing file_operations infrastructure or similar to provide either a virtual filesystem or just an ioctl that can return a new already-open fd that can then be used with read/write/close (but not lseek) from userland as you'd normally expect?
The device does have a concept of a filename (that's the string) but it is not possible to enumerate existing valid filenames (only to try to open a specific filename and see if it gives an error or not), and the filename is not sufficient to open a stream by itself, which is why I'm currently leaning more towards the "special open" ioctl on the parent device rather than trying to expose things directly in some userland-visible fs that can be opened directly. (Also there's no concept of subdirs and only basic write-protect permissions, so a full fs seems like overkill anyway.) But I'm willing to be persuaded otherwise if there's a better way to do it.
I have written basic char drivers from scratch myself before, so I'm reasonably confident that I can get the read/write ops and other supporting things to work; I'm just not sure how to best handle that initial step of opening the handle.
I'm currently targeting kernel 3.2+.
Edit: The main reason that I think making an actual filesystem (or trying to expose it via procfs or sysfs) wouldn't work is that there's no way to populate a directory -- the only ops available are "open for read" and "open for write", and there's no way to tell which names are valid prior to the open attempt (the files are stored in external hardware and accessed via a protocol I cannot change). If I'm missing something and it is possible to support this sort of thing, that would be useful to know as well.
You can most certainly create a file system where readdir() is not implemented, but the open() method is. It's normally not done because it's not particularly user-friendly, but it certainly is doable.
You're targetting really ancient kernels if you're looking at 3.2 -- the upstream kernel developers aren't even bother to try to backport security fixes that far back, so I certainly wouldn't recommend shipping something as ancient as 3.2, but it's technically doable.
All you need to do is to implement lookup() method in the inode_operations structure for directories. You'll need to figure out some way of creating inodes with unique inode numbers, that contains private information so you can identify the subtream. The inode will have a file_operations structure that implements the read/write methods for reading and writing the substream.
You can try looking at a simple file system such as cramfs or minix to see how things are done.
xmalloc can be used in the process environment only when I write a AIX kernel extension.
what's the memory allocation functions can be called from the interrupt environment in AIX?
thanks.
The network memory allocation routines. Look in /usr/include/net/net_malloc.h. The lowest level is net_malloc and net_free.
I don't see much documentation in IBM's pubs nor the internet. There are a few examples in various header files.
There is public no prototype that I can find for these.
If you look in net_malloc.h, you will see MALLOC and NET_MALLOC macros defined that call it. Then if you grep in all the files under /usr/include, you will see uses of these macros. From these uses, you can deduce the arguments to the macros and thus deduce the arguments to net_malloc itself. I would make one routine that is a pass through to net_malloc that you controlled the interface to.
On your target system, do "netstat -m". The last bucket size you see will be the largest size you can call net_malloc with the M_NOWAIT flag. M_WAIT can be used only at process time and waits for netm to allocate more memory if necessary. M_NOWAIT returns with a 0 if there is not enough memory pinned. At interrupt time, you must use M_NOWAIT.
There is no real checking for the "type" but it is good to pick an appropriate type for debugging purposes later on. The netm output from kdb shows the type.
In a similar fashion, you can figure out how to call net_free.
Its sad IBM has chosen not to document this. An alternative to get this information officially is to pay for an "ISV" question. If you are doing serious AIX development, you want to become an ISV / Partner. It will save you lots of heart break. I don't know the cost but it is within reach of small companies and even individuals.
This book is nice to have too.
I am aware that this is nothing new and has been done several times. But I am looking for some reference implementation (or even just reference design) as a "best practices guide". We have a real-time embedded environment and the idea is to be able to use a "debug shell" in order to invoke some commands. Example: "SomeDevice print reg xyz" will request the SomeDevice sub-system to print the value of the register named xyz.
I have a small set of routines that is essentially made up of 3 functions and a lookup table:
a function that gathers a command line - it's simple; there's no command line history or anything, just the ability to backspace or press escape to discard the whole thing. But if I thought fancier editing capabilities were needed, it wouldn't be too hard to add them here.
a function that parses a line of text argc/argv style (see Parse string into argv/argc for some ideas on this)
a function that takes the first arg on the parsed command line and looks it up in a table of commands & function pointers to determine which function to call for the command, so the command handlers just need to match the prototype:
int command_handler( int argc, char* argv[]);
Then that function is called with the appropriate argc/argv parameters.
Actually, the lookup table also has pointers to basic help text for each command, and if the command is followed by '-?' or '/?' that bit of help text is displayed. Also, if 'help' is used for a command, the command table is dumped (possible only a subset if a parameter is passed to the 'help' command).
Sorry, I can't post the actual source - but it's pretty simple and straight forward to implement, and functional enough for pretty much all the command line handling needs I've had for embedded systems development.
You might bristle at this response, but many years ago we did something like this for a large-scale embedded telecom system using lex/yacc (nowadays I guess it would be flex/bison, this was literally 20 years ago).
Define your grammar, define ranges for parameters, etc... and then let lex/yacc generate the code.
There is a bit of a learning curve, as opposed to rolling a 1-off custom implementation, but then you can extend the grammar, add new commands & parameters, change ranges, etc... extremely quickly.
You could check out libcli. It emulates Cisco's CLI and apparently also includes a telnet server. That might be more than you are looking for, but it might still be useful as a reference.
If your needs are quite basic, a debug menu which accepts simple keystrokes, rather than a command shell, is one way of doing this.
For registers and RAM, you could have a sub-menu which just does a memory dump on demand.
Likewise, to enable or disable individual features, you can control them via keystrokes from the main menu or sub-menus.
One way of implementing this is via a simple state machine. Each screen has a corresponding state which waits for a keystroke, and then changes state and/or updates the screen as required.
vxWorks includes a command shell, that embeds the symbol table and implements a C expression evaluator so that you can call functions, evaluate expressions, and access global symbols at runtime. The expression evaluator supports integer and string constants.
When I worked on a project that migrated from vxWorks to embOS, I implemented the same functionality. Embedding the symbol table required a bit of gymnastics since it does not exist until after linking. I used a post-build step to parse the output of the GNU nm tool for create a symbol table as a separate load module. In an earlier version I did not embed the symbol table at all, but rather created a host-shell program that ran on the development host where the symbol table resided, and communicated with a debug stub on the target that could perform function calls to arbitrary addresses and read/write arbitrary memory. This approach is better suited to memory constrained devices, but you have to be careful that the symbol table you are using and the code on the target are for the same build. Again that was an idea I borrowed from vxWorks, which supports both teh target and host based shell with the same functionality. For the host shell vxWorks checksums the code to ensure the symbol table matches; in my case it was a manual (and error prone) process, which is why I implemented the embedded symbol table.
Although initially I only implemented memory read/write and function call capability I later added an expression evaluator based on the algorithm (but not the code) described here. Then after that I added simple scripting capabilities in the form of if-else, while, and procedure call constructs (using a very simple non-C syntax). So if you wanted new functionality or test, you could either write a new function, or create a script (if performance was not an issue), so the functions were rather like 'built-ins' to the scripting language.
To perform the arbitrary function calls, I used a function pointer typedef that took an arbitrarily large (24) number of arguments, then using the symbol table, you find the function address, cast it to the function pointer type, and pass it the real arguments, plus enough dummy arguments to make up the expected number and thus create a suitable (if wasteful) maintain stack frame.
On other systems I have implemented a Forth threaded interpreter, which is a very simple language to implement, but has a less than user friendly syntax perhaps. You could equally embed an existing solution such as Lua or Ch.
For a small lightweight thing you could use forth. Its easy to get going ( forth kernels are SMALL)
look at figForth, LINa and GnuForth.
Disclaimer: I don't Forth, but openboot and the PCI bus do, and I;ve used them and they work really well.
Alternative UI's
Deploy a web sever on your embedded device instead. Even serial will work with SLIP and the UI can be reasonably complex ( or even serve up a JAR and get really really complex.
If you really need a CLI, then you can point at a link and get a telnet.
One alternative is to use a very simple binary protocol to transfer the data you need, and then make a user interface on the PC, using e.g. Python or whatever is your favourite development tool.
The advantage is that it minimises the code in the embedded device, and shifts as much of it as possible to the PC side. That's good because:
It uses up less embedded code space—much of the code is on the PC instead.
In many cases it's easier to develop a given functionality on the PC, with the PC's greater tools and resources.
It gives you more interface options. You can use just a command line interface if you want. Or, you could go for a GUI, with graphs, data logging, whatever fancy stuff you might want.
It gives you flexibility. Embedded code is harder to upgrade than PC code. You can change and improve your PC-based tool whenever you want, without having to make any changes to the embedded device.
If you want to look at variables—If your PC tool is able to read the ELF file generated by the linker, then it can find out a variable's location from the symbol table. Even better, read the DWARF debug data and know the variable's type as well. Then all you need is a "read-memory" protocol message on the embedded device to get the data, and the PC does the decoding and displaying.
On POSIX systems rename(2) provides for an atomic rename operation, including overwriting of the destination file if it exists and if permissions allow.
Is there any way to get the same semantics on Windows? I know about MoveFileTransacted() on Vista and Server 2008, but I need this to support Win2k and up.
The key word here is atomic... the solution must not be able to fail in any way that leaves the operation in an inconsistent state.
I've seen a lot of people say this is impossible on win32, but I ask you, is it really?
Please provide reliable citations if possible.
See ReplaceFile() in Win32 (http://research.microsoft.com/pubs/64525/tr-2006-45.pdf)
Win32 does not guarantee atomic file meta data operations. I'd provide a citation, but there is none - that fact that there's no written or documented guarantee means as much.
You're going to have to write your own routines to support this. It's unfortunate, but you can't expect win32 to provide this level of service - it simply wasn't designed for it.
In Windows Vista and Windows Server 2008 an atomic move function has been added - MoveFileTransacted()
Unfortunately this doesn't help with older versions of Windows.
Interesting article here on MSDN.
Starting with Windows 10 1607, NTFS does support an atomic superseding rename operation. To do this call NtSetInformationFile(..., FileRenameInformationEx, ...) and specify the FILE_RENAME_POSIX_SEMANTICS flag.
Or equivalently in Win32 call SetFileInformationByHandle(..., FileRenameInfoEx, ...) and specify the FILE_RENAME_FLAG_POSIX_SEMANTICS flag.
you still have the rename() call on Windows, though I imagine the guarantees you want cannot be made without knowing the filesystem you're using - no guarantees if you're using FAT for instance.
However, you can use MoveFileEx and use the MOVEFILE_REPLACE_EXISTING
and MOVEFILE_WRITE_THROUGH options. The latter has this description in MSDN:
Setting this value guarantees that a
move performed as a copy and delete
operation is flushed to disk before
the function returns. The flush occurs
at the end of the copy operation.
I know that's not necessarily the same as a rename operation, but I think it might be the best guarantee you'll get - if it does that for a file move, it should for a simpler rename.
The MSDN documentation avoids clearly stating which APIs are atomic and which are not, but Niall Douglas states in his Cppcon 2015 talk that the only atomic function is
SetFileInformationByHandle
with FILE_RENAME_INFO.ReplaceIfExists set to true. It's available starting with Windows Vista / 2008 Server.
Niall is the author of a highly complicated LLFIO library and is an expert in file system race conditions so I believe if you're writing an algorithm where atomicity is crucial, better be safe than sorry and use the suggested function even though nothing in ReplaceFile's description states it's not atomic.
A fair number of answers but not the one I was expecting... I had the understanding (perhaps incorrectly) that MoveFile could be atomic provided that the proper stars aligned, flags were used, and file system was the same on the source as target. Otherwise, the operation would fall back to a [Copy->Delete]File.
Given that; I was also had the understanding that MoveFile -- when it is atomic -- was just setting the file information which also could be done here: setfileinfobyhandle.
Someone gave a talk called "Racing the Filesystem" which goes into some more depth about this. (about 2/3rds down they talk about atomic rename)
There is std::rename and starting with C++17 std::filesystem::rename.
It's unspecified what happens if destination exists with std::rename:
If new_filename exists, the behavior is implementation-defined.
POSIX rename, however, is required to replace existing files atomically:
This rename() function is equivalent for regular files to that defined
by the ISO C standard. Its inclusion here expands that definition to
include actions on directories and specifies behavior when the new
parameter names a file that already exists. That specification
requires that the action of the function be atomic.
Thankfully, std::filesystem::rename requires that it behaves just like POSIX:
Moves or renames the filesystem object identified by old_p to new_p as
if by the POSIX rename
However, when I tried to debug, it appears that std::filesystem::rename as implemented by VS2019 (as of March 2020) simply calls MoveFileEx, which isn't atomic in some cases.
So, possibly, when all bugs in its implementation are fixed, we'll see portable atomic std::filesystem::rename.
I am currently porting a self-written network application from C++ to Ruby. This network application often needs to manage around 10.000 sockets at the same time, that means it needs quick access to any sockets that have readable data available, incoming connections, etc.
I have already experienced while writing it in C++, that select() does not work for this case, because internally it uses 32 DWORDs (128 byte) to manage maximally 1024 sockets with bitmasks. Since I sometimes have to work with more than 10.000 sockets, this function did not suffice. Thus I had to switch to poll() which also made the code more elegant because I did not always have add and remove all the file-descriptors again.
As I can see from the Ruby documentation, Ruby offers IO.select(), which would basically be a wrapper for the C-API (as far as I know). Unfortunately it seems like there is no IO.poll(), which I would need for this particular application.
Does IO.select() have the same limitations as select() on WinSocks and Berkeley Sockets? If yes, is there a way to work around that?
Select cannot safely be used with programs that have more than 1024 file descriptors on a Linux system. This is because the underlying fd_set that the select system call uses is a fixed sized buffer i.e. its size is allocated at compile time, not run time.
From man 2 select:
int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
An fd_set is a fixed size buffer. Executing FD_CLR() or
FD_SET() with a value of fd that is negative or is equal to or
larger than FD_SETSIZE will result in undefined behavior.
Moreover, POSIX requires fd to be a valid file descriptor.
This means that if you have more than 1024 file descriptors in your program, and you use the select system call, you will end up with memory corruption.
If you want to use more than 1024 file descriptors in your program, you must use poll or epoll, and ensure that you never use select, or you will get random memory corruption. Changing the size of the file descriptor table through ulimit is very dangerous if you are using select. Don't do it.
Ruby's select does seem to be actually implemented with the select system call, so while it may look like increasing ulimit works, under the hood corruption is happening:
https://github.com/ruby/ruby/blob/trunk/thread.c
Furthermore, some unrelated API's in ruby seem to use select (see thread_pthread.c) so it's probably also unsafe to use those, or any code that uses those API's within a ruby program running with a file descriptor table larger than 1024.
The limitations on IO.select() and in fact the number of open connections you can have per process appear to be determined primarily by the underlying operating system support. Definitely no fixed 1024 socket limit.
For example, under WinXP, I max out at 69 socket opens (even before I get to select). I'm sure that is probably tunable, I just don't know how.
Under Linux, the limitation is the number of open files allowed. By default, the limit is usually 1024 (run ulimit -a to check).
However, you can easily change this e.g. ulimit -n 10000.
I just ran a test and happily went well over 1024 active sockets created with TCPSocket.new, and using IO.select to test for ready data.
NB: there is a good example of IO.select usage in this GServer article.
IO::Reactor may do what you need. It has a poll method that looks similar to what you're describing.