On Unix-like operating systems, the file descriptors (file handles) used for things like pipes and sockets are small integers. It's common for a parent process to communicate with its child process by opening a pipe, socket or file at a particular file descriptor number. The number will either be agreed upon by the programmers of the parent and child program, or passed to the child program via the command line or environment variables. The child can then access the inherited file descriptor using the same number as the parent.
Is there a convention for passing handle numbers in a similar way on Windows? As far as I can tell, the relevant MSDN articles only talk about such conventions for the "standard" file handles (standard input, standard output, standard error). What would be a robust and idiomatic way to do something like "pass a pipe on file descriptor 3" as we sometimes do on Unix?
The MSDN article on Inheritance says (emphasis mine):
An inherited handle refers to the same object in the child process as it does in the parent process. It also has the same value and access privileges.
Does this mean that I can just cast the HANDLE value to an integer, convert that integer into a string and pass that string as a command line argument for the child process to parse and get the same handle back, and is this reliable or customary? Is there some other IPC mechanism that Windows programmers would normally use instead?
The main motivation for me would be to avoid temporary files and use pipes instead. This is conventional when all you have is stdin/stdout but less so when you have multiple pipes.
Related
Doing some tests in scm (a scheme interpreter), I've intentionally closed the current-input-port (equivalent to the standard input file descriptor). Once the program work in REPL, the things got crazy, printing systematically a error message. My question is: how could I recover the control of process, that means, how could I reestablish the input file descriptor of such process?
Search for "changing file descriptor of a running process" or something similar, I couldn't find a helpful article.
Thanks in advance
System information: Debian 10.
You almost certainly can't, although this does slightly depend on how the language-level ports are mapped to the underlying OS-level I/O system.
If what you do is close the OS-level standard input then all is lost:
the REPL tries to read from standard input, gets an error as it's closed;
it tries to raise some error which will involve prompting the user for input ...
... from standard input, which is closed, so it gets error;
game over.
The only way to survive this is for one of two things to be true:
either you've wrapped an error handler around the code which is already prepared to deal with this;
or the implementation is smart enough to recognise that it's getting closed-port errors in its closed-port error handler and gives up in some smart way.
Basically once the OS level standard input is gone anything that needs to get input from it is doomed: you can't put it back without OS-level surgery on the process.
However it's possible that the implementation maps a single OS-level I/O stream to multiple language-level streams, and closing only one of these streams would leave the system with some other stream-of-last-resort to which it can still talk, and which still refers to the OS-level standard input. Common Lisp is an example of a system which can (depending on configuration) do this. It has, for instance, *standard-input* *error-output*, *query-io*, *terminal-io* and other streams, and it's very possible to be in a situation where, for instance, *standard-input* has been closed causing read errors, but *query-io* still points somewhere with a human on the end of it.
I don't know if scm does that.
As I understand, on Windows, the process that creates a named pipe, owns the "server" handle (one side of the pipe).
Is it possible to split pipe creation from actually using it (opening it for reading and writing)?
I need to create a named pipe with one process, but use two other processes to open it for reading and writing.
But I'm concerned, that when the process that creates the pipe exits, the pipe gets removed from the file system.
Named pipes exist solely because of the sort of scenario you're talking about.
With anonymous pipes (see CreatePipe), it is necessary that there be a "factory" somewhere that creates the pipe, and then at least one of the handles is handed off to a "client" somewhere (within the same process or not).
With named pipes (see CreateNamedPipe), it is not necessary to have the "factory" which creates and distributes the handles. Instead, the "server" side of the named pipe directly does a CreateNamedPipe with the correct pipe name, and the "client" side of the named pipe directly uses CreateFile with the pipe name to open the client side of the named pipe. Each side deals with its error conditions appropriately. If implemented correctly, no additional coordination is necessary.
If your application requires a "factory" process for the named pipe (it shouldn't), then you'll have to figure out your own way to provide the handles to the "client" processes in such a way that the pipe is not destroyed in the interim. Or you can use the pipe name instead of distributing the handles, but you may have to explicitly coordinate between the processes using something like a named event (see CreateEvent).
If this doesn't answer your question, please update your question with more details.
I am working on making a program that will act in a similar way as a shell, but supports only foreground processes and pipes. I have multiple processes writing to the same pipe and some other properties that differ from the normal usage of pipes. Anyhow, my question is,
Is there any easy (automatic) way to close all file descriptors of a process except the three basic ones?
I am asking this question since I have a lot of difficulties keeping track of all file descriptors for every process. And sometimes they act in some unpredictable ways to me. It could be also because of the fact that I don't have a very thorough understanding of them.
Is there any easy way(automatic) to close all file descriptors of a process except the three basic ones?
The normal way to do this is to simply iterate over all of them and close them:
for (i = getdtablesize(); i > 3;) close(--i);
That's already a one-liner. It doesn't get any more "automatic" than that.
I am asking this question since I have a lot of difficulty keeping track of all file descriptors for every process.
It will be worth your time to think about the life cycle of each file descriptor you open, when it gets duplicated (e.g. dup2() and fork()), how it gets used, and make sure you account for how each one is going to get closed when it is no longer needed. Papering over a problem of leaked file descriptors by indiscriminately closing them all is not going to be sustainable.
I have multiple processes writing to the same pipe
If you do this, then you need to be aware that the order in which data arrive at the other end of the pipe is going to be unpredictable. It will be difficult to avoid corrupting the data stream.
Use the closefrom(3) C library function.
From the manpage:
The closefrom() system call deletes all open file descriptors greater
than or equal to lowfd from the per-process object reference table.
Any errors encountered while closing file descriptors are ignored.
Example usage:
#include <unistd.h>
int main() {
// Close everything except stdin, stdout and stderr
closefrom(3); // Were 3 is the lowest file descriptor you wish to close
printf("Clear of all, but the three basic file descriptors!\n");
return 0;
}
This works in most unices, but requires the libbsd support library for Linux.
Could somebody explain I/O to me? From everything I'm gathering, it can be summed up, abstractly, as the way computers interact with humans and vice versa. The I/O channel, or the "how", can run the gamut depending on external devices and/or internal OS management.
So what does the IO class in Ruby do? And how is it different from that of Java or C?
And take this code for instance:
x = IO.sysopen("file_name")
p x
The return is a Fixnum based on the file descriptor. In this case, the "file_name" is a pdf file and return a 7. What does the return object mean?
First of all, sysopen is a very low-level way of interacting with the system. For normal input and output in Ruby, you should use File.open instead.
The number returned by sysopen is called a "file descriptor". It's essentially an index into an array, but not a Ruby array; it lives inside the part of a process's memory which is maintained by the operating system. The first file descriptor, number 0, is called "standard input". Input calls will read from this input stream by default. The second, 1, is called "standard output"; output calls send their output there by default. And the third, 2, is called "standard error", which is where error messages go. All three of those are opened by the operating system before Ruby even starts. Normally they're all tied to the terminal, but you can change that with shell redirection.
As a general rule, when you open an extra file, the first one you open will get file descriptor 3, the next 4, and so on. So if you get a 7 back, that just means that Ruby has opened 4 other files by the time it gets to your code. And that's all it means. You can't tell anything else about an open file just based on the number. You have to hand that number off to a system call which can go look at the file descriptor array to see what's up.
But in Ruby, you usually have no reason to know or care about file descriptor numbers. You deal with instances of the IO class (and its subclasses like File for specific types of I/O). You call methods on the IO objects, and they handle the details of the system calls for you. The object referred to by the predefined constant STDIN (which is also the initial value of the global variable $stdin) knows that its file descriptor is 0, so you don't have to know that.
I need to send data from child processes to parent. Some of this data is HTML, plain text, etc. but it may also be necessary send image data, zip file data, etc.
As I understand it, anonymous pipes use the child process standard input and standard output. Conventionally stdin and stdout only convey textual data: would there be any problem with sending non printable characters using this mechanism?
There is no relation between anonymous pipes and stdin/out. As one process has only one stdin/out, you could create only one anonymous pipe per process that way, which sounds stupid, doesn't it? You can redirect stdin/out of a child process to the pipe, yes. But you don't have to, if the child process is able to report itself by another means (like logfile or network activity). A call to CreatePipe gives you reading and writing handles and it's up to you how you use them. Sending arbitrary binary data is indeed possible. Anonymous pipe is in no way different from named pipe in that respect.
Even if you do choose to use stdin/stdout redirection to pass the pipe handle(s) to the child process, you shouldn't have any problems provided the child process uses the Windows API to send the data rather than the C runtime library functions.
That is, WriteFile will work perfectly, but printf would not be a good idea.
You can use GetStdHandle to get the handle(s) to the pipe(s) for use with the Windows API functions.