I am looking for a way to dump input into my terminal from a file, but when EOF is reached I would like input returned back to my keyboard. Is there a way to do this with Bash (or any other commonly-available *nix shell)?
Details:
I am debugging a server program which executes a fork to start a child process. Every time I start a debugging session with gdb I have to type set follow-fork-mode child. I would like to use some sort of input redirection to have this pre-populated. There are other uses as well that I can think of, so I'd prefer a general solution - hence the reason this question is not about gdb.
Solution:
start-server.sh
#!/bin/bash
cat run-server.txt - |/bin/bash
run-server.txt
gdb ./Server
set follow-fork-mode child
run
You can do this:
cat input_file - | program
That will concatenate input_file followed by stdin to program, which I think is what you want.
maybe expect is what you want
Maybe use an intermediate file? Assuming you want to run the script myscript.sh:
INPUT_FILE=input.txt
TEMP_FILE=`mktemp -t input`
myscript.sh < $TEMP_FILE &
cat $INPUT_FILE >> $TEMP_FILE
cat >> $TEMP_FILE
Related
I have a few commands i run between brackets which i then redirect to a named pipe and tail the pipe however it looks like the redirection happens only after the block has finished executing as i don't see any output from the tail command for a while and it only shows the last command ouput when i do. Any ideas how view the output of the block in realtime?
Example Script
#!/usr/bin/env bash
mkfifo /tmp/why_you_no_out;
trap "rm /tmp/why_you_no_out" 0;
{
for ((i=1;i<=100;i++)); do
printf "$i";
done
sleep 10s;
printf "\n12356";
} >> /tmp/why_you_no_out &
printf "here";
tail -n 1 -f /tmp/why_you_no_out
Sounds like the issue is buffering. Most shells don't want to write data a byte at a time because it's wasteful. Instead, they wait until they have a sizable chunk of data before committing it unless the output is connected to your terminal.
If you're looking to unbuffer the output of an arbitrary command, you may find the "unbuffer" utility helpful or any of the solutions mentioned in this question: How to make output of any shell command unbuffered?
If you're dealing with specific applications, they may have options to reduce buffering. For example, GNU's grep includes the --line-buffered option.
I have a program that can output its results only to files, with -o option. This time I need to output it to console, i.e. stdout. Here is my first try:
myprog -o /dev/stdout input_file
But it says:
/dev/ not writable
I've found this question that's similar to mine, but /dev/stdout is obviously not going to work without some additional magic.
Q: How to redirect output from file to stdout?
P.S. Conventional methods without any specialized software are preferable.
Many tools interpret a - as stdin/stdout depending on the context of its usage. Though, this is not part of the shell and therefore depends on the program used.
In your case the following could solve your problem:
myprog -o - input_file
If the program can only write to a file, then you could use a named pipe:
pipename=/tmp/mypipe.$$
mkfifo "$pipename"
./myprog -o "$pipename" &
while read line
do
echo "output from myprog: $line"
done < "$pipename"
rm "$pipename"
First we create the pipe, we put it into /tmp to keep it out of the way of backup programs. The $$ is our PID, and makes the name unique at runtime.
We run the program in background, and it should block trying to write to the pipe. Some programs use a technique called "memory mapping" in which case this will fail, because a pipe cannot be memory mapped (a good program would check for this).
Then we read the pipe in the script as we would any other file.
Finally we delete the pipe.
You can cat the contents of the file written by myprog.
myprog -o tmpfile input_file && cat tmpfile
This would have the described effect -- allowing you to pipe the output of myprog to some subsequent command -- although it is a different approach than you had envisioned.
In the circumstance that the output of myprog (perhaps more aptly notmyprog) is too big to write to disk, this approach would not be good.
A solution that cleans up the temp file in the same line and still pipes the contents out at the end would be this
myprog -o tmpfile input_file && contents=`cat tmpfile` && rm tmpfile && echo "$contents"
Which stores the contents of the file in a variable so that it may be accessed after deleting the file. Note the quotes in the argument of the echo command. These are important to preserve newlines in the file contents.
I have a compiled program which i run from the shell; as i run it, it asks me for an input file in stdin. I want to run that program in a bash loop, with predefined input file, such as
for i in $(seq 100); do
input.txt | ./myscript
done
but of course this won't work. How can I achieve that? I cannot edit the source code.
Try
for i in $(seq 100); do
./myscript < input.txt
done
Pipes (|) are inter-process. That is, they stream between processes. What you're looking for is file redirection (e.g. <, > etc.)
Redirection simply means capturing output from a file, command,
program, script, or even code block within a script and sending it as
input to another file, command, program, or script.
You may see cat used for this e.g. cat file | mycommand. Given the above, this usage is redundant and often the winner of a 'Useless use of cat' award.
You can use:
./myscript < input.txt
to send content of input.txt on stdin of myscript
Based on your comments, it looks like myscript prompts for a file name and you want to always respond with input.txt. Did you try this?
for i in $(seq 100); do
echo input.txt | ./myscript
done
You might want to just try this first:
echo input.txt | ./myscript
just in case.
I need to write the time taken to execute this command in a txt file:
time ./program.exe
How can I do in bash script?
I try with >> time.txt but that doesn't work (the output does not go to file and does go to the screen).
Getting time in bash to write to a file is hard work. It is a bash built-in command. (On Mac OS X, there's an external command, /usr/bin/time, that does a similar job but with a different output format and less recalcitrance.)
You need to use:
(time ./program.exe) 2> time.txt
It writes to standard error (hence the 2> notation). However, if you don't use the sub-shell (the parentheses), it doesn't work; the output still comes to the screen.
Alternatively, and without a sub-shell, you can use:
{ time ./program.exe; } 2> time.txt
Note the space after the open brace and the semi-colon; both are necessary on a single line. The braces must appear where a command could appear, and must be standalone symbols. (If you struggle hard enough, you'll come up with ...;}|something or ...;}2>&1. Both of these identify the brace as a standalone symbol, though. If you try ...;}xyz, the shell will (probably) fail to find a command called }xyz, though.)
I need to run more command in more terminal. If I do this:
xterm -xrm '*hold: true' -e (time ./Program.exe) >> time.exe & sleep 2
it doesn't work and tells me Syntax error: "(" unexpected. How do I fix this?
You would need to do something like:
xterm -xrm '*hold: true' -e sh -c "(time ./Program.exe) 2> time.txt & sleep 2"
The key change is to run the shell with the script coming from the argument to the -c option; you can replace sh with /bin/bash or an equivalent name. That should get around any 'Syntax error' issues. I'm not quite sure what triggers that error, though, so there may be a simpler and better way to deal with it. It's also conceivable that xterm's -e option only takes a single string argument, in which case, I suppose you'd use:
xterm -xrm '*hold: true' -e 'sh -c "(time ./Program.exe) 2> time.txt & sleep 2"'
You can manual bash xterm as well as I can.
I'm not sure why you run the timed program in background mode, but that's your problem, not mine. Similarly, the sleep 2 is not obviously necessary if the hold: true keeps the terminal open.
time_elapsed=(time sh -c "./program.exe") 2>&1 | grep "real" | awk '{print $(NF)}'
echo time_elapsed > file.txt
This command should give you the exact time consumed in bash in a desired file..
You can also redirect this to a file usng 2 > file.txt as explained in another reply.
It's not easy to redirect the output of the bash builtin time.
One solution is to use the external time program:
/bin/time --append -o time.txt ./program.exe
(on most systems it's a GNU program, so use info time rather than man to get its documentation).
Just enclose the command to time in a { .. }:
{ time ./program.exe; } 2>&1
Of course, the output of builtin time goes to stderr, thus the needed redirection 2>&1.
Then, it may appear to be tricky to capture the output, let's use a second { .. } to read the command more easily, this works:
{ { time ./program.exe; } 2>&1; } >> time.txt # This works.
However, the correct construct should simply have the capture reversed, as this:
{ time ./program.exe; } >> time.txt 2>&1; # Correct.
To close any possible output from the command, redirect it's output to /dev/null, as this:
{ time ./program.exe >/dev/null 2>&1; } >> time.txt 2>&1 # Better.
And, as now there is only output on stderr, we could simply capture just it:
{ time ./program.exe >/dev/null 2>&1; } 2>> time.txt # Best.
The output from ./program should be redirected, or it may well end inside time.txt.
I need a way to make a process keep a certain file open forever. Here's an example of what I have so far:
sleep 1000 > myfile &
It works for a thousand seconds, but really don't want to make some complicated sleep/loop statement. This post suggested that cat is the same thing as sleep for infinite. So I tried this:
cat > myfile &
It almost looks like a mistake doesn't it? It seemed to work from the command line, but in a script the file connection did not stay open. Any other ideas?
Rather than using a background process, you can also just use bash to open one of its file descriptors:
exec 5>myfile
(The special use of exec here allows changing the current file descriptor redirections - see man bash for details). This will open file descriptor 5 to "myfile" (use >> if you don't want to empty the file).
You can later close the file again with:
exec 5>&-
(One possible downside of this is that the FD gets inherited by every program that the shell runs in the meantime. Mostly this is harmless - e.g. your greps and seds will generally ignore the extra FD - but it could be annoying in some cases, especially if you spawn any processes that stay around (because they will then keep the FD open).
Note: If you are using a newer version of bash (>4.1) you can use a slightly different syntax:
exec {fd}>myfile
This allocates a new file descriptor, and puts it in the variable fd. This can help ensure that scripts don't accidentally overwrite each other's file descriptors. To close the file later, use
exec {fd}>&-
The reason that cat>myfile& works is because it re-directs standard input into a file.
if you launch it with an ampersand (in background), it won't get ANY input, including end-of-file, which means it will forever wait and print nothing to the output file.
You can get an equivalent effect, except WITHOUT dependency on standard input (the latter is what makes it not work in your script), with this command:
tail -f /dev/null > myfile &
On the cat > myfile & issue running in terminal vs not running as part of a script: In a non-interactive shell the stdin of a backgrounded command & gets implicitly redirected from /dev/null.
So, cat > myfile & in a script actually gets translated into cat </dev/null > myfile, which terminates cat immediately.
See the POSIX standard on the Shell Command Language & Asynchronous Lists:
The standard input for an asynchronous list, before any explicit redirections are
performed, shall be considered to be assigned to a file that has the same
properties as /dev/null. If it is an interactive shell, this need not happen.
In all cases, explicit redirection of standard input shall override this activity.
# some tests
sh -c 'sleep 10 & lsof -p ${!}'
sh -c 'sleep 10 0<&0 & lsof -p ${!}'
sh -ic 'sleep 10 & lsof -p ${!}'
# in a script
- cat > myfile &
+ cat 0<&0 > myfile &
tail -f myfile
This 'follows' the file, and outputs any changes to the file. If you don't want to see the output of tail, redirect output to /dev/null or something:
tail -f myfile > /dev/null
You may want to use the --retry option, depending on your specific case. See man tail for more information.