I have a bash script that I am modifying. The script now also executes a binary. Say something like this
mybin arg1 arg1 The binary takes about 5 minutes to execute and when I execute it from bash directly, it does show the intermediate outputs. When I add it to my script as
`mybin arg1 arg1`
I get the output in the end and bash thinks the output is a command and tries to execute it. So I want to solve 2 things
Show the intermediate output on the screen when I execute the binary from the bash script.
And the output must not be treated to be a command for processing, just regular output
Remove the backticks.
`prog` means "collect the output of prog and interpolate it into the current command", so if `prog` is the only thing on the command line, its output will be executed as another command. This is known as command substitution.
In other words, the two things you don't want to happen are exactly what ` ` is designed to do.
Related
Beginner in bash and makefiles here. I have a course where we need to create a makefile where each rule calls one of the already compiled programs. All of the compiled programs take a command line argument. As the arguments can be quite large and mostly consists of the same character in a row (for example AAAAAAA) I made a script that uses python to print the argument. Example:
#!/bin/bash
python -c 'print "A"*1000 + "Q"*200'
I am wondering how to create the rule in the makefile so that the output of the above script will be passed as the command line argument. Essentially like this:
test:
./schoolprogram ./myprogram.sh
So when make test is executed then ./schoolprogram should be run with the argument 1000 A's followed by 200 Q's and not the literal string "./myprogram.sh".
I don't know why you have a script that does nothing but invoke python; why not just run python directly?
In any event, this isn't really a makefile question it's mostly a shell question. When writing makefile recipes, the best way is to get the command you want to run working at your shell prompt, then take that same command (with one small tweak) and put it into your makefile.
In the shell, you can use either $(...) or backticks (an older style) to run a command and replace it with the output of that command. So you can run this:
$ ./schoolprogram $(./myprogram.sh)
or more easily:
$ ./schoolprogram $(python -c 'print "A"*1000 + "Q"*200')
Now when you take a command and put it into a makefile recipe, the thing you have to remember is that the $ character is special to make, so if you want a $ to be passed to your command you have to escape it by doubling it, $$. So you'd write:
test:
./schoolprogram $$(./myprogram.sh)
or equivalently:
test:
./schoolprogram $$(python -c 'print "A"*1000 + "Q"*200')
Consider the following script:
#!/bin/bash
CMD="echo hello world > /tmp/hello.out"
${CMD}
The output for this is:
hello world > /tmp/hello.out
How can I modify CMD so that the output gets redirected to hello.out?
For my use case, it is not feasible to either do this:
${CMD} > /tmp/hello.out
or to add this at the top of the script:
exec > /tmp/hello.out
No, there is no way to make a redirection happen from a variable.
Why?
The first thing the shell does with a command line is:
Each line that the shell reads from the standard input or a script is called a pipeline; it contains one or more commands separated by zero or
more pipe characters (|). For each pipeline it reads, the shell breaks it up into commands, sets up the I/O for the pipeline, then does the following for each command (Figure 7-1):
From: Learning the bash Shell Unix Shell Programming . Chapter Preview / Figure . Pdf
That means that even before starting with the first word of a command line, the redirections are set up.
The "Parameter Expansion" happens quite a lot latter (in step 6 of the Figure).
There is no way to set up redirections after a variable is expanded.
Unless ...
The "command line is reprocessed" using eval.
eval "$CMD"
But this comes with a lot of danger.
The command line is changed by the first processing in the 12 steps detailed in the book (quotes are removed, variables expanded, words split, etc.).
It is usually quite difficult to estimate all the changes and consequences before the line is actually processed.
And then, it is processed again.
You can use eval to instruct the shell to reinterpret the variable content as a shell command:
eval $CMD
CMake's execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell. The thing is, I want to use pipes, file descriptor redirection, etc. - and that does not seem to be possible. The alternative would be very painful for me (I think)...
What should I do?
PS - CMake 2.8 and 3.x answer(s) are interesting.
You can execute any shell script, using your shell's support for taking in a script within a string argument.
Example:
execute_process(
COMMAND bash "-c" "echo -n hello | sed 's/hello/world/;'"
OUTPUT_VARIABLE FOO
)
will result in FOO containing world.
Of course, you would need to escape quotes and backslashes with care. Also remember that running bash would only work on platforms which have bash - i.e. it won't work on Windows.
execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell.
Yes, exactly this is written in documentation for that command:
All arguments are passed VERBATIM to the child process. No intermediate shell is used, so shell operators such as > are treated as normal arguments.
I want to use pipes
Different COMMAND within same execute_process invocation are actually piped:
Runs the given sequence of one or more commands with the standard output of each process piped to the standard input of the next.
file descriptor redirection, etc. - and that does not seem to be possible.
For complex things just prepare separate shell script and run it using execute_process. You can pass variables from CMake to this script using its parameters, or with prelimiary configure_file.
I needed to pipe two commands one after the other and actually learned that each COMMAND of the execute_process is piped already. So at least that much is resolved by simply adding commands one after the other:
execute_process(
COMMAND echo "Hello"
COMMAND sed -e 's/H/h/'
OUTPUT_VARIABLE GREETINGS
OUTPUT_STRIP_TRAILING_WHITESPACE)
Now the variable GREETINGS is set to hello.
If you indeed need a lot of file redirection (as you stated), you probably want to write an external script and then execute that script from CMakeLists.txt. It's really difficult to get all the escaping right in CMake.
If you can simplify your scripts to one command generating a file, then another handling that file, etc. then you can always use the INPUT_FILE and OUTPUT_FILE options. Or pass a filename to your command for the input.
It's often much cleaner to handle one file at a time. Although I understand that some commands may need multiple sources and destinations.
I have an rsync command that works as expected when I type it directly into a terminal. The command includes several --include='blah' and --exclude='foo' type arguments. However, if I save that command to a one-line file called "myfile" and I try `cat myfile` (or, equivalently $(cat myfile)), the rsync command behaves differently.
I'm sure it is the exact same command in both cases.
Is this behavior expected/explainable?
I've found the answer to this question. The point is that the cat command takes the contents of the file and treats it like a string. Any string operators (like the escape operator, ) are executed. Then, the final string output is what is passed to a command via the backticks.
As a solution, I've just made "myfile" a shell script that I can execute rather than trying to use cat.
I am trying to print out the output displayed from a command passed into a bash script. The problem I am trying to solve is how to get the output to look exactly like it would if you ran the command from the shell. For example, when I run ls, I see different colors for directories vs. files.
Here is some sample code of what I have so far:
#!/bin/bash
command="$#"
output=`$command`
echo "$output"
So my shell script takes in a command, runs the command, then prints the output. I know that I can customize the color of the output using color codes and echo -e, but I want the output to look just as it does when I run the command from the shell. Any idea of how I can do this?
If all you need is to display the output, you can run the command inline within your script (just let it write to stdout directly, without storing its output in some variable).
That is, you can replace:
output=`$command`
echo $output
with:
$command
or
eval $command
If you also need that output for some kind of processing, that would be a bit tricky. You can (for instance) use | tee /var/tmp/some-temp-file.$$ and then read the output from the temporary file.
Some programs, such as ls, check whether standard output isatty() and behave differently depending on that. If you are capturing the command's output in a variable, the shell redirects its standard output to a pipe which is not a TTY.
There is not much you can do about this except reading the manual page for each individual command to find out whether it supports special options that make its behavior independent of whether its standard output is piped. For ls in particular, you can use the dir program that will always produce human-friendly column formatted output as an alternative.
On a more general level: What you are trying to do seems to be a rather strange thing anyway. I'm sure there is a more robust solution to do what you are trying to accomplish.
Why not just have your script as:
#!/bin/bash
"$#"
It will run any command line passed as argument and print the output unmodified.