CMake's execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell. The thing is, I want to use pipes, file descriptor redirection, etc. - and that does not seem to be possible. The alternative would be very painful for me (I think)...
What should I do?
PS - CMake 2.8 and 3.x answer(s) are interesting.
You can execute any shell script, using your shell's support for taking in a script within a string argument.
Example:
execute_process(
COMMAND bash "-c" "echo -n hello | sed 's/hello/world/;'"
OUTPUT_VARIABLE FOO
)
will result in FOO containing world.
Of course, you would need to escape quotes and backslashes with care. Also remember that running bash would only work on platforms which have bash - i.e. it won't work on Windows.
execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell.
Yes, exactly this is written in documentation for that command:
All arguments are passed VERBATIM to the child process. No intermediate shell is used, so shell operators such as > are treated as normal arguments.
I want to use pipes
Different COMMAND within same execute_process invocation are actually piped:
Runs the given sequence of one or more commands with the standard output of each process piped to the standard input of the next.
file descriptor redirection, etc. - and that does not seem to be possible.
For complex things just prepare separate shell script and run it using execute_process. You can pass variables from CMake to this script using its parameters, or with prelimiary configure_file.
I needed to pipe two commands one after the other and actually learned that each COMMAND of the execute_process is piped already. So at least that much is resolved by simply adding commands one after the other:
execute_process(
COMMAND echo "Hello"
COMMAND sed -e 's/H/h/'
OUTPUT_VARIABLE GREETINGS
OUTPUT_STRIP_TRAILING_WHITESPACE)
Now the variable GREETINGS is set to hello.
If you indeed need a lot of file redirection (as you stated), you probably want to write an external script and then execute that script from CMakeLists.txt. It's really difficult to get all the escaping right in CMake.
If you can simplify your scripts to one command generating a file, then another handling that file, etc. then you can always use the INPUT_FILE and OUTPUT_FILE options. Or pass a filename to your command for the input.
It's often much cleaner to handle one file at a time. Although I understand that some commands may need multiple sources and destinations.
Related
My default shell is bash in Ubuntu 14.04. I have a csh script file named clean.sh with the following make command:
#! /bin/csh -f
make -f commande.make del
And commande.make has
CKHOME=../CHEMKIN/DATA_BASES
LIN_DATA=${CKHOME}/LIN_FILES/
LINK_CKTP=${CKHOME}/LINK_CKTP_FILES/
#-----------------------------------------------------
include schema_cinetique.make
LINKFILE=${NAME}_LINK
LINKTPFILE=${NAME}_LINKTP
LINKFILE_OLD=${NAME_OLD}_LINK
LINKFILE_NEW=${NAME_NEW}_LINK
#-----------------------------------------------------
cplink :
${COPY} ${LINK_CKTP}${LINKFILE} LINK
cplink2 :
${COPY} ${LINK_CKTP}${LINKFILE} LINKZ1
tplink :
${COPY} ${LINK_CKTP}${LINKTPFILE} LINKTPZ1
calcul :
${COPY} jobtimp1 LJOBNZ1
${COPY} unsteadyf.dat1 DATZ1
del :
${DELETE} LINKZ1 LINKTPZ1 LJOBNZ1 DATZ1 SOLASUZ1
I opened the terminal and moved to the location and tried
./clean.sh
or
csh clean.sh &
or
csh -f clean.sh
Nothing worked.
I got the following line in the terminal,
LINKZ1 LINKTPZ1 LJOBNZ1 DATZ1 SOLASUZ1
make: LINKZ1: Command not found
make: *** [del] Error 127
So, how to run clean.sh file ?
You are confused. The Csh script contains a single command which actually runs identically in Bash.
#!/bin/bash
make -f commande.make del
Or, for that matter, the same with #!/bin/sh. Or, in this individual case, even sh clean.sh, since the shebang is then just a comment, and the commands in the file are available in sh just as well as in csh.
Once make runs, that is what parses and executes the commands in commande.make. make is not a "Fortran command", it is a utility for building projects (but the makefile named commande.make probably contains some instructions for how to compile Fortran code).
In the general case, Csh and Bash are incompatible, but the differences are in the shell's syntax itself (so, the syntax of loops and conditionals, etc, as well as variable assignments and various other shell builtins).
As an aside, Csh command files should probably not have a .sh extension, as that vaguely implies Bourne shell (sh) syntax. File extensions on Unix are just a hint to human readers, so not technically important; but please don't confuse them/us.
(As a further aside, nobody should be using Csh in 2022. There was a time when the C shell was attractive compared to its competition, but that was on the order of 40 years ago.)
The subsequent errors you are reporting seem to indicate that the makefile depends on some utilities which you have not installed. Figuring that out is a significant enough and separate enough question that you should probably ask a new question about that, probably with more debugging details. But in brief, it seems that make needs to be run with parameters to indicate what NAME and COPY (and probably some other variables) should be. Try with make -f commande.make COPY=cp DELETE=rm NAME=foobar for a start, but it's probably not yet anywhere near sufficient.
(I would actually assume that there will be a README file or similar to actually instruct you how to use commande.make since it seems to have some local conventions of its own.)
It seems the script is written having portability in mind, i.e. the name of the cp and rm binaries is kept in variables rather than hard-coding it. My best guess is that this has been done to make it possible to run the script on non UNIX systems, like Windows.
To make it work, export the respective variables before running the script. For the del action you are calling, only the DELETE variable is needed. It should be set to rm which is the command used to remove files on Linux:
export DELETE=rm
./clean.sh
Note: exporting the variable can also be done in one line when invoking the script, by prepending it to the command line:
DELETE=rm ./clean.sh
This behaviour is described in the bash manual:
The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments, as described in Shell Parameters. These assignment statements affect only the environment seen by that command.
When posting this question originally, I totally misworded it, obtaining another, reasonable but different question, which was correctly answered here.
The following is the correct version of the question I originally wanted to ask.
In one of my Bash scripts, there's a point where I have a variable SCRIPT which contains the /path/to/an/exe which, when executed, outputs a line to be executed.
What my script ultimately needs to do, is executing that line to be executed. Therefore the last line of the script is
$($SCRIPT)
so that $SCRIPT is expanded to /path/to/an/exe, and $(/path/to/an/exe) executes the executable and gives back the line to be executed, which is then executed.
However, running shellcheck on the script generates this error:
In setscreens.sh line 7:
$($SCRIPT)
^--------^ SC2091: Remove surrounding $() to avoid executing output.
For more information:
https://www.shellcheck.net/wiki/SC2091 -- Remove surrounding $() to avoid e...
Is there a way I can rewrite that $($SCRIPT) in a more appropriate way? eval does not seem to be of much help here.
If the script outputs a shell command line to execute, the correct way to do that is:
eval "$("$SCRIPT")"
$($SCRIPT) would only happen to work if the command can be completely evaluated using nothing but word splitting and pathname expansion, which is generally a rare situation. If the program instead outputs e.g. grep "Hello World" or cmd > file.txt then you will need eval or equivalent.
You can make it simple by setting the command to be executed as a positional argument in your shell and execute it from the command line
set -- "$SCRIPT"
and now run the result that is obtained by expansion of SCRIPT, by doing below on command-line.
"$#"
This works in case your output from SCRIPT contains multiple words e.g. custom flags that needs to be run. Since this is run in your current interactive shell, ensure the command to be run is not vulnerable to code injection. You could take one step of caution and run your command within a sub-shell, to not let your parent environment be affected by doing ( "$#" ; )
Or use shellcheck disable=SCnnnn to disable the warning and take the occasion to comment on the explicit intention, rather than evade the detection by cloaking behind an intermediate variable or arguments array.
#!/usr/bin/env bash
# shellcheck disable=SC2091 # Intentional execution of the output
"$("$SCRIPT")"
By disabling shellcheck with a comment, it clarifies the intent and tells the questionable code is not an error, but an informed implementation design choice.
you can do it in 2 steps
command_from_SCRIPT=$($SCRIPT)
$command_from_SCRIPT
and it's clean in shellcheck
I have an rsync command that works as expected when I type it directly into a terminal. The command includes several --include='blah' and --exclude='foo' type arguments. However, if I save that command to a one-line file called "myfile" and I try `cat myfile` (or, equivalently $(cat myfile)), the rsync command behaves differently.
I'm sure it is the exact same command in both cases.
Is this behavior expected/explainable?
I've found the answer to this question. The point is that the cat command takes the contents of the file and treats it like a string. Any string operators (like the escape operator, ) are executed. Then, the final string output is what is passed to a command via the backticks.
As a solution, I've just made "myfile" a shell script that I can execute rather than trying to use cat.
I am trying to print out the output displayed from a command passed into a bash script. The problem I am trying to solve is how to get the output to look exactly like it would if you ran the command from the shell. For example, when I run ls, I see different colors for directories vs. files.
Here is some sample code of what I have so far:
#!/bin/bash
command="$#"
output=`$command`
echo "$output"
So my shell script takes in a command, runs the command, then prints the output. I know that I can customize the color of the output using color codes and echo -e, but I want the output to look just as it does when I run the command from the shell. Any idea of how I can do this?
If all you need is to display the output, you can run the command inline within your script (just let it write to stdout directly, without storing its output in some variable).
That is, you can replace:
output=`$command`
echo $output
with:
$command
or
eval $command
If you also need that output for some kind of processing, that would be a bit tricky. You can (for instance) use | tee /var/tmp/some-temp-file.$$ and then read the output from the temporary file.
Some programs, such as ls, check whether standard output isatty() and behave differently depending on that. If you are capturing the command's output in a variable, the shell redirects its standard output to a pipe which is not a TTY.
There is not much you can do about this except reading the manual page for each individual command to find out whether it supports special options that make its behavior independent of whether its standard output is piped. For ls in particular, you can use the dir program that will always produce human-friendly column formatted output as an alternative.
On a more general level: What you are trying to do seems to be a rather strange thing anyway. I'm sure there is a more robust solution to do what you are trying to accomplish.
Why not just have your script as:
#!/bin/bash
"$#"
It will run any command line passed as argument and print the output unmodified.
I make a complex and long line command to successful login in a site. If I execute it in Console it work. But if I copy and paste the same line in a bash script it not work.
I tried a lot of thing, but accidentally discovery that if I NOT use the line
#!/bin/sh
it work! Why this happens in my mac OSX Lion? What this config line do in a bash script?
A bash script that is run via /bin/sh runs in sh compatibility mode, which means that many bash-specific features (herestrings, process substitution, etc.) will not work.
sh-4.2$ cat < <(echo 123)
sh: syntax error near unexpected token `<'
If you want to be able to use full bash syntax, use #!/bin/bash as your shebang line.
"#!/bin/sh" is a common idiom to insure that the correct interpreter is used to run the script. Here, "sh" is the "Bourne Shell". A good, standard "least common denominator" for shell scripts.
In your case, however, "#!/bin/sh" seems to be the wrong interpreter.
Here's a bit more info:
http://www.unix.com/answers-frequently-asked-questions/7077-what-does-usr-bin-ksh-mean.html
Originally, we only had one shell on unix. When you asked to run a
command, the shell would attempt to invoke one of the exec() system
calls on it. It the command was an executable, the exec would succeed
and the command would run. If the exec() failed, the shell would not
give up, instead it would try to interpet the command file as if it
were a shell script.
Then unix got more shells and the situation became confused. Most
folks would write scripts in one shell and type commands in another.
And each shell had differing rules for feeding scripts to an
interpreter.
This is when the "#! /" trick was invented. The idea was to let the
kernel's exec() system calls succeed with shell scripts. When the
kernel tries to exec() a file, it looks at the first 4 bytes which
represent an integer called a magic number. This tells the kernel if
it should try to run the file or not. So "#! /" was added to magic
numbers that the kernel knows and it was extended to actually be able
to run shell scripts by itself. But some people could not type "#! /",
they kept leaving the space out. So the kernel was exended a bit again
to allow "#!/" to work as a special 3 byte magic number.
So #! /usr/bin/ksh and
#!/usr/bin/ksh now mean the same thing. I always use the former since at least some kernels might still exist that don't understand the
latter.
And note that the first line is a signal to the kernel, and not to the
shell. What happens now is that when shells try to run scripts via
exec() they just succeed. And we never stumble on their various
fallback schemes.
The very first line of the script can be used to select which script interpreter to use.
With
#!/bin/bash
You are telling the shell to invoke /bin/bash interpreter to execute your script.
Assure that there are not spaces or empty lines before #!/bin/bash or it will not work.