I need to use "&&" to execute multiple commands in one step. So I create factory as below:
f1 = factory.BuildFactory()
f1.addStep(shell.ShellCommand, command=["sh", "-c", "pwd", "&&", "cd", "/home/xxx/yyy", "&&", "pwd"])
But during execution it's found that buildbot processes it as below, which makes it impossible to execute
sh -c pwd '&&' cd /home/xxx/yyy '&&' pwd
What I expected is
sh -c pwd && cd /home/xxx/yyy && pwd
Could anyone help me out of this please? Thanks.
Since you're using /bin/sh anyway just call it with a single string:
f1.addStep(shell.ShellCommand, command="pwd && cd /home/xxx/yyy && pwd")
As documentation says:
The ShellCommand arguments are:
command
a list of strings (preferred) or single string (discouraged) which specifies the command to be run. A list of strings is preferred
because it can be used directly as an argv array. Using a single
string (with embedded spaces) requires the buildslave to pass the
string to /bin/sh for interpretation, which raises all sorts of
difficult questions about how to escape or interpret shell
metacharacters.
It's not recommended but you'd still need the shell anyway as && is only interpreted within the shell. Calling sh would just be redundant.
Related
I have a command like this
bash -c 'cd \"/Users/Shammon/Projects/t2i-tokenisation-corda/build/nodes/AGCSIT\" ; \"/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre/bin/java\" \"-Dcapsule.jvm.args=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5008 -javaagent:drivers/jolokia-jvm-1.6.0-agent.jar=port=7008,logHandlerClass=net.corda.node.JolokiaSlf4jAdapter\" \"-Dname=AGCSIT\" \"-jar\" \"/Users/Shammon/Projects/t2i-tokenisation-corda/build/nodes/AGCSIT/corda.jar\" && exit'
When I run this in match terminal, I am getting the following error
bash: line 0: cd:
"/Users/Shammon/Projects/t2i-tokenisation-corda/build/nodes/AGCSIT":
No such file or directory bash:
"/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre/bin/java":
No such file or directory
But the paths are valid ones
Escaping the double quotes causes the shell to look treat them as literal parts of the filenames and parameters, instead of just it into a single token. There's no need to escape the quotes.
bash -c 'cd "/Users/Shammon/Projects/t2i-tokenisation-corda/build/nodes/AGCSIT" ; "/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre/bin/java" "-Dcapsule.jvm.args=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5008 -javaagent:drivers/jolokia-jvm-1.6.0-agent.jar=port=7008,logHandlerClass=net.corda.node.JolokiaSlf4jAdapter" "-Dname=AGCSIT" "-jar" "/Users/Shammon/Projects/t2i-tokenisation-corda/build/nodes/AGCSIT/corda.jar"'
There's also no need for && exit at the end. When the command is done, the shell automatically exits.
I have the most simple issue and I am so not certain what I am doing wrong.
I have a simple shell script using /bin/sh
inside the script I have the following:
exec_as_wwwdata() {
if [ $(whoami) = ${WWW_USER} ]]; then
$#
else
su -s /bin/sh -c '$#' ${WWW_USER}
fi
}
I am calling it with
exec_as_wwwdata composer config -g github-oauth.github.com $COMPOSER_GITHUB_TOKEN
it just does nothing, no error message nothing.
If I call the following directly inside the script
su -s /bin/sh -c 'composer config -g github-oauth.github.com $COMPOSER_GITHUB_TOKEN' ${WWW_USER}
it works.
What am I doing wrong here?
Based on feedback I have changed it to this
exec_as_wwwdata() {
if [ $(whoami) = ${WWW_USER} ]]; then
$#
else
su -s /bin/sh -c '"$#"' "$WWW_USER" _ "$#"
fi
}
Although when I am calling it with the following arguments
exec_as_wwwdata /usr/local/bin/php /usr/local/bin/composer create-project --repository-url=xxxx .
I receive the following error message
su: unrecognized option '--repository-url=
I think there is issue with -- in the string. How can I escape that ?
There are two overlapping uses of $# here, and you've inadvertently stumbled on a partial correct solution. -c expects a single word, while "$#" would produce multiple distinct words. The correct solution would be
su -s /bin/sh -c '"$#"' "$WWW_USER" _ "$#"
The short version: you don't want to build a command string from the current parameters; you want to pass them as arguments to a hard-coded command string to let the new shell expand things appropriately.
A breakdown:
-s /bin/sh - use /bin/sh instead of the appropriate users's login shell
-c '"$#"' run the command "$#", as desired. Note this is a hard-coded value; the new shell will expand its positional parameters correctly once it has started.
"$WWW_USER" - specify the user to run the shell as
_ - specify the value of $0 in the shell being run. You probably don't care what this value is; you just need some placeholder to prevent your first real argument from being treated as the value for $0.
"$#" pass the current positional parameters as arguments to the new shell, which will expand its "$#" to these values.
I want to call /bin/sh, and use the -c option to pass the command '+x', i.e., to execute a program called '+x', whose name begins with a plus sign.
Since '+x' is interpreted by /bin/sh as an option (specifically, disable the 'x' option), /bin/sh must be prevented from interpreting it as an option. I get the following different results depending on the /bin/sh I use:
(1) First variant:
/bin/sh -c -- +x
Using Dash and Bash on Linux: The command +x is executed.
Using FreeBSD's sh: The command -- is executed, and the +x option is set.
(2) Second variant:
/bin/sh -c +x
Using Dash and Bash on Linux: The +x option is set, and there is an error because the option -c is missing an argument.
Using FreeBSD's sh: The command +x is executed.
(3) Third variant:
/bin/sh -c - +x
Using Dash and Bash on Linux: +x is executed.
Using FreeBSD's sh: The command - is executed and the option +x is set.
(4) Fourth variant: (ADDED as suggested in the comments)
/bin/sh -c+x
Using Dash and Bash on Linux: Invalid/Illegal option '+'
Using FreeBSD's sh: Bad -c option
My question: What does POSIX prescribe?
I'm reading the POSIX specification for sh here: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html
Quoting from it: "A single hyphen shall be treated as the first operand and then ignored. If both '-' and "--" are given as arguments, or if other operands precede the single hyphen, the results are undefined."
I'm not sure whether that quote also applies to a single dash placed just after '-c'.
So, which one is right, Dash/Bash or FreeBSD? Or, if both are right because POSIX allows both, how to do this portably?
The answer to the question "What does Posix prescribe" is already present in the OP. But the important feature of the Posix standard is not highlighted: the -c option does not take an argument.
You can see this in the Synopsis:
sh -c [-abCefhimnuvx] [-o option]... [+abCefhimnuvx] [+o option]...
command_string [command_name [argument...]]
What the -c flag does is cause the positional parameters ("operands") to be interpreted in a different way. Without -c, they are interpreted as [command_file [argument...]]:
sh [-abCefhimnuvx] [-o option]... [+abCefhimnuvx] [+o option]...
[command_file [argument...]]
That, by the way, is why sh -c+x is an error. If -c took an argument, then it would be legal to include the argument in the same word.
So, to answer the more specific questions:
Posix says "A single hyphen shall be treated as the first operand and then ignored...". Does that apply to a - immediately following -c?
A: Yes, it does. -c is a complete option, and the - is therefore an operand. By contrast, - in -o - would not be treated as an operand. (It would be treated as an invalid option name.)
Which one is right, Dash/Bash or FreeBSD?
A: In this case, Dash and Bash are Posix-compliant, and FreeBSD's sh is not. FreeBSD's shell considerably predates the current Posix specification, and I don't believe it ever purported to be fully compliant to any Posix specification.
How do I portably use sh to run a command whose name begins with a +?
A: I would think the following would work on any shell:
sh -c " +x"
" +x" will not be recognized as an option because it doesn't start with a + or -, and sh -c causes the operand to be parsed as a shell command, so leading whitespace will be ignored. I don't have a copy of FreeBSD's ash to play with just now, so I welcome corrections.
Or you could use a simple compound command:
sh -c "{ +x; }"
Possibly clearest (assuming the shell you're using implements the Posix-standard builtin command) is:
sh -c "command +x"
I have an rsync command in my csh script like this:
#! /bin/csh -f
set source_dir = "blahDir/blahBlahDir"
set dest_dir = "foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
When I run this I get the following error:
rsync: No match.
If I remove the --exclude option it works. I wrote the equivalent script in bash and that works as expected
#/bin/bash -f
source_dir="blahDir/blahBlahDir"
dest_dir="foo/anotherFoo"
rsync -av --exclude=*.csv ${source_dir} ${dest_dir}
The problem is that this has to be done in csh only. Any ideas on how I can get his to work?
It's because csh is trying to expand --exclude=*.csv into a filename, and complaining because it cannot find a file matching that pattern.
You can get around this by enclosing the option in quotes:
rsynv -rv '--exclude=*.csv' ...
or escaping the asterisk:
rsynv -rv --exclude=\*.csv ...
This is a consequence of the way csh and bash differ in their default treatment of arguments with wildcards that don't match a file. csh will complain while bash will simply leave it alone.
You may think bash has chosen the better way but that's not necessarily so, as shown in the following transcript where you have a file matching the argument:
pax> touch -- '--file=xyzzy.csv' ; ls -- *.csv
--file=xyzzy.csv
pax> echo --file=*.csv
--file=xyzzy.csv
You can see there that the bash shell expands the argument rather than giving it to the program as is. Both sides have their pros and cons.
how I can use the second argument of previous command in a new command ?
example, with
$ mkdir test
I make a directory, how I can use the name of directory for change to this ?
$ mkdir test && cd use_var
$_ is the last (right-most) argument of the previous command.
mkdir gash && cd "$_"
(I don't create files or directories called test, that's the name of a shell built-in and can cause confusions)
With history expansion, you can refer to arbitrary words in the current command line
mkdir dir1 && cd "!#:1"
# 0 1 2 3 4
!# refers to the line typed so far, and :1 refers to word number one (with mkdir starting at 0).
If you use this in a script (i.e., a non-interactive shell), you need to turn history expansion on with set -H and set -o history.
Pressing Esc + . places the last argument of previous command on the current place of cursor. Tested in bash shell and ksh shell.
I use functions for this. Type this in your shell:
mkcd() { mkdir "$1" ; cd "$1" ; }
Now you have a new command mkcd.
If you need this repeatedly, put the line into the file ~/.bash_aliases (if you use bash; other shells use different names).