As an example, double dash or two hyphens -- is used like so:
npm test -- --coverage
Running npm without the double dash flag does not run in coverage mode so it seems to append subsequent flags, is this correct? I couldn't find the documentation on this.
-- as an argument on its own is standardized across all UNIX commands: It means that further arguments should be treated as positional arguments, not options. See Guideline 10 in POSIX Utility Syntax Conventions.
To give you a non-NPM-based example, ls -- -l will look for a file named -l, because the -- specified that all subsequent arguments are positional.
In this context, it means that --coverage isn't an option to npm itself; presumably, then, it's subsequently read by the test subcommand. For a tool that were following the conventions properly this wouldn't be necessary, because Guideline 9 specifies that all options shall be given before any arguments (thus that in this context --coverage should be treated as an argument since it comes after the argument test); however, inasmuch as NPM is only partially following the guidelines, this is a foreseeable result.
(Long --option-style options are actually a GNU extension as a whole, so what we have here is a mismash of multiple parsing styles; such is life, unfortunately).
I've done some further digging; according to the docs for my node version -
"--" Indicates the end of node options. Pass the rest of the arguments to the script. If no script filename or eval/print script is supplied prior to this, then the next argument will be used as a script filename.
But, a simple script that contains -
console.log(`process.execArgv:${process.execArgv}`);
console.log(`process.argv:${process.argv}`);
behaves as -
>node --prof argv.js --myArg
process.execArgv:--prof
process.argv:C:\Program Files\nodejs\node.exe,C:\Dev\Web\QA_Web_POC\argv.js,--myArg
>node --prof argv.js -- --myArg
process.execArgv:--prof
process.argv:C:\Program Files\nodejs\node.exe,C:\Dev\Web\QA_Web_POC\argv.js,--, --myArg
>node argv.js --prof -- --myArg
process.execArgv:
process.argv:C:\Program Files\nodejs\node.exe,C:\Dev\Web\QA_Web_POC\argv.js,--prof,--,--myArg
>node argv.js -- --prof --myArg
process.execArgv:
process.argv:C:\Program Files\nodejs\node.exe,C:\Dev\Web\QA_Web_POC\argv.js,--,--prof,--myArg
So, it seems there's a bug?
Related
I have a list/array like so:
['path/to/folder/a', 'path/to/folder/b']
This is an example, the array can be of any length. But for each item in the array I'd like to set up the following as a single command:
$ someTool <command> --flag <item-1> --flag <item-2> ... --flag <item-N>
At the moment I am currently doing a loop over the array but I am just wondering if doing them individually has a different behaviour to doing them all at once (which the tool specifies I should do).
for i in "${array[#]}"; do
someTool command --flag $i
done
Whether passing all flag arguments to a single invocation of the tool does the same thing as passing them one-at-a-time to separate invocations depends entirely on the tool and what it does. Without more information, it's impossible to say for sure, but if the instructions recommend passing them all at once, I'd go with that.
The simplest way to do this in bash is generally to create a second array with the flags and arguments as they need to be passed to the tool:
flagsArray=()
for i in "${array[#]}"; do
flagsArray+=(--flag "$i")
done
someTool command "${flagsArray[#]}"
Note: all of the above syntax -- all the quotes, braces, brackets, parentheses, etc -- matter to making this run properly and robustly. Don't leave anything out unless you know why it's there, and that leaving it out won't cause trouble.
BTW, if the option (--flag) doesn't have to be passed as a separate argument (i.e. if the tool allows --flag=path/to/folder/a instead of --flag path/to/folder/a), then you can use a substitution to add the --flag= bit to each element of the array in a single step:
someTool command "${array[#]/#/--flag=}"
Explanation: the /# means "replace at the beginning (of each element)", then the empty string for the thing to replace, / to delimit that from the replacement string, and --flag= as the replacement (/addition) string.
For example,
# Execute the pre-hook.
export SHELL=#shell#
param1=#param1#
param2=#param2#
param3=#param3#
param4=#param4#
param5=#param5#
if test -n "#preHook#"; then
. #preHook#
fi
For context, this is from a shell script in a commit from 2004 in the Nixpkgs repo; tried to see if this maybe a reference feature but string "shell" only occurs once (in a case-sensitive search) in the entire file.
The answer by Chris Dodd is correct, insofar as there's no intrinsic meaning to the shell -- and #foo# is thus commonly used as a sigil. Insofar as you encountered this in nixpkgs, it provides some stdenv tools specifically for implementing this pattern.
As documented at https://nixos.org/manual/nixpkgs/stable/#ssec-stdenv-functions, nixpkgs stdenv provides shell functions including substitute, substituteAll, substituteInPlace &c. which will replace #foo# values with the content of corresponding variables.
In the context of the linked commit, subsitutions of that form can be seen being performed in pkgs/build-wrapper/gcc-wrapper/builder.sh:
sed \
-e "s^#gcc#^$src^g" \
-e "s^#out#^$out^g" \
-e "s^#bash#^$SHELL^g" \
-e "s^#shell#^$shell^g" \
< $gccWrapper > $dst
...is replacing #out# with the value of $out, #bash# with the value of $SHELL, etc.
The # symbol has no meaning to the shell -- it is a punctuation character that will pretty much never occur in any actual shell script.
This makes it a good choice to use for patterns in script templates -- the basic idea being that a simple search-and-replace process will be used (perhaps with a sed script as in the link you show) to rewrite the template into an actual shell script. Every string of the form #name# in the template will be replaced by some other string related to the environment in which the script is being installed.
I have to pass a command with its arguments in a scheduled task, while separating the arguments from the command. I used:
split(/(?=\s-)/)
to do this, but it won't work when the argument is not passed as -arg format.
Example of commands can be passed in format:
"ping http://www.google.com" here url is argument
"abc-abc -V"
"abc-abc -L c:\\folder name\\test.log"
'"C:\\Program Files\\example\\program.exe" -arg1 -arg2'
"C:\\Program Files\\example\\program.exe"
To make this more clear these commands are not passed as command line argument which can get in ARGV
The command gets set in command property which accepts input in string format
command '"C:\\Program Files\\example\\program.exe" -arg1 -arg2'
Use Shellwords.split, from the standard library:
Shellwords.split("ping http:\\www.google.com here url is argument")
#=> ["ping", "http:www.google.com", "here", "url", "is", "argument"]
Shellwords.split("abc-abc -V")
#=> ["abc-abc", "-V"]
Shellwords.split("abc-abc -L c:\\folder name\\test.log")
#=> ["abc-abc", "-L", "c:folder", "nametest.log"]
Shellwords.split('"C:\\Program Files\\example\\program.exe" -arg1 -arg2')
#=> ["C:\\Program Files\\example\\program.exe", "-arg1", "-arg2"]
Shellwords.split('"C:\\Program Files\\example\\program.exe"')
#=> ["C:\\Program Files\\example\\program.exe"]
No need to reinvent the wheel with a custom regex/splitter, or an external system call.
It seems to me that if there's no consistent pattern to your command syntax, then any regex based approach will inevitably fail. It seems better instead to solve this problem the way a human would, i.e. with some knowledge of context.
In a *nix terminal, you can use the compgen command to list available commands. This Ruby script invokes that command to print the first 5 options from that list:
list = `cd ~ && compgen -c`
list_arr = list.split("\n")
list_arr[0,6].each{|x| puts x }
(The cd in the first line seems to be needed because of the context in which my Ruby is running with rvm.) For Windows, you may find this thread a useful starting point.
I'd match against the elements of this list to identify my commands, and take it from there.
Tom Lord's answer is far better than this one.
You probably want to look at OptionParser or GetOptLong if you need parsing of command line arguments provided to a ruby program.
If you are interested in parsing some strings that may or may not be commands with arguments, here's a quick-and-dirty:
I'd use scan instead of split with the following regex: /(".*"|[\w\:\:\.\-\\]+)/.
Best results come from: 'some string'.scan(/(".*"|[\w\:\:\.\-\\]+)/).flatten:
["ping", "http:\\www.google.com"]
["abc-abc", "-V"]
["abc-abc", "-L", "c:\\folder\\", "name\\test.log"]
# Technically, this is wrong, but so is the non-escaped whitespace.
["\"C:\\Program Files\\example\\program.exe\"", "-arg1", "-arg2"]
["\"C:\\Program Files\\example\\program.exe\""]
I have a bash script which imports same variables from another static file, which itself uses environment variables set by another file when the script is invoked.
This is the file that gets imported and sets some variables.
# package.mk
PKG_NAME="binutils"
PKG_VERSION="2.24"
PKG_URL_TYPE="http"
PKG_URL="http://ftp.gnu.org/gnu/binutils/${PKG_NAME}-${PKG_VERSION}.tar.bz2"
PKG_DEPENDS=""
PKG_SECTION="devel"
PKG_CONFIGURE_OPTS="--prefix=${TOOLS} \
--target=${TARGET} \
--with-sysroot=${TOOLS}/${TARGET} \
--disable-nls \
--disable-multilib"
It is used by the builds script as so:
#!/bin/bash
# Binutils
. settings/config
pkg_dir="$(locate_package 'binutils')"
. "${pkg_dir}/package.mk"
# etc...
"${CLFS_SOURCES}/${PKG_NAME}-${PKG_VERSION}/configure" "${PKG_CONFIGURE_OPTS}"
# etc...
This script first imports the settings/config file which has a bunch of global variables used by this script and others, and exports them so they are available as environment variables. It then locates the correct package.mk file for the specific component we are building, and imports it as well. So far, so good.
However when I double-quote the options (PKG_CONFIGURE_OPTS) for the configure script:
"${CLFS_SOURCES}/${PKG_NAME}-${PKG_VERSION}/configure" "${PKG_CONFIGURE_OPTS}"`
I get the following error:
gcc: error: unrecognized command line option ‘--with-sysroot=/root/LiLi/target/cross-tools/arm-linux-musleabihf’
If I leave it not quoted like:
"${CLFS_SOURCES}/${PKG_NAME}-${PKG_VERSION}/configure" ${PKG_CONFIGURE_OPTS}`
it works fine (--with-sysroot= is indeed a valid configure flag for binutils).
Why is this? What can I change so that I can double-quote that portion (going by the bash wisdom that one should double-quote just about everything).
Quoting the variable means the entire thing is passed as a single argument, spaces and newlines included. You want word splitting to be performed so that the string is treated as multiple arguments. That's why leaving it unquoted works.
If you're looking for the "right" way to handle this, I recommend using an array. An array can hold multiple values while also preserving whitespace properly.
PKG_CONFIGURE_OPTS=(--prefix="$TOOLS"
--target="$TARGET"
--with-sysroot="$TOOLS/$TARGET"
--disable-nls
--disable-multilib)
...
"$CLFS_SOURCES/$PKG_NAME-$PKG_VERSION/configure" "${PKG_CONFIGURE_OPTS[#]}"
I'm writing a simple command line gem.
The library that does the actual work was developed with rspec and so far that works.
I'm trying to test the command line portion with Aruba/Cucumber, but I've come across some strange behaviour.
Just to test this, I've got a the binary file to puts ARGV, and I've got test files in tmp/aruba
When I run bundle exec gem_name tmp/aruba/*.* I am presented with the list of shell expanded file names.
Now my features file has:
Given files to work on # I set up files in tmp/aruba in this step
When I run `gem_name *.*` # standard step
Then the output should contain "Wibble"
The last step is obviously going to fail, but it shows me a diff between what it expects and the actual output. Rather than seeing a list of shell expanded filenames, all I get is "*.*"
So I'm left in the position of having an app that actually works as expected, but I can't get the tests to pass. I could take the "." and generate the list of files from there, but then I'm writing extra production code just to get the app to work under test - which I don't think is the correct way to go about it. And all because shell expansion isn't happening.
If you look at my profile, you'll see that Ruby isn't my main bag, feel free to point me at any resources that I may have missed about this, but is this just me missing something, or expected behaviour that somebody knows how to work around?
After a little digging in the Aruba source I figured out that the When I run step ends up in a code block like this:
def run!(&block)
#process = ChildProcess.build(*shellwords(#cmd))
...
begin
#process.start
...
Further digging into ChildProcess ends up here:
def launch_process
...
begin
exec(*#args)
...
And therein lies the problem. exec does not do shell expansion when the argument list is split into multiple array elements:
If exec is given a single argument, that argument is
taken as a line that is subject to shell expansion before being
executed. If multiple arguments are given, the second and
subsequent arguments are passed as parameters to command with no
shell expansion.
However playing with shellwords a bit we find:
Shellwords.shellwords('gem_name *.*')
=> ["gem_name", "*.*"] # No good
Shellwords.shellwords('"gem_name *.*"')
=> ["gem_name *.*"] # Aha!
Therefore the solution might be as simple as:
When I run `"gem_name *.*"`
If that doesn't work then you are pretty much out of luck. I would suggest you expand the file names manually since you're not really testing shell expansion here - we know that works: you are testing multiple arguments.
Therefore you should instead do:
When I run `gem_name your_file1 your_file2 your_file3`