I'm trying to join input $* which is one parameter consisting of all the parameters added together.
This works.
#!/bin/bash
foo() {
params="${*}"
echo "${params//[[:space:]]/-}"
}
foo 1 2 3 4
1-2-3-4
However, is it possible to skip the assignment of variable?
"${"${*}"//[[:space:]]/-}"
I'm getting bad substitution error.
I can also do
: "${*}"
echo "${_//[[:space:]]/-}"
But it feels hacky.
One option could be to set bash's internal field separator, IFS, to - locally and just echo "$*":
foo() {
local IFS=$'-'
echo "$*"
}
To answer your question, you can do global pattern substitutions on the positional parameters like this:
${*//pat/sub}
${#//pat/sub}
And also arrays like this:
${arr[*]//pat/sub}
${arr[#]//pat/sub}
This won’t join the parameters, but substitute inside them.
Setting IFS to dash adds a dash in between each parameter for echo "$*", or p=$*, but won’t replace anything inside a parameter.
Eg:
$ set -- aa bb 'cc cc'
$ IFS=-
$ echo "$*"
aa-bb-cc cc
To remove all whitespace, including inside a parameter, you can combine them:
IFS=-
echo "${*//[[:space:]]/-}"
Or just assign to a name first, like you were doing:
no_spaces=$*
echo "${no_spaces//[[:space:]]/-}"
Related
This piece of code works as expected:
for var in a 'b c' d;
do
echo $var;
done
The bash script loops through 3 arguments printing
a
b c
d
However, if this string is read in via jq , and then looped over like so:
JSON_FILE=path/to/jsonfile.json
ARGUMENTS=$(jq -r '.arguments' "${JSON_FILE}")
for var in ${ARGUMENTS};
do
echo $var;
done
The result is 4 arguments as follows:
a
'b
c'
d
Example json file for reference:
{
"arguments" : "a 'b c' d"
}
What is the reason for this? I tried putting quotes around the variable like suggested in other SO answers but that caused everything to just be handled as 1 argument.
What can I do to get the behavior of the first case (3 arguments)?
What is the reason for this?
The word splitting expansion is run over unquoted results of other expansions. Because ${ARGUMENTS} expansion in for var in ${ARGUMENTS}; is unquoted, word splitting is performed. No, word splitting ignores quotes resulted from variable expansion - it only cares about whitespaces.
What can I do to get the behavior of the first case (3 arguments)?
The good way™ would be to write your own parser, to parse the quotes inside the strings and split the argument depending on the quotes.
I advise to use xargs, it (by default, usually a confusing behavior) parses quotes in the input strings:
$ arguments="a 'b c' d"
$ echo "${arguments}" | xargs -n1 echo
a
b c
d
# convert to array
$ readarray -d '' arr < <(<<<"${arguments}" xargs printf "%s\0")
As presented in the other answer, you may use eval, but please do not, eval is evil and will run expansions over the input string.
Change IFS to a new line to make it work:
...
IFS='\n'; for var in $ARGUMENTS;
do
echo $var;
done
I use awk a lot to select single columns and after learning what an alias was I started off with
alias a1='awk '\{print $1}'\'
alias a2='awk '\{print $2}'\'
...
After I learned a little more I thought those were cheesy and replaced them with
function a() {
awk "{print \$$1}"
}
so now I can do a 3 or a 11 without needing to create explicit aliases.
So that's good, but sometimes I need to select more than one column, and when I do I have to resort to typing out the actual full awk '{print ...}' command (the horror!).
So I'm trying to come up with a way to do something similar to the a function but one that will accept different numbers of arguments, so I could do a 3 or a 5 7 or a 2 4 9.
I've tried diff things with $# and $* but can't get it right and everything I'm trying now I know are cheesy workarounds and so I'd rather just stop and ask how to do it the proper way.
Thanks all.
$ cat tst.sh
function a {
awk -v args="$*" '
BEGIN { n=split(args,f) }
{ for (i=1;i<=n;i++) printf "%s%s", $(f[i]), (i<n?OFS:ORS) }
'
}
echo "a b c d e f" | a 1 3 5
echo "---"
echo "a b c d e f" | a 1 3 4 6
$ ./tst.sh
a c e
---
a c d f
You could get arbitrary complicated with this sort of thing (what if you wanted to be able to say a 2-5 7 11-, as with cut?) but here's one that will work with a list of numbers:
a() { (IFS=,; awk '{print '"${*/#/$}"'}'); }
That requires a bit of explanation.
a() { ... }
defines a shell function, which differs from an alias in various ways, one of which being that you can give it parameters.
Inside the shell function, I want to change the value of IFS; to avoid having to remember the old value and change it back, I surround the command I actually want to execute with (...), which causes it to execute in a subshell. When the subshell finishes, all environmental changes finish with it, so it effectively makes the change to IFS local.
IFS is the set of characters used for word splitting, but it also defines the character used to separate elements in the expansion of "$*" (that is, the list of function or script arguments) when it is surrounded by quotes. So setting it to , means the $* expansion will be a comma-separated list.
The awk program I want to create is actually something like {print $1,$4,$7}, so aside from putting commas between the list, I need to add a $ before each number. I do that with the bash parameter expansion substitute syntax: ${parameter/pattern/replacement}. By specifying * as the parameter, I get $* with the substitution applied to each argument. (Note that the expansion is quoted. If it weren't, it wouldn't work.)
In the replacement expression, the pattern is empty because the # character at the beginning of the pattern indicates that the match must be at the beginning of the string. Since the actual pattern is empty, the first match always be at the beginning of the string and the replacement ($) will therefore be inserted at the beginning of each argument. The # is needed because // is syntactically different: it means "change all occurrences of the pattern", instead of just the first one.
Unlike many languages, in bash search-and-replace expressions are not terminated with a /, but rather with the matching }. If you type ${p/foo/bar/}, it will replace the first instance of foo with bar/.
I am having problems with expansion of command-line options containing spaces. They are not getting grouped as I expect them to be. How can I modify following code(below) to get the desired output(below).
function myFunction {
while getopts "a:b:A:" optionName; do
echo "$optionName::$OPTARG"
done
}
#dynamic variable, cannot be hardcoded into $MY_ARGS
MY_VAR="X1=162356374 X2=432876 X3=342724"
#$MY_ARGS is useful and will be used more than once,
#so we don't want to eliminate it and replace it's usage with its value everywhere
MY_ARGS="-a 24765437643 -b '$MY_VAR' -A jeeywewueuye"
myFunction $MY_ARGS
Actual Output:
a::24765437643
b::'X1=162356374
Desired Output:
a::24765437643
b::X1=162356374 X2=432876 X3=342724
A::jeeywewueuye
The best way to store a list of arguments is in an array. An array can handle arguments with whitespace without problem, and you don't have to figure out how to get the quotes and backslashes just right.
MY_ARGS=(-a 24765437643 -b "$MY_VAR" -A jeeywewueuye)
myFunction "${MY_ARGS[#]}"
The only unnatural part about arrays is the weird syntax to expand them: "${array[#]}". The quotes, curly braces, and [#] notation are all important.
I agree that arrays answer the question in the best way.
Perhaps you don't want to use arrays (colleagues will not understand), or you must obey the Google Guidelines for bash (nice work, I agree with for over 90%). that claims: "If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use Python. ".
When you must look for another solutions:
An ugly solution is changing the IFS:
function myFunction {
while getopts "a:b:A:" optionName; do
echo "$optionName::$OPTARG"
done
}
#dynamic variable, cannot be hardcoded into $MY_ARGS
MY_VAR="X1=162356374 X2=432876 X3=342724"
MY_ARGS="-a/24765437643/-b/"$MY_VAR"/-A/jeeywewueuye"
IFS=/
myFunction ${MY_ARGS}
Perhaps you want to do something with myFunction
function myFunction {
while getopts "a:bA:" optionName; do
case "${optionName}" in
b) echo "${optionName}::${MY_VAR}" ;;
*) echo "${optionName}::${OPTARG}" ;;
esac
done
}
or you could tr the spaces into another character before calling myFunction and tr the characters back to spaces in myFunction().
I know how to parse a string into variables in the manner of this SO question, e.g.
ABCDE-123456
becomes:
var1=ABCDE
var2=123456
via, say, cut. I can do that in one script, no problem.
But I have a few dozen scripts which parse strings/arguments all in the same fashion (same arguments & variables, i.e. same parsing strategy).
And sometimes I need to make a change or add a variable to the parsing mechanism.
Of course, I could go through every one of my dozens of scripts and change the parsing manually (even if just copy & paste), but that would be tedious and more error-prone to bugs/mistakes.
Is there a modular way to do parse strings/arguments as such?
I thought of writing a script which parses the string/args into variables and then exports, but the export command does not work form child-to-parent, (only vice-versa).
Something like this might work:
parse_it () {
SEP=${SEP--}
string=$1
names=${#:2}
IFS="$SEP" read $names <<< "$string"
}
$ parse_it ABCDE-123456 var1 var2
$ echo "$var1"
ABCDE
$ echo "$var2"
123456
$ SEP=: parse_it "foo:bar:baz" id1 id2 id3
$ echo $id2
bar
The first argument is the string to parse, the remaining arguments are names of variables that get passed to read as the variables to set. (Not quoting $names here is intentional, as we will let the shell split the string into multiple words, one per variable. Valid variable names consist of only _, letters, and numbers, so there are no worries about undesired word splitting or pathname generation by not quoting $names). The function assumes the string uses a single separator of "-", which can be overridden via the environment.
For more complex parsing, you may want to use a custom regular expression (bash 4 or later required for the -g flag to declare):
parse_it () {
reg_ex=$1
string=$2
shift 2
[[ $string =~ $reg_ex ]] || return
i=1
for name; do
declare -g "$name=${BASH_REMATCH[i++]}"
done
}
$ parse_it '(.*)-(.*):(.*)' "abc-123:xyz" id1 id2 id3
$ echo "$id2"
123
I think what you really want is to write your function in one script and include it in all of your other scripts.
You can include other shell scripts by the source or . command.
For example, you can define your parse function in parseString.sh
function parseString {
...
}
And then in any of your other script, do
source parseString.sh
# now we can call parseString function
parseString abcde-12345
Regardless of the number of arguments passed to my script, I would like for the second to the last argument to always represent a specific variable in my code.
Executing the program I'd type something like this:
sh myprogram.sh -a arg_a -b arg_b special specific
test=("${3}")
echo $test
The results will show 'special'. So using that same idea if I try this (since I won't know that number of arguments):
secondToLastArg=$(($#-1))
echo $secondToLastArg
The results will show '3'. How do I dynamically assign the second to last argument?
You need a bit of math to get the number you want ($(($#-1))), then use indirection (${!n}) to get the actual argument.
$ set -- a b c
$ echo $#
a b c
$ n=$(($#-1))
$ echo $n
2
$ echo ${!n}
b
$
Indirection (${!n}) tells bash to use the value of n as the name of the variable to use ($2, in this case).
You can use $# as array & array chopping methods:
echo ${#:$(($#-1)):1}
It means, use 1 element starting from $(($#-1))...
If some old versions of shells do not support ${array:start:length} syntax but support only ${array:start} syntax, use below hack:
echo ${#:$(($#-1))} | { read x y ; echo $x; } # OR
read x unused <<< `echo ${#:$(($#-1))}`