How to use bash function with arguments in make file - bash

I want to use the bash function in the make file. Without argument, it's working, but how I can use it with arguments.
all :
foo () { echo $1} ; foo "hello"

Writing bash code inside Makefile require special handling for all the characters that are special to Make. In particular, '$', which must be escaped (doubled). Also note that ';' is required before '}'.
In general, when writing bash/sh snipplet inside Makefile, ${XYZ} (or $(XYZ)) will refer to MAKE variable, and $$XYZ (or $${XYZ}}) will refer to the sh variable.
all:
foo() { echo $$1 ; } ; foo "hello"

Related

Parameter expansion with replacement, avoid additional variable

I'm trying to join input $* which is one parameter consisting of all the parameters added together.
This works.
#!/bin/bash
foo() {
params="${*}"
echo "${params//[[:space:]]/-}"
}
foo 1 2 3 4
1-2-3-4
However, is it possible to skip the assignment of variable?
"${"${*}"//[[:space:]]/-}"
I'm getting bad substitution error.
I can also do
: "${*}"
echo "${_//[[:space:]]/-}"
But it feels hacky.
One option could be to set bash's internal field separator, IFS, to - locally and just echo "$*":
foo() {
local IFS=$'-'
echo "$*"
}
To answer your question, you can do global pattern substitutions on the positional parameters like this:
${*//pat/sub}
${#//pat/sub}
And also arrays like this:
${arr[*]//pat/sub}
${arr[#]//pat/sub}
This won’t join the parameters, but substitute inside them.
Setting IFS to dash adds a dash in between each parameter for echo "$*", or p=$*, but won’t replace anything inside a parameter.
Eg:
$ set -- aa bb 'cc cc'
$ IFS=-
$ echo "$*"
aa-bb-cc cc
To remove all whitespace, including inside a parameter, you can combine them:
IFS=-
echo "${*//[[:space:]]/-}"
Or just assign to a name first, like you were doing:
no_spaces=$*
echo "${no_spaces//[[:space:]]/-}"

How to use a single bash command like $1 or $2 or $3 as a string of text instead of one word?

How can I use a bashrc command like below
# .bashrc file
# google_speech
say () {
google_speech -l en "$1"
}
as a string of text, since the above code only reads out the first word of the sentence or paragraph i paste.
like for example if i go into terminal and type:
$ say hello friends how are you
then the script only thinks i typed
$ say hello
Try to use "$#" (with double quotes arround) to get all the arguments of your function:
$ declare -f mysearch # Showing the definition of mysearch (provided by your .bashrc)
mysearch ()
{
echo "Searching with keywords : $#"
}
$ mysearch foo bar # execution result
Searching with keywords : foo bar
Function or scripts arguments are like arrays, so you can use:
1) $# / ${#array[#]} to getting the number of arguments / array's elements.
2) $1 / ${array[1]} to getting the first argument / array's element.
3) $# / ${array[#]} to getting all arguments / array's elements.
EDIT: according to chepner's comment:
Using $# inside a larger string can sometimes produce unintended results. Here, the intent is to produce a single word, so it would be better to use $*
And here's a good topic with great answers explaining the differences.
EDIT 2: Do not forget to put double quotes around $# or $*, maybe your google_speach is taking only one arg. Here's a demo to give you a better understanding:
$ mysearch () { echo "Searching with keywords : $1"; }
$ mysearch2 () { mysearch "$*"; }
$ mysearch2 Hello my dear
Searching with keywords : Hello my dear
$ mysearch3 () { mysearch $*; } # missing double quotes
$ mysearch3 Hello my dear
Searching with keywords : Hello # the other arguments are at least ignored (or sometimes the program will fail when he's controlling the args).

How to parse a string into variables?

I know how to parse a string into variables in the manner of this SO question, e.g.
ABCDE-123456
becomes:
var1=ABCDE
var2=123456
via, say, cut. I can do that in one script, no problem.
But I have a few dozen scripts which parse strings/arguments all in the same fashion (same arguments & variables, i.e. same parsing strategy).
And sometimes I need to make a change or add a variable to the parsing mechanism.
Of course, I could go through every one of my dozens of scripts and change the parsing manually (even if just copy & paste), but that would be tedious and more error-prone to bugs/mistakes.
Is there a modular way to do parse strings/arguments as such?
I thought of writing a script which parses the string/args into variables and then exports, but the export command does not work form child-to-parent, (only vice-versa).
Something like this might work:
parse_it () {
SEP=${SEP--}
string=$1
names=${#:2}
IFS="$SEP" read $names <<< "$string"
}
$ parse_it ABCDE-123456 var1 var2
$ echo "$var1"
ABCDE
$ echo "$var2"
123456
$ SEP=: parse_it "foo:bar:baz" id1 id2 id3
$ echo $id2
bar
The first argument is the string to parse, the remaining arguments are names of variables that get passed to read as the variables to set. (Not quoting $names here is intentional, as we will let the shell split the string into multiple words, one per variable. Valid variable names consist of only _, letters, and numbers, so there are no worries about undesired word splitting or pathname generation by not quoting $names). The function assumes the string uses a single separator of "-", which can be overridden via the environment.
For more complex parsing, you may want to use a custom regular expression (bash 4 or later required for the -g flag to declare):
parse_it () {
reg_ex=$1
string=$2
shift 2
[[ $string =~ $reg_ex ]] || return
i=1
for name; do
declare -g "$name=${BASH_REMATCH[i++]}"
done
}
$ parse_it '(.*)-(.*):(.*)' "abc-123:xyz" id1 id2 id3
$ echo "$id2"
123
I think what you really want is to write your function in one script and include it in all of your other scripts.
You can include other shell scripts by the source or . command.
For example, you can define your parse function in parseString.sh
function parseString {
...
}
And then in any of your other script, do
source parseString.sh
# now we can call parseString function
parseString abcde-12345

Exporting the full environment to GNU Parallel

I find it somewhat annoying that I cannot use aliases in GNU Parallel:
alias gi="grep -i"
parallel gi bar ::: foo
/bin/bash: gi: command not found
I had somewhat come to terms with that it is just the way it is. But reading Accessing Associative Arrays in GNU Parallel I am starting to think: Does it really have to be this way?
Is is possible to make a bash function, that collects all of the environment into a function, exports that function and calls GNU Parallel, which will then import the environment in the spawned shell using that function?
So I am not talking about a specialized solution for the gi-alias, but a bash function that will take all aliases/functions/variables (without me having to name them explicitly), package those into a function, that can be activated by GNU Parallel.
Something similar to:
env_parallel() {
# [... gather all environment/all aliases/all functions into parallel_environment() ...]
foreach alias in all aliases {
append alias definition to definition of parallel_environment()
}
foreach variable in all variables (including assoc arrays) {
append variable definition to definition of parallel_environment()
# Code somewhat similar to https://stackoverflow.com/questions/24977782/accessing-associative-arrays-in-gnu-parallel
}
foreach function in all functions {
append function definition to definition of parallel_environment()
}
# make parallel_environment visible to GNU Parallel
export -f parallel_environment
# Running parallel_environment will now create an environment with
# all variables/all aliases/all functions set in current state
# (with the exception of the function parallel_environment of course)
# Inside GNU parallel:
# if set parallel_environment(): prepend it to the command to run
`which parallel` "$#"
}
# Set an example alias
alias fb="echo fubar"
# Set an example variable
BAZ=quux
# Make an example function
myfunc() {
echo $BAZ
}
# This will record the current environment including the 3 examples
# put it into parallel_environment
# run parallel_environment (to set the environment)
# use the 3 examples
env_parallel parallel_environment\; fb bar {}\; myfunc ::: foo
# It should give the same output as running:
fb bar foo; myfunc
# Outputs:
# fubar bar foo
# quux
Progress: This seems to be close to what I want activated:
env_parallel() {
export parallel_environment='() {
'"$(echo "shopt -s expand_aliases"; alias;typeset -p | grep -vFf <(readonly);typeset -f)"'
}'
`which parallel` "$#"
}
VAR=foo
myfunc() {
echo $VAR $1
}
alias myf=myfunc
env_parallel parallel_environment';
' myfunc ::: bar # Works (but gives errors)
env_parallel parallel_environment';
' myf ::: bar # Works, but requires the \n after ;
So now I am down to 1 issue:
weed out all the variables that cannot be assigned value (e.g BASH_ARGC)
How do I list those?
GNU Parallel 20140822 implements this. To activate it you will need to run this once (e.g. in .bashrc):
env_parallel() {
export parallel_bash_environment='() {
'"$(echo "shopt -s expand_aliases 2>/dev/null"; alias;typeset -p | grep -vFf <(readonly; echo GROUPS; echo FUNCNAME; echo DIRSTACK; echo _; echo PIPESTATUS; echo USERNAME) | grep -v BASH_;typeset -f)"'
}'
# Run as: env_parallel ...
`which parallel` "$#"
unset parallel_bash_environment
}
And call GNU Parallel as:
env_parallel ...
That should put the myth to rest that it is impossible to export aliases: all you need is a little Behändigkeit (Thanks a lot to #rici for the inspiration).
In principle, it should be possible. But, as usual, there are a lot of details.
First, it is quite possible in bash for a name to be simultaneously a function, a variable (scalar or array) and an alias. Also, the function and the variable can be exported independently.
So there would be a certain ambiguity in env_parallel foo ... in the case that foo has more than one definition. Possibly the best solution would be to detect the situation and report an error, using a syntax like:
env_parallel -a foo -f bar
in order to be more specific, if necessary.
A simpler possibility is to just export the ambiguity, which is what I do below.
So the basic logic to the importer used in env_parallel might be something like this, leaving out lots of error checking and other niceties:
# Helper functions for clarity. In practice, since they are all short,
# I'd probably in-line all of these by hand to reduce name pollution.
get_alias_() { alias "$1" 2>/dev/null; }
get_func_() { declare -f "$1" 2>/dev/null; }
get_var_() { [[ -v "$1" ]] && declare -p "$1" | sed '1s/--\?/-g/'; }
make_importer() {
local name_
export $1='() {
'"$(for name_ in "${#:2}"; do
local got_=()
get_alias_ "$name_" && got_+=(alias)
get_func_ "$name_" && got_+=(function)
get_var_ "$name_" && got_+=(variable)
if [[ -z $got_ ]]; then
echo "Not found: $name_" >>/dev/stderr
elif (( ${#got_[#]} > 1 )); then
printf >>/dev/stderr \
"Ambiguous: %s is%s\n" \
$name_ "$(printf " %s" "${got_[#]}")"
fi
done)"'
}'
}
In practice, there's no real point defining the function in the local environment if the only purpose is to transmit it to a remote shell. It would be sufficient to print the export command. And, while it is convenient to package the import into a function, as in Accessing Associative Arrays in GNU Parallel,
it's not strictly necessary. It does make it a lot easier to pass the definitions through utilities like Gnu parallel, xargs or find, which is what I typically use this hack for. But depending on how you expect to use the definitions, you might be able to simplify the above by simply prepending the list of definitions to the given command. (If you do that, you won't need to fiddle the global flag with the sed in get_var_.)
Finding out what is in the environment
In case it is useful, here is how to get a list of all aliases, functions and variables:
Functions
declare -F | cut -d' ' -f3
Aliases (Note 1)
alias | awk '/^alias /{print substr($2,1,index($2,"=")-1)}'
Variables (Note 1)
declare -p | awk '$1=="declare"{o=(index($3, "="));print o?substr($3,1,o-1):$3}'
In the awk program, you could check for variable type by looking at $2, which will is usually -- but could be -A for an associative array, -a for an array with integer keys, -i for an integer, -x for exported and -r for readonly. (More than one option may apply; -aix is an "exported" (not implemented) integer array.
Note 1
The alias and declare -p commands produce "reusable" output, which could be eval'ed or piped into another bash, so the values are quoted. Unfortunately, the quoting is just good enough for eval; it's not good enough to avoid confusion. It is possible to define, for example:
x='
declare -a FAKE
'
in which case the output of declare -p will include:
declare -x='
declare -a FAKE
'
Consequently, the lists of aliases and variables need to be treated as "possible names": all names will be included, but it might be that everything included is not a name. Mostly that means being sure to ignore errors:
for a in "${_aliases[#]}"; do
if
defn=$(alias $a 2>>/dev/null)
then
# do something with $defn
fi
done
As is often the case, the solution is to use a function, not an alias. You must first export the function (since parallel and bash are both developed by GNU, parallel knows how to deal with functions as exported by bash).
gi () {
grep -i "$#"
}
export -f go
parallel gi bar ::: foo

How can I escape an arbitrary string for use as a command line argument in Windows?

I have a list of strings and I want to pass those strings as arguments in a single Windows command line call. For simple alphanumeric strings it suffices to just pass them verbatim:
> script.pl foo bar baz yes no
foo
bar
baz
yes
no
I understand that if an argument contains spaces or double-quotes, I need to backslash-escape the double-quotes and backslashes, and then double-quote the argument.
> script.pl foo bar baz "\"yes\"\\\"no\""
foo
bar
baz
"yes"\"no"
But when I try to pass an argument with literal percent signs, this happens:
> script.pl %PATH%
C:\Program
Files\PHP\;C:\spaceless\perl\bin\;C:\Program
Files\IBM\Java60\bin;
(...etc.)
Double quoting doesn't work:
> script.pl "%PATH%"
C:\Program Files\PHP\;C:\spaceless\perl\bin\;C:\Program Files\IBM\Java60\bin; (...etc.)
Nor does backslash-escaping (notice how the backslashes are present in the output):
> script.pl \%PATH\%
\%PATH\%
Also, the rules are inconsistent for backslash-escaping backslashes:
> script.pl "\\yes\\"
\\yes\
> script.pl "\yes\\"
\yes\
> script.pl "\yes\"
\yes"
Also, doubtless there are special characters in the Windows command line shell, much like there are in all shells. What, then, is the general procedure for safely escaping arbitrary command line arguments for use at the Windows command line?
The ideal answer will describe a function escape() which can be used in situations like the following (a Perl example):
$cmd = join " ", map { escape($_); } #args;
Here are some more example strings which should be safely escaped by this function (I know some of these look Unix-like, that's deliberate):
yes
no
child.exe
argument 1
Hello, world
Hello"world
\some\path with\spaces
C:\Program Files\
she said, "you had me at hello"
argument"2
\some\directory with\spaces\
"
\
\\
\\\
\\\\
\\\\\
"\
"\T
"\\T
!1
!A
"!\/'"
"Jeff's!"
$PATH
%PATH%
&
<>|&^
()%!^"<>&|
>\\.\nul
malicious argument"&whoami
*#$$A$##?-_
Here is an msdn blogpost showing how. It however assumes that every command line program internally uses CommandLineToArgvW to parse it's command line (not a shabby assumption, since it's part of the Shell32 library).
Original link (may not work): http://blogs.msdn.com/b/twistylittlepassagesallalike/archive/2011/04/23/everyone-quotes-arguments-the-wrong-way.aspx
Web archive link: https://web.archive.org/web/20190109172835/https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
To escape a command line argument, use the following:
sub escapeArg {
my $arg = shift;
# Sequence of backslashes followed by a double quote:
# double up all the backslashes and escape the double quote
$arg =~ s/(\\*)"/$1$1\\"/g;
# Sequence of backslashes followed by the end of the arg,
# which will become a double quote later:
# double up all the backslashes
$arg =~ s/(\\*)$/$1$1/;
# All other backslashes do not need modifying
# Double-quote the whole thing
$arg = "\"".$arg."\"";
# Escape shell metacharacters
$arg =~ s/([()%!^"<>&|;, ])/\^$1/g;
return $arg;
}
To escape the actual command line command, for example when invoking a command with a ridiculous name such as ()!&%PATH%^;, .exe (which is perfectly legal), use the following:
sub escapeCmd {
my $arg = shift;
# Escape shell metacharacters
$arg =~ s/([()%!^"<>&|;, ])/\^$1/g;
return $arg;
}
Note that using escapeArg() for the command will not work.
Sources:
http://blogs.msdn.com/b/twistylittlepassagesallalike/archive/2011/04/23/everyone-quotes-arguments-the-wrong-way.aspx
http://support.microsoft.com/kb/103368

Resources