how to keep `$#` arguments seperated when using them in a string? - bash

Hey I'm writing a wrapper script for git that applies one git command to all its submodules e.g.: supergit commit -m "change message" commits to all submodules.
The script essentially does:
function git_foreach () {
git submodule foreach "git \"$#\" || : "
}
git_foreach "$#"
The problem is when the supergit call contains an argument with spaces (like in the commit message above) the space separated calls are interpreted as multiple arguments.
I read in this answer that the way to do it is to use "$#" but that doesn't work within a string.
Is there a way to expand $# to keep the quotes so that my function works as expected?
EDIT:
What I want is to pass the arguments to git submodule foreach, with supergit commit -m "commit message" I want to run:
git submodule foreach "git commit -m \"commit message\""

If Your /bin/sh Is Provided By Bash
printf %q will generate a version of your data correctly escaped to be parsed by a shell.
printf '%q ' "$#" generates a string containing an individually shell-escaped word for each argument in your array, with spaces after each word.
Thus:
git_foreach() {
local cmd_q
printf -v cmd_q '%q ' "$#"
git submodule foreach "git $cmd_q ||:"
}
...or, with bash 5.0 or newer (which adds a ${var#Q} expansion), we can make this one line:
git_foreach() { git submodule foreach "git ${##Q} ||:"; }
With current implementations of printf %q, git_foreach commit -m "commit message" invokes git submodule foreach 'git commit -m commit\ message' or a semantic equivalent; the escaping isn't identical to how you would write it by hand, but the effect of the command is exactly the same.
If You Need Broader Compatibility
Unfortunately, both printf %q and the newer ${variable#Q} expansion are able to generate strings that use bash-only syntax (particularly if your string contains newlines, tabs, or similar). If you don't control which shell git starts to run the foreach commands, then we need to generate a string that's escaped for consumption by any POSIX-compliant shell.
Bash doesn't have a feature for doing that... but Python does!
posix_escape() {
python3 -c 'import sys, shlex; print(" ".join([shlex.quote(s) for s in sys.argv[1:]]))' "$#"
}
git_foreach() {
local cmd_q
cmd_q=$(posix_escape "$#")
git submodule foreach "git $cmd_q ||:"
}

Related

Git alias for commit with branch name

I'm trying to create a simple bash alias to commit with my branch name in MacOs. For instance, if my branch if CS-12 I'd usually commit as follows:
git commit /file/location/myfile -m "CS-12 my message goes in here"
So I'm trying to create an alias which will receive only the file name and the message, ie:
gcm /file/location/myfile "my message goes in here"
I've got the following but it's not working:
alias gcm="echo git commit $1 -m \"$(current_branch) - $2\""
where current_branch is the function:
function current_branch() {
ref=$(git symbolic-ref HEAD 2> /dev/null) || \
ref=$(git rev-parse --short HEAD 2> /dev/null) || return
echo ${ref#refs/heads/}
}
which does work.
The output of running my alias:
gcm src/pages/register/Register.js "aasdasd asdasd"
is giving me back:
git commit -m master - src/pages/register/Register.js aasdasd asdasd
any idea what I'm doing wrong? Bash is not my area of expertise.
Thanks
The escaped quotes are 'stripped' by alias, so you need to escape them once more:
alias x="echo \\\"foo\\\""
x
"foo"
aliases do not take parameters. Just write a function:
gcm() { git commit "$1" -m "$(current_branch) - $2"; }
Note that there's really no need for aliases, and you shouldn't use them. Since at least 1996, the bash man page has stated: "For almost every purpose, aliases are superseded by shell functions."

How does eval stop the pathspec errors in this script?

I want to automate the many version control steps of Git. I was successful until I used git commit -S -m ${var} in my Bash script. This line gives me (pathspec errors x # of word) - 1... unless I use eval. How does eval make my script work?
I thought this article had the answer, but my issue involves a string, not an array.
Gif video of the broken vs. working Bash script
Broken code
brokenCommitCode () {
# Give it a multi-word, space-separated message
read -p 'Commit message (use quotes): ' commitMsg
commitMsg="'${commitMsg}'"
echo ${commitMsg}
git add -A &&
git commit -S -m ${commitMsg}
}
Working code
workingCommitCode () {
read -p 'Commit message (use quotes): ' commitMsg
commitMsg="'${commitMsg}'"
echo ${commitMsg}
git add -A &&
eval git commit -S -m ${commitMsg}
}
I expected the brokenCommitCode to commit properly with the message I enter on the prompt. The actual result is a pathspec error when it reaches git commit -S -m ${commitMsg}. How does eval make this work?
I'm using GNU bash, version 4.4.19(1)-release (x86_64-pc-msys) with git version 2.16.2.windows.1 on a Windows 8.1 PC.
Correct fix is
funname() {
read -p 'Commit message (use quotes): ' commitMsg
echo "${commitMsg}"
git add -A &&
git commit -S -m "${commitMsg}"
}
Why eval seems to fix:
single quotes where added to commitMsg variable (seems intent was to prevent message argument to be split on a whitespace)
looking what happens with the following message:
commitMsg="this is a message"
git commit -S -m ${commitMsg}
git commit -S -m this is a message
[error because "is" "a" "message" are taken as different additional arguments]
however it doesn't prevent because single quote is not re-interpreted but is like any other character in variable content
following with the example
git commit -S -m ${commitMsg}
git commit -S -m \'this is a message\'
[error "is" "a" "message'" are taken as different additional arguments]
with eval the single quotes are re-interpreted but also any other character which has a particular meaning in bash (;, &, ${..}, ..)
Suppose for example the following commit message which can inject arbitrary command.
commitMsg="message'; ls -l; echo 'done"
git commit -S -m 'message'; ls -l; echo 'done'

Perform command from command line inside directories from glob in bash shell script

In a bash shell script do-for.sh I want to perform a command inside all directories named in a glob using bash. This has been answered oodles of times, but I want to provide the command itself on the command line. In other words assuming I have the directories:
foo
bar
I want to enter
do-for * pwd
and have bash print the working directory inside foo and then inside bar.
From reading the umpteen answers on the web, I thought I could do this:
for dir in $1; do
pushd ${dir}
$2 $3 $4 $5 $6 $7 $8 $9
popd
done
Apparently though the glob * gets expanded into the other command line arguments variable! So the first time through the loop, for $2 $3 $4 $5 $6 $7 $8 $9 I expected foo pwd but instead it appears I get foo bar!
How can I keep the glob on the command line from being expanded into the other parameters? Or is there a better way to approach this?
To make this clearer, here is how I want to use the batch file. (This works fine on the Windows batch file version, by the way.)
./do-for.sh repo-* git commit -a -m "Added new files."
I will assume you are open to your users having to provide some kind of separator, like so
./do-for.sh repo-* -- git commit -a -m "Added new files."
Your script could do something like (this is just to explain the concept, I have not tested the actual code) :
CURRENT_DIR="$PWD"
declare -a FILES=()
for ARG in "$#"
do
[[ "$ARG" != "--" ]] || break
FILES+=("$ARG")
shift
done
if
[[ "${1-}" = "--" ]]
then
shift
else
echo "You must terminate the file list with -- to separate it from the command"
(return, exit, whatever you prefer to stop the script/function)
fi
At this point, you have all the target files in an array, and "$#" contains only the command to execute. All that is left to do is :
for FILE in "${FILES[#]-}"
do
cd "$FILE"
"$#"
cd "$CURRENT_DIR"
done
Please note that this solution has the advantage that if your user forgets the "--" separator, she will be notified (as opposed to a failure due to quoting).
In this case the problem is not the expansion of metacharacter, is just that your script has an undefined number of arguments of which the last one is the command to execute for all previous arguments.
#!/bin/bash
CMND=$(eval echo "\${$#}") # get the command as last argument without arguments or
while [[ $# -gt 1 ]]; do # execute loop for each argument except last one
( cd "$1" && eval "$CMND" ) # switch to each directory received and execute the command
shift # throw away 1st arg and move to the next one in line
done
Usage: ./script.sh * pwd or ./script.sh * "ls -l"
To have the command followed by arguments (ex. ./script.sh * ls -l) the script has to be longer because each argument has to be tested if it's a directory until the command is identified (or backwards until a dir is identified).
Here is an alternative script that would accept the syntax: ./script.sh <dirs...> <command> <arguments...>
For example: ./script.sh * ls -la
# Move all dirs from args to DIRS array
typeset -i COUNT=0
while [[ $# -gt 1 ]]; do
[[ -d "$1" ]] && DIRS[COUNT++]="$1" && shift || break
done
# Validate that the command received is valid
which "$1" >/dev/null 2>&1 || { echo "invalid command: $1"; exit 1; }
# Execute the command + it's arguments for each dir from array
for D in "${DIRS[#]}"; do
( cd "$D" && eval "$#" )
done
Here is how I would do it:
#!/bin/bash
# Read directory arguments into dirs array
for arg in "$#"; do
if [[ -d $arg ]]; then
dirs+=("$arg")
else
break
fi
done
# Remove directories from arguments
shift ${#dirs[#]}
cur_dir=$PWD
# Loop through directories and execute command
for dir in "${dirs[#]}"; do
cd "$dir"
"$#"
cd "$cur_dir"
done
This loops over the arguments as seen after expansion, and as long as they are directories, they are added to the dirs array. As soon as the first non-directory argument is encountered, we assume that now the command starts.
The directories are then removed from the arguments with shift, and we store the current directory in cur_dir.
The last loop visits each directory and executes the command consisting of the rest of the arguments.
This works for your
./do-for.sh repo-* git commit -a -m "Added new files."
example – but if repo-* expands to anything other than directories, the script breaks because it will try to execute the filename as part of the command.
It could be made more stable if, for example, the glob and the command were separated by an indicator such as --, but if you know that the glob will always be just directories, this should work.
I will begin with the Windows batch file that you mentioned twice as working. The big difference is that on Windows, the shell doesn’t make any globbing, leaving it to the various commands (and each of them does it differently), while on Linux/Unix the globbing is usually done by the shell, and can be prevented by quoting or escaping. Both the Windows approach and the Linux approach have their merits, and they compare differently in different use cases.
For regular bash users, quoting
./do-for.sh repo-'*' git commit -a -m "Added new files."
or escaping
./do-for.sh repo-\* git commit -a -m "Added new files."
are the simplest solution, because they are what they consistently use on a daily basis. If your users need a different syntax, you have all the solutions proposed so far, that I will classify into four categories before proposing my own (note that in each example below do-for.sh stands for a different script adopting the respective solution, which can be found in one of the other answers.)
Disable shell globbing. This is clumsy, because, even if you remember which shell option does it, you have to remember to reset it to default to have the shell working normally afterwards.
Use a separator:
./do-for.sh repo-* -- git commit -a -m "Added new files."
This works, is similar to the solution adopted in similar situations with other shell commands, and fails only if your expansion of directory names includes a directory name exactly equal to the separator (an unlikely event, which wouldn’t happen in the above example, but in general might happen.)
Have the command as the last argument, all the rest are directories:
./do-for.sh repo-* 'git commit -a -m "Added new files."'
This works, but again, it involves quoting, possibly even nested, and there is no point in preferring it to the more usual quoting of globbing characters.
Try to be smart:
./do-for.sh repo-* git commit -a -m "Added new files."
and consider to be dealing with directories till you hit a name which is not a directory. This would work in many cases, but might fail in obscure ways (e.g. when you have a directory named like the command).
My solution doesn’t belong to any of the mentioned categories. In fact, what I propose is not to use * as a globbing character in the first argument of your script. (This is similar to the syntax used by the split command where you provide a non-globbed prefix argument for the files to be generated.) I have two versions (code below). With the first version, you would do the following:
# repo- is a prefix: the command will be excuted in all
# subdirectories whose name starts with it
./do-for.sh repo- git commit -a -m "Added new files."
# The command will be excuted in all subdirectories
# of the current one
./do-for.sh . git commit -a -m "Added new files."
# If you want the command to be executed in exactly
# one subdirectory with no globbing at all,
# '/' can be used as a 'stop character'. But why
# use do-for.sh in this case?
./do-for.sh repo/ git commit -a -m "Added new files."
# Use '.' to disable the stop character.
# The command will be excuted in all subdirectories of the
# given one (paths have to be always relative, though)
./do-for.sh repos/. git commit -a -m "Added new files."
The second version involves using a globbing character the shell knows nothing about, such as SQL’s % character
# the command will be excuted in all subdirectories
# matching the SQL glob
./do-for.sh repo-% git commit -a -m "Added new files."
./do-for.sh user-%-repo git commit -a -m "Added new files."
./do-for.sh % git commit -a -m "Added new files."
The second version is more flexible, as it allows non-final globs, but is less standard for the bash world.
Here is the code:
#!/bin/bash
if [ "$#" -lt 2 ]; then
echo "Usage: ${0##*/} PREFIX command..." >&2
exit 1
fi
pathPrefix="$1"
shift
### For second version, comment out the following five lines
case "$pathPrefix" in
(*/) pathPrefix="${pathPrefix%/}" ;; # Stop character, remove it
(*.) pathPrefix="${pathPrefix%.}*" ;; # Replace final dot with glob
(*) pathPrefix+=\* ;; # Add a final glob
esac
### For second version, uncomment the following line
# pathPrefix="${pathPrefix//%/*}" # Add a final glob
tmp=${pathPrefix//[^\/]} # Count how many levels down we have to go
maxDepth=$((1+${#tmp}))
# Please note that this won’t work if matched directory names
# contain newline characters (comment added for those bash freaks who
# care about extreme cases)
declare -a directories=()
while read d; do
directories+=("$d")
done < <(find . -maxdepth "$maxDepth" -path ./"$pathPrefix" -type d -print)
curDir="$(pwd)"
for d in "${directories[#]}"; do
cd "$d";
"$#"
cd "$curDir"
done
As in Windows, you would still need to use quotes if the prefix contains spaces
./do-for.sh 'repository for project' git commit -a -m "Added new files."
(but if the prefix does not contain spaces, you can avoid quoting it and it will correctly deal with any space-containing directory names beginning with that prefix; with obvious changes, the same is true for %-patterns in the second version.)
Please note the other relevant differences between a Windows and a Linux environment, such as case sensitivity in pathnames, differences in which characters are considered special, and so on.
In bash you may execute "set -o noglob" which will inhibit the shell to expand path names (globs). But this has to be set on the running shell before you execute the script, otherwise you should quote any meta character which you provide in the arguments.
find-while-read combination is one of the safest combination to parse file names. Do something like below
#!/bin/bash
myfunc(){
cd "$2"
eval "$1" # Execute the command parsed as an argument
}
cur_dir=$(pwd) # storing the current directory
find . -type d -print0 | while read -rd '' dname
do
myfunc "pwd" "$dname"
cd "$cur_dir" #Remember myfunc changes the current working dir, so you need this
done
Why not keep it simple and create a shell function that uses find but eases the burden for your users of typing out its commands, for example:
do_for() { find . -type d \( ! -name . \) -not -path '*/\.*' -name $1 -exec bash -c "cd '{}' && "${#:2}" " \; }
So they can type something like do_for repo-* git commit -a -m "Added new files."
Note, if you want to use the * by itself, you'll have to escape it:
do_for \* pwd
Wildcards are evaluated by the shell before being passed to any program or script. There is nothing you can do about that.
But if you accept quoting the globbing expression then this script should to do the trick
#!/usr/bin/env bash
for dir in $1; do (
cd "$dir"
"${#:2}"
) done
I tried it out with two test directories and it seems to be working. Use it like this:
mkdir test_dir1 test_dir2
./do-for.sh "test_dir*" git init
./do-for.sh "test_dir*" touch test_file
./do-for.sh "test_dir*" git add .
./do-for.sh "test_dir*" git status
./do-for.sh "test_dir*" git commit -m "Added new files."
Nobody proposing a solution using find ? Why not try something like this:
find . -type d \( -wholename 'YOURPATTERN' \) -print0 | xargs -0 YOURCOMMAND
Look at man find for more options.

ZSH alias with parameter

I am trying to make an alias with parameter for my simple git add/commit/push.
I've seen that a function could be used as an alias, so I tried but I didn't make it.
Before I had:
alias gitall="git add . ; git commit -m 'update' ; git push"
But I want to be able to modify my commits:
function gitall() {
"git add ."
if [$1 != ""]
"git commit -m $1"
else
"git commit -m 'update'"
fi
"git push"
}
If you really need to use an alias with a parameter for some reason, you can hack it by embedding a function in your alias and immediately executing it:
alias example='f() { echo Your arg was $1. };f'
I see this approach used a lot in .gitconfig aliases.
You can't make an alias with arguments*, it has to be a function. Your function is close, you just need to quote certain arguments instead of the entire commands, and add spaces inside the [].
gitall() {
git add .
if [ "$1" != "" ] # or better, if [ -n "$1" ]
then
git commit -m "$1"
else
git commit -m update
fi
git push
}
*: Most shells don't allow arguments in aliases, I believe csh and derivatives do, but you shouldn't be using them anyway.
I used this function in .zshrc file:
function gitall() {
git add .
if [ "$1" != "" ]
then
git commit -m "$1"
else
git commit -m update # default commit message is `update`
fi # closing statement of if-else block
git push origin HEAD
}
Here git push origin HEAD is responsible to push your current branch on remote.
From command prompt run this command: gitall "commit message goes here"
If we just run gitall without any commit message then the commit message will be update as the function said.
"git add ." and the other commands between " are just strings for bash, remove the "s.
You might want to use [ -n "$1" ] instead in your if body.
I tried the accepted answer (Kevin's) but was getting the following error
defining function based on alias `gitall'
parse error near `()'
Hence changed the syntax to this, based on the git issue and it worked.
function gitall {
git add .
if [ "$1" != "" ]
then
git commit -m "$1"
else
git commit -m update
fi
git push
}
I can easily add params just using $1.
Eg:
alias gsf="git show --name-only $1"
works just fine. To call it I just use gsf 2342aa225

bash how to make command work

I have following bash function in my ~/.bashrc
function gitlab {
MSG='first commit'
CMD="git commit -m '${MSG}'"
echo $CMD
$CMD
}
Here is the result
$ gitlab
git commit -m 'first commit'
error: pathspec 'commit'' did not match any file(s) known to git.
What's the fix?
BASH FAQ entry #50: "I'm trying to put a command in a variable, but the complex cases always fail!"
Definitely read BashFAQ/050 that Ignacio linked to.
You could try this, though:
function gitlab {
local PS4='Running: '
local msg='first commit'
bash -xc "git commit -m '$msg'"
}
I suppose you should use \" instead of ' so it should be something like:
CMD="git commit -m \"${MSG}\""
Try putting your commit message in double quotes, as single and double quotes mean different things to bash.
function gitlab {
MSG="first commit"
CMD=`git commit -m \"${MSG}\"`
echo $CMD
$CMD
}

Resources