I wrote a script to ease the syncing and building of Android source. I tried adding a function to cherrypick patches, but I can't get it to work properly. I know it's because of the forward slashes, but I don't know how to protect/escape them.
Part of the code is:
echo "Copy/paste the project folder, i.e. 'frameworks/base'"
read folder
echo ""
echo "Now paste the cherry-pick git link, i.e. 'git fetch <someproject> refs/changes/... && git cherry-pick FETCH_HEAD'"
read cherry
echo ""
Begin
clear
echo ""
export IFS="&&"
for x in $cherry
do
cd ${CM}/${folder}
CHERRY=$(trim "$x")
$CHERRY
done
Let's say that the 'cherry' variable is:
git fetch http://r.cyanogenmod.com/CyanogenMod/android_frameworks_base refs/changes/68/22968/2 && git cherry-pick FETCH_HEAD
I would get this error:
/home/tristan202/bin/build_cm.sh: line 159: git fetch http://r.cyanogenmod.com/CyanogenMod/android_frameworks_base refs/changes/91/23491/2: No such file or directory
/home/tristan202/bin/build_cm.sh: line 159: git cherry-pick FETCH_HEAD: command not found
I cannot figure out why it fails.
The 'trim' function it calls is a function that trims leading and trailing spaces. If I do echo "$CHERRY" within the for loop, the commands are printed correctly, but it still fails.
I will give your another example:
cmd='echo hello && echo world'
$cmd
The result is:
hello && echo world
bash parses the command $cmd as Simple Commands not Lists of Commands.
After Parameter Expansion, && is passed as argument to echo(1st word after Word Splitting).
The solution is pulling && out:
cmd1='echo hello'
cmd2='echo world'
$cmd1 && $cmd2
Once you put && in a variable it ceases to be interpreted as separating two commands:
$ A="echo a && echo b"
$ echo $A
echo a && echo b
$ echo c && ${A}
c
a && echo b
So you need to avoid putting && into a variable.
git was even telling you that the && was the problem in its error message.
Related
This question already has answers here:
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 5 months ago.
This is probably a simple one for a bash scripter, which I am not.
I'm running a cron job that downloads some data, and then depending on that data, may or may not modify a second file. After the job, I want to git commit one or both files. For the conditional commit, I tried this in a .sh script:
# attempt to capture whether MyNotes.txt was changed
# by counting lines in git status output
mywc=(git status -s MyNotes.txt | wc -l)
echo $mywc found!
if [ $mywc = 1 ]; then
echo Add file for commit
else
echo Nothing to add
fi
I'm pretty much getting nowhere; this thing seems to fail on the first line with syntax error near unexpected token '|'. If I run git status -s MyNotes.txt | wc -l on the command line, I get the numeric output I expect.
What am I doing wrong and how can I make this work?
If there's a more elegant way to determine whether a file changed, feel free to share.
Also, for my edification, how could I get this to work without the interim mywc variable? I.e., if I wanted to just do the command within the if, something like this:
if [[ $(git status -s MyNotes.txt | wc -l) = 1 ]]; then
...
Thanks!
What am I doing wrong and how can I make this work?
put a dollar before parenthesis.
foo=$(command)
The thing you are using looks like a bash array
declare -a letters=(a b c d)
If there's a more elegant way to determine whether a file changed, feel free to share.
Consider this:
$ git diff -s --exit-code README.md || echo has changed
has changed
$ git checkout README.md
Updated 1 path from the index
$ git diff -s --exit-code README.md || echo has changed
The OR (||) runs if the first command exits with a non-zero code.
Same thing essentially:
$ false || echo false exits with 1
false exits with 1
$ true || echo will not trigger
An aspect of bash that people overlook is that [[, ]], [ and ] are separate commands. They have return codes too. With this knowledge, you can leverage the return codes with if and any other command.
$ if true; then echo yes; else echo no; fi
yes
$ if false; then echo yes; else echo no; fi
no
So for detecting changes in a tracked file:
$ if git diff -s --exit-code README.md; then echo same as in index; else echo changed; fi
same as in index
$ echo 123 >> README.md
$ if git diff -s --exit-code README.md; then echo same as in index; else echo changed; fi
changed
With all of that said...
Just add the file. You don't need to check anything. If it hasn't changed, nothing will happen.
$ echo foo >> myfile
$ git add myfile
$ git commit -m 'maybe changed' myfile
[master b561cc1] maybe changed
1 file changed, 1 insertion(+), 1 deletion(-)
$ git add myfile
$ git commit -m 'maybe changed' myfile
no changes added to commit (use "git add" and/or "git commit -a")
if you need to avoid a non-zero exit code (such as with set -e), just put a || true after the command that you want to ignore the exit status of:
$ cat foo.sh
#!/bin/basho
set -e
echo foo >> myfile
git add myfile
git commit -m 'maybe changed' myfile
git add myfile
git commit -m 'maybe changed' myfile > /dev/null || true
echo no error here. it\'s fine..
false
echo fill never reach this.
Try running that script and see what happens
I search for a way for checking if file changed.
git diff --exit-code -s <path>
Now the bash scripter knows that every command returns a status code which can be checked with $?. In case everything went smoothly, 0 is returned. In that case we get 0 if file is not changed.
Every bash scripter knows too that you can use that with && and || operators (because of lazy evaluation) to write such construct:
git diff --exit-code -s <path> && echo "should add file"
About your edification, what you wrote is perfectly fine!
As CryptoFool pointed out in a comment, I failed to include a $ in my variable assignment. Simple fix in the first line of my script:
mywc=$(git status -s MyNotes.txt | wc -l)
As matt pointed out in a subsequent comment, doing a git add on a file that hasn't changed has no effect. It won't stage the file for commit. So instead of doing conditional logic to determine whether to git add myfile.txt, I'll just blindly execute git add myfile.txt, which will either stage the file if there are changes, or do nothing if there are no changes. Therefore, my entire script can be replaced with one line:
git add MyNotes.txt
I have this for loop
for repository in ./*/; do
echo $repository && cd $repository && git checkout -b prod && cd - >/dev/null;
done
But if branch prod already exists it prints a message and exit the loop.
How can ignore this error and just go to the next directory ?
Thanks
So the problem is that git checkout -b prod returns failure to the shell if the branch already exists. Since it's connected to the next command (cd -) with the conditional operator &&, that next command only runs if git succeeds. So when git fails, the cd doesn't run, and your shell is left in the wrong directory to continue its loop.
In general, when you want your code to continue even if a command fails, separate the commands with ; or newlines instead of &&.
But a better solution in this case is to just do the cd in a subshell so that it doesn't affect the outer loop's working directory and you don't have to cd - at all:
for repository in ./*/; do
echo "$repository" && (
cd "$repository" && git checkout -b prod
)
done
That will work fine even if the branch creation fails. It will still print out the error message; if you don't want to see those, add the redirect:
for repository in ./*/; do
echo "$repository" && (
cd "$repository" && git checkout -b prod
) 2>/dev/null
done
I've also quoted the expansion of $repository in the commands, which you should almost always do in shell scripts. With the unquoted version, you would get an error if any of the repo directory names had spaces in them, for instance.
Also, that "no side effects in a subshell" thing is great for doing part of your work in a different directory, but it applies more widely. If you had a more complicated loop that set any shell variables or anything while it was in the subdir, those would also be lost. Just something to keep in mind.
Like this
home=$PWD
for repository in "$home"/*/; do
basename "$repository" # to 'echo' $repository
cd "$repository" && git checkout -b prod
done
Better use pushd and popd and additionally it is saver to use find:
while read -r repository; do
pushd "${repository}"
if git checkout -b prod; then
echo "git checkout success"
else
echo "git chechout error"
fi
popd
done < <( find . -mindepth 1 -maxdepth 1 -type d -print )
I'm experimenting a little bit with a shell script, which should run git commands for multiple repositories on the same level. This project structure might be a bad idea, but this is another story.
Everything works fine until I've run into this problem:
DETAIL="test test" && command="commit -m '${DETAIL}'" && echo $(git ${command})
# -> error: pathspec 'test'' did not match any file(s) known to git.
I've also tried other opportunities like
DETAIL="test test" && command="commit -m ${DETAIL}" && echo $(git ${command})
DETAIL="test test" && command="commit -m $DETAIL" && echo $(git ${command})
All give the same result (see above). I've also scanned these docs about string expansion, but I don't have the problem, that the variables/strings might be null or undefined. The last echo is not the problem, you can also store the result of $(git status) in a variable and echo this one (my way in the script).
I know, there are similar questions, but I did not found a similar scenario yet, since I'm just dealing with simple and non-null strings, but with (too?) many quotes.
Interesting variant:
DETAIL="test test" && command="commit -m '${DETAIL}'" && echo $("git ${command}")
# -> git commit -m 'test test': command not found # WHAT?
Also interesting, just:
command="commit -m 'test'" && echo $(git ${command})
works fine.
Use bash arrays with proper quoting...
DETAIL="test test" && command=(commit -m "$DETAIL") && git "${command[#]}"
To your code:
echo "$(command)" is the same as command (ok, trailing empty newlines are removed)
"command blabla" does not execute file command with the first argument blabla. It will execute a filename named exactly with space command blabla.
Inside $("git ${command}") you want to execute a filename named git commit -m 'test test' (exactly, this is the whole filename name, with spaces, after ${command} is expanded). As on you system there is no file named git commit -m 'test test' bash returns command not found.
For example, I can do this with a subshell:
VAL=$( do_something )
but how do I achieve the same thing with curly braces so the command is NOT executing in a subshell? I.e. this does not work:
VAL={ do_something; }
TIA.
I'm not sure I understand the reasoning for what you're trying to accomplish, but if you can elaborate a bit more I might be able to help you.
I do recommend reading this fantastic write up about what's actually going on though, and why I don't think you want to invoke a process without a subshell.
However, to try and answer what you've asked:
You can't really run a command inside ${}, except in the fallback clause for when a value is not set (in POSIX sh or bash; might be feasible in zsh, which allows all manner of oddball syntax).
However, you can call cd like this if you really wanted this:
cdr() {
if (( $# )); then
command cd "$#"
else
local home
home=$(git rev-parse --show-toplevel 2>/dev/null) || home=$HOME
command cd "$home"
fi
}
Note
Using a function lets us test our argument list, use branching logic, have local variables, &c.
command cd is used to call through to the real cd implementation rather than recursing.
set -e is kinda stiff. Try something like
trap 'err=$?;
echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
exit $err;' ERR
This is a lot more informative when reading through your logs, and you can put a similar command inside the subshell. Yes, it means adding it inside the subshell... but I often do this sort of thing in function definitions that get called in subshells. Works well.
In use:
$ trap 'echo BOOM' ERR # parent shell trap for demo
$ false # trigger manually for demo
BOOM
$ x="$( trap 'err=$?;
> echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
> exit $err;' ERR
> date
> pwd
> false
> echo "I shan't"
> )"
ERROR 1 in bash at line 7, Aborting
BOOM
$ echo "$x"
Thu, Jan 10, 2019 8:35:57 AM
/c/Users/P2759474/repos/Old/deploy_microservices
$
If the outer shell had the same or a similar trap, it would have aborted too, with another message. (It's usually helpful to make the messages different.)
If you just don't like that, then as a clumsy workaround you can drop the data to a tempfile. Here's a script that will do it.
set -ex
{ pwd
date
false
echo "will this happen?"
} > foo
x=$(<foo)
echo "$x"
Put that in a script, it successfully bails.
$: ./sete
+ pwd
+ date
+ false
$: echo $?
1
I'd still use the trap, but the logic works.
I'd also use mktemp, and a trap to delete the temp on exit, etc.... but you get the idea.
How to recurse into all submodules and save the info to an array? that array should be accessible from outside of git submodule foreach, in the below example, I am trying to save all the paths which has foo in it.
$ declare -a paths
$ git submodule foreach --recursive '[[ "$name" = *"foo"* ]] && \
( echo $path; paths+=($path) ) || true'
Entering 'bar-1'
Entering 'foo-1'
foo-1
Entering 'foo-2'
foo-2
Entering 'foo-8'
foo-8
Entering 'foo'
foo
Entering 'baz'
$
$ echo ${paths[#]}
$
git submodule foreach runs in a sub-shell. This means there is no direct way to affect the parent shell, and that, in turn, means you need to affect the parent shell indirectly.
There are any number of ways to do this, but a simple one is to write to a file, then use source or . to read the file. Given your syntax above, you are presumably using bash, so:
git submodule foreach --recursive '[[ "$name" = *"foo"* ]] && \
( echo $path; echo "paths+=($path)" >> /tmp/paths ) || true'
source /tmp/paths
rm /tmp/paths
echo ${paths[#]}
Another way to do this is to eval the output of the foreach, but this is trickier since you then have to be careful with all output. There's a handy trick with exec for this, to redirect various file descriptors:
exec 1>&3
eval $(command)
where command expands (via alias or shell function, or script, or whatever) to:
command() {
exec 4>&1 1>&3 3>&-
echo now we can print normally
echo var=value 1>&4 # this is a directive for the "eval"
}
The outer 3>&1 makes a copy of stdout for the inner command, which then moves its fd 1 to fd 4, moves 3 to 1, and closes 3. Now the inner command's stdout is the same as the outer stdout, while fd 4 is where the items to be eval-ed go.
Write the values as assignment statements to a temp file. Source the temp file.