Git check file contents across branches - bash

I'm currently using the following two bash functions to chunk through our git repo to determine what the latest version number is on each branch. Since we have several git hooks, including post-checkout, the gitTrackAll function can take quite a while to run. Is there any way to do what I'm doing without checking out every branch, pulling it in, then running a git show?
To clarify, I'm currently checking out each branch and getting the contents of the VERSION file. This essentially tells me what the last version that was merged into that branch is. We occasionally have some branches that stagnate while the developer is working on them. This list provides us a quick way to see what's way behind.
gitTrackAll && gitBranchVersions
function gitTrackAll(){
remote='origin';
for brname in `git branch -r | grep $remote | grep -v master | grep -v HEAD| awk '{gsub(/^[^\/]+\//,"",$1); print $1}'`; do
git checkout "$brname"
git branch --set-upstream-to $remote/$brname $brname;
git pull
done
git checkout master
}
function gitBranchVersions(){
line='--------------------------------------------------------------------------------'
line="$line$line"
for branch in $(git for-each-ref --format='%(refname)' refs/heads/); do
VER=$(git show "$branch":VERSION);
printf "%s %s $VER\n" $branch "${line:${#branch}}"
done
}
The resulting output looks a little like the following (branch names changed)
refs/heads/1954-branch-a ---------------------------------------------------------------------------------------------------------------------------------------- 2.9.27
refs/heads/1955-branch-b ---------------------------------------------------------------------------------------------------------------------------------------- 2.9.43
refs/heads/1965-branch-c ---------------------------------------------------------------------------------------------------------------------------------------- 2.9.32
refs/heads/1968-branch-d ---------------------------------------------------------------------------------------------------------------------------------------- 2.9.101
refs/heads/1969-branch-e ---------------------------------------------------------------------------------------------------------------------------------------- 2.9.114
refs/heads/master ----------------------------------------------------------------------------------------------------------------------------------------------- 2.9.115

I would rewrite gitTrackAll the following way without any checkout at all:
function gitTrackAll(){
remote='origin';
for brname in `git branch -r | grep $remote | grep -v master | grep -v HEAD| awk '{gsub(/^[^\/]+\//,"",$1); print $1}'`; do
git branch --set-upstream-to $remote/$brname $brname;
# git fetch cannot update the current branch - use git pull
git fetch $remote $brname:$brname || git pull $remote $brname
done
}

It looks like I can read files straight from remote. Using pieces from both functions, I've come up with the following, which works perfectly for the application I need.
function gitBranchVersions(){
line='--------------------------------------------------------------------------------'
line="$line$line"
remote='origin' ;
git fetch --all --prune
for brname in `git branch -r | grep $remote | grep -v HEAD| awk '{gsub(/^[^\/]+\//,"",$1); print $1}'`; do
VER=$(git show "$remote/$brname":VERSION);
printf "%s %s $VER\n" $brname "${line:${#brname}}"
done
}

Related

Bitbucket Pipeline trying to get list of files changed compared to

I have the following bash script I use to perform linting checks in a bitbucket pipeline.
#!/bin/bash
set -e
git fetch origin master:refs/remotes/origin/master
FORK_POINT=$(git merge-base HEAD origin/master)
PY_FILES=$(git --no-pager diff --name-only HEAD $FORK_POINT | grep "\.py$")
if [ "$PY_FILES" ]; then
flake8 $PY_FILES
else
echo "No *.py files found"
exit 0
fi
This was working fine up until December 4th around 4pm UTC. After some experimenting, I have narrowed the problem down to this line
PY_FILES=$(git --no-pager diff --name-only HEAD $FORK_POINT | grep "py$"). When I remove the grep and echo $PY_FILES the script completes successfully but nothing shows up in the log. So, it appears that git --no-pager diff --name-only HEAD $FORK_POINT isn't returning anything when run by the pipeline, which I guess causes grep to error? When I run this locally it is all fine, so git --no-pager diff --name-only HEAD $FORK_POINT should definitely be returning results.
Can anyone help me find out what went wrong here, or advise me on another way I can get a list of python files changed in a branch to run flake8 on?
It looks like you simply need to debug your pipeline ; one obvious difference between your local clone and the one used by bitbucket pipeline is that you are very probably not working on the same active commit (HEAD).
Have your pipeline output what it is working with :
its active commit : git rev-parse HEAD
its version of origin/master : git rev-parse origin/master
the reuslt of git merge-base : echo $FORK_POINT
From your comments : an empty FORK_POINT could indicate that the pipeline was trigered from a commit which hasn't a common ancestor with origin/master anymore.
Check the history of the remote HEAD :
git log --graph --oneline <sha from the pipeline>
You can for example compare it to the history of origin/master :
git log --graph --oneline <sha> origin/master

change directory in bash is not affecting the post-commit hook

i've made the following bash script to commit the parent repo after some change in submodule.
it's all about that the script want to cd .. to check the parent repo current branch but the problem is that the cd .. is not affecting the upcoming commands because i guess the subshell
i've tried to run
1- cd ../ && before each command
2- make alias but didn't succeed
3- run exec but the script didn't continued
#!/bin/sh
#
# An example hook script to verify what is about to be committed.
# Called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if
# it wants to stop the commit.
#
# To enable this hook, rename this file to "post-commit".
commit_msg= git log -1 --pretty=%B
if [[ $(git branch | grep \* | cut -d ' ' -f2) == "int1177/next" ]]; then
cd ..
if [[ $(git branch | grep \* | cut -d ' ' -f2) == "B0/next" ]]; then
git add 6_Tests
git commit -m "bs esss"
echo "development branch B0/next has now new commit"
else
echo "development branch isn't B0/next"
fi
else
echo "current branch isn't int1177/next"
fi
Actually, this particular problem is not a bash issue, but rather a Git issue.
Why doesn't "cd" work in a shell script? is valid in general, and is a suitable answer to many other questions. But this particular post-commit hook is trying to chdir out of a submodule into its parent superproject, then make a commit within the parent superproject. That is possible. It may be a bad idea for other reasons—in general it's unwise to have Git commit hooks create commits, even in other repositories1—but in this particular case you're running into the fact that Git finds its directories through environment variables.
In particular, there's an environment variable GIT_DIR that tells Git: The .git directory containing the repository is at this path. When Git runs a Git hook, Git typically sets $GIT_DIR to . or .git. If $GIT_DIR is not set, Git will find the .git directory by means of a directory-tree search, but if $GIT_DIR is set, Git assumes that $GIT_DIR is set correctly.
The solution is to unset GIT_DIR:
unset GIT_DIR
cd ..
The rest of the sub-shell commands will run in the one-step-up directory, and now that $GIT_DIR is no longer set, Git will search the superproject's work-tree for the .git directory for the superproject.
As an aside, this:
$(git branch | grep \* | cut -d ' ' -f2)
is a clumsy way to get the name of the current branch. Use:
git rev-parse --abbrev-ref HEAD
instead, here. (The other option is git symbolic-ref --short HEAD but that fails noisily with a detached HEAD, while you probably want the quiet result to be just the word HEAD, which the rev-parse method will produce.)
1The main danger in this case is that the superproject repository is not necessarily in any shape to handle a commit right now. Edit: or, as discovered in this comment, is not even set up to be a superproject for that submodule, yet, much less to have a submodule-updating commit added.

Bash - making decisions based upon output?

I am scripting a lot of processes in BASH. I want to not build certain items if they are already up to date in Git.
if a git pull is issued and the results are something like:
Already on 'master'
Your branch is up-to-date with 'origin/master'.
From https://github.com/xxx/yyyy
* branch master -> FETCH_HEAD
Already up-to-date.
How can I base decisions on the output of git pull?
Attempts:
upstream ⚡ master) 17≡ $ git status
On branch master
nothing to commit, working tree clean
(upstream ⚡ master) 17≡ $ git pull origin master
From github.com:xxx/yyy m
* branch master -> FETCH_HEAD
Already up to date.
(upstream ⚡ master) 17≡ $ git pull origin master 2>/dev/null |grep -q "Already up-to-date"
(upstream ⚡ master) 17≡ $
Attempt 2:
$ git pull origin master 2>/dev/null | grep -q "Already up-to-date"; echo $?
1
$
Attempt 3:
$ git pull origin master 2>/dev/null |grep -q "Already up-to-date"; echo $?
1
$ git pull origin master
From github.com:cocos2d/cocos2d-x-docs
* branch master -> FETCH_HEAD
Already up to date.
$ git status
On branch master
nothing to commit, working tree clean
$ git pull origin master 2>/dev/null |grep -q "Already up-to-date"; echo $?
1
Attempt 4:
$ git pull origin master 2>/dev/null |cat
Already up to date.
$ git pull origin master 2>/dev/null | grep -q "Already up-to-date"; echo $?
1
This will return true (0) if the output of git pull contains Already up-to-date:
git pull | grep -q "Already up-to-date"
Then you can put it into an if-statement to take action conditionally. For example:
if git pull | grep -q "Already up-to-date"; then
echo "Building stuff"
else
echo "Not building stuff"
fi
Your attempts are failing because git is printing up to date without the dashes, but the grep command is searching for up-to-date with the dashes, which is what your example output says. (BTW I'm using git 2.7.4 which prints up-to-date.) This sort of output might change between versions of git, so there is a more stable way of checking this sort of output, but I'm not sure what it is :/

Is there any way to type some text automatically on terminal?

When I use git via terminal I open a tab dedicated to git command only, then I don't want to type "git " every time. Is there any way to make some text automatically typed on every line?
You can define every git command as an alias, so that for example typing diff mybranch will invoke git diff mybranch. To invoke the normal shell command, type a backslash before it, for example \diff file ../elsewhere/file invokes /usr/bin/diff and not git diff.
Put the following code in a file ~/.git.bashrc. Configure your git terminal to run bash --rcfile ~/.git.bashrc instead of just running bash.
. ~/.bashrc
for c in $(COLUMNS=4 git help -a | sed -n 's/^ \([a-z]\)/\1/p';
git config --get-regexp '^alias.' | sed 's/alias\.//; s/ .*//')
do
alias "$c=git $c"
complete -F _complete_alias foo
done
The complete line requires the _complete_alias function.
I created this .bashrc function that pushes the code and tags it.
All you need to give it is the comment you want for the push.
The alias of the function is "gp" (which stands for git push).
So if you want to push and tag some code all you need after you add this code to your .bashrc is:
$ gp "test my new git push function"
gpfunction() {
git status
echo [Enter to continue...]
read a
git pull
git commit -am"$1"
git push
tag_major_min=$(git tag |sort -V|tail -1|awk -F. '{print $1 "." $2 "."}')
echo Tag major min $tag_major_min
latest_tag_number=$(git tag |sort -V|tail -1|awk -F. '{print $3}')
echo Latest tag number $latest_tag_number
next=$(echo $latest_tag_number + 1 | bc)
echo Next $next
new_tag=$(echo $tag_major_min $next | sed 's/ //g')
echo New tag $new_tag
git tag $new_tag
git push origin $new_tag
}
alias gp=gpfunction
This script uses a major.minor.patch version standard and increments the patch version.
You can tweak it as you please.

Git fast undo all changed with one command

We have a big repo... and drive encryption. So git reset --(whatever) takes quite long. Let's imagine a situation:
you're on a feature branch
you have some configuration changed
you want to checkout master a-clean && pull
checking out master is not possible straight away because you have made some changes
There are several options I know to revert those changes:
git reset --hard --> slow
git checkout . in root dir --> seems it's identical to reset --hard, and slow as well the same way
git stash - takes even longer
git status and then git checkout -- (filename). Now, that's fast, but you have to repeat it for every file!
Myself and bash don't understand each other very well, so doing something fancy like git status | grep modified: | awk "git checkout -- {%2}" is something beyond my current knowledge.
However, maybe there's a command in mgit that does git checkout -- to all the "modified:" files?
git status -s | grep -Po '^ ?M ?\K.*' | xargs git checkout --
-s short format useful to parse
grep -Po : -P perl regex \K keep out of match left of \K, -o print matches
xargs repeat argument as much shell command can accept

Resources