So I am trying to clean up a script I have that gets the list of currently staged files using git status to return the relative file path of each staged file. I have looked at the documentation for git status to see if there was a way to get it to return the absolute path of each staged file, but there doesn't seem to be an option for that and git ls files nor git diff will work for this since the specific use case is during a merge.
During the merge, using git diff returns nothing, while git status does show all of the staged files, so I am stuck using git status.
From this list I have, I want to then grep through the list of files to extract any line(s) that contain the string "Path: " and output it. Basically, I have a list of staged .yml files and I want to extract all changes to the Path property in those ymls. Heres what I have so far:
IFS=$'\n'
for file in `git status -s -uno | sed s/^..//`
do
relativePath=$(echo $file | sed 's/^[ \t]*//;s/[ \t]*$//' | tr -d '"')
startPath=`pwd`
grep "Path: " "$startPath/$relativePath"
done
unset IFS
Explanation:
git status -s -uno | sed s/^..//
I am piping the result of git status into sed to remove any extra whitespace
relativePath=$(echo $file | sed 's/^[ \t]*//;s/[ \t]*$//' | tr -d '"')
I echo the file path and pipe it into sed to remove any extra spaces that weren't removed from the initial sed call in the start of the for loop. I then pipe that into tr to remove the first and last double quotes from the string since I need to combine that relative path with my starting path in order to get the complete path.
startPath=`pwd`
grep "Path: " "$startPath/$relativePath"
Store the current working directory in a variable, then combine it with our relative path and pass that into grep
This script works and extracts the data that I need, but I feel like there is a much cleaner way I could be doing this. Is there a way I can get git status to return the full path of each staged file so I don't have to have my second $startPath variable that I combine with my $relativePath thats passed into grep?
The simplest (correct) way to do this is by using git grep combined with git ls-files. The latter is used as a selector for grep.
Recursive search of modified tracked files using a pattern:
git grep -r 'pattern' -- `git ls-files -m`
Recursive search of all tracked files using a pattern:
git grep -r "pattern" .
Note that this grep search doesn't cover untracked files. You must add them first so that git can see them.
Since you probably call git status from inside the repo, all paths will be relative to $PWD, so you can just add it in place, yes?
$: git status -s | sed "s#^[^ ]* #$PWD/#"
Related
I'm trying to produce a shell script which is formatting commits in Shell.
After making a git log, I use grep to delete useless lines and now I want to change this :
feat(my-feat): my commit
to this:
my-feat: my commit
I want to write directly in a file: git log ... | grep -v ... >> CHANGELOG.md
Do you please have an idea how to achieve this ?
Thank you.
Using sed
$ sed 's/[^(]*(\([^)]*\))/\1/' input_file > new_file
$ cat new_file
my-feat: my commit
I am trying to split the output of the following code into two fields and insert it from the 3rd row of a csv file
#!/bin/bash
cid=`git log -n 1 --pretty=format:"%H"`
git diff-tree --no-commit-id --name-only -r $cid | xargs -I {} echo '\'{} | xargs -I {} md5sum > final.csv
Current Output comes as a single line ( need to be separated into fields)
title,Path
l34sdg232f00b434532196298ecf8427e /path/to/file
sg35s3456f00b204e98324998ecsdf3af /path/to/file
Expected Output
final.csv
title,Path
l34sdg232f00b434532196298ecf8427e,/path/to/file
sg35s3456f00b204e98324998ecsdf3af,/path/to/file
I am thinking of placing the output of the script in a third file and then reading that file line by line using awk. Not sure if that's the correct way to proceed.
Thanks in advance.
You seem to be overcomplicating things.
#!/bin/sh
cid=$(git log -n 1 --pretty=format:"%H")
git diff-tree --no-commit-id --name-only -r "$cid" |
xargs md5sum |
sed 's/ /,/' > final.csv
This simply replaces the two spaces in the md5sum output with a comma.
Because nothing here is Bash-specific, I changed the shebang to #!/bin/sh; obviously, still feel free to use Bash if you prefer.
I also switched from the obsolescent `backtick` syntax to modern $(command substitution) syntax.
If you absolutely require the CSV header on top, adding that in the sed script should be trivial. Generally, header lines are more of a nuisance than actually useful, so maybe don't.
This kind of does what you're asking:
#!/bin/bash
cid=$(git log -n 1 --pretty=format:"%H")
git diff-tree --no-commit-id --name-only -r "$cid" | while read -r path
do
md5sum "${path}"
done | awk 'BEGIN{printf "%s,%s\n", "title", "path";printf "\n"}{printf "%s,%s\n",$1,$2}' > final.csv
Is it possible to use pipe output as input for grep or git grep? The data im trying to pass to grep/git grep is the following
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/"
argocd
default
kube-node-lease
kube-public
kube-system
nsx-system
pks-system
I've tried to extent the command but this results in an error:
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/" | xargs git grep -i
fatal: ambiguous argument 'default': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Using just grep results in:
kubectl get namespace -o name -l app.kubernetes.io/instance!=applications | cut -f2 -d "/" | xargs grep -i
grep: default: No such file or directory
grep: kube-node-lease: No such file or directory
grep: kube-public: No such file or directory
grep: kube-system: No such file or directory
grep: nsx-system: No such file or directory
grep: pks-system: No such file or directory
The issue im facing with grep in general in this particular case is, that even if i soley use grep within my directory, it takes ages till it's done, whereas git grep is done within seconds. If I'm not doing something terrible wrong that would explain the slow results of grep, getting git grep to work would be preferred.
I've found this other Stackoverflow Question that somewhat explains what the issue is, but I don't know how to "process" the output into git grep properly.
The problem is that (as your screenshot shows) the result is multiple terms which I'm guessing you want to be OR-ed together, and not searching for the first term in the files identified by the last terms (which is what the current xargs command does)
Since OR in regex is via the | character, you can use xargs echo to fold the vertical list into a space delimited horizontal list then replace the spaces with | and be pretty close to what you want
printf 'alpha\nbeta\ncharlie\n' | xargs echo | tr ' ' '|' | xargs git grep -i
although due to the folding operation, that command is an xargs of one line, and thus would be conceptually easier to reason about using just normal $() interpolation:
git grep -i $(printf 'alpha\nbeta\ncharlie\n' | xargs echo | tr ' ' '|')
The less "whaaa" shell pipeline would be to use kubectl get -o go-template= to actually emit a pipe-delimited list and feed that right into xargs (or $()), bypassing the need to massage the output text first
I have a alias that does a short status, parses it with sed then adds the files to the 'assume-unchanged' index of git.
However, the issue seems to be a simple problem with my understanding of escaping single quotes in OS X bash.
irm = !sh -c 'git ignore $(git st --short -u | sed '\''/^ D/s/^ D//g'\'')'
This is the full line in gitconfig. I can issue the command in the shell (with sh and the quote), but I get bad git config when I try to run it via git irm
based on advice below, I have configured this a little differently. However, it still doesn't work in gitconfig. So I added this to my ~/.profile
alias irm="git ignore $(git st --short | grep '^ D' | sed 's/^ D //')"
You should be able to use double quotes, but you'll have to escape them:
irm = !sh -c 'git ignore $(git st --short -u | sed \"s/^ D//\")'
You don't need to select the line since the operation is the same as the selection. You may want to use -n and p with sed as Chris suggests in the comment if you only want to output the lines that match and exclude any others.
Also, since the pattern is anchored you don't need the global option.
I have a rather complicated deploy setup for our Drupal site that is a combination of CVS and SVN. We use CVS to get the newest versions of modules, and we deploy with SVN. Unfortunately, when CVS updates remove files, Subversions complains because they weren't removed in SVN. I am trying to do some shell scripting and Perl to run an svn rm command on all these files that have already been deleted from the filesystem, but I have not gotten far. What I have so far is this:
svn st | grep !
This outputs a list of all the files deleted from the filesystem like so:
! panels_views/panels_views.info
! panels_views/panels_views.admin.inc
! contexts/term.inc
! contexts/vocabulary.inc
! contexts/terms.inc
! contexts/node_edit_form.inc
! contexts/user.inc
! contexts/node_add_form.inc
! contexts/node.inc
etc. . .
However, I want to somehow run an svn del on each of these lines. How can I get this output into my Perl script, or alternatively, how can I run svn del on each of these lines?
Edit: The exact command I used, with some help from all, was
svn st | grep ^! | cut -c 9- | xargs svn del
Try using xargs, like this:
svn st | grep ^! | cut -f2 | xargs svn rm
The xargs command takes lines on its standard input, and turns them around and uses those lines as command line parameters on the svn rm. By default, xargs uses multiple lines with each invocation of its command, which in the case of svm rm is fine.
You may also have to experiment with the cut command to get it just right. By default, cut uses a tab as a delimiter and Subversion may output spaces there. In that case, you may have to use cut -d' ' -f6 or something.
As always when building such a command pipeline, run portions at a time to make sure things look right. So run everything up to the cut command to ensure that you have the list of file names you expect, before running it again with "| xargs svn rm" on the end.
svn st | egrep ^! | cut -b 9- | xargs svn del
Just as an alternative to the ones above, I'd use something like:
svn st | awk '$1=="!"{print $2}' | xargs svn del
I find awk's pattern-matching language very handy for tasks like this.