Below is my pre-commit git hoook
#!/bin/bash
....
# if git diff -U0 "$FILE_PATH" | grep -iq 'todo'; # Double quoting $FILE_PATH doesnt' change anything
if git diff -U0 $FILE_PATH | grep -iq 'todo';
then
echo $FILE_PATH ' -> Contains TODO'
exit 1
else
echo 'nooooooooooooooooooooooooooooooooooo'
fi
I'm always getting the noooooooooooooooooooo message, however the command below, tried directly on my terminal, works well:
git diff -U0 my/file/path.php | grep -iq 'todo' && echo 'true' || echo 'false'
Output
true
UPDATE
When running bash .git/hooks/pre-commit it works, very strange!!
FYI
I don't know if it's an important information but .git/hooks/pre-commit is a symbolik link
Most likely, your pipe does not return status 0. To verify that this is the case (and not the way you write your compound statement), you could rewrite it as
git diff -U0 "$FILE_PATH" | grep -iq 'todo'
grep_status=$?
echo grep status is $grep_status
if (( grep_status == 0 ))
then
echo contains todo
else
echo no
fi
I also noticed that your code contains an unnecessary semicolon in the if line. I first thought that this semicolon might cause the weird behaviour, but at least on my bash, where I tried your code, it does not seem to do any harm. Still, I would remove it for the safe side.
Related
This question already has answers here:
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 5 months ago.
This is probably a simple one for a bash scripter, which I am not.
I'm running a cron job that downloads some data, and then depending on that data, may or may not modify a second file. After the job, I want to git commit one or both files. For the conditional commit, I tried this in a .sh script:
# attempt to capture whether MyNotes.txt was changed
# by counting lines in git status output
mywc=(git status -s MyNotes.txt | wc -l)
echo $mywc found!
if [ $mywc = 1 ]; then
echo Add file for commit
else
echo Nothing to add
fi
I'm pretty much getting nowhere; this thing seems to fail on the first line with syntax error near unexpected token '|'. If I run git status -s MyNotes.txt | wc -l on the command line, I get the numeric output I expect.
What am I doing wrong and how can I make this work?
If there's a more elegant way to determine whether a file changed, feel free to share.
Also, for my edification, how could I get this to work without the interim mywc variable? I.e., if I wanted to just do the command within the if, something like this:
if [[ $(git status -s MyNotes.txt | wc -l) = 1 ]]; then
...
Thanks!
What am I doing wrong and how can I make this work?
put a dollar before parenthesis.
foo=$(command)
The thing you are using looks like a bash array
declare -a letters=(a b c d)
If there's a more elegant way to determine whether a file changed, feel free to share.
Consider this:
$ git diff -s --exit-code README.md || echo has changed
has changed
$ git checkout README.md
Updated 1 path from the index
$ git diff -s --exit-code README.md || echo has changed
The OR (||) runs if the first command exits with a non-zero code.
Same thing essentially:
$ false || echo false exits with 1
false exits with 1
$ true || echo will not trigger
An aspect of bash that people overlook is that [[, ]], [ and ] are separate commands. They have return codes too. With this knowledge, you can leverage the return codes with if and any other command.
$ if true; then echo yes; else echo no; fi
yes
$ if false; then echo yes; else echo no; fi
no
So for detecting changes in a tracked file:
$ if git diff -s --exit-code README.md; then echo same as in index; else echo changed; fi
same as in index
$ echo 123 >> README.md
$ if git diff -s --exit-code README.md; then echo same as in index; else echo changed; fi
changed
With all of that said...
Just add the file. You don't need to check anything. If it hasn't changed, nothing will happen.
$ echo foo >> myfile
$ git add myfile
$ git commit -m 'maybe changed' myfile
[master b561cc1] maybe changed
1 file changed, 1 insertion(+), 1 deletion(-)
$ git add myfile
$ git commit -m 'maybe changed' myfile
no changes added to commit (use "git add" and/or "git commit -a")
if you need to avoid a non-zero exit code (such as with set -e), just put a || true after the command that you want to ignore the exit status of:
$ cat foo.sh
#!/bin/basho
set -e
echo foo >> myfile
git add myfile
git commit -m 'maybe changed' myfile
git add myfile
git commit -m 'maybe changed' myfile > /dev/null || true
echo no error here. it\'s fine..
false
echo fill never reach this.
Try running that script and see what happens
I search for a way for checking if file changed.
git diff --exit-code -s <path>
Now the bash scripter knows that every command returns a status code which can be checked with $?. In case everything went smoothly, 0 is returned. In that case we get 0 if file is not changed.
Every bash scripter knows too that you can use that with && and || operators (because of lazy evaluation) to write such construct:
git diff --exit-code -s <path> && echo "should add file"
About your edification, what you wrote is perfectly fine!
As CryptoFool pointed out in a comment, I failed to include a $ in my variable assignment. Simple fix in the first line of my script:
mywc=$(git status -s MyNotes.txt | wc -l)
As matt pointed out in a subsequent comment, doing a git add on a file that hasn't changed has no effect. It won't stage the file for commit. So instead of doing conditional logic to determine whether to git add myfile.txt, I'll just blindly execute git add myfile.txt, which will either stage the file if there are changes, or do nothing if there are no changes. Therefore, my entire script can be replaced with one line:
git add MyNotes.txt
Due to some problems with a script which commits and pushes automatically, i'd like to implement a whitelist.
The plan is, that only commits with the pattern 'foo' and 'bar' in path, are allowed.
#!/bin/sh
WHITELIST="foo bar"
WRKDIR=/home/athur/workwork/test/repo
cd $WRKDIR
git add -A
for file in `git diff --cached -p --name-status | cut -c3-`; do
if [[ "$file" == *"$WHITELIST"* ]] ; then
echo "$file is on whitelist"
else
echo "$file is not on whitelist. Commit aborted."
exit 1
fi
done
The problem is, it's always uses the 'else' clause.
I can't find the problem. Thanks
As a best-practices approach, consider:
#!/usr/bin/env bash
# ^^^^ important: [[ ]] is not guaranteed to work with bin/sh
whitelist_re='(foo|bar)'
workdir=/home/athur/workwork/test/repo
cd -- "$workdir" || exit
git add -A
while IFS= read -r filename; do
if [[ $file =~ $whitelist ]]; then
echo "$file is on whitelist" >&2
else
echo "$file is not on whitelist; commit aborted." >&2
exit 1
fi
done < <(git diff --cached --name-only)
To walk through the changes:
The shebang specifies bash as a shell, which guarantees that extensions like [[ ]] and <(...) will be available -- a guarantee not made with /bin/sh.
A while read loop is used rather than attempting to iterate over line-oriented data with for; see DontReadLinesWithFor for an explanation of the reasoning behind this change.
The whitelist is specified as an ERE-compliant regular expression, such that =~ can be used to test whether a value matches.
Instead of using git diff --cached --name-status and then using cut to remove the status data after-the-fact, we use --name-only to generate only names in the first place.
Using lowercase variable names complies with the conventions given in http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html, specifying that POSIX-defined tools will use all-caps shell and environment variable names for their own purposes, and that names with at least one lowercase character are reserved for application use. (Keep in mind that setting a shell variable overwrites any like-named environment variable, so these conventions apply even when export is not in use).
By the way, if you just wanted to find out if any non-matches exist, without knowing which files those are, you could use:
#!/bin/sh
# ^^ actually safe here, as no non-POSIX functionality is used
whitelist_re='foo|bar'
if git diff --cached --name-only | grep -qEv "$whitelist_re"; then
echo "At least one file is not on whitelist; commit aborted" >&2
exit 1
fi
Using an explicit list
The == is not symmetric in this case and ** seems to be used badly.
Try "$WHITELIST" == *"$file"*.
(Inspired by How do I check if a variable exists in a list in BASH)
Note that using your WHITELIST, only files foo and bar will be whitelisted.
Detecting a pattern
If you need to detect individual patterns, you may need to construct a function such as:
for entry in $WHITELIST ; do
if [[ "$file" =~ $entry ]] ; then
return 0
fi
done
return 1
Using the Ampersand (&) to place it in the background. But in this script for some reason it doesnt work. My programming skills are not great, so please remember im a noob trying to get stuff working.
#!/bin/bash
# Date in format used by filenaming
date=$(date '+%Y%m%d')
# Location where the patch files should be downloaded
patches=~/lists/patches
# Location of the full list
blacklist=~/lists/list
while :
do
# Fetching last download date from downloaded patches
ldd=$(cd $patches && printf '%s\n' * | sed "s/[^0-9]*//g"); echo $ldd
if [ "$ldd" = "" ]
then
break
else
if [ "$ldd" = "$date" ]
then
break
else
ndd=$(date +%Y%m%d -d "${ldd}+1 days")
# Cant have multiple patches in $patches directory, otherwise script wont work
rm -rf $patches/*
sleep 1
file=$patches/changes-$ndd.diff.gz
curl -s -o "$file" "http://url.com/directory/name-$ndd.diff.gz" &
sleep 1
done=$(jobs -l | grep curl | wc -l)
until [ "$done" == 1 ]
do
echo "still here"
done
gunzip "$file"
# Apply patch directory to list's file directories
cat $(echo "$file" | sed "s/.gz//g") | sed 's/.\/yesterday//' | sed 's/.\/today//' > $patches/$ndd.diff
rm $(echo $file | sed "s/.gz//g")
cd $blacklist
patch -p1 --batch -r /root/fail.patch < $patches/$ndd.diff
rm /root/fail.patch
fi
fi
done
What i want to do is let the script wait for each command until the one before is finished. As you can see i used 'sleep' sometimes but i know that isnt a solution. I also read about the wait command, but then you have to place a command in the background using the Ampersand. And thats the problem. For some reason this script doesnt recognize the ampersand at the end of my curl command. I also tried wget, same results. Who can point me in the right direction?
It would never change done after first check. So you need to check every iteration, that's why you should test for command, not for variable
And while will be better, because you need to check before entering
while [ "$(jobs -l | grep curl | wc -l)" -ne 0 ]; do
echo "Still there"
sleep 1
done
I've added sleep because otherwise it wold just flood your console.
I'm wondering if I've done this right. I'm trying to learn BASH and really want to learn the "Best Practices" the first time, so I don't adopt the sloppy/easy way.
What I'm wondering, can I nest an IF/THEN statement like I've done below? Why or why not? Would the block below be served better by using an elif instead?
Lastly, I was hoping someone could shed some light for me on the use of "${foo}" and "$(bar)" ... curly braces or parenthesis? I've (so far) used curly braces when I'm defining a variable "foo='bar'" is later called as "${foo} and parenthesis when I'm capturing a command "foo=$(find . -type f -name bar)" would be called as "$foo" ... or maybe I'm just way off and doing the same thing twice, I don't know ... I'd love to hear what you've all got to say! :D
# Downloading the script bundle
echo "Lets get the script bundle and get to work!"
wget http://place.to.get/att.tar
# Logic switch, checking if the TAR bundle exists. If it does
# verify the MD5 Checksum (to prevent corruption).
# If verfied, then un-tar the bundle in our working directory
# otherwise, exit with an error code, otherwise
if [[ -f att.tar ]]
then
echo "Okay, we have the bundle, lets verify the checksum"
sum=$(md5sum /root/att/att.tar | awk '{print $1}')
if [[ $sum -eq "xxxxINSERT-CHECKSUM-HERExxxx" ]]
then
tar -xvf att.tar
else
clear
echo "Couldn't verify the MD5 Checksum, something went wrong" | tee /tmp/att.$time.log
sleep 0.5
exit 1;
fi
else
clear
echo "There was a problem getting the TAR bundle, exiting now ..." | tee /tmp/att.$time.log
sleep 0.5
exit 1;
fi
Overall comments
Nothing wrong with nested "if's," but early exit would be clearer
cut is cheaper than awk, but read is cheaper still
Simple string equality tests are marginally cheaper with "[" rather than "[["
Write error messages to STDERR
Use read and < <() rather than $( | cut -f1 -d' ') because it avoids a pipe and second fork/exec
Use functions
A simplified version
bail () {
clear
echo "${#}" | tee /tmp/att.${time}.log >&2
exit 1
}
# Downloading the script bundle
echo "Lets get the script bundle and get to work!" >&2
wget http://place.to.get/att.tar || bail "There was a problem getting the TAR bundle, exiting now ..."
sum=''
read sum rest < <(md5sum /root/att/att.tar)
[ $sum == "xxxxINSERT-CHECKSUM-HERExxxx" ] || bail "Couldn't verify the MD5 Checksum, something went wrong"
tar -xvf att.tar || bail "Extract failed"
How can I test if a command outputs an empty string?
Previously, the question asked how to check whether there are files in a directory. The following code achieves that, but see rsp's answer for a better solution.
Empty output
Commands don’t return values – they output them. You can capture this output by using command substitution; e.g. $(ls -A). You can test for a non-empty string in Bash like this:
if [[ $(ls -A) ]]; then
echo "there are files"
else
echo "no files found"
fi
Note that I've used -A rather than -a, since it omits the symbolic current (.) and parent (..) directory entries.
Note: As pointed out in the comments, command substitution doesn't capture trailing newlines. Therefore, if the command outputs only newlines, the substitution will capture nothing and the test will return false. While very unlikely, this is possible in the above example, since a single newline is a valid filename! More information in this answer.
Exit code
If you want to check that the command completed successfully, you can inspect $?, which contains the exit code of the last command (zero for success, non-zero for failure). For example:
files=$(ls -A)
if [[ $? != 0 ]]; then
echo "Command failed."
elif [[ $files ]]; then
echo "Files found."
else
echo "No files found."
fi
More info here.
TL;DR
if [[ $(ls -A | head -c1 | wc -c) -ne 0 ]]; then ...; fi
Thanks to netj
for a suggestion to improve my original:if [[ $(ls -A | wc -c) -ne 0 ]]; then ...; fi
This is an old question but I see at least two things that need some improvement or at least some clarification.
First problem
First problem I see is that most of the examples provided here simply don't work. They use the ls -al and ls -Al commands - both of which output non-empty strings in empty directories. Those examples always report that there are files even when there are none.
For that reason you should use just ls -A - Why would anyone want to use the -l switch which means "use a long listing format" when all you want is test if there is any output or not, anyway?
So most of the answers here are simply incorrect.
Second problem
The second problem is that while some answers work fine (those that don't use ls -al or ls -Al but ls -A instead) they all do something like this:
run a command
buffer its entire output in RAM
convert the output into a huge single-line string
compare that string to an empty string
What I would suggest doing instead would be:
run a command
count the characters in its output without storing them
or even better - count the number of maximally 1 character using head -c1(thanks to netj for posting this idea in the comments below)
compare that number with zero
So for example, instead of:
if [[ $(ls -A) ]]
I would use:
if [[ $(ls -A | wc -c) -ne 0 ]]
# or:
if [[ $(ls -A | head -c1 | wc -c) -ne 0 ]]
Instead of:
if [ -z "$(ls -lA)" ]
I would use:
if [ $(ls -lA | wc -c) -eq 0 ]
# or:
if [ $(ls -lA | head -c1 | wc -c) -eq 0 ]
and so on.
For small outputs it may not be a problem but for larger outputs the difference may be significant:
$ time [ -z "$(seq 1 10000000)" ]
real 0m2.703s
user 0m2.485s
sys 0m0.347s
Compare it with:
$ time [ $(seq 1 10000000 | wc -c) -eq 0 ]
real 0m0.128s
user 0m0.081s
sys 0m0.105s
And even better:
$ time [ $(seq 1 10000000 | head -c1 | wc -c) -eq 0 ]
real 0m0.004s
user 0m0.000s
sys 0m0.007s
Full example
Updated example from the answer by Will Vousden:
if [[ $(ls -A | wc -c) -ne 0 ]]; then
echo "there are files"
else
echo "no files found"
fi
Updated again after suggestions by netj:
if [[ $(ls -A | head -c1 | wc -c) -ne 0 ]]; then
echo "there are files"
else
echo "no files found"
fi
Additional update by jakeonfire:
grep will exit with a failure if there is no match. We can take advantage of this to simplify the syntax slightly:
if ls -A | head -c1 | grep -E '.'; then
echo "there are files"
fi
if ! ls -A | head -c1 | grep -E '.'; then
echo "no files found"
fi
Discarding whitespace
If the command that you're testing could output some whitespace that you want to treat as an empty string, then instead of:
| wc -c
you could use:
| tr -d ' \n\r\t ' | wc -c
or with head -c1:
| tr -d ' \n\r\t ' | head -c1 | wc -c
or something like that.
Summary
First, use a command that works.
Second, avoid unnecessary storing in RAM and processing of potentially huge data.
The answer didn't specify that the output is always small so a possibility of large output needs to be considered as well.
if [ -z "$(ls -lA)" ]; then
echo "no files found"
else
echo "There are files"
fi
This will run the command and check whether the returned output (string) has a zero length.
You might want to check the 'test' manual pages for other flags.
Use the "" around the argument that is being checked, otherwise empty results will result in a syntax error as there is no second argument (to check) given!
Note: that ls -la always returns . and .. so using that will not work, see ls manual pages. Furthermore, while this might seem convenient and easy, I suppose it will break easily. Writing a small script/application that returns 0 or 1 depending on the result is much more reliable!
For those who want an elegant, bash version-independent solution (in fact should work in other modern shells) and those who love to use one-liners for quick tasks. Here we go!
ls | grep . && echo 'files found' || echo 'files not found'
(note as one of the comments mentioned, ls -al and in fact, just -l and -a will all return something, so in my answer I use simple ls
Bash Reference Manual
6.4 Bash Conditional Expressions
-z string
True if the length of string is zero.
-n string
string
True if the length of string is non-zero.
You can use shorthand version:
if [[ $(ls -A) ]]; then
echo "there are files"
else
echo "no files found"
fi
As Jon Lin commented, ls -al will always output (for . and ..). You want ls -Al to avoid these two directories.
You could for example put the output of the command into a shell variable:
v=$(ls -Al)
An older, non-nestable, notation is
v=`ls -Al`
but I prefer the nestable notation $( ... )
The you can test if that variable is non empty
if [ -n "$v" ]; then
echo there are files
else
echo no files
fi
And you could combine both as if [ -n "$(ls -Al)" ]; then
Sometimes, ls may be some shell alias. You might prefer to use $(/bin/ls -Al). See ls(1) and hier(7) and environ(7) and your ~/.bashrc (if your shell is GNU bash; my interactive shell is zsh, defined in /etc/passwd - see passwd(5) and chsh(1)).
I'm guessing you want the output of the ls -al command, so in bash, you'd have something like:
LS=`ls -la`
if [ -n "$LS" ]; then
echo "there are files"
else
echo "no files found"
fi
sometimes "something" may come not to stdout but to the stderr of the testing application, so here is the fix working more universal way:
if [[ $(partprobe ${1} 2>&1 | wc -c) -ne 0 ]]; then
echo "require fixing GPT parititioning"
else
echo "no GPT fix necessary"
fi
Here's a solution for more extreme cases:
if [ `command | head -c1 | wc -c` -gt 0 ]; then ...; fi
This will work
for all Bourne shells;
if the command output is all zeroes;
efficiently regardless of output size;
however,
the command or its subprocesses will be killed once anything is output.
All the answers given so far deal with commands that terminate and output a non-empty string.
Most are broken in the following senses:
They don't deal properly with commands outputting only newlines;
starting from Bash≥4.4 most will spam standard error if the command output null bytes (as they use command substitution);
most will slurp the full output stream, so will wait until the command terminates before answering. Some commands never terminate (try, e.g., yes).
So to fix all these issues, and to answer the following question efficiently,
How can I test if a command outputs an empty string?
you can use:
if read -n1 -d '' < <(command_here); then
echo "Command outputs something"
else
echo "Command doesn't output anything"
fi
You may also add some timeout so as to test whether a command outputs a non-empty string within a given time, using read's -t option. E.g., for a 2.5 seconds timeout:
if read -t2.5 -n1 -d '' < <(command_here); then
echo "Command outputs something"
else
echo "Command doesn't output anything"
fi
Remark. If you think you need to determine whether a command outputs a non-empty string, you very likely have an XY problem.
Here's an alternative approach that writes the std-out and std-err of some command a temporary file, and then checks to see if that file is empty. A benefit of this approach is that it captures both outputs, and does not use sub-shells or pipes. These latter aspects are important because they can interfere with trapping bash exit handling (e.g. here)
tmpfile=$(mktemp)
some-command &> "$tmpfile"
if [[ $? != 0 ]]; then
echo "Command failed"
elif [[ -s "$tmpfile" ]]; then
echo "Command generated output"
else
echo "Command has no output"
fi
rm -f "$tmpfile"
Sometimes you want to save the output, if it's non-empty, to pass it to another command. If so, you could use something like
list=`grep -l "MY_DESIRED_STRING" *.log `
if [ $? -eq 0 ]
then
/bin/rm $list
fi
This way, the rm command won't hang if the list is empty.
As mentioned by tripleee in the question comments , use moreutils ifne (if input not empty).
In this case we want ifne -n which negates the test:
ls -A /tmp/empty | ifne -n command-to-run-if-empty-input
The advantage of this over many of the another answers when the output of the initial command is non-empty. ifne will start writing it to STDOUT straight away, rather than buffering the entire output then writing it later, which is important if the initial output is slowly generated or extremely long and would overflow the maximum length of a shell variable.
There are a few utils in moreutils that arguably should be in coreutils -- they're worth checking out if you spend a lot of time living in a shell.
In particular interest to the OP may be dirempty/exists tool which at the time of writing is still under consideration, and has been for some time (it could probably use a bump).