Bash tab completion adds extra space after the first completion - bash

Bash tab completion adds extra space after the first completion which stops further completion if the compeletion target is a file in multi-level folders.
For example, I have a file in the path ~/Documents/foo/bar.txt, and I want to list it.
I face the following problem, when input
a#b:~$ls Docu <TAB>
I get
a#b:~$ls Documents |(<-this is the cursor, so there is an extra space afer Documents)
So I cannot further tab complete. I have to backspace to delete the extra space.
Normally I want to get:
a#b:~$ls Docu <TAB>
a#b:~$ls Documents/ <TAB>
a#b:~$ls Documents/foo/ <TAB>
a#b:~$ls Documents/foo/bar.txt

Just for the record: There is also a bug in the adobereader-enu (acroread) package that breaks bash completion. In this case you can just delete the symlink:
rm /etc/bash_completion.d/acroread.sh
See also: https://bugs.launchpad.net/ubuntu/+source/acroread/+bug/769866

I have had this same problem with my bash completion in both Ubuntu 11.10 and 12.04. I found that I was able to get many commands to start working correctly by editing /etc/bash_completion. Specifically, I commented out the following section:
####
# makeinfo and texi2dvi are defined elsewhere.
#
#for i in a2ps awk bash bc bison cat colordiff cp csplit \
# curl cut date df diff dir du enscript env expand fmt fold gperf gprof \
# grep grub head indent irb ld ldd less ln ls m4 md5sum mkdir mkfifo mknod \
# mv netstat nl nm objcopy objdump od paste patch pr ptx readelf rm rmdir \
# sed seq sha{,1,224,256,384,512}sum shar sort split strip tac tail tee \
# texindex touch tr uname unexpand uniq units vdir wc wget who; do
# have $i && complete -F _longopt -o default $i
#done
Now ls works well again. I have not figured out yet why mv is still mis-behaving.

This has been answered here at askubuntu. It is related to the bug here
Relevant answer from the above thread:
edit /etc/bash_completion line 1587, change default to filenames (make a backup first).

i also got around the problem by changing
_filedir with _filedir_pdf
in /etc/bash_completion.d/acroread.sh
(Ubuntu 12.04)
acroread bash completion changes the _filedir function thereby altering the behaviour of a lot of other alsobash completion functions

Related

Understanding a docker entrypoint script

The script is located here: https://github.com/docker-library/ghost/blob/master/docker-entrypoint.sh
#!/bin/bash
set -e
if [[ "$*" == npm*start* ]]; then
baseDir="$GHOST_SOURCE/content"
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
mkdir -p "$targetDir"
if [ -z "$(ls -A "$targetDir")" ]; then
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
fi
done
if [ ! -e "$GHOST_CONTENT/config.js" ]; then
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
fi
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
chown -R user "$GHOST_CONTENT"
set -- gosu user "$#"
fi
exec "$#"
From what I know, it says that if you use some variation of npm start to move some files around from $GHOST_SOURCE to $GHOST_CONTENT, do something to the config.js file, link the config file, set ownership of the content files, and then execute npm start as the user user. Otherwise, it just runs your commands normally.
The specifics are what are hard for me to understand because there are a lot of things from bash that I've never seen before. So I have a lot of questions.
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
In the above, why do they specify both /*/ and /themes/*/? Shouldn't /*/ contain themes? Is * not a wildcard for some reason?
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
In the above, what is the point of # in the variable expansion?
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
In the above, does this somehow save time? Why not use something like rsync? I understand the point of -C, but why -c and --one-file-system?
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
What does this sed command do? I know it's a replacement, but why the "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js" as the end?
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
In the above, what is the point of this symlink? Why try to link them to each other if both files already exist?
set -- gosu user "$#"
In the above what does calling set with no args do?
I hope that's not too much. I felt making a separate question for each of these would be too much especially since it's all related to each other.
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
In the above, why do they specify both /*/ and /themes/*/? Shouldn't
/*/ contain themes? Is * not a wildcard for some reason?
themes/ is in the first match, but themes/*/ is not, so you need the second entry to include the contents of themes.
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
In the above, what is the point of # in the variable expansion?
It removes the $baseDir prefix from $dir. So for example:
bash$ dir=/home/bmitch/data/docker
bash$ echo $dir
/home/bmitch/data/docker
bash$ echo ${dir#/home/bmitch}
/data/docker
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
In the above, does this somehow save time? Why not use something like
rsync? I understand the point of -C, but why -c and --one-file-system?
rsync may not be installed on every machine by default, tar is fairly universal. The -c is to create, vs extract, and --one-file-system avoids tar continuing to an outside mount point (nfs, symlink to root, etc).
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
What does this sed command do? I know it's a replacement, but why the
"$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js" as the
end?
config.example.js is the input (last arg to the sed), config.js is the output (after the >). So it takes the config.example.js, change the ip address from 127.0.0.1 to 0.0.0.0, effectively listening on all interfaces/ip's instead of just internally on the loopback. The second half of the sed is changing the path.join arguments from __dirname to process.env.GHOST_CONTENT.
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
In the above, what is the point of this symlink? Why try to link them
to each other if both files already exist?
The $GHOST_SOURCE/config.js is replaced (-f) with a link to $GHOST_CONTENT/config.js. Symbolic links give a file name reference to another actual file, so there will be two names, but one copy of the data, which means you will only have a single configuration in this situation.
set -- gosu user "$#"
In the above what does calling set with no args do?
This changes the values of $1, $2, ... $n to be $1=gosu, $2=user, $3=the old $1, $4=the old $2..., essentially adding the gosu and user to the beginning of the passed parameters to the script. The -- makes sure that set doesn't interpret any values from $# as a flag for itself.

Makefile autocompletion on Mac

Makefile's targets are available by completion on Linux but, AFAICS, not on Mac OS (10.8.5).
Is it possible to get completion working with this OS?
This seems to achieve simple bash completions for me on El Capitan:
# .bashrc
function _makefile_targets {
local curr_arg;
local targets;
# Find makefile targets available in the current directory
targets=''
if [[ -e "$(pwd)/Makefile" ]]; then
targets=$( \
grep -oE '^[a-zA-Z0-9_-]+:' Makefile \
| sed 's/://' \
| tr '\n' ' ' \
)
fi
# Filter targets based on user input to the bash completion
curr_arg=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -W "${targets[#]}" -- $curr_arg ) );
}
complete -F _makefile_targets make
Here's how this works:
complete -F [function name] [command name] -- this bash builtin register a new completion for [command name] which is generated by the bash function [function name]. So in my code above, if you type make [TAB][TAB] into your shell, you'll trigger the _makefile_targets() function.
if [[ -e "$(pwd)/Makefile" ]]; then -- make sure there's a Makefile in the current directory, otherwise don't try a bash completion.
grep -oE '^[a-zA-Z0-9_-]+:' Makefile -- filter every line of Makefile using the regex for a target name like "test:". -o means only return the part of the line that matches. For example, given a Makefile target like "test: build package", only "test:" will be returned
| sed 's/://' -- taking the grep results, remove the colon from the end of line
| tr '\n' ' ' -- smoosh all targets onto one line, separated by one space
Inside a bash completion function, complete sets several environment variables for you. COMP_WORDS is an array of the list of available bash completion choises based on what the user typed. COMP_CWORD is the index of the currently selected word.
Another very magical builtin compgen will take a list of space separately strings and filter them using the currently selected word. I'm not at all clear how that works.
So, the bottom line is that the last two lines in the function filter our list of makefile targets (stored inside $targets) and shoves them into an array COMPREPLY. The bash completion reads and displays COMPREPLY as choices in the shell.
Inspired by:
https://gist.github.com/tlrobinson/1073865
http://www.thegeekstuff.com/2013/12/bash-completion-complete/ (Esp 9.)
This worked for me on Catalina in the standard zsh terminal
Edit the file named .zshrc in you home directory this can be done with this command in the terminal nano ~/.zshrc
Add the following lines
zstyle ':completion:*:*:make:*' tag-order 'targets'
autoload -U compinit && compinit
exit and save by pressing ctrl + x and then save Y
For macOS Monterey, the following works:
Edit the .zprofile file in your home directory. This can be done with a text editor. Path: ~/.zprofile
Add the following lines to the end of the file:
zstyle ':completion:*:*:make:*' tag-order 'targets'
autoload -Uz compinit && compinit
After that, everything works correctly, tips work when you press TAB.
This method does not display anything extra when entering the terminal, unlike the methods indicated above.
If you use bash and homebrew:
brew install bash-completion
follow their documentation:
Add the following line to your ~/.bash_profile:
[[ -r "$(brew --prefix)/etc/profile.d/bash_completion.sh" ]] && . "$(brew --prefix)/etc/profile.d/bash_completion.sh"
restart your terminal or
source ~/.bash_profile

Self-extracting script in sh shell

How would I go about making a self extracting archive that can be executed on sh?
The closest I have come to is:
extract_archive () {
printf '<archive_contents>' | tar -C "$extract_dir" -xvf -
}
Where <archive_contents> contains a tarball with null characters, %, ' and \ characters escaped and enclosed between single quotes.
Is there any better way to do this so that no escaping is required?
(Please don't point me to shar, makeself etc. I want to write it from scratch.)
Alternative variant is to use marker for end of shell script and use sed to cut-out shell script itself.
Script selfextract.sh:
#!/bin/bash
sed '0,/^#EOF#$/d' $0 | tar zx; exit 0
#EOF#
How to use:
# create sfx
cat selfextract.sh data.tar.gz >example_sfx.sh
# unpack sfx
bash example_sfx.sh
Since shell scripts are not compiled, but executed statement by statement, you can mix binary and text content using a pattern like this (untested):
#!/bin/sh
sed -e '1,/^exit$/d' "$0" | tar -C "${1-.}" -zxvf -
exit
<binary tar gzipped content here>
You can add those two lines to the top of pretty much any tar+gzip file to make it self extractable.
To test:
$ cat header.sh
#!/bin/sh
sed -e '1,/^exit$/d' "$0" | tar -C "${1-.}" -zxvf -
exit
$ tar -czf header.tgz header.sh
$ cat header.sh header.tgz > header.tgz.sh
$ sh header.tgz.sh
header.sh
Some good articles on how to do exactly that could be found at:
http://www.linuxjournal.com/node/1005818.
https://community.linuxmint.com/tutorial/view/1998
Yes, you can do it natively with xtar.
Build xtar elf64 tar self-extractor header (you free to modify it to support elf32, pe and other executable formats), it is based on lightweight bsdtar untar and std elf lib.
cc contrib/xtar.c -o ./xtar
Copy xtar binary to yourTar.xtar
cp ./xtar yourTar.xtar
Append yourTar.tar archive to the end of yourTar.xtar
cat yourTar.tar >> yourTar.xtar
chmod +x yourTar.xtar

bash completion of makefile target

Suppose I have a simple makefile like:
hello:
echo "hello world"
bye:
echo "bye bye"
Then in bash I want something like:
make h < tab >
so it can complete to
make hello
I found a simple way like creating empty files hello and bye but I'm looking for something more sophisticated.
Add this in your ~/.bash_profile file or ~/.bashrc file
complete -W "\`grep -oE '^[a-zA-Z0-9_.-]+:([^=]|$)' ?akefile | sed 's/[^a-zA-Z0-9_.-]*$//'\`" make
This searches for a target in your Makefile titled 'Makefile' or 'makefile' (note the capital ? wildcard in ?akefile) using grep, and pipes it over to the complete command in bash which is used to specify how arguments are autocompleted. The -W flag denotes that the input to the complete command will be a wordlist which is accomplished by passing the results of grep through sed which arranges it into the desirable wordlist format.
Caveats and gotchas:
Your make file is named 'GNUMakefile' or anything else other than 'Makefile' or 'makefile'. If you frequently encounter such titles consider changing the regular expression ?akefile accordingly.
Forgetting to source your ~/.bash_profile or ~/.bashrc file after making the changes. I add this seemingly trivial detail since, to the uninitiated it is unfamiliar.
For any change to your bash files to take effect, source them using the command
source ~/.bashrc
or
source ~/.bash_profile
PS. You also now have the added ability to display the possible make targets by pressing [Tab] twice just like in bash completion. Just make sure you add a space after the command make before typing [Tab] twice.
This answer from 2010 is outdated - the project mentioned here seems to have been discontinued.
Could this be what you're looking for?
http://freshmeat.net/projects/bashcompletion/
make [Tab] would complete on all
targets in Makefile. This project was
conceived to produce programmable
completion routines for the most
common Linux/UNIX commands, reducing
the amount of typing sysadmins and
programmers need to do on a daily
basis.
There's a useful package called bash-completion available for most every OS. It includes Makefile completion.
(If you're using macOS and Homebrew, you can get this via brew install bash-completion.)
This seems to be default in at least Debian Lenny:
$ grep Makefile /etc/bash_completion
# make reads `GNUmakefile', then `makefile', then `Makefile'
elif [ -f ${makef_dir}/Makefile ]; then
makef=${makef_dir}/Makefile
# before we scan for targets, see if a Makefile name was
# deal with included Makefiles
The header of this file states:
# The latest version of this software can be obtained here:
#
# http://bash-completion.alioth.debian.org/
#
# RELEASE: 20080617.5
Here is a completion script that looks at the .PHONY: declaration.
_make_phony_words() {
local opt_revert
if [ -n "${BASH_VERSION:-}" ]; then
shopt -q nullglob || {
opt_revert=1 ; shopt -s nullglob ;
}
elif [ -n "${ZSH_VERSION:-}" ]; then
[[ -o nullglob ]] || {
opt_revert=1 ; setopt nullglob
}
fi
for f in ./?akefile ./*.make ; do
sed -nEe '/^.PHONY/ { s/^.PHONY:[ ]?// ; p ; } ' "$f" | tr ' ' $'\n' | sort -u
done
if [ -n "$opt_revert" ]; then
[ -n "${ZSH_VERSION:-}" ] && unsetopt nullglob
[ -n "${BASH_VERSION:-}" ] && shopt -u nullglob
fi
unset opt_revert
}
_make_phony_complete() {
local cur="${COMP_WORDS[COMP_CWORD]}"
COMPREPLY+=( $(compgen -W "$( _make_phony_words )" -- ${cur}) )
}
complete -F _make_phony_complete make
Makefile completion on steroids!
I had 2 problems with the normal completions:
Problem #1
Sometimes you have targets you want to call like make greet:hi and make greet:hola sort of like namespacing Makefile target names. So your Makefile ends up looking like:
greet\:hola:
echo "hola world"
# OR a .PHONY target
.PHONY: greet\:hi
greet\:hi:
echo "hi world"
In this case the auto-completions after : don't show up as it uses \: in the Makefile as shown above.
Problem #2
There wasn't a way to navigate through the list of all Makefile targets that match my input using arrow keys (or CTRL-p / CTRL-n) in my bash shell.
Basically, I wanted to use fuzzy search like approach on the targets (i.e. fzf).
FZF Repo: https://github.com/junegunn/fzf
Solution
Install FZF Dependency
Using Homebrew
You can use Homebrew (on macOS or Linux)
to install fzf.
brew install fzf
$(brew --prefix)/opt/fzf/install
Using Linux package managers
Package Manager
Linux Distribution
Command
APK
Alpine Linux
sudo apk add fzf
APT
Debian 9+/Ubuntu 19.10+
sudo apt-get install fzf
Conda
conda install -c conda-forge fzf
DNF
Fedora
sudo dnf install fzf
Nix
NixOS, etc.
nix-env -iA nixpkgs.fzf
Pacman
Arch Linux
sudo pacman -S fzf
pkg
FreeBSD
pkg install fzf
pkgin
NetBSD
pkgin install fzf
pkg_add
OpenBSD
pkg_add fzf
XBPS
Void Linux
sudo xbps-install -S fzf
Zypper
openSUSE
sudo zypper install fzf
FZF and : compatible auto-complete command
Put this in your .bashrc
complete -W "\`grep -oE '^[a-zA-Z0-9_.-]+[\\:]*[a-zA-Z0-9_.-]+:([^=]|$)' ?akefile | sort | uniq | sed 's/[^a-zA-Z0-9_.-]*$//' | sed 's/[\]//g' | fzf\`" make
Now just typing make and then hitting the key will work!
DEMO: in action!
Then you can use as following:
make using fzf
I added so I follow "include" directives in Makefile. So my .bashrc looks like this:
function followMakefile() {
grep -oE '^[a-zA-Z0-9_.-]+:([^=]|$)' ?akefile | sed 's/[^a-zA-Z0-9_.-]*$//'
for x in `grep -E '^include' ?akefile | sed 's/include //'`
do
grep -oE '^[a-zA-Z0-9_.-]+:([^=]|$)' $x | sed 's/[^a-zA-Z0-9_.-]*$//'
done
}
complete -W "\`followMakefile\`" make
In Ubuntu 10.04, source the following file:
. /etc/bash_completion
or uncomment it in
/etc/bash.bashrc

Inline comments for Bash?

I'd like to be able to comment out a single flag in a one-line command. Bash only seems to have from # till end-of-line comments. I'm looking at tricks like:
ls -l $([ ] && -F is turned off) -a /etc
It's ugly, but better than nothing. Is there a better way?
The following seems to work, but I'm not sure whether it is portable:
ls -l `# -F is turned off` -a /etc
My preferred is:
Commenting in a Bash script
This will have some overhead, but technically it does answer your question
echo abc `#put your comment here` \
def `#another chance for a comment` \
xyz etc
And for pipelines specifically, there is a cleaner solution with no overhead
echo abc | # normal comment OK here
tr a-z A-Z | # another normal comment OK here
sort | # the pipelines are automatically continued
uniq # final comment
How to put a line comment for a multi-line command
I find it easiest (and most readable) to just copy the line and comment out the original version:
#Old version of ls:
#ls -l $([ ] && -F is turned off) -a /etc
ls -l -a /etc
$(: ...) is a little less ugly, but still not good.
Here's my solution for inline comments in between multiple piped commands.
Example uncommented code:
#!/bin/sh
cat input.txt \
| grep something \
| sort -r
Solution for a pipe comment (using a helper function):
#!/bin/sh
pipe_comment() {
cat -
}
cat input.txt \
| pipe_comment "filter down to lines that contain the word: something" \
| grep something \
| pipe_comment "reverse sort what is left" \
| sort -r
Or if you prefer, here's the same solution without the helper function, but it's a little messier:
#!/bin/sh
cat input.txt \
| cat - `: filter down to lines that contain the word: something` \
| grep something \
| cat - `: reverse sort what is left` \
| sort -r
Most commands allow args to come in any order. Just move the commented flags to the end of the line:
ls -l -a /etc # -F is turned off
Then to turn it back on, just uncomment and remove the text:
ls -l -a /etc -F
How about storing it in a variable?
#extraargs=-F
ls -l $extraargs -a /etc
If you know a variable is empty, you could use it as a comment. Of course if it is not empty it will mess up your command.
ls -l ${1# -F is turned off} -a /etc
ยง 10.2. Parameter Substitution
For disabling a part of a command like a && b, I simply created an empty script x which is on path, so I can do things like:
mvn install && runProject
when I need to build, and
x mvn install && runProject
when not (using Ctrl + A and Ctrl + E to move to the beginning and end).
As noted in comments, another way to do that is Bash built-in : instead of x:
$ : Hello world, how are you? && echo "Fine."
Fine.
It seems that $(...) doesn't survive from ps -ef.
My scenario is that I want to have a dummy param that can be used to identify the very process. Mostly I use this method, but the method is not workable everywhere. For example, python program.py would be like
mkdir -p MyProgramTag;python MyProgramTag/../program.py
The MyProgramTag would be the tag for identifying the process started.

Resources