I am writing a command line tool in python and using pip to distribute it. I have written some scripts (one for bash and one for zsh) to allow the tool to have tab completion. Is there a way to get pip to install these scripts when someone does a pip install?
For example:
I have a completion.bash file. When someone does
pip install mypackage
It will also source the bash file.
I'm pretty sure I can do this for linux and bash by putting the script in my data_files section in the setup.py script.
data_file=[
('/etc/bash_completion.d', ['bin/completion.bash'])
],
But how can I do this so it is platform and shell independent? I need it to work for mac/linux in both bash and zsh. If possible, even support windows.
Is this possible? And if so, how?
In case it matters, here is my bash script:
_foo_complete() {
COMPREPLY=()
local words=( "${COMP_WORDS[#]}" )
local word="${COMP_WORDS[COMP_CWORD]}"
words=("${words[#]:1}")
local completions="$(foo completer --cmplt=\""${words[*]}"\")"
COMPREPLY=( $(compgen -W "$completions" -- "$word") )
}
complete -F _foo_complete foo
I am currently installing it by just running source completion.bash
You are asking several different questions.
First, there's no cross-platform or cross-shell solution for defining custom shell-completions. The one you posted works for bash, but in tcsh, for example, you use tcsh's complete command, which works differently than bash's.
Second, sourcing the files which contain those completion-definitions at the time of pip install wouldn't do much good. The completions might work in that very session, but what you probably want is for them to take effect in future sessions (i.e. shell invocations) as well. For that, your files would have to be sourced each time the shell starts (e.g. from within user's .bashrc, in case of bash).
This measn that "installing" your files simply means placing them somewhere, and suggesting the users should source them from their respective dot-rc file. Even if you could, you shouldn't try to "force" it. Give your users the option to add that to their dot-rc file if they want.
The best approach would probably be to include in your package a completion-definitions file per supported shell, e.g. complete.bash, complete.tcsh, and god knows what for windows (sorry, I'm not a windows user).
Related
I've been trying to make a package for the Linux command line. I have a few questions that will probably seem silly to those of you seasoned developers out there, but the documentation and guides online have been unclear to say the least.
I'm actually a developer myself and have been for the past several years but have never dipped my toe into making applications/packages for the Linux terminal, but I now have to do it for work. I would prefer to do it in C# or Python if possible, but the only thing that I've been able to get working is Bash scripts. There's no initialization or template I could find online for making one of these Linux Terminal packages. I might be bad at googling, but after three hours, I've concluded that there just some sort of secret society that exclusively knows how to make these.
I'm fine with using Bash if I can't make terminal apps/packages in other languages, but it's still a bit annoying. For example, I can't run a file without ".sh" at the end. If I run my app, test.sh, I simply type test.sh [args] but if I run another package, such as cat, I can just use cat filename without having to type cat.sh filename.
In short,
1) Do I have to use Bash?
2) If so, how can I run scripts without the ".sh" extension?
3) If not, how can I make terminal apps with C#, Python, etc.? Is there some kind of template to get you started?
Thanks for your time.
1) Do I have to use Bash (to make applications/packages for the Linux terminal)?
No, you can program in any programming language installed in your machine.
2) If so, how can I run scripts without the ".sh" extension?
The .sh extension is a convention for Bash scripts, not a requirement. You can use any extension. Suppose my_script, my_script.sh and my_script.bash have the same contents, then these commands are equivalent:
$ bash my_script.sh
$ bash my_script.bash
$ bash my_script
You could also make the script executable with chmod +x my_script.sh and run it as ./my_script.sh
3) If not, how can I make terminal apps with C#, Python, etc.? Is there some kind of template to get you started?
To run programs in a given language, you have to install the language first. Installation is typically done with a package manager. Alternatively you could download the language source code and build it. Then you can create programs with that language. This is a sample Python script and its output:
$ cat my_script.py
#!/usr/bin/env python
print "test"
$ ./my_script.py
test
I have a bunch of bash completion files in
/usr/local/etc/bash_completion.d
most of the scripts in there have something like this at the bottom of them:
complete -F _tmux tmux
the above is for tmux.
My question is - it doesn't look like bash by default sources these files?
I see some instructions online about doing something like this:
for f in '/usr/local/etc/bash_completion.d/'*; do
source "$f"
done;
do I need to do this manually or should bash be doing this out of the box?
This varies depending on your platform and/or versions of bash and bash-completion. For example:
Ubuntu
On Ubuntu 20.04 the file /etc/bash_completion does this:
. /usr/share/bash-completion/bash_completion
And in that file I find
for dir in ${XDG_DATA_DIRS:-/usr/local/share:/usr/share}; do
dirs+=( $dir/bash-completion/completions )
done
Which indicates that /usr/local/share/bash-completion/completions is scanned for completion scripts. Empirical experiments supports this.
MacOS/Brew
I could not find anything about bash completion in /etc or /usr/share on my MacOS 11.2.3. Which indicates that bare Darwin does not have bash completion, which makes sense since Apple have left Bash at 3.2 because of licensing. Might have for zsh, though, I didn't look.
/usr/local/etc/bash-completion.d, which you reference, is a part of the Homebrew installation under /usr/local. In there I found some completion scripts, but not the activation script. You should not have to activate those explicitly yourself.
I did find /usr/local/share/bash-completion, again from Homebrew, and it has the script bash_completion. In it are the same lines as Ubuntu, which also makes sense since Homebrew is kinda complete "GNU" but under /usr/local. But it also reference the directory /usr/local/etc/bash-completion.d. Sourcing /usr/local/share/bash-completion/bash_completion added the completion from that directory also.
But /usr/local/share/bash-completion/bash_completion is not executed by default, so you have to add that to your ~/bash_profile or ~/profile as described here. It also describes how to handle zsh and fish.
Cygwin
Cygwin is another Posix-compliant environment which has bash completion. (I haven't checked if bash completion is part of the Posix standard, though) After installing the bash-completion package there is /usr/share/bash-completion/bash_completion as Ubuntu and Homebrew has. Here there is no /etc/bash_completion and as the ~/.bashrc I had (generated long ago) did only look for this completions wasn't activated.
Summary
Many GNU-like environments support bash_completion but you might have to
install a package
ensure that it is sourced when you log in, which is not always the case by default
If it is not activated by default in your environment, you can activate it by sourcing the "root" script (in /etc, /usr/share/bash-completion or where it might be located) from your .bashrc, bash_profile or similar.
I'm guessing YMMV for all other possible platforms (other Linux distros, MSYS2 etc.) but the above might help you to figure out how to enable completion. Which is really helpful when available.
Is there some way to get zsh to complete long flag names on the command line?
$ command --reall<tab>
$ command --really-long-flag-name
Seems like a zshy thing to do.
The short answer is yes, or course it can.
To turn on zsh’s completion system, you need to do the following – probably in a startup file like your ~/.zshrc:
autoload -U compinit && compinit
On most modern Unix-like systems, once you do that you ought to find that many commands already have their flags and parameters completed, because zsh ships with a library of completion functions for common Unix commands and utilities. This library ought to be installed in a location like /usr/local/share/zsh/function (or similar, depending on your system) and consists of a bunch of scripts with filenames starting in a _ character, each of which defines the completion for a specific command.
If a command or utility you’re interested in is not yet completed correctly by zsh, you have several options:
Look into the zsh-completions package. (It may well be installable by your operating system or distribution’s package manager.)
Read the documentation for the tool you wish to have completion. Many Unix utilities ship with completion scripts for bash and/or zsh, or with some way of generating completion scripts.
If all else fails, read the documentation on zsh’s completion system (or find a good book or online tutorial) and write it yourself. This can — obviously — be non-trivial!
Reading that zsh documentation might also show you how to do other things that you may not even know yet that you want, like turning on menu-based completion.
I've seen in a number of places, including recommendations on this site (What is the preferred Bash shebang?), to use #!/usr/bin/env bash in preference to #!/bin/bash. I've even seen one enterprising individual suggest using #!/bin/bash was wrong and bash functionality would be lost by doing so.
All that said, I use bash in a tightly controlled test environment where every drive in circulation is essentially a clone of a single master drive. I understand the portability argument, though it is not necessarily applicable in my case. Is there any other reason to prefer #!/usr/bin/env bashover the alternatives and, assuming portability was a concern, is there any reason using it could break functionality?
#!/usr/bin/env searches PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on my OpenBSD system, it's in /usr/local/bin, since it was installed as an optional package.
If you are absolutely sure bash is in /bin and will always be, there's no harm in putting it directly in your shebang—but I'd recommend against it because scripts and programs all have lives beyond what we initially believe they will have.
The standard location of bash is /bin, and I suspect that's true on all systems. However, what if you don't like that version of bash? For example, I want to use bash 4.2, but the bash on my Mac is at 3.2.5.
I could try reinstalling bash in /bin but that may be a bad idea. If I update my OS, it will be overwritten.
However, I could install bash in /usr/local/bin/bash, and setup my PATH to:
PATH="/usr/local/bin:/bin:/usr/bin:$HOME/bin"
Now, if I specify bash, I don't get the old cruddy one at /bin/bash, but the newer, shinier one at /usr/local/bin. Nice!
Except my shell scripts have that !# /bin/bash shebang. Thus, when I run my shell scripts, I get that old and lousy version of bash that doesn't even have associative arrays.
Using /usr/bin/env bash will use the version of bash found in my PATH. If I setup my PATH, so that /usr/local/bin/bash is executed, that's the bash that my scripts will use.
It's rare to see this with bash, but it is a lot more common with Perl and Python:
Certain Unix/Linux releases which focus on stability are sometimes way behind with the release of these two scripting languages. Not long ago, RHEL's Perl was at 5.8.8 -- an eight year old version of Perl! If someone wanted to use more modern features, you had to install your own version.
Programs like Perlbrew and Pythonbrew allow you to install multiple versions of these languages. They depend upon scripts that manipulate your PATH to get the version you want. Hard coding the path means I can't run my script under brew.
It wasn't that long ago (okay, it was long ago) that Perl and Python were not standard packages included in most Unix systems. That meant you didn't know where these two programs were installed. Was it under /bin? /usr/bin? /opt/bin? Who knows? Using #! /usr/bin/env perl meant I didn't have to know.
And Now Why You Shouldn't Use #! /usr/bin/env bash
When the path is hardcoded in the shebang, I have to run with that interpreter. Thus, #! /bin/bash forces me to use the default installed version of bash. Since bash features are very stable (try running a 2.x version of a Python script under Python 3.x) it's very unlikely that my particular BASH script will not work, and since my bash script is probably used by this system and other systems, using a non-standard version of bash may have undesired effects. It is very likely I want to make sure that the stable standard version of bash is used with my shell script. Thus, I probably want to hard code the path in my shebang.
There are a lot of systems that don't have Bash in /bin, FreeBSD and OpenBSD just to name a few. If your script is meant to be portable to many different Unices, you may want to use #!/usr/bin/env bash instead of #!/bin/bash.
Note that this does not hold true for sh; for Bourne-compliant scripts I exclusively use #!/bin/sh, since I think pretty much every Unix in existence has sh in /bin.
For invoking bash it is a little bit of overkill. Unless you have multiple bash binaries like your own in ~/bin but that also means your code depends on $PATH having the right things in it.
It is handy for things like python though. There are wrapper scripts and environments which lead to alternative python binaries being used.
But nothing is lost by using the exact path to the binary as long as you are sure it is the binary you really want.
#!/usr/bin/env bash
is definitely better because it finds the bash executable path from your system environment variable.
Go to your Linux shell and type
env
It will print all your environment variables.
Go to your shell script and type
echo $BASH
It will print your bash path (according to the environment variable list) that you should use to build your correct shebang path in your script.
I would prefer wrapping the main program in a script like below to check all bash available on system. Better to have more control on the version it uses.
#! /usr/bin/env bash
# This script just chooses the appropriate bash
# installed in system and executes testcode.main
readonly DESIRED_VERSION="5"
declare all_bash_installed_on_this_system
declare bash
if [ "${BASH_VERSINFO}" -ne "${DESIRED_VERSION}" ]
then
found=0
all_bash_installed_on_this_system="$(\
awk -F'/' '$NF == "bash"{print}' "/etc/shells"\
)"
for bash in $all_bash_installed_on_this_system
do
versinfo="$( $bash -c 'echo ${BASH_VERSINFO}' )"
[ "${versinfo}" -eq "${DESIRED_VERSION}" ] && { found=1 ; break;}
done
if [ "${found}" -ne 1 ]
then
echo "${DESIRED_VERSION} not available"
exit 1
fi
fi
$bash main_program "$#"
Normally #!path/to/command will trigger bash to prepend the command path to the invoking script when executed. Example,
# file.sh
#!/usr/bin/bash
echo hi
./file.sh will start a new process and the script will get executed like /bin/bash ./file.sh
Now
# file.sh
#!/usr/bin/env bash
echo hi
will get executed as /usr/bin/env bash ./file.sh which quoting from the man page of env describes it as:
env - run a program in a modified environment
So env will look for the command bash in its PATH environment variable and execute in a separate environment where the environment values can be passed to env like NAME=VALUE pair.
You can test this with other scripts using different interpreters like python, etc.
#!/usr/bin/env python
# python commands
Your question is biased because it assumes that #!/usr/bin/env bash is superior to #!/bin/bash. This assumption is not true, and here's why:
env is useful in two cases:
when there are multiple versions of the interpreter that are incompatible.
For example python 2/3, perl 4/5, or php 5/7
when the location depends on the PATH, for instance with a python virtual environment.
But bash doesn't fall under any of these two cases because:
bash is quite stable, especially on modern systems like Linux and BSD which form the vast majority of bash installations.
there's typically only one version of bash installed under /bin.
This has been the case for the past 20+ years, only very old unices (that nobody uses any longer) had a different location.
Consequently going through the PATH variable via /usr/bin/env is not useful for bash.
Add to these three good resons to use #!/bin/bash:
for system scripts (when not using sh) for which the PATH variable may not contain /bin.
For example cron defaults to a very strict PATH of /usr/bin:/bin which is fine, sure, but other context/environments may not include /bin for some peculiar reason.
when the user screwed-up his PATH, which is very common with beginners.
for security when for example you're calling a suid program that invokes a bash script. You don't want the interpreter to be found via the PATH variable which is entirely under the user's control!
Finally, one could argue that there is one legitimate use case of env to spawn bash: when one needs to pass extra environment variables to the interpreter using #!/usr/bin/env -S VAR=value bash.
But this is not a thing with bash because when you're in control of the shebang, you're also in control of the whole script, so just add VAR=value inside the script instead and avoid the aforementioned problems introduced by env with bash scripts.
I have made a few python scripts, but is there an easier way to run them? I am using cygwin.
python "C:\Users\Desk\Dropbox\scripts\wsort.py" > data11414_unsorted.txt < data11414_sorted.txt
I want something like this (not typing the path name or "python"):
wsort > data11414_unsorted.txt < data11414_sorted.txt
where wsort is a link to my real wsort.py
Add a
Shebang
to the script
#!/bin/python
then invoke like this
wsort.py > data11414_unsorted.txt < data11414_sorted.txt
First, your question has a Windows-style path (backslashes, beginning with C:) rather than a Cygwin path (/cygdrive/c/Users/Desk/Dropbox/scripts/wsort.py). That implies you're not actually using Cygwin, or if you are, you're ignoring a bunch of warnings.
The below assumes you're using Cygwin Bash (which should be what you get if you start Cygwin Terminal from the Start Menu) and Cygwin Python (which you've installed using Cygwin's setup.exe, not a Windows Python installer). If your not, you're making life more difficult for yourself than you need to.
That out the way, there's a bunch of steps you need to take:
First, make the script executable. Use the chmod command for that, from a Cygwin Bash shell:
chmod +x /cygdrive/c/Users/Desk/Dropbox/scripts/wsort.py
Second, tell the system how to execute it. Add the following line to the top of the script:
#!/bin/python
(That's a "shebang". Python sees it as a comment, so doesn't do anything with it, but Cygwin and other Linux-like systems will use that line to see which program to run the script with. In this case, Python.)
Third, make sure your line endings are correct. Cygwin expects Linux line endings and will fail without them. This may not be a problem, but there's no harm in doing this. Run the following command:
dos2unix /cygdrive/c/Users/Desk/Dropbox/scripts/wsort.py
At this point, you'll be able to call the script by specifying the full path to it in Cygwin. You can't yet run it without specifying where the script is explicitly.
The fourth step is making sure the script is "in your path", ie in one of the folders where Cygwin looks for scripts to run. There are lots of ways to do this, but the most sensible is probably to just add your scripts directory to your path. The following command will add your scripts directory to your path whenever you start a new Cygwin session:
echo 'PATH="/cygdrive/c/Users/Desk/Dropbox/scripts:$PATH"' >>~/.bashrc
You will need to restart your Cygwin terminal for that to take effect, however.
At that point, you'll be able to run the script in Cygwin just by typing wsort.py (and thus use it with redirections and so forth as in your question).
Finally, to be able to call it simply as wsort, there's a number of options. The obvious one is just renaming the file. More usefully (and without copying the file or doing anything liable to break with Dropbox syncing things), try creating an alias:
echo 'alias wsort=wsort.py' >>~/.bashrc
Again, you'll need to restart your Cygwin terminal for that to take effect.
Maybe use an alias ?
alias wsort = "Command_Used"