I see this all the time at my place of work:
#!/bin/sh
.....
CAT=/usr/bin/cat # An alias for cat
MAIL=/usr/bin/mail # An alias for mail
WC=/usr/bin/wc # An alias for word count
GREP=/usr/bin/grep # An alias for grep
DIRNAME=/usr/bin/dirname # An alias for dirname
RM=/usr/bin/rm # An alias for rm
MV=/usr/bin/mv # An alias for mv
.....
Is it just my company that does this? Is there a reason why you would want to spell out where these extremely common commands are? Why would I want $CAT to refer to /usr/bin/cat when cat already refers to /usr/bin/cat? Am I missing something? It seems like its needlessly redundant.
Using the full pathname ensures that the script operates correctly even if it's run by a user who customizes their PATH environment variable so that it finds different versions of these commands than the script expects.
Using variables simplifies writing the script, so you don't have to write the full pathname of a command each time it appears in the script.
Is it just my company that does this?
No.
Is there a reason why you would want to spell out where these extremely common commands are?
Yes.
Why would I want $CAT to refer to /usr/bin/cat when cat already refers to /usr/bin/cat?
Are you sure cat always refers to /usr/bin/cat? What if your script happens to be run in an environment where there is a different cat earlier in the path? Or where there is simply a user-controlled directory earlier in the path, where a user could install a rogue cat command? If your script ever happens to be run with elevated privileges, then do you really want to give random users the ability to do anything they want to your system?
Are you sure cat is supposed always to refer to /usr/bin/cat? If ever the script were installed in an environment where a different cat were needed (say /usr/local/bin/gnucat), then would you prefer to modify one line or twenty?
Am I missing something? It seems like its needlessly redundant.
Yes, you are missing something.
One would like to avoid writing out /usr/bin/cat everywhere they want to run cat, and one would like to be able to choose a different cat where needed (or more likely a different make or grep or sed). On the other hand, one wants to avoid potentially unsafe external influence on the behavior of a trusted script. Defining the full path to the command in a shell variable and then using that variable to run the command accomplishes these objectives.
One way to avoid this and still have the safety of ignoring the user's environment is to explicitly spell out the variables in the script
#!/bin/sh
PATH=/bin:/usr/bin # maybe you need something in /usr/sbin, add that
LC_ALL=C # ignore the user's locale
LD_LIBRARY_PATH=something # or unset it if you want nothing
# then
cat /a/file # have confidence you're using /bin/cat
There may well be others: check the man pages of the programs you use in your code.
Welcome to the enterprise where nothing is taken for granted.
These are commonly defined to ensure correct version, or to enforce env setup across many boxes.
https://github.com/torvalds/linux/blob/master/tools/scripts/Makefile.include#L15
This way you can check if dir exist.
Related
Consider the following folder structure:
$ tree ~/test_path
test_path
`-- sub_folder
`-- script.sh
1 directory, 1 file
Let's say that you have added test_path to your path by
export PATH=$PATH:~/test_path
$ whereis sub_folder
sub_folder: /home/murtraja/test_path/sub_folder
Now how to execute script.sh by calling sub_folder/script.sh?
$ sub_folder/script.sh
bash: sub_folder/script.sh: No such file or directory
EDIT: I don't want to change the call sub_folder/script.sh because this is called by another script which I cannot (am avoiding to) change.
Short answer: You can't, but depending on the set of constraints you're facing, there might be another way to handle it.
Long answer: When a command name contains at least one slash character, it treated as a path to the executable (i.e. it doesn't search the directories in $PATH). With the command name sub_folder/script.sh, it contains a slash, but doesn't start with a slash, so it'll be resolved relative to the current working directory.
So there are a couple of possibilities for making this work:
If you can cd to ~/test_path before running this, it'll find it directly. Of course, this may break other things (i.e. anything else that uses relative paths and/or plain filenames and expects them to be resolved somewhere else). Also, be sure to check for errors when you cd, or the script could execute in an unexpected directory, with unexpected consequences.
If the script needs to execute from a different working directory, you might be able to create a symbolic link from sub_folder in that working directory to ~/test_path/sub_folder. But depending on where the script's working directory is, this may be impossible or unsafe. I'd avoid using this option if possible.
There's also an option that depends on a weird/nonstandard feature of bash: the ability to define function names with slash in them. But this has weird limitations depending on the version of bash you have:
You can define a function like this:
sub_folder/script.sh() { ~/test_path/sub_folder/script.sh "$#"; }
and then either use export -f sub_folder/script.sh (so bash subprocesses inherit it), or do this in a wrapper script and then source the script you can't change from there (so it's the same shell, and inheritance isn't necessary).
Difficulty: some versions of bash refuse to export functions with weird names, and some refuse to inherit them. So the export method might or might not work (and might break unexpectedly due to an update). The source method might be better, but also might cause other trouble.
If there's any way at all to change the other script, that'd really be the best option.
Since you've added it to your path, you can just call the script by name. It should also let you tab complete as well.
$ ./script.sh
Is there a way to preprocess a line entered into bash in interactive mode before it is processed by bash?
I'd like to introduce some custom shorthand syntax to deal with long paths. For example, instead of writing 'cd /one/two/three/four/five', I'd like to be able to write something like 'cd /.../five', and then my preprocessing script would replace this by the former command (if a unique directory 'five' exists somewhere below /).
I found http://glyf.livejournal.com/63106.html which describes how to execute a hook function before a command is executed. However, the approach does not allow to alter the command to be executed.
There's no good way of doing this generally for all commands.
However, you can do it for specific commands by overriding them with a function. For your cd case, you can stick something like this in your .bashrc:
cd() {
path="$1"
[[ $path == "/.../five" ]] && path="/one/two/three/four/five"
builtin cd "$path"
}
In bash 4 or later, you can use the globstar option.
shopt -s globstar
cd /**/five
assuming that five is a unique directory.
The short answer is not directly. As you have found, the PROMPT_COMMAND environment variable allows you to issue a command before the prompt is displayed, which can allow for some very creative uses, e.g. Unlimited BASH History, but nothing that would allow you to parse and replace input directly.
What you are wanting to do can be accomplished using functions and alias within your .bashrc. One approach would be to use either findutils-locate or simply a find command to search directories below the present working directory for the last component in the ellipsed path, and then provide the full path in return. However, even with the indexing, locate would take a bit of time, and depending on the depth, find itself may be to slow for generically doing this for all possible directories. If however, you had a list of specific directories you would like to implement something like this for, then the solution would be workable and relatively easy.
To provide any type of prototype or further detail, we would need to know more about how you intent to use the path information, and whether multiple paths could be provided in a single command.
Another issue arises if the directory five is non-unique...
I have a number of scripts which need to specify the python binary which runs them:
#! /home/nargle/python/bin/python2.6
I need to adapt these scripts to work at two different sites. Lots of tools are installed in different places, so at new site 2 the script needs to start with:
#! /user/nargle/python/bin/python2.6
..
I want to replace directly-quoted paths with environment variables which are set differently for each site. What I would like is for this to work:
#! $MY_PYTHON_PATH
but it doesn't! I am slightly hazy on where to research this. Is it the executing shell (be it bash, csh or whatever) which detects the '#!' at the start of a script (be it bash, python or whatever) and fires up the interpreter/shell to run it?
I feel that there must be some way to do this. Please advise!
Oh yes, there is one more constraint: we cannot use the path for this. This may seem like a stupid restriction but this is for a large environment with many users
The environment is RHEL 5.7.
EDIT It has been suggested to use a shell script and that is the current plan: it works fine:
$MY_PYTHON_PATH some_script file.py $#
The problem is really that we have lots of people using the python files, and lots of automated tests which need to changed. If it has to be done it has to be done but I if possible I want to minimise the impact of a change of working practice for scores of people.
EDIT It would also be possible to have a link in a location which is the same on both systems, and which links to the real binary in a different target on each system. This is quite feasible but seems kind of messy: we use the linux 'modules' package to setup environment variables for many tools and it would be nice if we could take the python path from our modulefiles.
EDIT It isn't the answer but this feels like the kind of evil hack I was looking for:
http://docs.nscl.msu.edu/daq/bluebook/html/x3237.html
.. see "Example 4-2. #! lines for bash and for tclsh"
EDIT I hoped this might work but it didn't:
!# /usr/bin/env PATH=$PATH:$MY_PYTHON_PATH python2.6
The common solution is to change the shebang to
#!/usr/bin/env python2.6
Then, just arrange your $PATH to point to the right python2.6 on each machine.
Write a wrapper shell script. If you have script.py, write a script.py.sh with the following content:
#!/bin/bash
PYTHON_SCRIPT=$( echo "$0" | sed -e 's/\.sh$//' )
exec $MY_PYTHON_PATH $PYTHON_SCRIPT "$#"
Disclaimer: This isn't tested, just wrote it off the top of my head.
Now just set up your MY_PYTHON_PATH on each machine, and call script.py.sh instead of script.py.
Summary This solution is only second-best, since it requires a lot of script calls to be changed from script.py to script.py.sh, something that should be avoided if at all possible.
Alternative
Use env to call a python-finder script, which just calls the python binary contained in $MY_PYTHON_PATH. The python-finder script has to be in the same location on both machines, use symlinks if necessary.
#!/usr/bin/env /usr/local/bin/python-finder.sh
The contents of python-finder.sh:
#!/bin/bash
exec $MY_PYTHON_PATH "$#"
This works because for interpreter scripts (those starting with a shebang) execve calls the interpreter and passes the filename to env, which in turn passes it on to the command it calls.
I was being silly: using variable expansion with env does work.
#! /usr/bin/env PATH="$PATH:$MY_PYTHON_PATH" python2.6
We can do:
#!/bin/bash
"exec" "python" "$0"
print "Hello World"
from http://rosettacode.org/wiki/Multiline_shebang#Python
I have a set of tools which I need to pass parameters depending on the project I'm working on. I'd like to be able to automatically set a couple of environment variables based on the current directory. So when I switched between directories, my commonly used env vars would also change. Example:
Let's current directory is foo, thus if I do:
~/foo$ ./myscript --var1=$VAR1
VAR1 would have some foo based value.
Then, let's say I switched to bar directory. If I do:
~/bar$ ./myscript --var1=$VAR1
VAR1 should now have some bar based value.
Is that possible? How?
the ondir program lets you specify actions to run when you enter and leave directories in a terminal
There is direnv which helps you do this stuff much easily and in an elegant way. Just define a .envrc file in your project directory with all the env variables needed and it will source it once you cd into that folder.
I've written another implementation of this, which is somewhat similar to ondir. I didn't actually know about ondir when I started working on it. There are some key differences that may be useful, however.
smartcd is written entirely in shell, and is fully compatible with bash and zsh, even the more esoteric options
smartcd will run scripts all the way down and up the directory hierarchy down to their common ancestor, not just for the two directories you're entering and leaving. This means you can have a ~/foo script that will execute whether you "cd ~/foo" or "cd ~/foo/bar"
it has "variable stashing" which is a more automatic way of dealing with your environment variables, whereas ondir requires you to explicitly and manually remove and/or reset your variables
smartcd can work with "autocd" turned on by hooking your prompt command (PROMPT_COMMAND in bash, precmd in zsh)
You can find smartcd at https://github.com/cxreg/smartcd
This is not something that is directly supported with the built-in features of bash or any other common shell. However, you can create your own "cd" command that will do whatever you want. For example, you could alias cd to do the cd and then run a special script (eg: ~/bin/oncd). That script could look up the new directory in a database and run some commands, or see if there's a special file (eg: .env) in the directory and load it, etc.
I do this sort of thing a lot. I create several identically named batch files in directories where I need them that only set the variables and call the common script. I even have a batch file that creates the other small files.
This is not pretty, but you can use a combination of exported environment variables and the value of $PWD.
For example:
export VAR1=prefix
export prefix${HOME////_}_foo=42
export prefix${HOME////_}_bar=blah
Then myscript needs only to eval echo \${$VAR1${PWD////_}} to get at the directory based value.
How about wrap your script with a function (the function can be placed either in your bash profile/bashrc file in the system ones to make available for all the users ).
myscript () { case $PWD in
/path/to/foo) path/to/myscript --var1=$VAR1 ;;
/path/to/bar) path/to/myscript --var2=$VAR1 ;;
*) ;;
case
}
Hence the function myscript will call the real "myscript" knowing what to do based on the current working directory.
Take this as an example:
hmontoliu#ulises:/tmp$ myscript () { case $PWD in /tmp) echo I\'m in tmp;; /var) echo I\'m in var;; *) echo I\'m neither in tmp nor in bar; esac; }
hmontoliu#ulises:/tmp$ myscript
I'm in tmp
hmontoliu#ulises:/tmp$ cd /var
hmontoliu#ulises:/var$ myscript
I'm in var
hmontoliu#ulises:/var$ cd /etc
hmontoliu#ulises:/etc$ myscript
I'm neither in tmp nor in bar
How can I set variables that work everywhere? My .bashrc has:
Questions='/Users/User/Stackoverflow/Questions'
Address='My address'
My file has:
$Questions
$Addres
The command "$ cat my_file" in Shell prints "$Questions" and "$Address", instead of "/Users/User/Stackoverflow/Questions" and "My address". Other places where I would like have the variables are Email and Browser. But I cannot get them working.
How can I have my variables to work in every program?
cat doesn't interpret variables. It simply prints out the contents of a file, byte-for-byte.
If you want to be able to print out the values of the variables with my_file, I would suggest changing it to read
echo "$Questions"
echo "$Address"
Then you can "source" it (kind of like running it in the current shell environment) with
$ source my_file
or
$ . my_file
You might also have to use export in .bashrc, like so:
export Questions='/Users/User/Stackoverflow/Questions'
export Address='My address'
How can I have my variables to work in every program?
You can't. Bash and cat are two separate programs. You set a bash variable, it doesn't mean that cat will behave like bash to interpret it. $VARNAME is a shell syntax, cat is a different program, that share almost nothing with the shell.
You can export the shell variable as an environment variable, but cat is not programmed to replace any text templates, it goes way beyond its purpose.
Instead, you may use sed to perform text template substitutions:
sed -e "s|#QUESTIONS#|$Questions|g; s|#ADDRESS#|$Address|g" file.txt
This will replace all instances of #QUESTIONS# and #ANSWERS# in the file with the contents of $Questions and $Address shell variables. Note that these shell variables must not contain any pipe ("|") symbols, since the suggestion uses them as delimiters.
You can get the "cat" thing to work using some nasty hackery:
eval "$(printf 'cat << END\n%s\nEND' "$(< foo)")"
Where "foo" is the file that contains your text you want the bash parameters expanded from. This solution basically just converts the text into a here document, which does expand bash parameters.
cat <<END
[your text]
END
Limitations:
You can't have a line with just "END" in the text file or the solution will break. It'll think the line with "END" in the text file ends the here document instead of the END in the printf command, and the ouput will end early.
TBH:
This is something you just shouldn't want to do. If you want to make template files, go find a templating system that's built for this. You shouldn't be raping bash into doing something that it isn't built to do. It's a scripting language, not a templating system. It's built to parse scripts with a well defined syntax, not arbitrary text files.
You can't. In order to do that, you'd need the cooperation of every email client writer, every browser writer, every utility writer.
Type env see if the variables exist.
If so, try calling them with ${VAR_NAME}.
After re-iterating I now see your problem. You just can't have a text file, cat out it's content and expect the environments variables to be parsed. You just can't.
If it was a bash script file, and you were to run it, you could echo out the variables and they would be parsed like you want.
From your comment on Paul Tomblin's answer:
Are you sure? I have heard that you can have all kind of things like RSS-reader in Emacs. Perhaps, there is a better cooperation. I am a VIM-user, so I don't know. If someone knows whether Emacs has great cooperation between programs, please do not hesitate to share your knowledge. – UnixBasics (3 mins ago)
Do you want (or would you be satisfied with) an editor that can expand your environment variables?
That is a different (i.e. simpler, and possible) problem.
On purely theoretical level, I suppose a hacked filesystem could do what you want, but it would certainly break all kinds of binary storage. So we add a filesystem flag for text (expandable) and non-text file types. And we'd need a way for the filesystem to know what set of variable to expand, and ad nauseum...
Just add to your ~/.profile or to ~/.bashrc:
export Questions='/Users/User/Stackoverflow/Questions'
export Address='My address'