I have a couple of scripts for remote control of a server. Those scripts all need some environment variables set. My idea is to write a simple interpreter wrapper which first sets the variables and then calls the actual interpreter.
I tried as follows:
One of the scripts:
#! ../common/pywrapper.sh
import argparse
import parse_shell_vars
# ....and some more lines
The wrapper (../common/pywrapper.sh relative to the scripts):
#!/bin/bash
# echo params just to see what's going on
echo $#
# here would be some more configuration
python3.5 $#
Now, when I invoke the script, I get:
$ ./my_script.py param1 param2
./my_script.py: line 3: import: command not found
./my_script.py: line 4: import: command not found
How can I properly create an "interpreter wrapper"?
Remark: I know that I could export the variables in a previously-run configuration script or install them globally by some initialization script of the shell. But I think a wrapper is a good solution in this case, because I can start the scripts directly on any machine without previously installing something. Plus the scripts are self explaining for someone reading them. No guessing about required installation steps. All configuration and installation can be done by editing the wrapper.
#! ../common/pywrapper.sh
import argparse
You are mixing Python and shell. Use a Python interpreter in the shebang line for Python scripts.
Suggest doing this entirely in Python. Simpler and more portable to have one scripting language rather than several. You can get and modify environment variables with os.environ.
Related
I use a lot of computing clusters and these often use a module system for making software packages available. Basically, you use the module command like module load sample_software and the sample_software path is added to $PATH. On a cluster, this command can be invoked during interactive usage and job submission usage.
I have a linux box with PBS/Torque queueing system installed so that I can sandbox software for later use on clusters. I need a very similar module system on this box. I started by making a file called modules.sh in my `/etc/profile.d/ directory that looks like this:
module()
{
if [ $2 == "softwareX" ]; then
PATH=$PATH:/home/me/dir/softwareX
export PATH
fi
}
I then put the following line in my .bash_profile script:
source /etc/profile.d/modules.sh
Now, this works great for the following usages: 1) If I submit a job and my job script uses module load softwareX, no problem, the job runs perfectly. 2) If I am working interactively on the command line and I type module load softwareX, then the path to softwareX is loaded into my $PATH and everything works great.
However, this doesn't work for the following situation: If I make a simple bash script that contains the line module load softwareX, when the bash script executes I get an error. For example, here is my bash script:
#!/usr/bin/env bash
echo $PATH
module load softwareX
echo $PATH
When I execute this I receive the error script.sh: line 3L module: command not found
...and the $PATH never changes. Does anyone know how I can solve this problem to work in all three situations? Thanks for any help!
A bash script won't invoke your startup files. You have to do that explicitly.
See http://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files
Invoked non-interactively
When Bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for the file name.
As noted above, if a non-interactive shell is invoked with the --login option, Bash attempts to read and execute commands from the login shell startup files.
When you create a sub-shell, you create a new environment. When you exit back to your existing shell, you lose that environment.
I suspect this is what is going on with your module function call. If you added echo $PATH to the bottom of your module function, do you see the PATH get changed while inside the function, but changes again when you leave the function? If so, the problem is a sub-shell issue:
What you SHOULD do is have your module function print out the new path, and then do this:
PATH=$(module load softwareX)
I am new to TCL scripting and shell scripting. I want to invoke a TCL script from the shell script. I have tried as below.
#!/bin/sh
for i in {1..5}
do
my_script
test_script
done
If I run the script, it is throwing error as follows,
./sample.sh: line 5: my_script: command not found
./sample.sh: line 5: test_script: command not found
Can anyone help me out with this ?
Thanks in advance.
If they cannot be found in your $PATH you have to provide a path to your scripts, e.g.:
./my_myscript # current directory
/path/to/test_script # absolute path
If you haven't made your script executable (with chmod +x) then you need to use:
tclsh my_script.tcl
Or maybe tclsh8.5 /path/to/script.tcl or many variations on that.
If you have made the script executable, check that the directory containing the script is on your PATH (if not, use the full filename of the script or adjust your PATH) and that you've got a suitable #! line. The usual recommended one is:
#!/usr/bin/env tclsh8.5
as that will search your path for the tclsh8.5 executable instead of hard-coding it.
From man tclsh. I guess the second block answers your question.
If you create a Tcl script in a file whose first line is
#!/usr/local/bin/tclsh
then you can invoke the script file directly from your shell if you mark the file as executable. [...]
An even better approach is to start your script files with the following three lines:
#!/bin/sh
# the next line restarts using tclsh \
exec tclsh "$0" ${1+"$#"}
This approach has three advantages over the approach in the previous paragraph [...]
You should note that it is also common practice to install tclsh with its version number as part of the name.This has the advantage of allowing multiple
versions of Tcl to exist on the same system at once, but also the disadvantage of making it harder to write scripts that start up uniformly across different versions of Tcl.
I have a number of scripts which need to specify the python binary which runs them:
#! /home/nargle/python/bin/python2.6
I need to adapt these scripts to work at two different sites. Lots of tools are installed in different places, so at new site 2 the script needs to start with:
#! /user/nargle/python/bin/python2.6
..
I want to replace directly-quoted paths with environment variables which are set differently for each site. What I would like is for this to work:
#! $MY_PYTHON_PATH
but it doesn't! I am slightly hazy on where to research this. Is it the executing shell (be it bash, csh or whatever) which detects the '#!' at the start of a script (be it bash, python or whatever) and fires up the interpreter/shell to run it?
I feel that there must be some way to do this. Please advise!
Oh yes, there is one more constraint: we cannot use the path for this. This may seem like a stupid restriction but this is for a large environment with many users
The environment is RHEL 5.7.
EDIT It has been suggested to use a shell script and that is the current plan: it works fine:
$MY_PYTHON_PATH some_script file.py $#
The problem is really that we have lots of people using the python files, and lots of automated tests which need to changed. If it has to be done it has to be done but I if possible I want to minimise the impact of a change of working practice for scores of people.
EDIT It would also be possible to have a link in a location which is the same on both systems, and which links to the real binary in a different target on each system. This is quite feasible but seems kind of messy: we use the linux 'modules' package to setup environment variables for many tools and it would be nice if we could take the python path from our modulefiles.
EDIT It isn't the answer but this feels like the kind of evil hack I was looking for:
http://docs.nscl.msu.edu/daq/bluebook/html/x3237.html
.. see "Example 4-2. #! lines for bash and for tclsh"
EDIT I hoped this might work but it didn't:
!# /usr/bin/env PATH=$PATH:$MY_PYTHON_PATH python2.6
The common solution is to change the shebang to
#!/usr/bin/env python2.6
Then, just arrange your $PATH to point to the right python2.6 on each machine.
Write a wrapper shell script. If you have script.py, write a script.py.sh with the following content:
#!/bin/bash
PYTHON_SCRIPT=$( echo "$0" | sed -e 's/\.sh$//' )
exec $MY_PYTHON_PATH $PYTHON_SCRIPT "$#"
Disclaimer: This isn't tested, just wrote it off the top of my head.
Now just set up your MY_PYTHON_PATH on each machine, and call script.py.sh instead of script.py.
Summary This solution is only second-best, since it requires a lot of script calls to be changed from script.py to script.py.sh, something that should be avoided if at all possible.
Alternative
Use env to call a python-finder script, which just calls the python binary contained in $MY_PYTHON_PATH. The python-finder script has to be in the same location on both machines, use symlinks if necessary.
#!/usr/bin/env /usr/local/bin/python-finder.sh
The contents of python-finder.sh:
#!/bin/bash
exec $MY_PYTHON_PATH "$#"
This works because for interpreter scripts (those starting with a shebang) execve calls the interpreter and passes the filename to env, which in turn passes it on to the command it calls.
I was being silly: using variable expansion with env does work.
#! /usr/bin/env PATH="$PATH:$MY_PYTHON_PATH" python2.6
We can do:
#!/bin/bash
"exec" "python" "$0"
print "Hello World"
from http://rosettacode.org/wiki/Multiline_shebang#Python
I'm trying to write a bash script which will behave as a basic interpreter, but it doesn't seem to work: The custom interpreter doesn't appear to be invoked. What am I doing wrong?
Here's a simple setup illustrating the problem:
/bin/interpreter: [owned by root; executable]
#!/bin/bash
echo "I am an interpreter running " $1
/Users/zeph/script is owned by me, and is executable:
#!/bin/interpreter
Here are some commands for the custom interpreter.
From what I understand about the mechanics of hashbangs, the script should be executable as follows:
$ ./script
I am an interpreter running ./script
But this doesn't work. Instead the following happens:
$ ./script
./script: line 3: Here: command not found
...It appears that /bin/bash is trying to interpret the contents of ./script. What am I doing wrong?
Note: Although it appears that /bin/interpreter never invoked, I do get an error if it doesn't exist:
$ ./script
-bash: ./script: /bin/interpreter: bad interpreter: No such file or directory
(Second note: If it makes any difference, I'm doing this on MacOS X).
To make this work you could add the interpreter's interpreter (i.e. bash) to the shebang:
#!/bin/bash /bin/interpreter
Here are some commands for the custom interpreter.
bash will then run your interpreter with the script path in $1 as expected.
You can't use a script directly as a #! interpreter, but you can run the script indirectly via the env command using:
#!/usr/bin/env /bin/interpreter
/usr/bin/env is itself a binary, so is a valid interpreter for #!; and /bin/interpreter can be anything you like (a script of any variety, or binary) without having to put knowledge of its own interpreter into the calling script.
Read the execve man page for your system. It dictates how scripts are launched, and it should specify that the interpreter in a hash-bang line is a binary executable.
I asked a similar question in comp.unix.shell that raised some pertinent information.
There was a second branch of the same thread that carried the idea further.
The most general unix solution is to have the shebang point to a binary executable. But that executable program could be as simple as a single call to execl(). Both threads lead to example C source for a program called gscmd, which is little more than a wrapper to execv("gs",...).
My ruby script requires connection to an Oracle database. So I need to export ORACLE_HOME and LD_LIBRARY_PATH correctly before the script would run. Is there a way that I can export those env variables without using shell script? I tried to put ENV['ORACLE_HOME'] = '/usr/local/oracle_client' at the first line of the script and it doesn't work.
Now the only way it would work is to write a shell script, where export those variables and then run ruby there. The shell script looks like:
export ORACLE_HOME='/usr/local/oracle_client'
export LD_LIBRARY_PATH='/usr/local/oracle_client/lib'
ruby myscript.rb --options
It's kinda ugly because user has to go inside the shell script to change options. I'm wondering whether there's a better way of doing it. So user can just do at command line: ruby myscript.rb --options
Why not supply the ruby options as arguments to the shell script? E.g.,
#!/bin/bash
export ORACLE_HOME='/usr/local/oracle_client'
export LD_LIBRARY_PATH='/usr/local/oracle_client/lib'
ruby myscript.rb $*
Obviously you may want to add argument reasonableness checks, etc., but this gives the idea.
Why not call it out via Kernel.system?
system("export ORACLE_HOME='/usr/local/oracle_client'")
system("export LD_LIBRARY_PATH='/usr/local/oracle_client/lib'")