I am looking to execute a script but have it include another script before it executes. The problem is, the included script would be generated and the executed script would be unmodifiable. One solution I came up with, was to actually reverse the include, by having the include script as a wrapper, calling set to set the arguments for the executed script and then dotting/sourcing it. E.g.
#!/bin/bash
# Generated wrapper or include script.
: Performing some setup...
target_script=$1 ; shift
set -- "$#"
. "$target_script"
Where target_script is the script I actually want to run, importing settings from the wrapper.
However, the potential problem I face is that callers of the target script or even the target script itself may be expecting $0 to be set to the path of it's location on the file system. But because this wrapper approach overrides $0, the value of $0 may be unexpected and could produce undefined behaviour.
Is there another way to perform what is in effect, an LD_PRELOAD but in the scripted form, through bash without interfering with its runtime parameters?
I have looked at --init-file or --rcfile, but these only seem to be included for interactive shells.
Forcing interactive mode does seem to allow me to specify --rcfile:
$ bash --rcfile /tmp/x-include.sh -i /tmp/xx.sh
include_script: $0=bash, $BASH_SOURCE=/tmp/x-include.sh
target_script: $0=/tmp/xx.sh, $BASH_SOURCE=/tmp/xx.sh
Content of the x-include.sh script:
#!/bin/bash
echo "include_script: \$0=$0, \$BASH_SOURCE=$BASH_SOURCE"
Content of the xx.sh script:
#!/bin/bash
echo "target_script: \$0=$0, \$BASH_SOURCE=$BASH_SOURCE"
From the bash documentation:
When bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in
the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read
and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for the file name.
So that settles it then:
BASH_ENV=/tmp/x-include.sh /bin/bash /tmp/xx.sh
Related
I have a Jenkins job in which I want to read a file from a directory using the shell and pass that file in ant test step.
Say the file I want to read is /home/xxx/y.txt. The name of the file always changes but there will be only single file with .txt extension at any given point in that directory.
So, I am trying to pass that file in the "Execute Shell" build action as ant -Dfile=/home/xxx/*.txt but the build is "unable to read the file".
The shell won't expand -Dfile=/home/xxx/*.txt into -Dfile=/home/xxx/y.txt because -Dfile=/home/xxx/y.txt is not a file. However, the shell will expand /home/xxx/*.txt into /home/xxx/y.txt. You can get the result you want using command substitution:
ant -Dfile=`echo /home/xxx/*.txt`
To protect against whitespace in the file path, you can use double quotes around the backticks:
ant -Dfile="`echo /home/xxx/*.txt`"
General tip: If you are having trouble with a shell script running in a Jenkins job, try enabling command tracing and view the console output to help debug. Command tracing can be enabled in one of two ways (take your pick):
Pass -x as an option to the shebang at the beginning of the script. For example, replace #!/bin/sh with #!/bin/sh -x. All commands will be output on standard error before they are executed.
Place set -x somewhere in your script. Commands after this line will be traced.
Consider:
set -- /home/xxx/*.txt
{ [ "$#" -eq 1 ] && [ -e "$1" ]; } || {
echo "ERROR: There should be exactly one file matching /home/xxx/*.txt" >&2
exit 1
}
ant -Dfile="$1"
This has several advantages:
You're actually detecting the unexpected cases instead of letting it passed unnoticed when (not if) an impossible thing happens.
Everything is happening in a single shell -- there's no subshell performance impact.
Your filenames aren't being mangled at all -- all the odd corner cases (ie. names with literal backslashes, which echo is allowed by POSIX to mangle) are fully supported.
It's fully compliant with any POSIX shell.
There's also a caveat:
set -- /home/xxx/*.txt overrides "$#", the argument vector, in the current context. If you need to refer to arguments as "$1", "$2", etc. in the outside script, you might put this code inside a function.
file_name=(`/home/xxx/*.txt`)
ant -Dfile=${file_name}
I have a number of Bash and Perl scripts which are unrelated in functionality, but are related in that they work within the same project.
The fact that they work in the same project means that I commonly specify the same directories, the same project specific commands, the same keywords at the top of every script.
Currently, this has not bitten me, but I understand that it would be easier to have all of these values in one place, then if something changes I can change a value once and have the various scripts pick up on those changes.
The question is - how is best to declare these values? A single Perl script that is 'required' in each script would require less changes to the Perl scripts, though doesn't provide a solution to the Bash script. A configuration file using a "key=value" format would perhaps be more universal, but requires each script to parse the configuration and has the potential to introduce issues. Is there a better alternative? Using environmental variables? Or a Bash specific way that Perl can easily execute and interpret?
When you run a shell script, it's done in a sub-shell so it cannot affect the parent shell's environment. So when you declare a variable as key=value its scope is limited to the sub-shell context. You want to source the script by doing:
. ./myscript.sh
This executes it in the context of the current shell, not as a sub shell.
From the bash man page:
. filename [arguments]
source filename [arguments]
Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename.
If filename does not contain a slash, file names in PATH are used to find the directory containing filename.
Also you can use the export command to create a global environment variable. export governs which variables will be available to new processes, so if you say
FOO=1
export BAR=2
./myscript2.sh
then $BAR will be available in the environment of myscript2.sh, but $FOO will not.
Define environments variables :
user level : in your ~/.profile or ~/.bash_profile or ~/.bash_login or ~/.bashrc
system level : in /etc/profile or /etc/bash.bashrc or /etc/environment
For example add tow lines foreach variable :
FOO=myvalue
export FOO
To read this variable in bash script :
#! /bin/bash
echo $FOO
in perl script :
#! /bin/perl
print $ENV{'FOO'};
You could also source another file, so you do not create extra env variables, that may lead to unexpected behaviours.
source_of_truth.sh:
FOO="bar"
scritp1.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something
scritp2.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something else
I use a lot of computing clusters and these often use a module system for making software packages available. Basically, you use the module command like module load sample_software and the sample_software path is added to $PATH. On a cluster, this command can be invoked during interactive usage and job submission usage.
I have a linux box with PBS/Torque queueing system installed so that I can sandbox software for later use on clusters. I need a very similar module system on this box. I started by making a file called modules.sh in my `/etc/profile.d/ directory that looks like this:
module()
{
if [ $2 == "softwareX" ]; then
PATH=$PATH:/home/me/dir/softwareX
export PATH
fi
}
I then put the following line in my .bash_profile script:
source /etc/profile.d/modules.sh
Now, this works great for the following usages: 1) If I submit a job and my job script uses module load softwareX, no problem, the job runs perfectly. 2) If I am working interactively on the command line and I type module load softwareX, then the path to softwareX is loaded into my $PATH and everything works great.
However, this doesn't work for the following situation: If I make a simple bash script that contains the line module load softwareX, when the bash script executes I get an error. For example, here is my bash script:
#!/usr/bin/env bash
echo $PATH
module load softwareX
echo $PATH
When I execute this I receive the error script.sh: line 3L module: command not found
...and the $PATH never changes. Does anyone know how I can solve this problem to work in all three situations? Thanks for any help!
A bash script won't invoke your startup files. You have to do that explicitly.
See http://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files
Invoked non-interactively
When Bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for the file name.
As noted above, if a non-interactive shell is invoked with the --login option, Bash attempts to read and execute commands from the login shell startup files.
When you create a sub-shell, you create a new environment. When you exit back to your existing shell, you lose that environment.
I suspect this is what is going on with your module function call. If you added echo $PATH to the bottom of your module function, do you see the PATH get changed while inside the function, but changes again when you leave the function? If so, the problem is a sub-shell issue:
What you SHOULD do is have your module function print out the new path, and then do this:
PATH=$(module load softwareX)
when I am in a Cygwin terminal, I can easily use the "source" command.
For example, let's say I have a file called "my_aliases.sh", that contains the following
#!/bin/bash -f
alias clear='cmd /c cls'
#unalias clear
Then on the Cygwin terminal, I can type
$source my_aliases.sh
And it just works fine, and whenever I type "clear", I can see that it works.
But I don't know why doing the same thing inside another shell script, and calling that shell script doesn't work.
For example, let's say that I have a file called "run_alias.sh", with the following content:
#!/bin/bash -f
#
a=`source my_aliases.sh`
b=`ls -ltr`
echo $a
echo $b
And when I try to run my file
$ ./run_alias.sh
It just doesn't do anything. For example, I can see that the command (b) takes place, but nothing happens for command (a).
But after I run "run_alias.sh", and type "clear", I get the following error:
$ clear
bash: clear: command not found
I even tried to change run_alias.sh as follows:
#!/bin/bash -f
echo `source my_aliases.sh`
But now when run run_alias.sh, and type clear, I get the exact same error message !!!
Any idea how to call the "source" command from some other shell script in Cygwin?
A child process cannot alter its parent's environment.
When you execute the run_alias.sh script, you launch a new bash process, which sources your alias file. Then the script ends, that bash process terminates and it takes its modified environment with it.
If you want your aliases to be automatically available, source it from your $HOME/.bashrc file.
Backticks create a subshell. The changes made to your environment in that subshell do not affect the calling environment.
Id you want your script (run_alias.sh) to have access to the environment in my_aliases.sh, call source directly.
source my_aliases.sh
b=`ls -lrt`
echo $b
and if you want the changes that run_alias.sh makes to its environment to propagate to it's parent, run source on the command line.
$ source run_alias.sh
I'm trying to modify an existing shell script to accept user input and handle some system exports. The below is an excerpt from a larger script. After running this script, I echo $TEST_DIR and I don't get anything back. Any ideas?
#!/bin/sh
if [ -z "$TEST_DIR" ]
then
echo "TEST_DIR was not set, please enter the path: "
read input_variable
export TEST_DIR=$input_variable
exit 1
fi
Save this as script.sh.
#!/bin/bash
if [ -z "$TEST_DIR" ]
then
echo "TEST_DIR was not set, please enter the path: "
read input_variable
export TEST_DIR=$input_variable
fi
And run it like this:
. ./script.sh
Or, equivalently:
source ./script.sh
source runs the script in the current environment and consequently lets you modify the environment in the script. Without it every command, which runs as a child of the shell process, is only given a copy of the current environment.
Note I removed the exit line, as it would terminate the shell in this case.
The problem is that you're running the script in a subshell - it's setting your TEST_DIR properly, but then the shell exits, and the parent shell doesn't keep the change.
You might be better off making this a function you can source from the parent shell. Shell scripts can't modify the environment of the calling shell, but a function can modify the shell it's executing in.
You probably don't want to do exit 1; that is used to indicate failure. You'd use exit 0 to indicate success — but, as described below, you don't want to do that, either, in this case.
The other problem is that if you run the script as child process, it cannot affect the environment of the parent process.
To work around that, in POSIX shells, you need to use:
. your_script.sh
or, in bash, you can also use:
source your_script.sh
The script does not have to be executable, just readable, and will be searched for on $PATH unless you specify a name containing a slash, just like other commands.
And, when you do the . or source, you definitely do not want any exit in the script because that will cause the main shell to exit. The shell reads the file as if you were typing it at standard input (more or less), so an exit that's executed will exit the shell.