I'm using a shell script in git bash to call sqlcmd to run some sql scripts. The script names are based on the git branch name, so the command is sqlcmd -E -S mySQLServer -d myDB "$branchsql"
It works fine from the command line, but I want to repeat it for several git branches, so I have a script that calls this script for a list of branches:
While read branch
do
. C:/sqlScript.sh $branch
done < "$1"
The file with the list of branches is passed in $1
What happens is that is reads the first branch from the list, but never moves on to the next one. It repeatedly executes sqlScript.sh with the same value in $branch.
If I change sqlScript to just echo $1, everything works as expected. When I call sqlcmd, the first branch only is passed.
So why does sqlcmd mess things up ?
Drop the leading dot, invoke C:/sqlScript.sh ... rather than . C:/sqlScript.sh ...
. scripts.sh is short for source script.sh ,: it will execute the commands listed in script.sh in the current shell. If you have a command such as exit it will exit the current shell.
A regular invocation will start a separate shell, which won't mess with your current one.
I have the bash script that will be installed on some coworker machines to prevent them from running potentially dangerous things in production. For a command that starts with dbt, it will run the dbt_safety_filter() function.
The important portion is command+=" --exclude tag:subject_to_qa" which is what appends the safeguard to their command.
For some reason the output of this command is erroring because it is adding quotes around the appended section. The debugging output is:
+ dbt_filter.sh:95 #dbt_safety_filter:10> dbt run --models bcnc_pqr_breast_cancer_screening ' --exclude tag:subject_to_qa'
The issue in particular is dbt run --models bcnc_pqr_breast_cancer_screening ' --exclude tag:subject_to_qa' where quotations are added to the last portion of the command, which is in turn breaking the command
Why is this happening?
How can i prevent the literal quotes from being added to the end of the command?
#!/bin/bash
function dbt_safety_filter {
if [[ "$AWS_ENVIRONMENT" = "production" ]]; then
local command=("$#")
command+=" --exclude tag:subject_to_qa"
echo '\nProceeding on production on latest clean master.'
echo '\n-----------------------------------------------'
set -x
"`whence -p dbt`" $command
set +x
fi
}
alias dbt='dbt_safety_filter'
Your specific question's answer is that you are adding --x y to your argument list, which is of course an error, as you actually want two separate arguments --x y. To rectify it, use:
command+=( --exclude tag:subject_to_qa )
Your code might still have other problems, I haven't checked.
I have this shellscrip to deploy code from travis on another machine.
#!/bin/sh
codecov
function sshDeploy {
printf -v __ %q "$1"
ssh -oStrictHostKeyChecking=no deployuser#178.62.252.23 "cd api; git pull origin master; git checkout $__;./sbt clean; ./sbt stage; ./neeedo restart; exit $?"
}
sshDeploy $TRAVIS_COMMIT
exit $?
This script runs without any errors on my macbook. However when I run it on travis(unfortunately I can't tell you which unix they run on their buildagents because I dont have direct access to them) I observe strange behaviour.
Errors are thrown but the script is somehow still being executed.
./after-success.sh: 3: ./after-success.sh: function: not found
./after-success.sh: 4: printf: Illegal option -v
Warning: Permanently added '178.62.252.23' (ECDSA) to the list of known hosts.
From https://github.com/HTW-Projekt-2014-Commercetools/api
* branch master -> FETCH_HEAD
... Git Log ...
Stopping Neeedo-API: ..done.
Starting Neeedo-API: ....done.
./after-success.sh: 6: ./after-success.sh: Syntax error: "}" unexpected
Now I have 2 problems/question:
Why are these errors thrown and how can I make this shellscript as plattform independent as possible? I dont know which buildagent is running my script so it should run on the common unix systems.
How can I make sure that the script returns an errorcode different than 0 if something fails to make sure that the travis build fails. At the moment its still returning a green build and also the app is started somehow as you can see in the logs...
(3.) Is there a way to format the commands that are sent through ssh in a better way than in a long string?
Change it to a bash script.
First line: #!/bin/bash.
sh
$ printf -v ___ %q "$1"
sh: 1: printf: Illegal option -v
bash
$ printf -v ___ %q "$1"
<no error>
The errors are thrown because the function keyword and the -v argument to printf aren't being understood by /bin/sh on the remote host.
Changing the shebang line to #!/bin/bash will fix those errors.
The reason the script still "works" even with the above errors (quotes because it isn't really working I don't think) is because you got lucky. /bin/sh didn't understand the function keyword so it ignored that line entirely which then meant that it likely ran the ssh command directly (not through the function call) and didn't use $__ correctly (because of the printf -v error) so didn't run the correct git checkout command.
I would have expected to see an error about not knowing what the sshDeploy command was but maybe it didn't do that or maybe it got hidden by something else.
That all being said there's no real reason for that function at all.
The exit status from a script is the exit status of the last command that ran so there's no need to end with exit $?.
With that and without the function your script just becomes.
codecov
printf -v __ %q "$TRAVIS_COMMIT"
ssh -oStrictHostKeyChecking=no deployuser#178.62.252.23 "cd api; git pull origin master; git checkout $__;./sbt clean; ./sbt stage; ./neeedo restart"
And you can even avoid the printf -v most likely since git refs are unlikely (if not incapable) of containing any shell metacharacters that would need escaping when expanded in a shell string like that (I believe at least). Though this is certainly safer if you don't mind the bash requirement.
I am working on a project that is hosted in Heroku. The app is hard coded to use Amazon S3 and looks for the keys in environment variables. This is what I wrote after looking at some examples and I am not sure why its not working.
echo $1
if [ $1 != "unset" ]; then
echo "set"
export AMAZON_ACCESS_KEY_ID=XXXXXXXXXXXX
export AMAZON_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export S3_BUCKET_NAME=XXXXXXXXX
else
echo "unset"
export AMAZON_ACCESS_KEY_ID=''
export AMAZON_SECRET_ACCESS_KEY=''
export S3_BUCKET_NAME=''
fi
When running the script it goes to the set section. But when inspecting through echo $AMAZON_ACCESS_KEY_ID # => ''.
I am not sure what is causing the issue. I will be interested in...
A fix for this...
A way to extract and add heroku config variables in the the env in an easier way.
You need to source the script, not run it as a child. If you run the script directly, its environment disappears when it ends. Sourcing the script causes it to be executed in the current environment. help source for more information.
Example:
$ VAR=old_value
$ cat script.sh
#!/bin/bash
export VAR=new_value
$ ./script.sh
$ echo $VAR
old_value
$ source script.sh
$ echo $VAR
new_value
Scripts executed with source don't need to be executable nor do they need the "shebang" line (#!/bin/bash) because they are not run as separate processes. In fact, it is probably a good idea to not make them executable in order to avoid them being run as commands, since that won't work as expected.
I have two shell scripts, a.sh and b.sh.
How can I call b.sh from within the shell script a.sh?
There are a couple of different ways you can do this:
Make the other script executable with chmod a+x /path/to/file(Nathan Lilienthal's comment), add the #!/bin/bash line (called shebang) at the top, and the path where the file is to the $PATH environment variable. Then you can call it as a normal command;
Or call it with the source command (which is an alias for .), like this:
source /path/to/script
Or use the bash command to execute it, like:
/bin/bash /path/to/script
The first and third approaches execute the script as another process, so variables and functions in the other script will not be accessible.
The second approach executes the script in the first script's process, and pulls in variables and functions from the other script (so they are usable from the calling script).
In the second method, if you are using exit in second script, it will exit the first script as well. Which will not happen in first and third methods.
Check this out.
#!/bin/bash
echo "This script is about to run another script."
sh ./script.sh
echo "This script has just run another script."
There are a couple of ways you can do this. Terminal to execute the script:
#!/bin/bash
SCRIPT_PATH="/path/to/script.sh"
# Here you execute your script
"$SCRIPT_PATH"
# or
. "$SCRIPT_PATH"
# or
source "$SCRIPT_PATH"
# or
bash "$SCRIPT_PATH"
# or
eval '"$SCRIPT_PATH"'
# or
OUTPUT=$("$SCRIPT_PATH")
echo $OUTPUT
# or
OUTPUT=`"$SCRIPT_PATH"`
echo $OUTPUT
# or
("$SCRIPT_PATH")
# or
(exec "$SCRIPT_PATH")
All this is correct for the path with spaces!!!
The answer which I was looking for:
( exec "path/to/script" )
As mentioned, exec replaces the shell without creating a new process. However, we can put it in a subshell, which is done using the parantheses.
EDIT:
Actually ( "path/to/script" ) is enough.
If you have another file in same directory, you can either do:
bash another_script.sh
or
source another_script.sh
or
. another_script.sh
When you use bash instead of source, the script cannot alter environment of the parent script. The . command is POSIX standard while source command is a more readable bash synonym for . (I prefer source over .). If your script resides elsewhere just provide path to that script. Both relative as well as full path should work.
Depends on.
Briefly...
If you want load variables on current console and execute you may use source myshellfile.sh on your code. Example:
#!/bin/bash
set -x
echo "This is an example of run another INTO this session."
source my_lib_of_variables_and_functions.sh
echo "The function internal_function() is defined into my lib."
returned_value=internal_function()
echo $this_is_an_internal_variable
set +x
If you just want to execute a file and the only thing intersting for you is the result, you can do:
#!/bin/bash
set -x
./executing_only.sh
bash i_can_execute_this_way_too.sh
bash or_this_way.sh
set +x
You can use /bin/sh to call or execute another script (via your actual script):
# cat showdate.sh
#!/bin/bash
echo "Date is: `date`"
# cat mainscript.sh
#!/bin/bash
echo "You are login as: `whoami`"
echo "`/bin/sh ./showdate.sh`" # exact path for the script file
The output would be:
# ./mainscript.sh
You are login as: root
Date is: Thu Oct 17 02:56:36 EDT 2013
First you have to include the file you call:
#!/bin/bash
. includes/included_file.sh
then you call your function like this:
#!/bin/bash
my_called_function
Simple source will help you.
For Ex.
#!/bin/bash
echo "My shell_1"
source my_script1.sh
echo "Back in shell_1"
Just add in a line whatever you would have typed in a terminal to execute the script!
e.g.:
#!bin/bash
./myscript.sh &
if the script to be executed is not in same directory, just use the complete path of the script.
e.g.:`/home/user/script-directory/./myscript.sh &
This was what worked for me, this is the content of the main sh script that executes the other one.
#!/bin/bash
source /path/to/other.sh
The top answer suggests adding #!/bin/bash line to the first line of the sub-script being called. But even if you add the shebang, it is much faster* to run a script in a sub-shell and capture the output:
$(source SCRIPT_NAME)
This works when you want to keep running the same interpreter (e.g. from bash to another bash script) and ensures that the shebang line of the sub-script is not executed.
For example:
#!/bin/bash
SUB_SCRIPT=$(mktemp)
echo "#!/bin/bash" > $SUB_SCRIPT
echo 'echo $1' >> $SUB_SCRIPT
chmod +x $SUB_SCRIPT
if [[ $1 == "--source" ]]; then
for X in $(seq 100); do
MODE=$(source $SUB_SCRIPT "source on")
done
else
for X in $(seq 100); do
MODE=$($SUB_SCRIPT "source off")
done
fi
echo $MODE
rm $SUB_SCRIPT
Output:
~ ❯❯❯ time ./test.sh
source off
./test.sh 0.15s user 0.16s system 87% cpu 0.360 total
~ ❯❯❯ time ./test.sh --source
source on
./test.sh --source 0.05s user 0.06s system 95% cpu 0.114 total
* For example when virus or security tools are running on a device it might take an extra 100ms to exec a new process.
pathToShell="/home/praveen/"
chmod a+x $pathToShell"myShell.sh"
sh $pathToShell"myShell.sh"
#!/bin/bash
# Here you define the absolute path of your script
scriptPath="/home/user/pathScript/"
# Name of your script
scriptName="myscript.sh"
# Here you execute your script
$scriptPath/$scriptName
# Result of script execution
result=$?
chmod a+x /path/to/file-to-be-executed
That was the only thing I needed. Once the script to be executed is made executable like this, you (at least in my case) don't need any other extra operation like sh or ./ while you are calling the script.
Thanks to the comment of #Nathan Lilienthal
Assume the new file is "/home/satya/app/app_specific_env" and the file contents are as follows
#!bin/bash
export FAV_NUMBER="2211"
Append this file reference to ~/.bashrc file
source /home/satya/app/app_specific_env
When ever you restart the machine or relogin, try echo $FAV_NUMBER in the terminal. It will output the value.
Just in case if you want to see the effect right away, source ~/.bashrc in the command line.
There are some problems to import functions from other file.
First: You needn't to do this file executable. Better not to do so!
just add
. file
to import all functions. And all of them will be as if they are defined in your file.
Second: You may be define the function with the same name. It will be overwritten. It's bad. You may declare like that
declare -f new_function_name=old_function_name
and only after that do import.
So you may call old function by new name.
Third: You may import only full list of functions defined in file.
If some not needed you may unset them. But if you rewrite your functions after unset they will be lost. But if you set reference to it as described above you may restore after unset with the same name.
Finally In common procedure of import is dangerous and not so simple. Be careful! You may write script to do this more easier and safe.
If you use only part of functions(not all) better split them in different files. Unfortunately this technique not made well in bash. In python for example and some other script languages it's easy and safe. Possible to make partial import only needed functions with its own names. We all want that in next bush versions will be done the same functionality. But now We must write many additional cod so as to do what you want.
Use backticks.
$ ./script-that-consumes-argument.sh `sh script-that-produces-argument.sh`
Then fetch the output of the producer script as an argument on the consumer script.