I have a bash script which is deployed with cloud init, my bash script contains the following part of code
GO111MODULE=on go install . ./cmd/...
when running my bash script directly in the terminal of the deplyed server, it works as expected. But when i run it with runcmd in the cloud config, this part of the script:
GO111MODULE=on go install . ./cmd/...
does not get executed, anyone knows why?
runcmd:
- [ bash, /usr/local/bin/myscript.sh ]
A proper shell execution in runcmd would be (as seen in Cloud config examples):
- [ bash, -c, /usr/local/bin/myscript.sh ]
or:
- [ /usr/local/bin/myscript.sh ]
Assuming your script starts with a shebang #!/bin/bash
Plus, you need to add any environment variable inside the script, as Cloud config examples do not include any obvious way to set them.
#!/bin/bash
export GO111MODULE=on
export ...
Thanks to the tip from VonC, i was able to fix the issue. i added the following to myscript.sh
GOCACHE=/root/.cache/go-build
export GOCACHE
export GO111MODULE=on
go install . ./cmd/...
runcmd:
- [ bash, -c, /usr/local/bin/myscript.sh ]
the script now deploys and runs from cloud-init.
Related
When running Github actions on a self hosted runner machine, how do I access existing custom environment variables that have been set on the machine, in my Github action .yaml script?
I have set those variables and restarted the runner virtual machine several times, but they are not accessible using the $VAR syntax in my script.
If you want to set a variable only for one run, you can add an export command when you configure the self-hosted runner on the Github repository, before running the ./run.sh command:
Example (linux) with a TEST variable:
# Create the runner and start the configuration experience
$ ./config.sh --url https://github.com/owner/repo --token ABCDEFG123456
# Add new variable
$ export TEST="MY_VALUE"
# Last step, run it!
$ ./run.sh
That way, you will be able to access the variable by using $TEST, and it will also appear when running env:
job:
runs-on: self-hosted
steps:
- run: env
- run: echo $VAR
If you want to set a variable permanently, you can add a file to the etc/profile.d/<filename>.sh, as suggested by #frennky above, but you will also have to update the shell for it be aware of the new env variables, each time, before running the ./run.sh command:
Example (linux) with a HTTP_PROXY variable:
# Create the runner and start the configuration experience
$ ./config.sh --url https://github.com/owner/repo --token ABCDEFG123456
# Create new profile http_proxy.sh file
$ sudo touch /etc/profile.d/http_proxy.sh
# Update the http_proxy.sh file
$ sudo vi /etc/profile.d/http_proxy.sh
# Add manually new line in the http_proxy.sh file
$ export HTTP_PROXY=http://my.proxy:8080
# Save the changes (:wq)
# Update the shell
$ bash
# Last step, run it!
$ ./run.sh
That way, you will also be able to access the variable by using $HTTP_PROXY, and it will also appear when running env, the same way as above.
job:
runs-on: self-hosted
steps:
- run: env
- run: echo $HTTP_PROXY
- run: |
cd $HOME
pwd
cd ../..
cat etc/profile.d/http_proxy.sh
The etc/profile.d/<filename>.sh will persist, but remember that you will have to update the shell each time you want to start the runner, before executing ./run.sh command. At least that is how it worked with the EC2 instance I used for this test.
Reference
Inside the application directory of the runner, there is a .env file, where you can put all variables for jobs running on this runner instance.
For example
LANG=en_US.UTF-8
TEST_VAR=Test!
Every time .env changes, restart the runner (assuming running as service)
sudo ./svc.sh stop
sudo ./svc.sh start
Test by printing the variable
I'm unable to run bash scripts in "runcmd:" that aren't inline.
runcmd:
- [ bash, -c, echo "=========hello world=========" >>foo1.bar ]
- [ bash, -c, echo "=========hello world=========" >>foo2.bar ]
- [ bash, -c, /usr/local/bin/foo.sh ]
The first two lines are successfully run on the deployed Ubuntu instance. However, the foo.sh doesn't seem to run.
Here is /usr/local/bin/foo.sh:
#!/bin/bash
echo "=========hello world=========" >>foosh.bar
foo.sh has executable permissions for root and resides on the MAAS server.
I've looked at the following but they don't seem to sort out my issue:
Cannot make bash script work from cloud-init
run GO111MODULE=on go install . ./cmd/... in cloud init
https://gist.github.com/aw/40623531057636dd858a9bf0f67234e8
Any help or tips would be greatly appreciated.
Anything you run using runcmd must already exist on the filesystem. There is no provision for automatically fetching something from a remote host.
You have several options for getting files there. Two that come to mind immediately are:
You could embed the script in your cloud-init configuration using the write-files directive:
write_files:
- path: /usr/local/bin/foo.sh
permissions: '0755'
content: |
#!/bin/bash
echo "=========hello world=========" >>foosh.bar
runcmd:
- [bash, /usr/local/bin/foo.sh]
You could fetch the script from a remote location using curl (or similar tool):
runcmd:
- [curl, -o, /usr/local/bin/foo.sh, http://somewhere.example.com/foo.sh]
- [bash, /usr/local/bin/foo.sh]
I'm trying to learn how to use pipenv, and while I can start it and use it from the shell, I can't work out how to open in from a script, the tutorial I am using says to write the following file:
#!/bin/sh
export FLASK_APP=./myapp/index.py
source $(pipenv --venv)/bin/activate
flask run -h 0.0.0.0
as a shell script, but I get the following error:
./bootstrap.sh: 5: source: not found
* Serving Flask app "./catalogueService/index.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not import "index".
The could not import index, is fine, I can work with that, what I don't know is what:
source $(pipenv --venv)/bin/activate
Does, and what it should really be so it can find pipenv, I installed it using pip the usual way. Why is it not finding it please
On Linux or macOS:
source "$(pipenv --venv)/bin/activate"
On Windows:
source "$(pipenv --venv)/Scripts/activate"
OR
activate(){
activate_file=$(pipenv --venv)/bin/activate
if [ -e "$activate_file" ]; then
. $activate_file
# the pipenv shell normally enables these as well
export PYTHONDONTWRITEBYTECODE=1
export PIPENV_ACTIVE=1
if [ -f "${VIRTUAL_ENV}/.project" ]; then
cd $(cat "${VIRTUAL_ENV}/.project")
fi
return
fi
}
I want to set env variable in shell script. Shell script content is:
#!/bin/bash
export XDEBUG_CONFIG="idekey=PHPSTORM"
I tried both bash bin/enable_debug and bin/enable_debug. After both command I get:
$ echo $XDEBUG_CONFIG
$
However if I run export XDEBUG_CONFIG="idekey=PHPSTORM" directly in cli it works. What's wrong with my method?
You can try running your script as below:
. bin/enable_debug
OR
source bin/enable_debug
as indicated by #Aserre
I have a nodejs javascript project, but I would like to set a bunch of environment variables locally. created a bash file that just exports some variables:
#!/usr/bin/env bash
export waka=flaka
export fat=booty
When I use the dot to source and run the file from the command line it works fine:
. ./env.sh
And I can see the variable has been set
echo $waka # prints "flaka"
But then I try to take this command and make it an npm script by adding it to my package.json
scripts: {
"set-env": ". ./env.sh",
...
}
and then run it:
npm run set-env
The script is run but the environment variables are not saved:
echo $waka # prints undefined (assuming you didn't already run it from command line)
So, I'm wondering why it doesn't save the envrionment variables as an npm script and if it's possible to run the bash script from an npm script in a way such that the environment variables will be saved for the rest of the command prompt session. Thanks!
npm is not a shell command; it runs in a separate process that forks another shell in order to run the command specified by set-env. env.sh is executed, but then that shell immediately exits, at which point the changes are gone (and then npm itself exits).