Is it possible to get a unique key per slug / release from a running dyno? I was following this article to setup RAILS_CACHE_ID (for expiring etags after deployments) but found that the dynos no longer ship with GIT configured (which causes this error):
fatal: Not a git repository (or any parent up to mount point /app)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
I also considered setting it in a config/initializers to the current Time but obviously that wouldn't work across multiple dynos. Any ideas?
There's a new (Nov 2015) labs feature which does just what you need "Dyno metadata" https://devcenter.heroku.com/changelog-items/768
heroku labs:enable runtime-dyno-metadata -a <app name>
Then on heroku:
~ $ env
HEROKU_APP_ID: 9daa2797-e49b-4624-932f-ec3f9688e3da
HEROKU_APP_NAME: example-app
HEROKU_DYNO_ID: 1vac4117-c29f-4312-521e-ba4d8638c1ac
HEROKU_RELEASE_VERSION: v42
HEROKU_SLUG_COMMIT: 2c3a0b24069af49b3de35b8e8c26765c1dba9ff0
HEROKU_SLUG_DESCRIPTION: Deploy 2c3a0b2
One solution is to use a git pre-push hook to set a heroku config value. Since it's done before the push and slug compilation, the config variable will be available as an ENV var to your rails app.
.git/hooks/pre-push:
#!/bin/sh
remote="$1"
url="$2"
while read local_ref local_sha remote_ref remote_sha
do
if [[ $url =~ heroku ]] ; then
app=$(git remote show -n $remote | sed -n -E -e 's/[[:space:]]+(Push[[:space:]]+URL)(\/|:).+(:|\/)(.*)\.git$/\4/gp')
echo Setting RAILS_CACHE_ID to $local_sha on app $app
heroku config:set RAILS_CACHE_ID=$local_sha --app $app
fi
done
exit 0
The pre-push.sample file has some documentation about the parameters that the hook is called with. I'm using verbose output of git remote to determine which app to set the config value on. The '-E' option for sed is for Mac OS X -- if you are using GNU sed replace that with '-r'.
Another solution is to use the heroku-api via a profile.d script to get the unique release id. This example uses curl to get the latest release id using the RANGE header. It's not the commit reference but it will be unique to every release, including rollbacks and config changes. You'll want to set the API_KEY and the APP_NAME as heroku config variables.
.profile.d/release.sh
# get release id and set as RAILS_CACHE_ID
# Heroku config variables that need to be set
# API_KEY: heroku api key (get from dashboard or `heroku auth:token`
# APP_NAME: set this to your app_name (this could be hardcoded in the profile.d script but
# would make it harder to manage apps with multiple environments
res=$(curl -s -H "Accept: application/vnd.heroku+json; version=3"\
-H "Authorization: Bearer $API_KEY"\
-H "Range: version ..; order=desc, max=1"\
-X GET https://api.heroku.com/apps/$APP_NAME/releases)
release_id=$(ruby -rjson -e "j = JSON.parse('$res'); puts j[0]['id']")
export RAILS_CACHE_ID=$release_id
In the rails app, ENV['RAILS_CACHE_ID'] should now be set to the most recent release id. You could also use this same strategy in a rails initializer.
Related
TL;DR: update your bin/qgtunnel.
I've recently noticed an increase in my web dyno's memory usage. After digging a bit, I could see that the LD_PRELOAD variable that should be set with heroku-buildpack-jemalloc was not set correctly. I used a tiny script (bin/show_preload) that helped me debug that and trace which program was overriding LD_PRELOAD.
#!/usr/bin/env bash
echo "buildpack=foo preload='$LD_PRELOAD' at=start-app cmd='$#'"
$#
I introduced that in our Procfile:
web: bin/show_preload bin/qgtunnel bin/show_preload bin/start-nginx bin/show_preload bin/start-pgbouncer bin/show_preload bundle exec puma -C config/puma.rb
And when lauching on heroku I can see that bin/qgtunnel overrides our LD_PRELOAD configuration.
I created a tiny helper for the time being which makes sure I keep original value as well as what is added by bin/qgtunnel:
#!/usr/bin/env bash
after_qgtunnel_script=$(mktemp)
echo <<-BASH > $after_qgtunnel_script
# Retrieve previous LD_PRELOAD value
export LD_PRELOAD="\$LD_PRELOAD $LD_PRELOAD"
# Clean after usage
rm $after_qgtunnel_script
# Start following commands
$#
BASH
chmod +x $after_qgtunnel_script
bin/qgtunnel $after_qgtunnel_script $#
If you ever need this script use it in place of bin/qgtunnel
After reaching out to Quotaguard, they patched the qgtunnel binary and there is no error anymore:
curl https://quotaguard.s3.amazonaws.com/qgtunnel-2.4.1.tar.gz | tar xz
git add bin/qgtunnel vendor/nss_wrapper/libnss_wrapper.so
git commit -m "Update qgtunnel to fix LD_PRELOAD"
NOTE: new versions may occur since that one, see the related documentation
I'm developing a Node application using Docker and docker-compose, which all have awesome ways to import multiple environment variables from a .env file.
Just wondering if there's a shortcut for updating a Heroku deployment's env vars instead of typing them out.
I've noticed there's no --env-file option on the official docs: https://devcenter.heroku.com/articles/config-vars
Using some xargs magic
cat .env.prod | xargs heroku config:set
# Output:
# API_PORT: 3000
# API_KEY_1: ab-cd
# DB_PORT: 6379
# REDISCLOUD_URL: xxx
Happily sets multiple args from a local .env file
I'm trying to upload coverage reports to codecov.io using the codecov-bash script provided by Codecov. The bash script fails to run on Gitlab CI running an alpine:edge docker image.
Below is the error:
$ /bin/bash <(curl -s https://codecov.io/bash)
/bin/sh: eval: line 107: syntax error: unexpected "("
And here is the relevant part of my .gitlab-ci.yml file:
after_script:
- apk -U add git curl bash findutils
- /bin/bash <(curl -s https://codecov.io/bash)
Line 107 of the script is inside the show_help() function, just under This is non-exclusive, use -s "*.foo" to match specific paths.:
show_help() {
cat << EOF
Codecov Bash $VERSION
Global report uploading tool for Codecov
Documentation at https://docs.codecov.io/docs
Contribute at https://github.com/codecov/codecov-bash
-h Display this help and exit
-f FILE Target file(s) to upload
-f "path/to/file" only upload this file
skips searching unless provided patterns below
-f '!*.bar' ignore all files at pattern *.bar
-f '*.foo' include all files at pattern *.foo
Must use single quotes.
This is non-exclusive, use -s "*.foo" to match specific paths.
-s DIR Directory to search for coverage reports.
Already searches project root and artifact folders.
-t TOKEN Set the private repository token
(option) set environment variable CODECOV_TOKEN=:uuid
-t #/path/to/token_file
-t uuid
-n NAME Custom defined name of the upload. Visible in Codecov UI
-e ENV Specify environment variables to be included with this build
Also accepting environment variables: CODECOV_ENV=VAR,VAR2
-e VAR,VAR2
-X feature Toggle functionalities
-X gcov Disable gcov
-X coveragepy Disable python coverage
-X fix Disable report fixing
-X search Disable searching for reports
-X xcode Disable xcode processing
-X network Disable uploading the file network
-X gcovout Disable gcov output
-X html Enable coverage for HTML files
-X recursesubs Enable recurse submodules in git projects when searching for source files
-N The commit SHA of the parent for which you are uploading coverage. If not present,
the parent will be determined using the API of your repository provider.
When using the repository provider's API, the parent is determined via finding
the closest ancestor to the commit.
-R root dir Used when not in git/hg project to identify project root directory
-F flag Flag the upload to group coverage metrics
-F unittests This upload is only unittests
-F integration This upload is only integration tests
-F ui,chrome This upload is Chrome - UI tests
-c Move discovered coverage reports to the trash
-Z Exit with 1 if not successful. Default will Exit with 0
-- xcode --
-D Custom Derived Data Path for Coverage.profdata and gcov processing
Default '~/Library/Developer/Xcode/DerivedData'
-J Specify packages to build coverage. Uploader will only build these packages.
This can significantly reduces time to build coverage reports.
-J 'MyAppName' Will match "MyAppName" and "MyAppNameTests"
-J '^ExampleApp$' Will match only "ExampleApp" not "ExampleAppTests"
-- gcov --
-g GLOB Paths to ignore during gcov gathering
-G GLOB Paths to include during gcov gathering
-p dir Project root directory
Also used when preparing gcov
-k prefix Prefix filepaths to help resolve path fixing: https://github.com/codecov/support/issues/472
-x gcovexe gcov executable to run. Defaults to 'gcov'
-a gcovargs extra arguments to pass to gcov
-- Override CI Environment Variables --
These variables are automatically detected by popular CI providers
-B branch Specify the branch name
-C sha Specify the commit sha
-P pr Specify the pull request number
-b build Specify the build number
-T tag Specify the git tag
-- Enterprise --
-u URL Set the target url for Enterprise customers
Not required when retrieving the bash uploader from your CCE
(option) Set environment variable CODECOV_URL=https://my-hosted-codecov.com
-r SLUG owner/repo slug used instead of the private repo token in Enterprise
(option) set environment variable CODECOV_SLUG=:owner/:repo
(option) set in your codecov.yml "codecov.slug"
-S PATH File path to your cacert.pem file used to verify ssl with Codecov Enterprise (optional)
(option) Set environment variable: CODECOV_CA_BUNDLE="/path/to/ca.pem"
-U curlargs Extra curl arguments to communicate with Codecov. e.g., -U "--proxy http://http-proxy"
-A curlargs Extra curl arguments to communicate with AWS.
-- Debugging --
-d Don't upload, but dump upload file to stdout
-q PATH Write upload file to path
-K Remove color from the output
-v Verbose mode
EOF
}
I've tried many things to solve the issue, but I can't find a solution. On their GitHub repo, there is this issue that seems linked but the proposed solution has not worked for me: Failing on busybox 1.26, incorrect flags passed to find.
You can find the full log of the job here, line 434: https://gitlab.com/gaspacchio/back-to-the-future/-/jobs/788303704
Based on KamilCuk's comment, below is the full line needed to properly upload code coverage reports to codecov:
bash -c '/bin/bash <(curl -s https://codecov.io/bash)'
As pointed out by KamilCuk, notice the closing '.
The -c flag is documented as such in the man pages for bash:
-c string
If the -c option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0.
As of today, I don't know why this works. Feel free to edit this answer if you have any clues.
I'd like to print all environment variables set on my Heroku server. How can I do that with command line only?
Ok, I found the way:
heroku config
The heroku run command runs a one-off process inside a Heroku dyno. The unix command that prints environment variables is printenv (manual page). Thus
heroku run -a app-name printenv
is the command you are looking for.
step 1 : list your apps
heroku apps
Copy the name of your app
step 2 : view config variables of this app
heroku config -a acme-web
Append --json to get the output as JSON.
heroku config -a acme-web --json
Append -s to get the output in shell format, to paste directly to a .env file, for example.
heroku config -a your-app -s
I'm using the presence of an environmental variable to determine if my app is deployed or not (as adversed to running on my local machine).
My test script can find and display the variable value but my according to my app the variable isn't present.
test.rb
Secret_Key_Path = ENV['APPLICATION_VERSION'] ? '/path/to/encrypted_data_bag_secret' : File.expand_path('~/different/path/to/encrypted_data_bag_secret')
puts ENV['APPLICATION_VERSION']
puts Secret_Key_Path
puts File.exists? Secret_Key_Path
info.rb (the relevant bit)
::Secret_Key_Path = ENV['APPLICATION_VERSION'] ? '/path/to/encrypted_data_bag_secret' : File.expand_path('~/different/path/to/encrypted_data_bag_secret')
If I log the value of Secret_Key_Path it logs as the value I don't expect (i.e. '~/different/path/to/encrypted_data_bag_secret' instead of '/path/to/encrypted_data_bag_secret')
Here's how I start my app (from inside of my main executable script, so I can just run app install from any where instead of having to go to the folder)
exec "(cd /path/to/app/root && exec sudo rackup --port #{80} --host #{'0.0.0.0'} --pid /var/run/#{NAME}.pid -O NAME[#{NAME}] -D)"
if I do env | grep APP I get:
APPLICATION_VERSION=1.0.130
APPLICATION_NAME=app-name
It was suggested that it was an execution context problem but I'm not sure how to fix that if it were that.
So Whats going on? Any help & suggestion would be appreciated.
You can keep your environment variables with sudo by using the -E switch:
From the manual:
-E, --preserve-env
Indicates to the security policy that the user wishes to preserve their existing environment variables. The security policy may
return an error if the user does not have permission to preserve the environment.
Example:
$ export APPLICATION_VERSION=1.0.130
$ export APPLICATION_NAME=app-name
Check the variables:
$ sudo -E env | grep APP
and you should get the output:
APPLICATION_NAME=app-name
APPLICATION_VERSION=1.0.130
Also if you want to keep variables permanently keeped you can add to the /etc/sudoers file:
Defaults env_keep += "APPLICATION_NAME APPLICATION_VERSION"