I know the slug compiler removes the .git directory when creating a heroku slug, but is there any way to configure Heroku so that I can access the currently running git commit number from within my scripts?
I'd like to be able to have a small link on my sinatra app (run within Heroku) which says "running version e72fb274a0" (or something similar). How can I retrieve this, or force the slug compiler to add it to an environment variable?
PROGRESS:
I reckon the best way to do this is to make a custom buildpack which writes the git commit version number to the heroku slug before the .git directory is deleted.
I've tried to do this (see my fork of the ruby buildpack) but the line I've added – line 23 – doesn't seem to be doing the job. Heroku sees & uses the new buildpack, but doesn't seem to write the file to the slug.
Anyone have any idea why my custom buildpack isn't working as expected?
Thanks,
JP
A couple of options...
SOURCE_VERSION environment variable (build-time)
Since 1st April 2015, there's a SOURCE_VERSION environment variable available to builds running on Heroku. For git-pushed builds, this is the git commit SHA-1 of the source being built:
https://devcenter.heroku.com/changelog-items/630
(thanks to #srtech for pointing that out!)
An example of me using that variable in a build - if you look at the HTML served by the deployed app, you'll see the commit id is coming though in an HTML comment near the very bottom: https://gu-who.herokuapp.com/
/etc/heroku/dyno metadata file (run-time)
Heroku have beta functionality to write out a /etc/heroku/dyno metadata file onto your running dyno. If you email support you can probably get added to the beta. Here's a place where Heroku themselves are using it:
https://github.com/heroku/fix/blob/6c8ab7a/lib/heroku_dyno_metadata.rb
The contents look like this:
{
"dyno":{
"physical_id":"161bfad9-9e83-40b7-b385-78305db2f168",
"size":1,
"name":"run.7145"
},
"app":{
"id":null
},
"release":{
"id":50,
"commit":"2c3a0b24069af49b3de35b8e8c26765c1dba9ff0",
"description":null
}
}
..so release.commit is the field you're after. I used to use this method until the SOURCE_VERSION variable became available.
In 2018 this is what you want:
https://devcenter.heroku.com/articles/dyno-metadata
heroku labs:enable runtime-dyno-metadata -a <app name>
You can run a script before deploy that store this information (maybe on a YAML)
using these a = `ls` (note that is not ' "apostrophe" sign is ` "inverse accute" sign)
the a variable will have the result of this bash command,so you can do
git = `git log`
and then find the information you want it and store it.
So you will be able to retrieve it later.
Did this helped ?
Related
I am a newbie in go and go-swagger. I am following steps in Simple Server tutorial in goswagger.io.
I am using Ubuntu 18.04, swagger v0.25.0 and go 1.15.6.
Following the same steps, there are a few differences of the files generated. For instance, goswagger.io's has find_todos_okbody.go and get_okbody.go in models but mine does not. Why is that so?
Link to screenshot of my generated files vs
Link to screenshot of generated files by swagger.io
Starting the server as written in the tutorial go install ./cmd/todo-list-server/ gives me the following error. Can anyone please help with this?
# my_folder/swagger-todo-list/restapi
restapi/configure_todo_list.go:41:8: api.TodosGetHandler undefined (type *operations.TodoListAPI has no field or method TodosGetHandler)
restapi/configure_todo_list.go:42:6: api.TodosGetHandler undefined (type *operations.TodoListAPI has no field or method TodosGetHandler)
The first step in goswagger.io todo-list is swagger init spec .... Which directory should I run this command in? I ran it in a newly created folder in my home directory. However, from the page, it shows the path to be ~/go/src/github.com/go-swagger/go-swagger/examples/tutorials/todo-list. I am not sure whether I should use go get ..., git clone ... or create those folders. Can someone advise me?
Thanks.
This is likely the documentation lagging behind the version of the code that you are running. As long as it compiles, the specific files the tool generates isn't so crucial.
This is a compilation error. When you do go install foo it will try to build the foo package as an executable and then move that to your GOPATH/bin directory. It seems that the generated code in restapi/configure_todo_list.go isn't correct for the operations code generated.
All you need to run this tutorial yourself is an empty directory and the swagger tool (not its source code). You run the commands from the root of this empty project. In order not to run into GOPATH problems I would initialise a module with go mod init todo-list-example before doing anything else.
Note that while the todo-list example code exists inside the go-swagger source, it's there just for documenting example usage and output.
What I would advice for #2 is to make sure you're using a properly released version of go-swagger, rather than installing from the latest commit (which happens when you just do a go get), as I have found that to be occasionally unstable.
Next, re-generate the entire server, but make sure you also regenerate restapi/configure_todo_list.go by passing --regenerate-configureapi to your swagger generate call. This file isn't always refreshed because you're meant to modify it to configure your app, and if you changed versions of the tool it may be different and incompatible.
If after that you still get the compilation error, it may be worth submitting a bug report at https://github.com/go-swagger/go-swagger/issues.
Thanks #EzequielMuns. The errors in #2 went away after I ran go get - u -f ./... as stated in
...
For this generation to compile you need to have some packages in your GOPATH:
* github.com/go-openapi/runtime
* github.com/jessevdk/go-flags
You can get these now with: go get -u -f ./...
I think it's an error of swagger code generation. You can do as folloing to fix this:
delete file configure_todo_list.go;
regenerate code.
# swagger generate server -A todo-list -f ./swagger.yml
Then, you can run command go install ./cmd/todo-list-server/, it will succeed.
First of all: I am somewhat new to cloud build. Compared to previously used methods, I find it a wrenching, unripe and fairly annoying framework. Endless time is spend getting builders to work that supposedly work out of the box (like the helm builder for example), and it's limitations are astonishing and frustrating. Perhaps the following problem is a good example:
I would like to build and push a docker image. According to the documentation, the images to be pushed to the docker repository at the end (I'm using GCR for this) reside in the following configuration section in my cloudbuild.yaml file:
images:
- 'eu.gcr.io/$PROJECT_ID/my-project:${_TAG}'
- 'eu.gcr.io/$PROJECT_ID/my-project:latest'
I can set the _TAG substitution manually by using the section:
substitutions:
_TAG: x.y.z
but that means I have to manually fix the version number in this file every time. Worse still: if I branch out, I need to maintain the version number all the time. I have a python project in this case and it uses setuptools, the version is naturally contained in the setup.py file and I can parse it out with no problem. Attempts to parse the number into a specific file and use $(cat VERSION) in the images section fail, because the system claims it can't substitute the $(cat VERSION) part. So how can I overwrite the _TAG variable inside of another build step such, that it appears correct in the 'images' section?
If you are using triggered builds from Cloud Source Repositories, GitHub, or Bitbucket you can tag your commit and use the $TAG_NAME default substitution variable.
images:
- 'eu.gcr.io/$PROJECT_ID/my-project:$TAG_NAME'
- 'eu.gcr.io/$PROJECT_ID/my-project:latest'
On the other hand if you are using the Cloud SDK to submit the Cloud Build build you can provide values with the --substitutions argument:
gcloud builds submit [SOURCE] --config config.yaml --substitutions _TAG=x.y.z
Also I believe you would find this GitOps-style continuous delivery with Cloud Build tutorial very helpful. It explains how to create a continuous integration and delivery (CI/CD) pipeline on Google Cloud Platform using Cloud Build.
You can tag your image with several tags using cloudbuild.yaml and define the Docker build step with:
steps:
- name: gcr.io/cloud-builders/docker
args:
- build
- '--no-cache'
- '-t'
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- '-t'
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:latest'
- .
- '-f'
- Dockerfile.prod
id: Build
And resulting images with:
images:
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:latest'
I am using golang 1.13 .
I have a project that depends on a private gitlab project.
I have the ssh keys for the same.
When I try to retrieve the dependencies for a newly created module, I am getting the following error:
$ go version
go version go1.13 linux/amd64
$ go mod why
go: downloading gitlab.com/mycompany/myproject v0.0.145
verifying gitlab.com/mycompany/myproject#v0.0.145: gitlab.com/mycompany/myproject#v0.0.145: reading https://sum.golang.org/lookup/gitlab.com/mycompany/myproject#v0.0.145: 410 Gone
I have no idea why it is trying to ping sum.golang.org/lookup since it is a private gitlab project.
My ~/.gitconfig contains the following (based on my looking up in google search for similar errors)
# Enforce SSH
[url "ssh://git#github.com/"]
insteadOf = https://github.com/
[url "ssh://git#gitlab.com/"]
insteadOf = https://gitlab.com/
[url "ssh://git#bitbucket.org/"]
insteadOf = https://bitbucket.org/
[url "git#gitlab.com:"]
insteadOf = https://gitlab.com/
The error still persists.
I would expect the package to be downloaded from my private gitlab project repository to the current project.
Is there anything I need to do in my private gitlab project repository to make it ready for 'go get' ?
The private gitlab project repository already contains the go.sum and go.mod for the project as well.
Anything that I am missing ?
edit: 1) The private repo name and the company name contains no asterisks or any other special characters. only alphabets and not even numeric characters.
Answering my own question after looking up,
Setting the GOPRIVATE variable seems to help.
GOPRIVATE=gitlab.com/mycompany/* go mod why
"
The new GOPRIVATE environment variable indicates module paths that are not publicly available. It serves as the default value for the lower-level GONOPROXY and GONOSUMDB variables, which provide finer-grained control over which modules are fetched via proxy and verified using the checksum database.
" from https://golang.org/doc/go1.13
Aliter:
Setting the env variable GONOSUMDB also seems to work.
Specifically, invoking the following command seems to help.
GONOSUMDB=gitlab.com/mycompany/* go mod why
The above env variable prevents the ping to sum.golang.org/lookup for a checksum match. It also prevents leaking the names of private repos to a public checksum db. [ Source - https://docs.gomods.io/configuration/sumdb/ ]
Also - here at
* GONOSUMDB=prefix1,prefix2,prefix3 sets a list of module path prefixes, again possibly containing globs, that should not be looked up using the database.
source: https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md
Related Issues:
https://github.com/golang/go/issues/32291
https://github.com/golang/go/issues/33985
["Go 1.13 has been released, and this issue was filed well after the freeze window. The proposed changes will not happen in 1.13, but don't assume they will necessarily happen in 1.14 either." from issue 33985 above. ]
Basically it failed to verify private repository. However I don't like turning off checksum, but you can easily set GOSUMDB to off before trying to get module. something like this:
GOSUMDB=off go get github.com/mycompany/myproject
ref: https://github.com/golang/go/issues/35164#issuecomment-546503518
A second and better solution is to set GOPRIVATE environment variable that controls which modules the go command considers to be private (not available publicly) and should therefore NOT use the proxy or checksum database. The variable is a comma-separated list of glob patterns (same syntax of Go's path.Match) of module path prefixes. For example,
export GOPRIVATE=*.corp.example.com,rsc.io/private
Or
go env -w GOPRIVATE=github.com/mycompany/*
Last solution you can try is to turn off such checks for all private repositories that you don't want to go public or being verified through sum.golang.org/lookup/github.com/mycompany/...
GONOSUMDB=gitlab.com/mycompany/* go mod why
Note that:
If you have issues fetching modules or repos over https, you may want to add the following to your ~/.gitconfig to make go get/fetch repositories using ssh instead of https
[url "ssh://git#github.com/"]
insteadOf = https://github.com/
Change following go variable's setting and then upgrade your package,
$ export GO111MODULE=on
$ export GOPROXY=direct
$ export GOSUMDB=off
$ go get -u <your dependency package>
I have this scenario too and this works for me.
edit your .git/config and add two lines in it.( I have this in a global .gitconfig in home dir)
[url "ssh://youprivate.com"]
insteadOf = https://yourprivate.com
export GOSUMDB=off
Then everything will OK.
When I was using Arcanist and landing some code I found out that my terminal does not escape the colors. The <ESC>[1;32m9912da1<ESC>[m is supposed 32m9912da1 or something similar. Is this a problem with my terminal? Other people at my work don't find this problem.
Landing current branch 'some-branch'.
TARGET Landing onto "master", selected by following tracking branches upstream to the closest remote.
REMOTE Using remote "origin", selected by following tracking branches upstream to the closest remote.
FETCH Fetching origin/master...
These commits will be landed:
- <ESC>[1;32m9912da1<ESC>[m some commit
- <ESC>[1;32m687f799<ESC>[m some other commit
If you are using arcanist on windows with git bash this should fix it:
# add the following function to your .bash_profile
function arc(){
command arc --ansi $#|cat
}
For detailed instructions see: https://thomas-barthelemy.github.io/2015/04/23/phabricator-arcanist-gitbash/
To set config vars for a Heroku app, you do this:
$ heroku config:set GITHUB_USERNAME=joesmith
How would I set a config var with the contents of a file?
Take a look at the heroku-config plugin, which adds a heroku config:push command to push key-value pairs in a file named .env to the app.
It also has a heroku config:pull command to do the opposite and works very well with foreman for running the app locally with the config in .env.
https://github.com/xavdid/heroku-config
Example
heroku config:push --file=.env.production
I know this is too late but still it will be helpful for future users who land here.
I also wanted a quick solution to add variable to heroku app but copy-paste is so boring.. So wrote a script to read values from the .env file and set it at the requested app - all things passed as an option:
https://gist.github.com/md-farhan-memon/e90e30cc0d67d0a0cd7779d6adfe62d1
Usage
./bulk_add_heroku_config_variables.sh -f='/path/to/your/.environment/file' -s='bsc-server-name' -k='YOUR_CONFIG_KEY1 YOUR_CONFIG_KEY2'
A simple pure-python solution using honcho and invoke:
from honcho.environ import parse
from invoke import run
def push_env(file='.env'):
"""Push .env key/value pairs to heroku"""
with open(file, 'r') as f:
env = parse(f.read())
cmd = 'heroku config:set ' + ' '.join(
f'{key}={value}' for key, value in env.items())
run(cmd)
The idea here is that you will get the same configuration as if you ran the project locally using honcho. Then I use invoke to run this task easily from the command line (using #task and c.run) but I've adapted it here to stand alone.