I have a front-end build and deploy that I want to run on circleci with node. The deploy part needs a config file with api keys and passwords which I don't want to store in git. How do I add a config file to my build?
disclaimer: Developer Evangelist at CircleCI.
The CircleCI config file itself would be stored in Git. API keys, passwords, and secrets you'd store in private environment variables via the CircleCI UI.
Related
We are using Terraform Enterprise Cloud and Azure DevOps YML pipelines for Azure infra deployments.
Requirement: We want to separate .tfvar files completely from the main terraform folder and keep them in different Repo called config Repository.
Solution 1: We can refer tfvars from the config repository while running the below command,
terraform plan --var-fil -We cannot implement
Note: Since we are using global templates, these terraform commands like fmt, validate, plan, and apply are managed by the template itself, we are not allowed to edit the template.
Here is the logic,
template expects only .tfvars file in the current directory, then there are some bash commands to rename it to .auto.tfvars.
We know that these auto.tfvars files will be automatically identified by Terraform.
Solution 2: We are expecting and struggling to implement and need some help
By default Template copies all terraform folders to ADO Agent Container. we want to make sure the .tfvar file from the Config repository is available in the agent container. Then this solution will be good.
May be,
We can achieve it by Copying the .tfvars file from the config repository to the agent container by writing some shell script. but it has to be inside the terraform folder. because only terraform folder will be copied to the agent container.
Or is there any way that we can integrate a shell script to terraform configuration which can download tfvars file from config repository to container in run time.
Any other solution or approach will be appreciated.
To make sure the config repo files are available during runtime you can add a second artifact to the release pipeline. This will allow you to modify your var argument with the appropriate file.
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/artifacts?view=azure-devops
One approach is to have your tfvars file stored as a secure file, then just add a step in your pipeline to download it, however, if you're using Terraform Enterprise, is there any particular reason to not use Terraform workspace variables?
I'm using GitLab CI Auto DevOps to compile a project based on Maven based in a DinD (docker-in-docker) runner.
CI job start, and buildpack for Maven is correctly detected (based on herokuish).
How can I configure a custom settings file without switching to a custom .gitlab-ci.yml?
I would like to continue using Auto DevOps.
Because maven needs access to a private repository based on Nexus it is convenient to configure a custom settings.xml (and version it with your source code)
The easiest solution is to include a custom settings.xml. This feature was included in the corresponding buildpack used by Auto DevOps some time ago, as you can see in this article from Heroku buildpacks about "Using a Custom Maven Settings File".
So I defined MAVEN_SETTINGS_PATH variable in .gitlab-ci.yml config file:
variables:
- MAVEN_SETTINGS_PATH: ".m2/settings.xml"
Then, included the file settings.xml in the repository.
Avoid to include secrets or another sensible information
When using a private maven repository with credentials
Finally, you can define in Gitlab some variables to be used in settings.xml. Using Gitlab UI or API add variables for the user, password, and repository url, to be included as environment variables from Gitlab into the job. Then you can use it in settingx.xml like ${env.VARIABLE_NAME}
Example of Gitlab-CI configuration file:
include:
- template: Auto-DevOps.gitlab-ci.yml
variables:
MAVEN_SETTINGS_PATH: ".m2/settings.xml"
AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES: NEXUS_REPO_USER,NEXUS_REPO_PASSWORD,NEXUS_REPO_URL
As a final recommendation, you should avoid to use passwords in environment variables, use native methods from your environment for credentials storage is recommended.
I'm trying to set up automated build for go projects. Most people just use github dependencies which don't need credentials. We have some internal dependencies however available on our private git central server. Credentials are needed however to have go access these.
A possible workaround would be to configure a global git variable inside our build machines / build dockers; something like:
git config --global url."https://user:password#private.git.server/".insteadOf "https://private.git.server/"
however this doesn't seem to be the best solution to me, since the password would be stored in a human-readable text file.
I think the git-credentials plugin should be able to help me out; could I maybe export GIT_TERMINAL_PROMPT=1 and let the git-credentials plugin fill in for me?
How could I make sure go get or go install gets access to our private repository in a secure way?
I use a workaround with GITHUB_TOKEN to solve this.
Generate GITHUB_TOKEN here https://github.com/settings/tokens
export GITHUB_TOKEN=xxx
git config --global url."https://${GITHUB_TOKEN}:x-oauth-basic#github.com/mycompany".insteadOf "https://github.com/mycompany"
This way you don't expose the password and can revoke token at any time.
Note: Go uses http when downloading dependencies, not ssh.
travis has a config to set the buildpack source for heroku deployment using the Anvil deploy strategy, but it doesn't specify how to set that config value using the git deploy strategy.
how do you set buildpack config value as a heroku config variable before deployment if you're using the git deployment strategy in travis?
thanks.
For git deploys, Heroku relies on the BUILDPACK_URL environment variable to be configured for your repository.
Anvil doesn't utilize these settings, hence it needs to be manually specified in the deployment settings.
I am in process of moving configuration parameters out of Java application. I discover that the best approach is to extend your classpath and use .properties files (leave ZooKeeper alone for another requirement).
So my WAR file no longer have any hosts/IPs/URLs, users/passwords.
DevOps distribute configs manually across test, stage, stable installations.
Now time for Jenkins to run tests. But they fail as there are no required .propeties files in classpath.
How can I load this config files to Jenkins and how to make in available in test classpath?
maven-surefire-plugin allow extending classpath and passing system-properties.
So only question how to get separate directory in Jenkins hosting server and load files to this directory and create alias/placeholder/envvar per build job to refer to this path in build config.
This job can be done with SSH access, but I think that this is "wrong way". I expect that this can be done via Jenkins UI (any manager can upload file in WEB browser).
UPDATE I have no requirements for distributed slave/master builds but it whould nice to have solution that migrate configuration files to slaves automatically...
In this way sshing to host or ftp/scp - bad thing.
I read most of Jenkins docs, ask at mail list and IRC. Yea - Jenkins community is silent. At docs I found link to Config File Provider Plugin, after that I visit http://builder.evil.com/jenkins/pluginManager/available page and look for config keyword.
There are a lot related plug-ins with various usefulness to my subject (most useless first):
https://wiki.jenkins-ci.org/display/JENKINS/Envfile+Plugin - This plugin enables you to set environment variables via a file.
https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Binding+Plugin - Allows credentials to be bound to environment variables for use from miscellaneous build steps.
https://wiki.jenkins-ci.org/display/JENKINS/Environment+Script+Plugin - Allows you to run a script before each build that generates environment variables for it.
https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin - This plugin makes it possible to have an isolated environment for your jobs.
https://wiki.jenkins-ci.org/display/JENKINS/Copy+Data+To+Workspace+Plugin - Copies data to workspace directory for each project build.
https://wiki.jenkins-ci.org/display/JENKINS/Copy+To+Slave+Plugin - This plugin allows to copy a set of files, from a location somewhere on the master node, to jobs' workspaces. It also allows to copy files back from the workspaces of jobs located on a slave node to their workspaces on the master one.
https://wiki.jenkins-ci.org/display/JENKINS/Config+File+Provider+Plugin - Adds the ability to provide configuration files (i.e., settings.xml for maven, XML, groovy, custom files, etc.) loaded through the Jenkins UI which will be copied to the job's workspace.
Only last plug-in - Config File Provider Plugin allow editing configs via Jenkins WEB interface. And it have brother - Managed Script Plugin - for uploading/managing/editing custom scripts. No question now I use Config File Provider Plugin!
You should keep the configs required for the tests together with the rest of source code, so that after compilation, your unit tests can run.
After deploying the .war, the DevOps team should overwrite the in-war configs with whatever per-environment configs that they have.