With a Spring Boot application, the artifact of choice for deployment seems to be a single executable file, JAR or WAR. In one case, I have added my own encryption/decryption to the project so that I can check in my application configuration without plain-text credentials. If I build a deployment archive with a configuration file, that's checked in with empty credentials, I have to populate the credentials just before I package and deploy. Or, if I choose to use plain-text creds in the configuration file, I'm always forgetting to remove the creds and keep checking in the file with visible credentials.
I'm writing my question here to see if anyone has any bright ideas around how we can manage this, develop, checkin, package, and deploy without all this dancing around the YAML configuration files.
For a JHipster app my encryption/decryption solution at least protected the files from plain-text, but any sharp Java developer could use my encryption utility to decode the creds I check in.
What's the best strategy here? Could we easily add the creds with some scripts or Maven/Gradle operation? I'm game to change anything.
For me, the obvious solution is to externalize the config both local and deployed. And, if I can override the template config (without the creds) in my git repo, I'll still have the template version of my configs local and in the repo.
Related
I am close to publishing by first open-source library to the Sonatype repository. Everything works fine locally, but I want to automate the process so that whenever I create a new tag/version, the version get's published directly from Gitlab CI
Now, I have two problems: the secring.gpg file and the passwords.
I know that I can have protected variables for my CI, but I don't know two things:
How to use them during my build process (I am using gradle):
I need the signing.password, the signing.secretKeyRingFile path and the ossrhPassword, all of which are located in the gradle.properties file
How to store the sigring.gpg file, which is a binary file and I can't copy-paste it in the variable.
is there a way in the TeamCity to have application config files with production passwords stored in the TeamCity?
During build/deployment these files will be used for different environments.
For example in the project repository you have app.config file which is filled with Development env secrets. And you don't want to store production envs secrets in the repository.
Need exact file because it's not only a single values which could be covered with Configuration Parameters, it could be big XML pieces. So better to have a whole app.config file somewhere in the TeamCity configured which I can use to publish to production env.
Thanks in advance
As I know in Team city administration you can configure each build configuration as administrator. And for build configuration, you can find the parameters section where you can add parameters as secret values. These values would be available during the build process but they would not be logged. Hope it helps you.
I am using developing the application using Spring Boot, there are some secret keys has to be added to the files but while committing and pushing the file to Git it must not be visible to the public.
As those keys in the files need to be secure, how to provide the security or any encryption to those files??
As mentioned, don't push these files to a public repo, or any repo where you think other developers on the team should not have access.
You can easily ensure these files don't somehow get committed by using a .gitignore file:
gitignore - Specifies intentionally untracked files to ignore
See: .gitignore documentation
Here is a collection of useful .gitignore configurations: A collection of useful .gitignore templates
Rather than using encryption, it would be better to use a directory and repo structure as follows:
Project directory: All project files, no secrets.
Secrets directory: Secrets only.
Project repo: Public repository.
Secrets repo: Private repository giving access to trusted developers.
Then within your project you simply reference the necessary secrets in the secrets directory.
If you do decide to use GPG encryption (not recommended as then you have to remember not to by mistake push an un-encrypted file), you can use a basic symmetric cipher. On a Unix machine, install gpg and then use the following terminal command - it will request a password and generate an encrypted .gpg file.
-c, --symmetric encryption only with symmetric cipher
gpg -c secrets.txt
You should save that keys as an environment variables.
How to store and retrieve that values will depend on the tools you used.
In your case, I think Spring framework can do really.
Hope you get the logic and can find out how to achieve that in Spring or server that you used.
We currently use TeamCity to build a deployment artifact, then a further TeamCity task takes that artifact and deploys it to our development and testing servers on demand.
We can store the passwords and other secret data in properties files that we can check into source control, as these are all internal servers and the developers have full access to them.
However for release to Production (and our final test layer) there are secret passwords and configuration that we don't want checked into the normal source control, or to have development be able to discover the passwords. So to do 'real' deployments we have to hand the artifact over to another team and they maintain a properties file with the production values.
What methods exist to store these secrets and allow TeamCity to run a deploy without ever leaking the secrets out?
(note I am one of the devs and it is not a trust issue... I don't want to have the ability to find out prod passwords so I can never accidently know them and do some horrific damage!)
Probably what you need here, is to create a separate project with narrower scope of permissions (for example, allow only certain people to edit build configurations). In this project create a build configuration, responsible for deployment. In this configuration, you can define a Typed Parameter of type 'password' to store the password to the production environment.
Another option is to use Deployer Plugin, especially its ability to deploy over ssh with private key authentication
If you are OK to use a third party solution, consider using a solution like CloudMunch which can help you to perform release management functions with these secure parameters collected at deploy time and encrypted post deployment.
Disclaimer: I work with CloudMunch
You can do 2 things.
Use a teamcity project to deploy artefacts for production only. This will only be accessible to ops members.
Teamcity also supports running agents with different user ids. You can create a new user id which can have access to the production "secrets" (passwords and configuration). Use this id to run the targets in the 1st step.
How best should I accomplish the following deployment objectives with Git deployment for Azure?
Easily switch when working locally to either use fake in-memory data or (eventually) non-production snapshot of real data
Deploy to staging environment on Azure such that at first I could use fake in-memory data and eventually move to non-production snapshot of real data.
Deploy to production with real data
I currently deploy using Github and a staging branch to a staging Azure website. Since I deploy to a public repo, the web.config file is ignored by git. (EDIT: I just learned that ignoring web.config actually causes deployment error on azure)
Any help/suggestion is appreciated.
It's actually supposed to be simpler than that. Please see this page. Basically, the idea is that you set some AppSettings in the Azure portal to override the default values that are committed to your repo.
Well... Here's what I did that works for me right now.
To quickly switch between fake in-memory data locally, I use a compilation symbol LOCAL and a preprocessor directive #if LOCAL.
Same compilation symbol works when you deploy to Azure, so I can work on fake data until I'm ready to switch to real db. I can also use the app settings if I really want to make to switch it more easily.
The challenge was to keep a web.config with "secrets" (like connection string) locally and not expose it to Github. I added it to .gitignore, but then my deployments started failing on Azure because it could not find the web.config. Just copying it to wwwroot via ftp did not help - Azure was looking for web.config in the repository.
So, to make this work I "slightly" altered the deployment process by first copying the Web.config from wwwroot to the repository before running the default deploy.cmd. This was simple - this is what you do:
Create a .deployment file in the root of your repository with the following:
[config]
command = deploy.my.cmd
Create deploy.my.cmd with the following script:
xcopy %DEPLOYMENT_TARGET%\Web.config %DEPLOYMENT_SOURCE%\\ /Y
deploy.cmd
Now, I have web.config with secrets locally. Git ignores this file. I uploaded the correct web.config to Azure via FTP, and it gets used whenever I deploy.