Here is the link 12 factor
I am confused weather if I should store values inside my app.properties file vs environment variable.
App.properties
Memory_Folder_Test = Test
Memory_Folder_Prod = Production
Memory_Folder_Dev = Development
Strong_threshold = 10
Low_Threshold = 2
Username = FirstUser
Password = PasswordSecret
So theoretically where should I put these values in? application.properties or as environment variables? If I did not read wrong the purpose of 12 factor is to remove putting values in properties file and externalize it.
You can store the values in application.properties file, however, spring allows you to override those values using environment variables. Hence, it is a 12 factor compliant.
You store the properties externally using something like spring cloud config. You then use the environment properties to define the configurations (like the url) needed to access cloud config from your applications
I prefer to store environment variables in files, encrypt the files and check the encrypted files into git, via blackbox: https://github.com/StackExchange/blackbox
Blackbox will handle encryption/decryption so that it makes it rather difficult to check the unencrypted creds into your repo. Also, the way openpgp works, you can enable teams of devs to encrypt/decrypt the files.
That project is maintained by StackExchange (aka the guys who run this site). It takes some time figuring out openpgp/gpg (which blackbox depends on), but it has been well worth it for me. I've been using in linux and also in windows (via the windows linux subsystem).
Related
We are currently set path of properties file which contains secret/access key for Credentials File for AWSCredentialsProviderControlerService . Issue, is we are changing properties path for prod and non prod each time we run nifi workflow. trying to come up no change on Configuration on Credential File path, so that access/secret key would be read regardless of prod and non prod. Since credential file wont support Nifi Expresion language, trying to make use of ACCESS KEY/SECRET properties ${ENV:equalsIgnoreCase("prod"):ifElse(${ACESS_PROD},${ACESS_NONPROD})} Issue we are facing, we are not able to store these access key/secret keys to the registry. Hence unable to implement this change. Is there any way to read access/secret key regardless of environment in Nifi. Curently we are using 1 property file for non prod nifi and 2nd property file for prod properties. In this set up, manually need to changed to credential file path when switching from prod to non prod. Trying to seamlessly work without changing path of credential file. Is there any way to make this happen?
enter image description here
The process that uses the AWSCredentialsProviderControlerService does not support param or variables, but the AWSCredentialsProviderControlerService "credential file" property supports "Parameters Context" as entries, make use of this for your solution.
Example:
Trigger something --> RouteOnAttribute --> If Prod (run executestreamcmd and change the Parameter Context Value to point to prod credfile) else if DEV(run executestreamcmd and change the Parameter Context Value to point to prod credfile) --> then run you AWS Processor.
You can use the toolkit client to edit the parameter context, or event nipyapi python module. It will not be fast tohu.
I'm using the Golang google-cloud-sdk to get informations on resources (specifically here compute instances, but it doesn't really matter).
The gcloud cli allows to do something like this:
gcloud config set compute/zone ZONE
Which under the hood will write in ~/.config/gcloud/configurations/config_default those value as something that looks like an ini file.
Can the (go) sdk read config those config file ?
The cli also read the environment variable CLOUDSDK_COMPUTE_ZONE if not defined in the config file.
Can the sdk also read this variables ?
To sum up the question , how can I use the same config mechanism the gcloud cli uses with the Go sdk ?
To sum up the question , how can I use the same config mechanism the gcloud cli uses with the Go sdk ?
As far as I know, you can't. You need to specify the zone to all your operations.
Long time ago, someone asked about CLOUDSDK_CONFIG and the last response is cristal clear:
Resolved: we decided not to honor CLOUDSDK_CONFIG, in the interest of maintaining simplicity for the ADC spec.
https://github.com/googleapis/google-cloud-go/issues/288
And I think it's true for all the CLOUDSDK_* env.
I searched a lot but couldn't find any useful information about this:
What exactly setenv.sh is used for in Tomcat ?
Let's say we have a REST API (built with Java EE or Spring) which uses some parameters/variables like AWS Credentials, Database Credentials etc.
Does it make sense to parametrize the application with environment variables for these things I mentioned above and put their values to env vars on setenv.sh for each Tomcat instance in case we use more than one instance and with different parameters/variables ?
Or setenv.sh isn't for things like that ?
Thanks in advance
I've only ever seen it used for specifying CATALINA_OPTS, but I noticed that the RUNNING.txt file mentions using it to set JRE_HOME and JAVA_HOME.
RUNNING.txt also states under Advanced Configuration - Multiple Tomcat Instances:
In many circumstances, it is desirable to have a single copy of a
Tomcat binary distribution shared among multiple users on the same
server. To make this possible, you can set the CATALINA_BASE
environment variable to the directory that contains the files for your
'personal' Tomcat instance.
So I guess the answer is that a setenv file can be created for each app and can probably can be used to store credentials, but I don't think that it's commonly used for that purpose.
~/Programs/apache-tomcat-9.0.7/bin/setenv.sh
export CATALINA_OPTS="$CATALINA_OPTS -DENV_TARGET=prod -DMy_Env_Var1=Whatever -DMy_Env_Var2=CapitalOneIsTheBest"
So everything in between "" above in setenv.sh gets set as environmental variables for your applications in Tomcat. To retrieve/use those environmental variables, use this in your Java (Spring) applications:
Ex1) String myWhatever = java.lang.System.getProperties().getProperty("My_Env_Var1");
Ex2) String env= System.getProperties().getProperty("My_Env_Var2");
To change the environmental variable (for example when you are writing unit tests), do this:
System.setProperty("My_Env_Var1", "newEnv");
We deploy services as docker containers using Marathon. The containers include a base config file, Marathon pulls an environment config file (which has a subset of the base keys) at deployment time so when the app starts it has;
environment.toml
config.toml
when reading the config we need to conflate the values in both files to a single set, effectively masking/shadowing the values present in both files with those in the environment file.
I didn't find this functionality in the Viper docs. Unless I have missed something it seems my options are;
Write a package that uses Viper to read both files and perform the conflation.
Extend Viper
Before I start writing code, is there already a mechanism for doing this?
I wrote a helper script in Ruby to handle my file synchronization through some servers. It was used only in my intranet and authentication was made by SSH keys. But now I want to use it where I can't use SSH keys and I want to store the passwords in a config file.
I know, there are some encryption libraries like bcrypt or OpenSSL, but I have a problem with that:
I start my script and enter my passphrase and it is stored in a variable to decrypt my passwords.
My code is open source.
So everybody, who has access with my user to my computer (which would be the first barrier, which I'd like to extend) and looks into the memory (where my passphrase is stored) can decrypt my password file. How is that handled in applications which are relevant to security?
Edith says as a reply to DevDude (but here, because I want to keep my specifications in my question):
But then this configuration file would be plain text and not encrypted. And when I encrypt this file there are two more issues in my opinion:
The super_secret_pwd would be stored in a variable, so when I would search in the memory of the computer, I would find it, wouldnt I?
The master password for encryption would be in the memory as plain text, too.
So the big question is: Is it possible to read plain text variables from the memory? As I know it is possible in C and a big security issue.
What you are looking for is to use a YAML file with the password/API keys. and never check this file into your repo.
Then you can reference this file on your initializers, and maybe make the password a global variable or x, use configatron, etc.
This is basically how production applications work, they read their important settings from a YAML file stored on the server itself.
This is what I use:
#c = configatron
# Per environment settings
app_settings = YAML.load_file('config/secret_stuff.yml')
#c.password = app_settings['super_secret_pwd']
Do not use ENVIRONMENT variables because they have all sort of security issues. They are an antipattern.