I currently build web app which is using external MongoDb via Mongolabs.
The api is based on personal key using in urls. As docs says e.g.:
Here’s an example of a complete Resource URL:
https://api.mongolab.com/api/1/databases?apiKey=**2E81PUmPFI84t7UIc_5YdldAp1ruUPKye**
So the question is how to securely store such api key 2E81PUmPFI84t7UIc_5YdldAp1ruUPKye
Reading Django docs about Cross Site Request Forgery but stil do not understand where the key is recorded.
There are two ways to do this.
One way is to have a local_settings.py file that's imported in the main settings.py file and put into .gitignore so it's not in git. Some people however think this isn't good practice, because it might tempt to put complex things in there that aren't in VCS, so people effectively have different environments. I however am fine with it.
try:
from local_settings import *
except ImportError:
pass # No local_settings file
The other way (recommended by dislikers of the first way) is by setting it via environment variables, and reading these in settings.py.
MONGO_API_KEY = os.environ['MONGO_API_KEY']
You'd then have to pass the environment variable somehow though. E.g. via uwsgi's environ setting, or by setting it in your bash with export, or via another way.
I would load it in the settings file from an environment variable. Have a look at the Django Settings
One alternative is to use the library django-fernet-fields that uses the library cryptography.
The usage is very simple. In your model you need to add a new field:
from django.db import models
from fernet_fields import EncryptedTextField
class MyModel(models.Model):
apikey = EncryptedTextField()
By default, the field is going to be encrypted using the SECRET_KEY from your settings. So if you change it or lose it, you will not be able to access your data.
For better security, you can save your SECRET_KEY as an environment variable, and then pass it to the settings file.
import os
SECRET_KEY = os.environ.get('APP_SECRET_KEY', 'unsafe-secret-key')
django-fernet-fields
Quick answer:
Store in .env
Read in settings.py
Related
I'd like to incorporate GitLab CI into my Karate testing. I'd like to loop through my tests with different user names and passwords to ensure our API endpoints are responding correctly to different users.
With that in mind, I'd like to be able to store the usernames and passwords as secure environment variables in GitLab (rather than in the karate-config as plain text) and have Karate pull them as needed from either the karate-config or the feature files.
Looking through the docs and StackOverflow questions, I haven't seen an example where it's being done.
Updating with new information
In regards to Peter's comment below, which is what I need I am trying to set it up as follows:
set client id in karate-config:
var client_id = java.lang.System.getenv('client_id');
in the actual config object:
clientId: client_id
In my feature file tried to access it:
* def client_id = clientId
It still comes through as null, unfortunately.
You can read environment variables in karate using karate.properties,
eg,
karate.properties['java.home']
If this helps you to read the environment variables that you are keeping securely on your gitlab, then you can use it in your karate-config for authentication.
But your config and environment variable will look cumbersome if you are having too many users.
If you want to run a few features with multiple users, I would suggest you look into this post,
Can we loop feature files and execute using multiple login users in karate
EDIT:
Using java interop as suggested by peter:
var systemPath = java.lang.System.getenv('PATH');
to see which are all variables are actually exposed try,
var evars= java.lang.System.getenv();
karate.log(evars);
and see the list of all environment variables.
I am using Airflow for a few weeks now and I use a lot xcoms to send informations from one task to another.
I just figured out that these xcoms ( as the return values of the Python Operators tasks) are accessible directly from the web UI.
However, I send data like credentials, etc that I don't want to be displayed in the web UI of Airflow.
I have tried to look into the airflow config file but the only option that can be changed is the possibility to pickle xcoms.
Does anyone have a solution to my problem ?
Thank you
for data credentails, you can do it in different mode based on the security strategy.
hide the xcom menu and pages, you can change the app.py in www folder. just comments the xcom related parts. it should be 1-3 line of codes.
encrypt the xcom raw message, change you code when push or pull messages.
delete the historical xcom messages as above metioned.
If the credentials are static, then add it in variables and call them via Variables by importing from airflow.models import Variable. The other option that comes to my mind is adding a PostgresOperator to delete the XCOM once the dag is completed, something like below,
delete_xcom = PostgresOperator(
task_id='task_id',
postgres_conn_id='airflow_db',
sql="delete from xcom where dag_id=dag.dag_id and
task_id='your_task_id' and execution_date={{ ds }}",
dag=dag)
I have a Google Service Account that my app uses to retrieve data from Google Analytics.
When I created the account I downloaded a client_secrets file with all the necessary information for authorization via OAuth, and I recorded the path to this file in an environment variable called GOOGLE_APPLICATION_CREDENTIALS as per Google's documentation.
I can now get an authenticated client like this:
authorization = Google::Auth.get_application_default(scopes)
This method reads the credentials out of the file, which works locally, but my app is hosted on Heroku where file storage is impossible.
The documentation states that I can either provide this file (can’t), run my app on an official Google Service (won’t), or experience an error.
How can I authenticate my service account without the client_secrets file?
I found the answer in the source code of the google-auth-library-ruby gem.
It turns out that there is another option: take the values from the client_secrets file and put them in environment variables named GOOGLE_ACCOUNT_TYPE, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_EMAIL and GOOGLE_PRIVATE_KEY respectively.
If these keys are populated, the credentials will load from there. Not a whisper of this in the docs, though.
Since this is one of the main results that returns when searching google for "google service credentials ruby," I thought I would add my very recent experience to the list of possible answers.
Though you can do the method mentioned in the first answer, I found an alternate solution that works well with Heroku. I know it has been somewhat mentioned in another post, but the key thing that was left out was how to properly store the full GOOGLE_APPLICATION_CREDENTIALS .json file so that it can all be kept within one env on Heroku and not have special characters blow up your app when tryin to
I detail my steps below:
Obtain your GOOGLE_APPLICATION_CREDENTIALS json file by following Google's instructions here: Getting Started with Authentication
That will, of course, contain a json object with all the spaces, line returns, and quotations that heroku simply doesn't need. So, strip out all spaces and line breaks...AND PAY ATTENTION HERE -> EXCEPT FOR THE LINE BREAKS WITHIN THE 'BEGIN PRIVATE KEY' SEGMENT. Basically turn the json into one long string. Use whatever method you feel comfortable with.
Once you have a single line json file with whitespace and line breaks removed, you will need to add it to Heroku by running the following command:
heroku config:set GOOGLE_APPLICATION_CREDENTIALS="$(< /Users/whoever/Downloads/[CREDENTIAL_JSON_FILENAME].json)" --app your-app
For my situation, I needed to have the service account available on initialization, so I placed this in an initializer in my rails app:
GOOGLE_SERVICE_ACCOUNT_CREDENTIALS=Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: StringIO.new(ENV['GOOGLE_APPLICATION_CREDENTIALS'])
)
Notice the StringIO.new() method. the #make_creds wants a file. So, fake it as such by using StringIO.new.
This method works perfectly.
If you need this to work differently on your local machine, you can always store the .json somewhere in the project and reference it through a file location string. Here is my full initializer:
require 'googleauth'
#https://www.rubydoc.info/github/google/google-auth-library-ruby/Google/Auth/ServiceAccountCredentials
if Rails.env == "test"
GOOGLE_SERVICE_ACCOUNT_CREDENTIALS =
Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: File.open('lib/google/google_application_credentials.json')
)
elsif Rails.env != "development"
GOOGLE_SERVICE_ACCOUNT_CREDENTIALS =
Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: StringIO.new(ENV['GOOGLE_APPLICATION_CREDENTIALS'])
)
end
If you are using a gem like dotenv you can store the formatted json string as an ENV or you can just reference the file location in the ENV
I hope this helps someone.
I found this
require "google/cloud/bigquery"
ENV["BIGQUERY_PROJECT"] = "my-project-id"
ENV["BIGQUERY_CREDENTIALS"] = "path/to/keyfile.json"
bigquery = Google::Cloud::Bigquery.new
more detail:
https://github.com/googleapis/google-cloud-ruby/blob/master/google-cloud-bigquery/AUTHENTICATION.md
I am somewhat new to Ruby, especially the more advanced concepts like modules and mixins, so I might be using module totally out of context..
I am currently writing an internal test framework using Capybara and I am trying to figure out the best/easiest way of handling configuration data. I file a file called config.rb and within it I want to store configuration settings per environment. For example:
module QAConfiguration
config data goes here
end
module DevConfiguration
config data goes here
end
The simplest example of configuration data is usernames and password. QA and Dev of course use different users. I am thinking of two different ways of going about this but I want to make sure I am following at least a decent practice and not going into the weeds.
module QAConfiguration
USERNAME = 'test'
PASSWORD = 'test'
end
or..
module QAConfiguration
def username
'test'
end
end
And so on. Which is the best way of approaching this?
A module probably isn't the best way to implement this. Generally when testing (I use rspec) we use helper files that contain reusable code.
Object attributes like usernames and passwords are usually handled by Factories. A great gem for factories is FactoryGirl.
A common way to store configuration in ruby/rails is in a yaml file. You could for example create a file called config/test_conf.yml with yaml format:
username: 'your_user'
password: 'somepass'
Then where you need the config data:
config = YAML.load(File.read("#{Rails.root}/config/test_conf.yml"))
puts config['username']
And finally, you will usually only put a test_config.yml.example on git/svn and in your app setup readme note that they need to cp config/test_config.yml.example config/test_config.yml and edit the file.
I'm developing an API Server in Go and the server (at the moment) handles all translations for clients. When an API client fetches particular data it also asks for the translations that are available for the given section.
Ideally I want to have the following folder structure:
/messages
/home.en
/home.fr
/home.sv
/news.en
/news.fr
/news.sv
Where news and home are distinct modules.
Now the question I have for Revel is is it possible to fetch ALL language strings for a given module and given locale? For example pull all home strings for en-US.
EDIT:
I would like the output (something I can return to the client) a key:value string of translations.
Any guidance would be appreciated.
It seems to me that revel uses messaged based translation (just like gettext does), so you need
the original string to get the translation. These strings are stored in Config objects,
which are themselves stored in messages of i18n.go, sorted by language.
As you can see, this mapping is not exported, so you can't access it. The best way
to fix this is to write a function for what you want (getting the config by supplying a language)
or exporting one of the existing functions and create a pull request for revel.
You may workaround this by copying the code of loadMessageFile or by forking your version
of revel and exporting loadMessageFile or parseMessagesFile. This also is a great opportunity
to create a pull request.
Note that the localizations are stored in a INI file format parsed by robfig/config,
so manually parsing is also an option (although not recommended).