I have followed the documentation and in my Python code I've added:
import os
import psycopg2
DATABASE_URL = os.environ['postgres://sficoXXX:842YYY#ec2-54-165-184-219.compute-1.amazonaws.com:5432/database-name']
conn = psycopg2.connect(DATABASE_URL, sslmode='require')
The URL I took from the session Config Vars (I used: heroku pg:credentials:url).
When I check the connection to the database using heroku pg:psql everything seems to be working fine.
But after deploying it shows the following error:
Failed to create session:
'postgres://sficoXXX:842YYY#ec2-54-165-184-219.compute-1.amazonaws.com:5432/database-name'
Traceback (most recent call last): File
"/Users/spoleto/.pyenv/versions/3.8.13/lib/python3.8/site-packages/otree/session.py",
line 447, in create_session_traceback_wrapper
return create_session(**kwargs) File "/Users/spoleto/.pyenv/versions/3.8.13/lib/python3.8/site-packages/otree/session.py",
line 418, in create_session
func(subsession) File "/Users/spoleto/PycharmProjects/upstream-reciprocity/prototypes/Consent/init.py",
line 35, in creating_session
DATABASE_URL = os.environ[postgres://sficoXXX:842YYY#ec2-54-165-184-219.compute-1.amazonaws.com:5432/database-name']
File "/Users/spoleto/.pyenv/versions/3.8.13/lib/python3.8/os.py", line
675, in getitem
raise KeyError(key) from None
Am I doing this right? Where does the error come from? How is the connection supposed to be established?
This is almost a copy/paste from the documentation.
The whole point of putting a connection string into an environment variable is so it doesn't need to be in your source code.
Instead of looking up the value of DATABASE_URL manually and pasting it into your source code, use the name of the environment variable:
DATABASE_URL = os.environ['DATABASE_URL']
Now your code will look for an environment variable with that name dynamically every time it runs, and set your DATABASE_URL variable to the value of the DATABASE_URL environment variable.
(The code you show in your question looks for an environment variable named postgres://..., which is very unlikely to exist.)
Note that this will fail with an IndexError if an environment variable named DATABASE_URL cannot be found. A safer way of doing this might be to use the .get() method (make sure to use round parentheses instead of square brackets):
DATABASE_URL = os.environ.get('DATABASE_URL')
Now you can even provide a fallback, e.g. for local development:
DATABASE_URL = os.environ.get('DATABASE_URL') or "postgres://postgres:postgres#localhost:5432/postgres"
Note: You didn't leak your whole connection string in your question, but I suggest you rotate your credentials anyway:
heroku pg:credentials:rotate
This will invalidate the old connection string and generate a new one.
The good news is that your DATABASE_URL environment variable will automatically be updated, and since your code now reads that value at runtime, it will continue to work!
Related
I enter the following code to connect to my heroku database using Python:
urlparse.uses_netloc.append("postgres")
url = urlparse.urlparse(os.environ["DATABASE_URL"])
conn = psycopg2.connect(
database=rul.pat[1:],
user=url.username,
password=url.password,
host=url.hostname,
port=url.port
)
I get a key error on the second line. Can someone describe what is happening? Do I need to add the DATABASE_URL to the environment somehow? I am running the Linux bash on Windows 10.
I found the answer here:
KeyError: 'DATABASE_URL' in django 1.4
in the terminal I typed something like this:
export DATABASE_URL=<url to database>
I was following this guide on deploying to Heroku and this one for sending email.
Everything works fine in development. My variables are set in Heroku:
heroku config
...
MAILGUN_DOMAIN: https://api.mailgun.net/v3/xxxxxx.mailgun.org
MAILGUN_KEY: key-3-xxxxxx
...
And loaded from the config files like so:
config :take_two, Mailer,
domain: System.get_env("MAILGUN_DOMAIN"),
key: System.get_env("MAILGUN_KEY")
However when I try to send email on Heroku when the Mailgun config is set from environment variables I get this error:
** (FunctionClauseError) no function clause matching in IO.chardata_to_string/1
(elixir) lib/io.ex:346: IO.chardata_to_string(nil)
(elixir) lib/path.ex:467: Path.join/2
(elixir) lib/path.ex:449: Path.join/1
lib/client.ex:44: Mailgun.Client.send_without_attachments/2
This happens when the domain is not set for the Mailgun Client. But it is supposed to be set from the environment variable. I made a simple module to test:
defmodule TakeTwo.Mailer do
require Logger
use Mailgun.Client,
Application.get_env(:take_two, Mailer)
def blank_shot do
Logger.info Application.get_env(:take_two, Mailer)[:domain]
Logger.info Application.get_env(:take_two, Mailer)[:key]
send_email from: "steve#xxx.com", to: "speggy#xxx.com", subject: "Hello", text: "This is a blank shot"
end
When I run TakeTwo.Mailer.blank_shot I see the correct domain/key variables logged followed by the error. I am not sure how to debug the Mailgun client remotely.
Finally, if I recreate the above module in the shell (after running heroku run iex -S mix) it works just fine!?
I feel like when the original module is being loaded perhaps the environment variables have yet to be loaded??
The answer was a little buried in a comment so I wanted to make it easier to find. As the other answer mentions, the environment variables aren't available, but the buildpack lets you configure them to be:
I created a elixir_buildpack.config file and added the following:
config_vars_to_export=(DATABASE_URL MAILGUN_DOMAIN MAILGUN_KEY SECRET_KEY_BASE)
The environment variables aren't available at build time. I had the same issue and decided to get rid of the macro carrying the configuration. You can use this patch to move on.
To set config vars for a Heroku app, you do this:
$ heroku config:set GITHUB_USERNAME=joesmith
How would I set a config var with the contents of a file?
Take a look at the heroku-config plugin, which adds a heroku config:push command to push key-value pairs in a file named .env to the app.
It also has a heroku config:pull command to do the opposite and works very well with foreman for running the app locally with the config in .env.
https://github.com/xavdid/heroku-config
Example
heroku config:push --file=.env.production
I know this is too late but still it will be helpful for future users who land here.
I also wanted a quick solution to add variable to heroku app but copy-paste is so boring.. So wrote a script to read values from the .env file and set it at the requested app - all things passed as an option:
https://gist.github.com/md-farhan-memon/e90e30cc0d67d0a0cd7779d6adfe62d1
Usage
./bulk_add_heroku_config_variables.sh -f='/path/to/your/.environment/file' -s='bsc-server-name' -k='YOUR_CONFIG_KEY1 YOUR_CONFIG_KEY2'
A simple pure-python solution using honcho and invoke:
from honcho.environ import parse
from invoke import run
def push_env(file='.env'):
"""Push .env key/value pairs to heroku"""
with open(file, 'r') as f:
env = parse(f.read())
cmd = 'heroku config:set ' + ' '.join(
f'{key}={value}' for key, value in env.items())
run(cmd)
The idea here is that you will get the same configuration as if you ran the project locally using honcho. Then I use invoke to run this task easily from the command line (using #task and c.run) but I've adapted it here to stand alone.
Please forgive my near total ignorance of Ruby, but I am having an issue with Phusion Passenger in that it attempts to run something that relies on $PATH (and blindly assumes it is defined and a string). However, nginx evidently clears out the variable.
Error message:
private method `split' called for nil:NilClass
...
Backtrace:
# File Line Location
0 /usr/lib64/ruby/gems/1.8/gems/bundler-1.1.0/lib/bundler.rb 254 in `which'
That line reads:
path = ENV['PATH'].split(File::PATH_SEPARATOR).find do |p|
If there is a workaround for this, I would greatly appreciate it, as I would like to avoid using Apache if at all possible.
have you tried setting the path within your nginx config? Something to the effect:
env PATH=/some/path/expected:/another/path;
See: http://wiki.nginx.org/CoreModule#env
I have rake tasks for getting the production database from the remote server, etc. It's always the same tasks but the server info changes per project. I have the code here: https://gist.github.com/868423 In the last task, I'm getting a #local_db_dir_path = nil error.
I don't think want to use shell environment variables because I don't want to set them up each time I use rake or open a new shell.
Stick the settings in a YAML file, and read it like this:
require 'yaml'
config = YAML.load("config.yaml") # or wherever
$remote_host = config['remote_host']
$ssh_username = config['ssh_username']
# and so on
Or you can just read one big config hash:
$config = YAML.load("config.yaml")
Note that I'm using globals here, not instance variables, so there's no chance of being surprised by variable scope.
config.yaml would then look like this:
---
remote_host: some.host.name
ssh_username: myusername
other_setting: foo
whatever: bar
I tend to keep a config.yaml.sample checked in with the main body of the code which has example but non-working settings for everything which I can copy across to the non-versioned config.yaml. Some people like to keep their config.yaml checked in to a live branch on the server itself, so that it's versioned, but I've never bothered with that.
you should be using capistrano for this, you could use mulitsage or just separate host setting to a task, example capistrano would look like this:
task :development do
server "development.host"
end
task :backup do
run "cd #{current_path}; rake db:dump"
download "remote_path", "local_path"
end
and call it like this:
cap development backup