I'm playing around with deploying Clojure/Noir apps on Heroku and I've got my app mostly working. However, one final piece I need is to figure out the hostname of my app when deployed on Heroku. Ideally, I want to do this dynamically instead of hard-coding it.
So, if for example, my app's URL is 'http://freez-windy-1800.herokuapp.com', I want to be able to dynamically get this within my clojure code.
I know that I can look at the incoming request to figure this out, but ideally, I'd like to have some sort of 'setting' where I evaluate an expression once and save the value in a variable that I can then use (coming from the Python/Django world, I'm thinking of the settings.py equivalent in Clojure).
For reference, the code I'm deploying is available at https://github.com/rmanocha/cl-short.
You could set an environment variable in Heroku by
heroku config:add BASE_IRI=http://freez-windy-1800.herokuapp.com
and read it back in Clojure
(defn- base-iri []
(or (System/getenv "BASE_IRI") "http://localhost/"))
Heroku already sets the PORT you can use
(defn -main []
(let [port (Integer. (or (System/getenv "PORT") 8080))]
(run-jetty #'app {:port port})))
Works for me in different environments.
You would typically do this with InetAddress from the Java standard library.
(.getCanonicalHostName (java.net.InetAddress/getLocalHost))
This, however, does not do a DNS lookup.
3 ways to get the hostname. Use as you wish.
(ns service.get-host-name
(require [clojure.java.shell :as shell]
[clojure.string :as str])
(:import [java.net InetAddress]))
(defn- hostname []
(try
(-> (shell/sh "hostname") (:out) (str/trim))
(catch Exception _e
(try
(str/trim (slurp "/etc/hostname"))
(catch Exception _e
(try
(.getHostName (InetAddress/getLocalHost))
(catch Exception _e
nil)))))))
Related
I want have two different sets of API keys for development and production and I want to use them automatically depending on whether the app is run locally or on heroku.
For nodejs apps, I do the following:
if (process.env.NODE_ENV === 'production') {
// ./prod contains the production keys
module.exports = require('./prod');
} else {
// ./dev contains the development keys
module.exports = require('./dev');
}
How to do something similar in a Clojure Leiningen app? What would the process.env.NODE_ENV analogue be, and how to implement it?
Clojure runs on JVM, so you could use java System class. Assuming you have set APP_ENVIRONMENT variable in bash.
(if (= "production"
(or (System/getenv "APP_ENVIRONMENT")
"development"))
(start/production)
(start/development))
I would say the cleanest solution would be to use the environ library together with the lein-environ plugin.
With these you store development version of API keys in env vars in the leiningen dev profile and on production pass them as actual environment variables. E.g. you can have the following in your project.clj:
(defproject ...
...
:profiles {:dev {:env {:my-api-key "whatever-dev" } } }
...)
On the production environment you could simply set an environment variable MY_API_KEY to "whatever-prod".
In your code you will access the key like this:
(require '[environ.core :refer [env]])
(env :my-api-key) ;; would yield "whatever-dev" on dev and "whatever-prod" on prod environment
In particular, this allows you not to store the production keys under source control.
environ provides a couple more options - in particular with env files, which may also fit your needs. Do check its capabilities.
If you also need access to environment variables in your CLJS (on frontend), you can't simply access environment variables there indeed (because there's no such thing as environment anymore - the code runs on the client).
Still, if you just wish to drop the values into the resulting js during compilation, you can use environ + a macro for that. You will write a macro in a clj file that just returns the env var and then refer it in your cljs file. During compilation the appropriate value will be injected.
In a clj file:
(ns yourns.environment
(:require [environ.core :refer [env]]))
(defmacro my-api-key []
(env :my-api-key))
In cljs:
(ns yourns.core
(:require-macros [yourns.environment :refer [my-api-key]))
(my-api-key) ;; will return the appropriate value
Note, however, again that the variable value will be injected in compile time and can't be changed later.
Please see here for a complete code example.
I was following this guide on deploying to Heroku and this one for sending email.
Everything works fine in development. My variables are set in Heroku:
heroku config
...
MAILGUN_DOMAIN: https://api.mailgun.net/v3/xxxxxx.mailgun.org
MAILGUN_KEY: key-3-xxxxxx
...
And loaded from the config files like so:
config :take_two, Mailer,
domain: System.get_env("MAILGUN_DOMAIN"),
key: System.get_env("MAILGUN_KEY")
However when I try to send email on Heroku when the Mailgun config is set from environment variables I get this error:
** (FunctionClauseError) no function clause matching in IO.chardata_to_string/1
(elixir) lib/io.ex:346: IO.chardata_to_string(nil)
(elixir) lib/path.ex:467: Path.join/2
(elixir) lib/path.ex:449: Path.join/1
lib/client.ex:44: Mailgun.Client.send_without_attachments/2
This happens when the domain is not set for the Mailgun Client. But it is supposed to be set from the environment variable. I made a simple module to test:
defmodule TakeTwo.Mailer do
require Logger
use Mailgun.Client,
Application.get_env(:take_two, Mailer)
def blank_shot do
Logger.info Application.get_env(:take_two, Mailer)[:domain]
Logger.info Application.get_env(:take_two, Mailer)[:key]
send_email from: "steve#xxx.com", to: "speggy#xxx.com", subject: "Hello", text: "This is a blank shot"
end
When I run TakeTwo.Mailer.blank_shot I see the correct domain/key variables logged followed by the error. I am not sure how to debug the Mailgun client remotely.
Finally, if I recreate the above module in the shell (after running heroku run iex -S mix) it works just fine!?
I feel like when the original module is being loaded perhaps the environment variables have yet to be loaded??
The answer was a little buried in a comment so I wanted to make it easier to find. As the other answer mentions, the environment variables aren't available, but the buildpack lets you configure them to be:
I created a elixir_buildpack.config file and added the following:
config_vars_to_export=(DATABASE_URL MAILGUN_DOMAIN MAILGUN_KEY SECRET_KEY_BASE)
The environment variables aren't available at build time. I had the same issue and decided to get rid of the macro carrying the configuration. You can use this patch to move on.
In the boot wiki (https://github.com/boot-clj/boot/wiki/S3-Repositories), it specifies that you can in-line AWS credentials for using S3 as a Maven repo. This is sub-optimal from a security perspective, because I don't want to check in AWS creds, even if they have limited permissions.
In leiningen with s3-wagon-private, you could specify the access key and secret key through environment variables with:
{:url "s3p://acme/repo/"
:username :env
:passphrase :env}
Or, from specific env variables, with:
{:url "s3p://acme/repo/"
:username :env/aws_access_key_id
:passphrase :env/aws_secret_access_key}
Or with a GPG encrypted ~/.lein/credentials.clj.gpg file with:
{:url "s3p://acme/repo/"
:creds :gpg}
The push task in boot seems to support GPG encrypted credentials for deploying to Clojars (https://github.com/boot-clj/boot/wiki/Deploying-with-Boot) in $BOOT_HOME/credentials.clj.gpg. So, in general, boot supports GPG it would seem.
When I try either environment variable approach, I get the following error, suggesting this form of credentials is not supported:
java.lang.IllegalArgumentException: No matching ctor found for class org.sonatype.aether.repository.Authentication
...
cemerick.pomegranate.aether/set-authentication aether.clj: 165
cemerick.pomegranate.aether/make-repository aether.clj: 185
cemerick.pomegranate.aether/resolve-dependencies*/fn aether.clj: 712
...
The GPG approach seems to fail to pick up the credentials, and results in a 403 error from S3.
I could use (System/getenv "AWS_ACCESS_KEY_ID") to directly read in the env variables in the repository map I suppose, but I would rather use a supported mechanism if there is one. GPG-encrypted credentials would be the ideal solution for us if this can be achieved from both a security perspective as well as having multiple S3 Wagons set up without juggling environment variables.
I'm using the latest Boot (2.4.2) on OS X El-Capitan. GPG can decrypt the credentials successfully on the command line, even in quiet mode (gpg --quiet --batch --decrypt ~/.boot/credentials.clj.gpg works). Putting the credentials directly in the repository map does work, and the same credentials.clj.gpg file works from lein. I am new to Boot though, so it's possible I'm missing something obvious!
Please update to Boot-clj 2.5.0, which has greatly simplified gpg signing and encrypting. It now leverages the gpg binary, and picks up your setup without further configuration.
GPG-encrypted credentials and environment variables are supported via the built-in configure-repositories! facility. It accepts a function that will operate on the repositories map. You can do whatever you want in the body as long as you return a repositories map.
So, in your case, For GPG-encrypted credentials:
(configure-repositories!
(let [creds-file (File. (boot.App/bootdir) "credentials.gpg")
creds-data (gpg-decrypt creds-file :as :edn)]
(fn [{:keys [url] :as repo-map}]
(merge repo-map (creds-data url)))))
And for environment variables:
(configure-repositories!
(fn [{:keys [url] :as repo-map}]
(->> (condp re-find url
#"^https://example\.org/repo"
{:username (get-sys-env "EXAMPLE_USER" :required)
:password (get-sys-env "EXAMPLE_PASS" :required)}
#".*" nil)
(merge repo-map))))
A good place to put this is in your boot.profile.
More information on the wiki
I'm using
[ics#steamboy util]$ lein version
Leiningen 2.5.1 on Java 1.7.0_91 OpenJDK Server VM
with Clojure 1.6
In lein repl I used to be able to call a function from within repl.
util.core=> (load-file "src/util/core.clj")
#'util.core/-main
util.core=> (bldg-sqft-test)
And execute a function from within the repl.
(defn ret-val-from-sos
"Takes a value, a map key, an s-o-s, and returns first match."
[in-val map-key-1 map-key-2 s-o-s]
(doseq [x s-o-s]
(println (str (first x)))))
(defn bldg-sqft-test
[& args]
(let [bldg-cols (fetch-csv-data "bldg_sqft_cols.csv")
bldg-data (fetch-csv-data "Buildingsqft.csv")
mapped-data (xform-sos-in bldg-data bldg-cols)
my-bldg-sqft (ret-val-from-sos (str 70782) (keyword "Bill#") (keyword "Fin. Area") mapped-data)]
my-bldg-sqft))
Debugging was easier, when I could examine variables in repl. As a workaround, I've converted a library to run with a main, but it's not as effective as repl debugging
What settings/configuration do I need to do call functions from within the repl?
When you run $ lein repl a JVM instance is started and it loads all the namespaces in your classpath, so (load-file "example.clj") isn't necessary.
The correct sequence for what you're trying to do is:
$ lein repl
user=>(require 'util.core)
nil
user=>(in-ns 'my-ns.core)
nil
my-ns.core=>(bldg-sqft-test)
or
$ lein repl
user=>(require '[util.core :refer :all])
nil
user=>(bldg-sqft-test)
If you somehow need to load a external .clj file then (load-file "external.clj") will add the file to the classpath and then you can require the namespace as above.
My code is very simple:
(def form-test
"<html><body><form action=\"/\" method=\"POST\"><input type=\"text\" name=\"ss\"/><input type=\"submit\" value=\"submit\"/></form></body></html>")
(defroutes app-routes
(GET "/" [] form-test)
(POST "/" req (str "the req: " req))
(route/resources "/")
(route/not-found "Not Found"))
(def app
(handler/site app-routes))
whenever I try my app on my local machine it works fine, I can see the request parameters, but when I deploy the same thing to heroku the request parameters are always empty... what's going on?
OK so I resolved my issue, the problem was the way my program was being executed in Heroku.
My Procfile previously:
web: lein run -m myapp.core
all I did is change it to:
web: java $JVM_OPTS -jar myapp.jar
basically I had to execute my program as a compiled jar.