Play 2.2 application crashes on Heroku - heroku

After moving from Play 2.0.4 to Play 2.2.0 I get this error when deploying on Heroku:
Oct 15 13:23:12 heroku/web.1: Starting process with command `target/universal/stage/bin/demagog -Dhttp.port=${PORT} ${JAVA_OPTS} -Dconfig.resource=${DEMAGOG_ENVIRONMENT}.conf`
Oct 15 13:23:13 app/web.1: Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Djava.rmi.server.useCodebaseOnly=true
Oct 15 13:23:13 app/web.1: Bad application path: -Xmx384m
Oct 15 13:23:15 heroku/web.1: State changed from starting to crashed
Oct 15 13:23:15 heroku/web.1: Process exited with status 0
Oct 15 13:24:37 heroku/web.1: Starting process with command `target/universal/stage/bin/demagog -Dhttp.port=${PORT} -Dconfig.resource=${DEMAGOG_ENVIRONMENT}.conf`
Oct 15 13:24:37 app/web.1: Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Djava.rmi.server.useCodebaseOnly=true
Oct 15 13:24:37 app/web.1: Play server process ID is 2
Oct 15 13:24:37 app/web.1: Oops, cannot start the server.
Oct 15 13:24:37 app/web.1: java.lang.IllegalStateException: System property demagog.defaultUser must be set.
I don't understand this message
Bad application path: -Xmx384m
The second problem I can see is that my Play application can't found system property 'demagog.defaultUser', but this property is set in JAVA_OPTS environment variable. So it should work. Maybe it's just a consequence of the above problem? Any hints?
UPDATED
I have removed ${JAVA_OPTS} from the Procfile as #jan suggested. The first error
Bad application path: -Xmx384m
is not here anymore, but the system property 'demagog.defaultUser' is still not set.
Oct 16 10:50:35 heroku/web.1: Starting process with command `target/universal/stage/bin/demagog -Dhttp.port=${PORT} -Dconfig.resource=${DEMAGOG_ENVIRONMENT}.conf`
Oct 16 10:50:35 app/web.1: Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Djava.rmi.server.useCodebaseOnly=true
Oct 16 10:50:35 app/web.1: Play server process ID is 2
Oct 16 10:50:35 app/web.1: Oops, cannot start the server.
Oct 16 10:50:35 app/web.1: java.lang.IllegalStateException: System property demagog.defaultUser must be set.
...
Oct 16 10:50:35 app/web.1: at play.api.Play$.start(Play.scala:87)
Oct 16 10:50:35 app/web.1: at play.core.StaticApplication.<init>(ApplicationProvider.scala:52)
Oct 16 10:50:35 app/web.1: at play.core.server.NettyServer$.createServer(NettyServer.scala:243)
Oct 16 10:50:35 app/web.1: at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:279)
Oct 16 10:50:35 app/web.1: at play.core.server.NettyServer$$anonfun$main$3.apply(NettyServer.scala:274)
Oct 16 10:50:35 app/web.1: at scala.Option.map(Option.scala:145)
Oct 16 10:50:35 app/web.1: at play.core.server.NettyServer$.main(NettyServer.scala:274)
Oct 16 10:50:35 app/web.1: at play.core.server.NettyServer.main(NettyServer.scala)
Oct 16 10:50:35 heroku/web.1: Process exited with status 255
when I run heroku command
heroku config
I can see the system property is included in the JAVA_OPTS environment variable
JAVA_OPTS: -Xmx384m -Xss512k -XX:+UseCompressedOops -Ddemagog.defaultUser=xxx ...

You probably haven't removed ${JAVA_OPTS} from your Procfile. With Play 2.2 the JAVA_OPTS are included in the generated start script so you don't have to include them in the Procfile anymore.
What happens then is that the start script tries to interpret your JAVA_OPTS as app parameters.

Ok, I finally found it. The problem with setting my system property using environment variable JAVA_OPTS is that:
Environment variables are case sensitive in Unix while case insensitive in Windows.
with combination that the script generated by the sbt-native-packager reads java_opts environment variable. So you have to set java_opts (lower case) environment variable within Heroku.

answer to the update:
i do not know exactly what the problem is now. i would suggest to set the default user via the application.config. that'll be more play like anyways.
in your application.conf it could for example look
demagog.defaultUser="SOME_STD_DEFAULT_USER"
demagog.defaultUser=${?DEMAGOG_DEFAULTUSER}
then you can set the system specific variable via something like
heroku config:add DEMAGOG_DEFAULTUSER="yourdefaultuser"

Related

Start a spring boot application via a systemd service

I try to run a spring boot application like a service under almalinux
Content of /etc/systemd/system/tf.service
[Unit]
Description=tf application
After=syslog.target
[Service]
User=almalinux
ExecStart=/home/almalinux/app/tf.jar
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target
tf.service: Main process exited, code=exited, status=203/EXEC
Jun 04 18:52:15 vps-4210f039.vps.ovh.ca systemd[1]: tf.service: Failed with result 'exit-code'.
Info on the file
50052 -rwxr--r--. 1 almalinux almalinux 51249838 Jun 4 17:55 tf.jar
Info on the service
4 -rwxr-xr-x. 1 root root 204 Jun 4 19:05 /etc/systemd/system/tf.service
sudo systemctl start testamentfacile
Jun 04 19:11:30 vps-4210f039.vps.ovh.ca systemd[1]: Started tf application.
Jun 04 19:11:30 vps-4210f039.vps.ovh.ca systemd[1]: tf.service: Main process exited, code=exited, status=203/EXEC
Jun 04 19:11:30 vps-4210f039.vps.ovh.ca systemd[1]: tf.service: Failed with result 'exit-code'.
So application don't start correctly
If I start application manually with java -jar, I don't have any issue
./tf.jar work fine
I had the same issue, which was solved calling java by its path, you should probably add a java call before the jar:
ExecStart=/usr/bin/java -jar /home/almalinux/app/tf.jar

Macbook pro Manjaro bluetooth not working

I'm not sure if this is the write place to write my question, but here it is.
Before a week i updated my macbook pro early 2015 with manjaro gnom.
All good, but the bluetooth is not working.
I tried with bluetooth-manager but it's not opening and inside the journalctl -f it's showing
Oct 15 20:08:59 PPeev systemd[1095]: Started Application launched by gnome-shell.
Oct 15 20:09:00 PPeev blueman-manager.desktop[5014]: blueman-manager 20.09.00 ERROR Manager:118 on_dbus_name_appeared: Default adapter not found, trying first available.
Oct 15 20:09:00 PPeev blueman-manager.desktop[5014]: blueman-manager 20.09.00 ERROR Manager:122 on_dbus_name_appeared: No adapter(s) found, exiting
Oct 15 20:09:00 PPeev blueman-manager.desktop[5014]: blueman-manager version 2.1.3 starting
Oct 15 20:09:00 PPeev systemd[1095]: gnome-launched-blueman-manager.desktop-5014.scope: Succeeded.
Oct 15 20:09:04 PPeev systemd[1095]: gnome-terminal-server.service: Succeeded.
Also when i check the status systemctl status bluetooth
● bluetooth.service - Bluetooth service
Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2020-10-15 19:52:23 EEST; 19min ago
Docs: man:bluetoothd(8)
Main PID: 1083 (bluetoothd)
Status: "Running"
Tasks: 1 (limit: 9413)
Memory: 2.3M
CGroup: /system.slice/bluetooth.service
└─1083 /usr/lib/bluetooth/bluetoothd
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEScanIntervalConnect” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEScanWindowConnect” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEMinConnectionInterval” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEMaxConnectionInterval” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEConnectionLatency” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEConnectionSupervisionTimeout” in group “Controller”
Oct 15 19:52:23 PPeev bluetoothd[1083]: src/main.c:parse_controller_config() Key file does not have key “LEAutoconnecttimeout” in group “Controller”
Oct 15 19:52:23 PPeev systemd[1]: Started Bluetooth service.
Oct 15 19:52:23 PPeev bluetoothd[1083]: Starting SDP server
Oct 15 19:52:23 PPeev bluetoothd[1083]: Bluetooth management interface 1.17 initialized
Is running.
try disabling and re-enabling the bluetooth from the bios menu.
I checked some threads, and the only working option i got is this.
You may try this. I guess it will work for you too.

Failed to start Elasticsearch. Error opening log file '/gc.log': Permission denied

Dear StackOverflow community,
I was running Kibana/Elasticsearch without a problem until installing a Kibana plugin. Then the service failed and I noticed that the problem is that Elasticsearch stopped. I tried several ways to fix it, and then even reinstalled all. But the problem still avoiding to launch Elasticsearch, even with a fresh installation.
Installation on Debian 9 using apt install.
systemctl start elasticsearch.service
results on:
Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
[0.000s][error][logging] Error opening log file '/gc.log': Permission denied
Full log with journalctl -xe
-- Unit elasticsearch.service has begun starting up.
Feb 07 14:09:06 Debian-911-stretch-64-minimal kibana[576]: {"type":"log","#timestamp":"2020-02-07T13:09:06Z","tags":["warning","elasticsearch","admin"],"pid":576,"message":"Unable to revive connection: http://localhost:9200/"}
Feb 07 14:09:06 Debian-911-stretch-64-minimal kibana[576]: {"type":"log","#timestamp":"2020-02-07T13:09:06Z","tags":["warning","elasticsearch","admin"],"pid":576,"message":"No living connections"}
Feb 07 14:09:06 Debian-911-stretch-64-minimal kibana[576]: {"type":"log","#timestamp":"2020-02-07T13:09:06Z","tags":["warning","elasticsearch","admin"],"pid":576,"message":"Unable to revive connection: http://localhost:9200/"}
Feb 07 14:09:06 Debian-911-stretch-64-minimal kibana[576]: {"type":"log","#timestamp":"2020-02-07T13:09:06Z","tags":["warning","elasticsearch","admin"],"pid":576,"message":"No living connections"}
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: output:
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: [0.000s][error][logging] Error opening log file '/gc.log': Permission denied
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: [0.000s][error][logging] Initialization of output 'file=/var/log/elasticsearch/gc.log' using options 'filecount=32,filesize=64m' failed.
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: error:
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: Error: Could not create the Java Virtual Machine.
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: Error: A fatal exception has occurred. Program will exit.
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:118)
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:86)
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:59)
Feb 07 14:09:06 Debian-911-stretch-64-minimal elasticsearch[2312]: at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:92)
Feb 07 14:09:06 Debian-911-stretch-64-minimal systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Feb 07 14:09:06 Debian-911-stretch-64-minimal systemd[1]: Failed to start Elasticsearch.
-- Subject: Unit elasticsearch.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit elasticsearch.service has failed.
The mentioned gc.log file was not in that folder. And the permissions were:
drwxr-s--- 2 elasticsearch elasticsearch 4096 Jan 15 13:20 elasticsearch
I created the file and also played with permissions until having these:
-rwxrwxrwx 1 root elasticsearch 0 Feb 7 15:19 gc.log
...and even changed the ownership:
-rwxrwxrwx 1 root root 0 Feb 7 15:19 gc.log
But no success, I still having the same issue.
Thanks
Make sure you are running CMD as Administrator.
This error also happens if you are using docker & running the container as a different user. You have to add --group_add flag to docker command or set TAKE_FILE_OWNERSHIP environment variable as mentioned here
Using docker-compose:
user: 1007:1007
group_add:
- 0
Using docker:
--group-add 0
Firstly, I didn't know why gc.log file was not present. Have you changed the logs folder path or something? The gc.log path can be set in jvm.options file. By default ES logs and java garbage collection logs are fed into the logs folder inside $ES_HOME directory.
About user perspective, elastic search can't be run as root user. So from the ES directory details its showing you have an elasticsearch user created, and trying to run the cluster by that user.
The problem here can be solved by changing the permissions of files insdie the ES directory where all it belongs. Now the gc.log file is owned by root user and it cannot be accessed by the elasticsearch user.
Try this: sudo chown <user> <path/to/es/directory> -R
Here it becomes : sudo chown elasticsearch elasticsearch/ -R
If the issue still persists, check the jvm.options file whether its all configured correctly. Unless you change the -Xloggc:logs/gc.log option, the gc.log won't be pushing to /var/log.
Feb 09 17:09:02 server elasticsearch[2199]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Your log says, the option is given as file=/var/log/elasticsearch/gc.log. Correct any wrong configurations as per documentation : https://www.elastic.co/guide/en/elasticsearch/reference/master/jvm-options.html
sudo systemctl -l status elasticsearch.service
Returns this log:
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
└─override.conf
Active: failed (Result: exit-code) since Sun 2020-02-09 17:09:02 CET; 2min 48s ago
Docs: http://www.elastic.co
Process: 2199 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 2199 (code=exited, status=1/FAILURE)
Feb 09 17:09:02 server elasticsearch[2199]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Feb 09 17:09:02 server elasticsearch[2199]: Error: Could not create the Java Virtual Machine.
Feb 09 17:09:02 server elasticsearch[2199]: Error: A fatal exception has occurred. Program will exit.
Feb 09 17:09:02 server elasticsearch[2199]: at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:118)
Feb 09 17:09:02 server elasticsearch[2199]: at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:86)
Feb 09 17:09:02 server elasticsearch[2199]: at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:59)
Feb 09 17:09:02 server elasticsearch[2199]: at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:92)
Feb 09 17:09:02 server systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Feb 09 17:09:02 server systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Feb 09 17:09:02 server systemd[1]: Failed to start Elasticsearch.
At this point I'm doing a fresh install. Not able to find the solution I need to continue working...

Unable to access Jenkins from browser on Windows

I installed Jenkins on CentOS 7 running in VMware Workstation. Jenkins Service is running:
jenkins.service - Jenkins Service
Loaded: loaded (/etc/systemd/system/jenkins.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2016-06-21 08:47:46 PDT; 5s ago
Main PID: 68969 (java)
Memory: 82.3M
CGroup: /system.slice/jenkins.service
└─68969 java -jar /usr/local/bin/jenkins.war
Jun 21 08:47:48 server.centos.lan java[68969]: Jun 21, 2016 8:47:48 AM org.eclipse.jetty.util.log.JavaUtilLog info
Jun 21 08:47:48 server.centos.lan java[68969]: INFO: Started ServerConnector#34ddf859{HTTP/1.1}{0.0.0.0:8080}
Jun 21 08:47:48 server.centos.lan java[68969]: Jun 21, 2016 8:47:48 AM org.eclipse.jetty.util.log.JavaUtilLog info
Jun 21 08:47:48 server.centos.lan java[68969]: INFO: Started #2472ms
Jun 21 08:47:48 server.centos.lan java[68969]: Jun 21, 2016 8:47:48 AM winstone.Logger logInternal
Jun 21 08:47:48 server.centos.lan java[68969]: INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled
Jun 21 08:47:49 server.centos.lan java[68969]: Jun 21, 2016 8:47:49 AM jenkins.InitReactorRunner$1 onAttained
Jun 21 08:47:49 server.centos.lan java[68969]: INFO: Started initialization
Jun 21 08:47:49 server.centos.lan java[68969]: Jun 21, 2016 8:47:49 AM jenkins.InitReactorRunner$1 onAttained
Jun 21 08:47:49 server.centos.lan java[68969]: INFO: Listed all plugins
IP of VM is 192.168.139.160. Host operating system is Windows 7.
When I try to access Jenkins from browser on Windows using http://192.168.139.160:8080/jenkins I get error:
"This site can’t be reached". What am I doing wrong?
If you have installed Jenkins on CentOS 7.1, you should add a rule in firewall for port 8080, or at least disable firewalld.
To disable firewalld, you can use these commands:
systemctl stop firewalld
systemctl disable firewalld
After this you should be able to reach Jenkins on port 8080.
Stop Firewall + start tomcat
Find where you have the iptables, then put them down like this:
/etc/init.d/iptables stop
(might be some different location on your computer)
Then start up the tomcat you'll have a script like this:
startup.sh (in my computer the path is /usr/share/apache-tomcat-7.0.56/bin/)
Please try it out.
First
JENKINS_HTTPS_LISTEN_ADDRESS="Public IP" >> /etc/sysconfig/jenkins
Then
/etc/init.d/jenkins restart

Avoid New Relic attaching to leiningen on Heroku

I've enabled New Relic monitoring for my Clojure app running on Heroku. To avoid the overhead of nesting my app inside Leiningen's JVM process, I start up with lein trampoline run.
This apparently adds some overhead from New Relic attaching to the initial Leiningen process, which then shuts down and launches my app, causing delay for New Relic to attach once again. This can sometimes result in not starting up within the 30-second boot timeout window and results in downtime.
Log output showing both New Relic agents starting up:
heroku/web.1: Starting process with command `lein trampoline run`
app/web.1: [date] NewRelic 1 INFO: Agent is using Log4j
app/web.1: [date] NewRelic 1 INFO: Loading configuration file "/app/newrelic/./newrelic.yml"
app/web.1: [date] NewRelic 1 INFO: Agent Host: 866e2426-7a0f-4293-ae89-b55c0332253e IP: 10.159.0.212
app/web.1: [date] NewRelic 1 INFO: Setting audit_mode to false
app/web.1: [date] NewRelic 1 INFO: Setting protocol to "http"
app/web.1: [date] NewRelic 1 INFO: Configuration file is /app/newrelic/./newrelic.yml
app/web.1: [date] NewRelic 1 INFO: New Relic Agent v2.9.0 has started
app/web.1: [date] NewRelic 1 INFO: Java version: 1.6.0_20
app/web.1: [date] NewRelic 1 INFO: Agent class loader: sun.misc.Launcher$AppClassLoader#7ea2dfe
app/web.1: [date] NewRelic 5 INFO: JVM is shutting down
app/web.1: [date] NewRelic 5 INFO: New Relic Agent has shutdown
app/web.1: [date] NewRelic 1 INFO: Agent is using Log4j
app/web.1: [date] NewRelic 1 INFO: Loading configuration file "/app/newrelic/./newrelic.yml"
app/web.1: [date] NewRelic 1 INFO: Agent Host: 866e2426-7a0f-4293-ae89-b55c0332253e IP: 10.159.0.212
app/web.1: [date] NewRelic 1 INFO: Configured to connect to New Relic at collector.newrelic.com:80
app/web.1: [date] NewRelic 1 INFO: Setting audit_mode to false
app/web.1: [date] NewRelic 1 INFO: Setting protocol to "http"
app/web.1: [date] NewRelic 1 INFO: Configuration file is /app/newrelic/./newrelic.yml
app/web.1: [date] NewRelic 1 INFO: New Relic Agent v2.9.0 has started
app/web.1: [date] NewRelic 1 INFO: Java version: 1.6.0_20
app/web.1: [date] NewRelic 1 INFO: Agent class loader: sun.misc.Launcher$AppClassLoader#7ea2dfe
Is there a way to avoid having New Relic attach to the leiningen process?
Instead of having -javaagent:newrelic/newrelic.jar set in your Heroku config's JVM_OPTS, could you not set it in your production profile's :jvm-opts in your project.clj?

Resources