when I go to my appharbor site I get a 502 Bad Gateway
in the logs I see
appharbor nginx xx.xx.xxx.xxx - "POST / HTTP/1.1" 502 150 "-" "Logplex/unknown" "source=nginx measure#http.response_time=0.000s measure#http.bytes.sent=300"
it was a problem on appharbors side
Related
I tried deploying a spring boot application as a war on an Azure App Service (Windows, java 8 , tomcat 8.5) .
It's a spring webflux application working with Server Sent Events so I need to have http2 support.
However even when http2 is enabled in the App Service I noticed that SSE is not working.
Looking in the logs I find that it is still using HTTP/1.1 under the covers.
From kudu D:\home\LogFiles\http\RawLogs
127.0.0.1 - - [12/Nov/2019:20:19:43 +0000] "GET /api/rooms/someroom/users/Gms_2290/info/subscribe HTTP/1.1" 200 5 144
127.0.0.1 - - [12/Nov/2019:20:19:44 +0000] "GET /api/rooms/someroom/users/Gms_2290/music/subscribe HTTP/1.1" 200 5 16
127.0.0.1 - - [12/Nov/2019:20:19:46 +0000] "GET /api/rooms/someroom/users/Gms_2290/heartbeat HTTP/1.1" 200 5 142
127.0.0.1 - - [12/Nov/2019:20:19:46 +0000] "GET /api/rooms/someroom/users/Gms_2290/info/subscribe HTTP/1.1" 200 5 19
127.0.0.1 - - [12/Nov/2019:20:19:48 +0000] "GET /api/rooms/someroom/users/Gms_2290/music/subscribe HTTP/1.1" 200 5 32
127.0.0.1 - - [12/Nov/2019:20:19:49 +0000] "GET /api/rooms/someroom/users/Gms_2290/heartbeat HTTP/1.1" 200 5 16
What I've tried
I tried playing with the java versions (switching between 8 and 9)
I tried playing with the Tomcat versions (8.5 and 9)
Linux angle
I first tried deploying the application to a Linux App Service (with java 8 and Tomcat 8.5)
That resulted in following errors from the log Stream
java.lang.IllegalStateException: Async support must be enabled on a servlet and for all filters involved in async request processing
I suspect Tomcat might need some extra configuration in order to use the App Service SSL and thereby allowing http2, but I can't find a good way to configure the Tomcat used.
Any help is appreciated!
I verified that a Spring WebFlux application that uses SSE works fine in App Service Linux when run as a jar file (using the Java SE 8 offering with HTTP 1.1). So the issue you are experiencing seems to be related to Tomcat configuration.
For investigation, can you share the repro that works fine on Tomcat locally but fails when run on Tomcat in App Service?
I have some strange lags issue with a sinatra thin app when the trafic get a little high (30-40 users).
It's a small game using long-polling so http IOs can be high compared to the number of users.
CPU load stay low and there's a lot of free memory.
Here are some typical logs lines when lags happen :
1 - [17/Jul/2015:16:50:17 -0400] "POST /play?next=word HTTP/1.1" 200 1 0.0018
2 - [17/Jul/2015:16:50:17 -0400] "GET /update?_=1437166100579 HTTP/1.1" 200 304 15.0046
3 - [17/Jul/2015:16:50:17 -0400] "GET /update?_=1437166102348 HTTP/1.1" 200 286 15.0045
4 - [17/Jul/2015:16:50:17 -0400] "POST /accept_replay? HTTP/1.1" 200 - 0.0021
5 - [17/Jul/2015:16:50:18 -0400] "GET /core HTTP/1.1" 200 3719 0.0015
6 - [17/Jul/2015:16:50:18 -0400] "GET /join HTTP/1.1" 302 - 0.0640
7 - [17/Jul/2015:16:50:18 -0400] "GET /core HTTP/1.1" 200 3719 0.0024
8 - [17/Jul/2015:16:50:19 -0400] "POST /play?next=word HTTP/1.1" 200 1 0.0034
9 - [17/Jul/2015:16:50:19 -0400] "GET /update?_=1437166215907 HTTP/1.1" 200 248 10.0018
10- [17/Jul/2015:16:50:19 -0400] "GET /update?_=1437166222579 HTTP/1.1" 200 252 11.0029
11- [17/Jul/2015:16:50:31 -0400] "GET /core HTTP/1.1" 200 3719 0.0034
12- [17/Jul/2015:16:50:31 -0400] "POST /sentiment/bad? HTTP/1.1" 200 - 0.0024
13- [17/Jul/2015:16:50:31 -0400] "GET / HTTP/1.1" 200 4449 0.0086
14- [17/Jul/2015:16:50:31 -0400] "POST /decline_replay HTTP/1.1" 302 - 0.0020
And 30 more exactly at [17/Jul/2015:16:50:31 -0400]
( get /update are longpolling requests so it can take up to 40seconds)
Everythings stops for 12seconds between 10 and 11. And all the requests received during this time seems to be processed simultaneously.
I start the app that way
thin start -p 80
Can it be a thin issue ?
Do i need a custom thin config file ?
Do i need nginx?
Any indication is welcome...
edit :
Errors I find in ObectSpace [SystemStackError, 1][NoMemoryError, 1][IOError, 1]
The kind of behavior smells a lot like request queuing, which means there aren't enough web processes free to handle incoming requests. So the requests sit waiting, and then then the backlog is cleared they suddenly all get processed super fast and all at once.
This guy wrote up a good post on how to use Thin, EventMachine, and Async Sinatra to handle long-polling requests.
I have an annoying problem in Apache Karaf 3.0.2. I want to add a feature repository. It is located in my private Nexus Snapshot Repository and protected by username and password. Everything is correctly stated in my settings.xml
~/.m2/repository/settings.xml:
<servers>
<server>
<id>test</id>
<username>karaf</username>
<password>karafpass</password>
</server>
</servers>
org.ops4j.pax.url.mvn.cfg:
org.ops4j.pax.url.mvn.repositories= \
http://repo1.maven.org/maven2#id=central, \
http://repository.springsource.com/maven/bundles/release#id=spring.ebr.release, \
http://repository.springsource.com/maven/bundles/external#id=spring.ebr.external, \
http://zodiac.springsource.com/maven/bundles/release#id=gemini, \
http://repository.apache.org/content/groups/snapshots-group#id=apache#snapshots#noreleases, \
https://oss.sonatype.org/content/repositories/snapshots#id=sonatype.snapshots.deploy#snapshots#norelease, \
https://oss.sonatype.org/content/repositories/ops4j-snapshots#id=ops4j.sonatype.snapshots.deploy#snapshots#noreleases, \
http://192.168.178.25:8081/nexus/content/repositories/snapshots#id=test#snapshots#noreleases
The command to add the feature repository:
feature:repo-add mvn:com.test/com.test.feature/1.0.0-SNAPSHOT/xml/features
In the first try I get:
Error executing command: Error resolving artifact com.test:com.test.feature:xml:features:1.0.0-SNAPSHOT:
Could not find artifact com.test:com.test.feature:xml:features:1.0.0-SNAPSHOT in apache (http://repository.apache.org/content/groups/snapshots-group/)
Why does it fail? The error statement is right, but why is the artifact not found in my Snapshot-Repository? Here is what I can see in my Nexus request.log:
192.168.178.25 - - [25/Dez/2014:15:44:40 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/maven-metadata.xml HTTP/1.1" 401 0
192.168.178.25 - - [25/Dez/2014:15:44:43 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/com.test.feature-1.0.0-SNAPSHOT-features.xml HTTP/1.1" 401 0
192.168.178.25 - - [25/Dez/2014:15:44:43 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/com.test.feature-1.0.0-SNAPSHOT-features.xml HTTP/1.1" 404 1289
The problem is the 401 in the first line which hinders karaf from getting maven-metadata.xml to form the subsequent commands right. Why? All credentials are part of my M2 settings.xml, just use them!
The next try with the same command succeeds and the request.log says:
192.168.178.25 - - [25/Dez/2014:15:59:02 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/maven-metadata.xml HTTP/1.1" 200 1029
192.168.178.25 - - [25/Dez/2014:15:59:02 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/maven-metadata.xml.sha1 HTTP/1.1" 200 40
192.168.178.25 - - [25/Dez/2014:15:59:03 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/com.test.feature-1.0.0-20141224.100146-1-features.xml HTTP/1.1" 200 1086
192.168.178.25 - - [25/Dez/2014:15:59:04 +0100] "GET /nexus/content/repositories/snapshots/com/test/com.test.feature/1.0.0-SNAPSHOT/com.test.feature-1.0.0-20141224.100146-1-features.xml.sha1 HTTP/1.1" 200 40
Please, I require your help? How can I get rid of that problem?
I have an interesting issue occurring frequently in my Spring web app. Basically, a user logs into the site successfully, navigates around to protected pages for a while and then for some reason, the server starts to return 403 responses. The user calls into tech support and tech support asks them to refresh their browser and try again. Magically, everything works fine the second time around after the refresh. No one on my team is able to reproduce the issue in either our QA environment nor on Production. Does anyone have any ideas what may cause this?
UPDATE - these are logs from the apache access log file
These calls are what prompt the call to tech support (notice the 403s)
"GET /server-webapp/api/getCartContents? HTTP/1.1" 200 1273
"GET /js/config.js HTTP/1.1" 200 809
"GET /server-webapp/api/getCartContents? HTTP/1.1" 200 1273
"GET /server-webapp/api/getUserInfo? HTTP/1.1" 200 201
"GET /server-webapp/api/getPendingSalesOrder? HTTP/1.1" 200 183
"POST /server-webapp/api/getShoppingCartErrors HTTP/1.1" 200 40
"GET /server-webapp/generated/CountriesAndStates.json? HTTP/1.1" 200 3319
"GET /server-webapp/api/getAddresses? HTTP/1.1" 403 390
"POST /server-webapp/api/createPendingSalesOrder HTTP/1.1" 403 390
"GET /server-webapp/api/getAddresses?" 403 390
"POST /server-webapp/api/createPendingSalesOrder HTTP/1.1" 403 390
"GET /server-webapp/api/getAddresses? HTTP/1.1" 403 390
"POST /server-webapp/api/createPendingSalesOrder HTTP/1.1" 403 390
The user is asked to refresh their browser and those same calls are now returning 200s...
"GET /server-webapp/api/getCartContents? HTTP/1.1" 200 1273
"GET /server-webapp/api/getCartContents? HTTP/1.1" 200 1273
"GET /server-webapp/api/getUserInfo? HTTP/1.1" 200 261
"POST /server-webapp/api/getShoppingCartErrors HTTP/1.1" 200 40
"GET /server-webapp/api/getPendingSalesOrder? HTTP/1.1" 200 183
"GET /server-webapp/generated/CountriesAndStates.json? HTTP/1.1" 200 3319
"GET /server-webapp/api/getAddresses? HTTP/1.1" 200 50
"POST /server-webapp/api/createPendingSalesOrder HTTP/1.1" 200 184
"POST /server-webapp/api/updatePendingSalesOrderLines HTTP/1.1" 200 42
"GET /server-webapp/api/getPendingSalesOrder? HTTP/1.1" 200 206
The issue ended up not being with Tomcat or Apache and was, in fact, a nasty little bug in the authentication logic. In short, the user was being logged in but with no permissions but only if they took a very specific route to becoming logged in. Thanks to those that have taken a look and if there is way for me to delete the ticket then let me know since it turned out to be coding logic and not anything to do with Tomcat or Apache.
Is there a way to make and external call once a worflow is completed in Oozie. Reading some documentation it seems like possible. (Oozie can make HTTP callback notifications on action start/end/failure events and workflow end/failure events.)
http://archive.cloudera.com/cdh/3/oozie/WorkflowFunctionalSpec.html
But not sure how to do it and can it be an external api call on callback.
Thanks
Further down the document, section 5
What do you mean by external API call? It can only do an HTTP get - so if you configure a servlet endpoint or something similar to trigger that external API call when the URL is visited, then you should able to do pretty much anything you want.
It is also worth noting however that the docs say:
Oozie will make a best effort to deliver the notifications, in case of failure it will retry the notification a pre-configured number of times at a pre-configured interval before giving up.
There are two properties you can add to your oozie job.properties. The variables $jobId , $status and $nodeName are replaced at runtime.
1. oozie.wf.workflow.notification.url=http://yourserver.com/some/path/$jobId/$status
This will give you a HTTP GET call at http://yourserver.com/some/path/ at job level for each status change for example
myhost.softlayer.com - - [19/Feb/2016 10:35:02] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/RUNNING HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:35] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/SUCCEEDED HTTP/1.1" 200 -
This one gives more detailed transition between your oozie job steps
2. oozie.wf.action.notification.url=http://yourserver.com/some/path/$jobId/$nodeName/$status
Sample output at server side :
myhost.softlayer.com - - [19/Feb/2016 10:35:02] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/:start:/T:ReadConfig HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:03] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/ReadConfig/S:RUNNING HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:09] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/ReadConfig/T:CheckStep HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:10] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/sqoop-table-import/S:RUNNING HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:10] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/CheckStep/T:sqoop-table-import HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:29] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/javaStatsReporting/S:RUNNING HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:29] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/sqoop-table-import/T:javaStatsReporting HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:35] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/end/T:null HTTP/1.1" 200 -
myhost.softlayer.com - - [19/Feb/2016 10:35:35] "GET /dbgen/lineitem2/0000050-151213000221460-oozie-oozi-W/javaStatsReporting/T:end HTTP/1.1" 200 -