js client could not connect to elasticsearch behind nginx - elasticsearch

I am using nginx to protect the elasticsearch. When trying to access elasticsearch in client js. Its throwing Unable to revive connection: http://ubuntulocal:80:9200/ error.
My question is how to connect the elasticsearch using client js behind the proxy ?
node js code
var elasticsearch = require('elasticsearch');
var host = [
{
host: 'http://ubunutlocal:80',
auth: 'root:root'
}]
var client = new elasticsearch.Client({
host: host
});
client.search({
index : 'bank'
// undocumented params are appended to the query string
//hello: "elasticsearch"
}, function (error , response) {
if (error) {
console.error('elasticsearch cluster is down!' , error);
} else {
console.log('All is well' , response);
}
});
Error log
Elasticsearch ERROR: 2015-10-26T13:14:06Z
Error: Request error, retrying -- getaddrinfo ENOTFOUND ubuntulocal:80
at Log.error (/node_modules/elasticsearch/src/lib/log.js:213:60)
at checkRespForFailure (/node_modules/elasticsearch/src/lib/transport.js:192:18)
at HttpConnector.<anonymous> (
/node_modules/elasticsearch/src/lib/connectors/http.js:153:7)
at ClientRequest.wrapper (
node_modules/elasticsearch/node_modules/lodash/index.js:3111:19)
at ClientRequest.emit (events.js:107:17)
at Socket.socketErrorListener (_http_client.js:271:9)
at Socket.emit (events.js:107:17)
at net.js:950:16
at process._tickCallback (node.js:355:11)
Elasticsearch WARNING: 2015-10-26T13:14:06Z
Unable to revive connection: http://ubuntulocal:80:9200/
Elasticsearch WARNING: 2015-10-26T13:14:06Z
No living connections
elasticsearch cluster is down! { [Error: No Living connections] message: 'No Living connections' }
Nginx configuration
server {
listen 80;
server_name guidanzlocal;
location / {
rewrite ^/(.*) /$1 break;
proxy_ignore_client_abort on;
proxy_pass http://localhost:9200;
proxy_redirect http://localhost:9200 http://guidanzlocal/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
auth_basic "Elasticsearch Authentication";
auth_basic_user_file /etc/elasticsearch/user.pwd;
}
}

Could you try to override nodesToHostCallback in your config:
var host = [{
host: 'http://ubunutlocal:9200',
auth: 'root:root'
}]
var client = new elasticsearch.Client({
host: host,
nodesToHostCallback: function ( /*nodes*/ ) {
return [host];
}
});
When elasticsearch-js attempt to reconnect after failure, it pings your node to get url, which can lead to inconsistent result when you're using a proxy

Related

get request body in auth_request nginx

I want to have a request body in auth_request to check something in there, but probably have some issues when i try to pass request body, as a subrequest, it can't get any information about the body of request
The api http://localhost:8091/api/authen/checkLoggedIn works alone very well and don't have any errors about request body, but when i use this api to do auth_request, it gets errors
when i get rid of #Nullable, even pass some infor like
{
"name" : "thisistest",
"gender" : "Male"
}
it can't get request body and throw exceptions
org.apache.catalina.connector.ClientAbortException: java.io.IOException: An existing connection was forcibly closed by the remote host
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:310) ~[tomcat-embed-core-9.0.65.jar:9.0.65]
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:273) ~[tomcat-embed-core-9.0.65.jar:9.0.65]
at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:118) ~[tomcat-embed-core-9.0.65.jar:9.0.65]
at java.base/java.io.FilterOutputStream.flush(FilterOutputStream.java:153) ~[na:na]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1187) ~[jackson-core-2.13.4.jar:2.13.4]
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1009) ~[jackson-databind-2.13.4.jar:2.13.4]
this is my auth_request
location = /auth {
internal;
proxy_method POST;
proxy_pass_request_body on;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
proxy_pass http://localhost:8091/api/authen/checkLoggedIn;
}
auth_request /auth;
#PostMapping("/checkLoggedIn")
public ResponseEntity checkLoggedIn(#Nullable #RequestBody Object object) {
// do something
return ResponseEntity.status(HttpStatus.OK).body("The request was successfully completed.");
}

Spring Boot actuator page returning http links instead of https

I have a Spring Boot 2.0.2 application. When I browse to the following URL: https://my-domain-name/my-application-name/actuator, I'm getting the following output:
{
"_links": {
"self": {
"href": "http://my-domain-name/my-application-name/actuator",
"templated": false
},
"health": {
"href": "http://my-domain-name/my-application-name/actuator/health",
"templated": false
},
"info": {
"href": "http://my-domain-name/my-application-name/actuator/info",
"templated": false
}
}
}
As you can see, the content is OK but all links start with 'http', and not with https. Nevertheless, I'm accessing the URL with HTTPS.
The domain name I'm trying to reach is an AWS Route 53 record, with an alias to an AWS ELB. This ELB redirects the call to a target which is a K8S cluster. The pod itself is running Nginx which redirects the URL to another pod which runs Spring Boot with an embedded Tomcat and it's serving its content using HTTP and port 8080.
For the Nginx, there's a proxy pass configuration:
location /my-application-name { proxy_pass http://my-application-name; }
The following headers are being added:
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
... so that Spring Boot will know the 'original' request.
Anybody an idea what I'm doing wrong? It seems like the actuator implementation is not taking into account the HTTPS protocol
I solved the issue by adding this code block in a configuration class:
#Bean
public FilterRegistrationBean forwardedHeaderFilterRegistration() {
ForwardedHeaderFilter filter = new ForwardedHeaderFilter();
FilterRegistrationBean<ForwardedHeaderFilter> registration = new FilterRegistrationBean<>(filter);
registration.setName("forwardedHeaderFilter");
registration.setOrder(10000);
return registration;
}
It seems like the ForwardedHeaderFilterdoes take the X-Forwarded-Proto header into account. Although it's still very unclear why the actuator implementation doesn't do this by default (because the other X-Forwarded headers are being treated correctly)

ExpressJS/Node: 404 images

I've set up a basic node/express server which serves public static javascript and css files fine, but returns a 404 error when attempting to serve images.
The strangest part is that everything works fine when run locally. When run on my remote server (linode), the image problem arrises.
It's really got me scratching my head... What might be the problem?
Here's the server:
/**
* Module dependencies.
*/
var express = require('express')
, routes = require('./routes')
var app = module.exports = express.createServer();
// Configuration
app.configure(function(){
app.set('views', __dirname + '/views');
app.set('view engine', 'jade');
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(express.compiler({ src: __dirname + '/public', enable: ['less'] }));
app.use(express.static(__dirname + '/public'));
app.use(app.router);
});
app.configure('development', function(){
app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});
app.configure('production', function(){
app.use(express.errorHandler());
});
// Globals
app.set('view options', {
sitename: 'Site Name',
myname: 'My Name'
});
// Routes
app.get('/', routes.index);
app.get('/*', routes.fourohfour);
app.listen(3000);
console.log("Express server listening on port %d in %s mode", app.address().port, app.settings.env);
if it works fine locally, maybe it's a case sensitivity issue, do your files have capitals etc?
I had this issue, but it ended up being that I had caps in the trailing .JPG extension and was calling .jpg in html. Windows is not case sensitive on file types, CentOS is...
Alrighty, I got around the issue by renaming my images folder from "/public/images" to /public/image. I don't know why the naming would cause an issue, but I'm glad that's all that was needed.
I had this exact issue. All user generated images uploaded to /static/uploads were not being rendered by express. The strange thing is everything in static/images, static/js, static/css were rending fine. I ensured it wasn't a permissions issue but was still getting a 404. Finally I configured NGINX to render all of my static file (which is probably faster anyway) and it worked!
I'd still love to know why Express wasn't rending my images though.
Here's my NGINX conf if anyone is having this issue:
server {
# listen for connections on all hostname/IP and at TCP port 80
listen *:80;
# name-based virtual hosting
server_name staging.mysite.com;
# error and access outout
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
location / {
# redefine and add some request header lines which will be transferred to the proxied server
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
#set the location of the web root
root /var/www/mysite.com;
# set the address of the node proxied server. port should be the port you set in express
proxy_pass http://127.0.0.1:9001;
# forbid all proxy_redirect directives at this level
proxy_redirect off;
}
# do a case insensitive regular expression match for any files ending in the list of extentions
location ~* ^.+\.(html|htm|png|jpeg|jpg|gif|pdf|ico|css|js|txt|rtf|flv|swf)$ {
# location of the web root for all static files
root /var/www/mysite.com/static;
# clear all access_log directives for the current level
#access_log off;
# set the Expires header to 31 December 2037 23:59:59 GMT, and the Cache-Control max-age to 10 years
#expires max;
}
}

nginx rewrite rules

I'm trying to implement nginx rewrite rules for the following situtation :
- Request /testfa/styles/style.css should be redirected to /testfa/templates/styles/style.css
I've enabled rewrite logging for my server, created following rules:
location /testfa {
rewrite ^/styles/(.+)?$ /testfa/templates/styles/$1 last;
}
but when I'm trying to perform request, I'm getting 404 error from server, and log file doesn't contain any info about rewriting, only following message:
open() "......../testfa/styles/style.css" failed (2: No such file or directory)
What is the correct way to perform such a rewrite for nginx ?
location /testfa/ {
rewrite ^/testfa/styles/(.+)$ /testfa/templates/styles/$1 last;
}
does this work for you ?
my tested virtual >
server {
listen ...ip...:80;
server_name sub.domain.com;
root /usr/local/www/test;
error_log /usr/local/www/test/error_debug.log debug;
rewrite_log on;
location /testfa/ {
rewrite ^/testfa/styles/(.+)$ /testfa/templates/styles/$1 last;
}
}
this works. even log has reported :
2011/11/25 01:06:52 [notice] 35208#0: *456705 rewritten data: "/testfa/templates/styles/test.css", args: "", client: IP, server: sub.domain.com, request: "GET /testfa/styles/test.css HTTP/1.1", host: "sub.domain.com"

Nginx proxy_pass to a password protected upstream

I want to pass a request to an upstream server. The original url is not password protected but the upstream server is. I need to inject a Basic auth username/password into the request but get errors when doing:
upstream supportbackend {
server username:password#support.yadayada.com;
}
and
upstream supportbackend {
server support.yadayada.com;
}
location /deleteuser {
proxy_pass http://username:password#supportbackend;
}
you need to add proxy_set_header Authorization "Basic ...."; where the .... is base64 of user:pass.

Resources