I am a newbie in elastic-search.
I am a web developer with very less networking experience.
I have read the following documents -
https://netty.io/
https://stackoverflow.com/questions/23839437/what-are-the-netty-alternatives-for-high-performance-networking
I wasn't able to understand the purpose of netty for elastic search. Could anyone explain it to me in layman terms?
Elasticsearch offers two interfaces to talk to it. One is the HTTP interface and the other one is the transport interface.
The first usually runs on port 9200 and can be accessed via any HTTP capable tool e.g. curl or your favorite browser. The transport interface is used by cluster members to exchange data and state and runs on port 9300 using a custom protocol.
Both interfaces use netty as "socket / NIO" library.
Related
I was reading the the official RFC for the Web Sockets protocol in order to implement it for learning purposes, I wanted to make things a bit different somehow but I was not sure what to make different. While I was reading the document, I came across this:
The Web Socket Protocol attempts to address the goals of existing bidirectional HTTP technologies in the context of the existing HTTP infrastructure; as such, it is designed to work over HTTP ports 80 and 443 as well as to support HTTP proxies and intermediaries, even if this implies some complexity specific to then current environment. However, the design does not limit Web Socket to HTTP, and future implementations could use a simpler handshake over a dedicated port without reinventing the entire protocol.
Does this imply a custom web socket protocol can be implemented with a non http based handshake?.
If so, does it mean a the regular Java Script Web Socket client will not work with this and I would need to implement a custom client to communicate using this protocol?
Is it possible to create a filter of some sort in elasticsearch, such that the search request will be honored only if the request is from trusted IP (certain servers only).
I referred to this post however, would like to check if the latest versions has this feature, i couldn't find anything in the elastic document.
Note: i have a more then 1 elastic server in the cluster.
Sure, the network module is what you are looking for.
There are two kind of networks in elasticsearch clusters:
Transport: Communication between the nodes of the cluster
Http: Communication with all kind of clients
If you want to restrict searches to serveral IPs please have a look on xpack.security.http.filter.* settings.
A brief documentation of this feature is available here.
I've had a search over the internet but can't seem to find any straightforward instructions on how to use the Thrift protocol from behind a proxy.
To give you a bit of background - we have a Zipkin instance setup (https://github.com/twitter/zipkin) that uses a Cassandra instance (http://cassandra.apache.org/) to store Zipkin traces. Our intention is to negotiate over the thrift protocol to a collector that is then responsible for writing traces to Cassandra.
What conditions have to be in place for us to negotiate successfully via our corporate proxy? Do we just have to set certain proxy properties when trying to negotiate or do we have to set something else up that allows this negotiation to happen?
Any help people can give in this direction with regards to resources and/or an answer would be greatly appreciated.
The Apache Thrift TSocketTransport (almost certainly what you are using) uses TCP on a configurable port. Cassandra usually uses port 9160 for thrift. When using Thrift/TCP no HTTP setup is necessary. Just open 9160 (and any other ports your custom thrift servers may be listening on).
Though you can use Thrift over HTTP, Thrift is RPC, not REST, so proxy caching will cause problems, the client needs a direct comm channel with the server.
If you do need to access a thrift service via a proxy, something like this would work:
https://github.com/totally/thrift_goodies/blob/master/transport.py
You can kill the kerberos stuff if you don't need that.
we are working on few REST interfaces with Fabric8, however taking a different approach in exposing those. E.g. not using CXF, rather doing our own stuff based on netty.
I've been studying the API registry and particularly the examples which are coming with Fabric8 and all of them are of course based on CXF and it is easy to do it that way. However is there a way to register our own RESt API developed on top of camel-netty component?
Same would apply for any other API based on TCP connections only. For example SMPP, where there is persistent session based TCP connection.
Another question would be, is there an easy way to use Zookeepers registry for Connection registry where persistent TCP connections ( or session based ones like SMPP ) can be registered and known to the whole cluster - e.g. their state etc...?
Thanks, Tiho
This question got answered here on the mailing list
In elastic search i created multi-node setup.I use Java Api transport Client to communicate to Elasticsearch server.
Now i created transport client with only one IP[assume:192.129.129.12.9300]. If i request any query in single ip it communicates all nodes and returns results. What happen if it my node[192.129.129.12.9300] that i mentioned in transport Client fails. Can i communicate with other nodes. What is the optimum way configuration to use transport Client for multi node set up.
You need to activate sniff option.
See http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#transport-client