Is it possible to create a filter of some sort in elasticsearch, such that the search request will be honored only if the request is from trusted IP (certain servers only).
I referred to this post however, would like to check if the latest versions has this feature, i couldn't find anything in the elastic document.
Note: i have a more then 1 elastic server in the cluster.
Sure, the network module is what you are looking for.
There are two kind of networks in elasticsearch clusters:
Transport: Communication between the nodes of the cluster
Http: Communication with all kind of clients
If you want to restrict searches to serveral IPs please have a look on xpack.security.http.filter.* settings.
A brief documentation of this feature is available here.
Related
We are migrating from Elastic Search VM to Elastic cluster on Kubernetes, is there is any possible ways to separate some percentage of the incoming traffic to both ECK cluster and Elastic search on VM’s
Yes you can do it however it depends on the what you are using to Load balancer the traffic and handling the requests.
If you are using the Nginx you can use the split traffic and handle the traffic percentages accordingly.
http://nginx.org/en/docs/http/ngx_http_split_clients_module.html
If you are using the Istio also you can manage it with the
https://istio.io/latest/docs/concepts/traffic-management/
So it would be mostly depends on what you are accepting the traffic and proxy setup.
I am a newbie in elastic-search.
I am a web developer with very less networking experience.
I have read the following documents -
https://netty.io/
https://stackoverflow.com/questions/23839437/what-are-the-netty-alternatives-for-high-performance-networking
I wasn't able to understand the purpose of netty for elastic search. Could anyone explain it to me in layman terms?
Elasticsearch offers two interfaces to talk to it. One is the HTTP interface and the other one is the transport interface.
The first usually runs on port 9200 and can be accessed via any HTTP capable tool e.g. curl or your favorite browser. The transport interface is used by cluster members to exchange data and state and runs on port 9300 using a custom protocol.
Both interfaces use netty as "socket / NIO" library.
Let's say I have a cluster of database replicas that I would like to make available under a frontend. These databases replicate with each other. Can I have Traefik serve the same backend to the same client IP if possible, such that the UI can be made consistent even when the DBs are still replicating the newest state?
What you seem to be asking for is sticky sessions (aka session affinity) on a per-IP address basis.
Traefik supports cookie-based stickiness, which means that a cookie will be assigned on the initial request if the relevant Traefik option is enabled. Subsequent requests will then reach the same backend unless it fails to be reachable, at which point a new sticky backend will be selected.
The option can be enabled like this:
[backends]
[backends.backend1]
[backends.backend1.loadbalancer]
sticky = true
Documentation can be found here (search for "sticky sessions").
If you are running one of the dynamic providers with Traefik (e.g., Docker, Kubernetes, Marathon), there are usually labels/tags/annotations available you can set per-frontend. The TOML configuration file documentation contains all the details.
If you are looking for true IP address-based stickiness where the IP address space gets hashed and traffic evenly distributed across all backends: This isn't possible yet, although there's an open feature request.
In elastic search i created multi-node setup.I use Java Api transport Client to communicate to Elasticsearch server.
Now i created transport client with only one IP[assume:192.129.129.12.9300]. If i request any query in single ip it communicates all nodes and returns results. What happen if it my node[192.129.129.12.9300] that i mentioned in transport Client fails. Can i communicate with other nodes. What is the optimum way configuration to use transport Client for multi node set up.
You need to activate sniff option.
See http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#transport-client
I am trying to set up a Squid Proxy combined with DansGuardian Content filtering engine on EC2. I will be filtering traffic from mobile(IOS/Android) clients via this filtered proxy but that could mean a lot of traffic flowing through my system, since I will have to route all of the traffic through the DNS, which inturn could mean a lot Amazon EC2 costs!. Is there a known method/standard in which I can direct only known blacklisted traffic via this proxy in a cost effective manner?. Things I have explored include creating blacklists on the device and filtering right there , but that might mean I have to keep going back and changing (adding or removing sites) and this is not really feasible anyway.