When using the .NET Elasticsearch NEST client, I'm trying to figure out how to minimize the number of pings the client library does to our nodes. I know there are settings to disable the pings, but if we had a node down I think we would see a big negative performance impact without them. So what I'm really trying to figure out is if there is a way to use a singleton pattern around the ElasticClient object, connection state information or some other object to help achieve this.
Basically we need a shared object that has all the nodes and their up/down state that multiple ElasticClients can use without having each new client created having to figure it out. Another option would be using the ElasticClient as a singleton itself.
I am using the client in a multithreaded ASP.NET app and azure worker role so ensuring it works across threads is important.
I'm using nginx in front of ES to monitor it's traffic and you can see there are a ton of "/" hits which must be the client library pings. (This report snippet below is via Stackify from parsing our nginx logs.)
Has anyone had any success using ElasticClient as a singleton or have any suggestions?
The client itself is stateless so you should be able to use it as a singleton. You can also instantiate a new client every time but if your using an IConnectionPool you need to make sure each of the client instances receives the same instance of the IConnectionPool.
Related
I have a web app running in Tomcat that needs to connect to ES using the High Level Java API. I am usure about the best practices for managing the ES resources (client, transport) in that context.
In the past, I would create a brand new client for every request and close it (as well as its transport) when I was done with the request.
But now I read that it's best to use a single client in my app and across all threads (the client is apparently thread-safe).
I can see two issues with that approach.
Issue 1: client timing out
If the single client hasn't been used in a while, it may have timed out. So before I use the client, I need a way to check if the client is still alive. But I can't find clear doc on how to do that (at least not without pinging the server everytime).
Issue 2: can't tell when Tomcat is done with the client
When I run my app as a comnmand line main() app, I can close the client's trasport at the end of that main. But in a Tomcat context, my code has no way of knowing when Tomcat is done with the client and its transport.
I tried all sorts of tricks using finalize() but none of them work consistently. And from what I read, it's unwise to rely on finalize() to close resources as the JVM offers no garantee as to when an object will be GCed (if ever!).
Thx for your guidance.
we're using Azure Cloud Functions with the Java SDK and connect to the Cosmos DB using the following Java API
CosmosClient client = new CosmosClientBuilder()
.endpoint("https://my-cosmos-project-xyz.documents.azure.com:443/")
.key(key)
.consistencyLevel(ConsistencyLevel.SESSION)
.buildClient();
This buildClient() starts a connection to CosmosDB, which takes 2 to 3 seconds.
The subsequent database queries using that client are fast.
Only this first setup of the connection is pretty slow.
We keep the CosmosClient as a static variable, so we can reuse it between multiple http requests that go to our function.
But once the function is getting cold (when Azure shuts it down after a few minutes unused), the static variable gets lost and will be reconnected, when the function is started up again.
Is there a way to make this initial connection to cosmos DB faster?
Or do you think we need to increase the time a function stays online, if we need faster response times?
This is a expected behavior, see https://youtu.be/McZIQhZpvew?t=850.
The first request a client does needs to go through a warm-up step. This warm-up consists of fetching the account information, container information, routing and partitioning information in order to know where to route the requests (as you experienced, further requests do not get this extra latency). Hence the importance of maintaining a singleton instance.
In some Functions plan (Consumption) instances get de-provisioned if there is no activity, in which case, any existing instance of the client is destroyed, so when a new instance is provisioned, your first request will pay this warm-up cost.
There are currently no workaround I'm aware of in the Java SDK but this should not affect your P99 latency since it's just the first request on a cold client.
Hope this and the video help with the reason.
I want to use RestHighLevelClient on different clusters with commands which are not supported by Cross Cluster mechanizem (for example close and open index).
My question is if I use more than one instance of RestHighLevelClient for every cluster it will keep connections open for every cluster? (to be ensure I didn't choke the application)
by looking at various resources, it seems RestHighLevelClient keeps the connection open unless you explicitly call client.close(); on it.
From the official RestHighLevelClient initialization
The high-level client will internally create the low-level client used
to perform requests based on the provided builder. That low-level
client maintains a pool of connections and starts some threads so you
should close the high-level client when you are well and truly done
with it and it will in turn close the internal low-level client to
free those resources. This can be done through the close method:
In your case, if you having a lot of ES clusters and creating multiple RestHighLevelClient than as you are guessing, it might choke your application due to the hold of threads and its resources so you should explicitly call the close which would require more time when you again create it but would not choke your application in most of the cases.
I would suggest you do some resource benchmarking on your application and based on your trade-off choose the best possible approach.
Create multiple clients and don't close them but allocate more resources so that the application is fast and don't choke.
close clients frequently, this would not require over-allocating resources but when you create a new client for your request, latency will be more.
I am currently redirecting the socket io through a custom proxy. The server it actually gets send to changes from time to time. The "new" server is notfied that a client will connect/swap to it before it connects/swaps. The only issue is that this leads to the client timing out and reconnecting, which works, but takes 2 seconds that I dont want to client to wait on switch. I do not want the client to know that the server change so somehow i have to make the server to add the upcomming client/socket id to its internal list.
How could I achieve this?
Ive looked at the socket.io-adapter, but I wasn't sure if that is only for rooms/or if there is an easier way to do it
It appears using the adapter would fix it. Rather than adding my own I just ended up accessing the default namespace and doing a 'addAll' to add the client.
io.nsps["/"].adapter.addAll(socketId, new Set<Room>([socketId]));
Everything with my co-located cache works fine as long as there is one request at a time. But when I hit my service with several concurrent requests, my cache doesn't seem to work.
Preliminary analysis led me to this - https://azure.microsoft.com/en-us/documentation/articles/cache-dotnet-how-to-use-service/
Apparently, I would have to use maxConnectionsToServer to allow multiple concurrent connections to cache. But the document also talks about a useLegacyProtocol parameter which has to be set to false to enable connection pooling.
I have the following questions:
My service would be getting a few hundred concurrent requests. Would this be a
good setting for such a scenario:
<dataCacheClient name="default" maxConnectionsToServer="100"
useLegacyProtocol="false">
This is my understanding of the behavior I would get with this configuration - Each time a request comes in, an attempt would be made to retrieve a connection from the pool. If there is no available connection, a new connection would be created if there are less than 100 connections currently, else the request would fail. Please confirm if this is correct.
The above documentation says that one connection would be used per instance of DataCacheFactory. I have a cache manager class which manages all interactions with cache. This is a singleton class. It creates a DataCacheFactory object and uses it to get a handle to the cache during its instantiation. My service would have 2 instances. Looks like I would need only 2 connections to server. Is this correct? Do I even need connection pooling?
What is the maximum value maxConnectionsToServer can accept and what would be an ideal value for the given scenario?
I also see a boolean paramater named "ConnectionPool". This looks complementary to "useLegacyProtocol". Is this not redundant? How is setting useLegacyProtocol="false" different from connectionPool="true"? I am confused as to whether or not and how to use this parameter.
Are maxConnectionsToServer and ConnectionPool parameters related in any way? What does it mean when I have maxConnectionsToServer set to 5 and ConnectionPool=true?