From the high level standpoint
In the case if I would like to optimize totaljs application.
Is there a possibility I can use a proxy server to load balance requests across multiple instances of totaljs on different ports each on its own thread or even a separate machine on lan without deep dive into the totaljs core?
I've tried a dummy method such as. It appears to be working, but maybe there's a better way.
const ports = [8080, 8081, 8082, 8084]
cp = require('child_process')
ports.forEach((port) => {
cp.fork('./debug.js', [port])
})
//debug.js
options = {};
options.port = process.argv[2];
Is there any more details how most efficiently I can execute multiple totaljs instances to serve the single point of entrance in order to improve or at least have an option to decrease the overload of the single thread.
You can use Total.js Cluster:
https://docs.totaljs.com/latest/en.html#api~FrameworkCluster
Implementation is very easy:
require('total.js');
const options = {};
options.ip = '0.0.0.0';
// Run the framework in the cluster
F.cluster.http(4, 'release', options);
IMPORTANT: Total.js products aren't optimized for the cluster.
Related
Under load in production we receive "RabbitMQ.Client.Exceptions.ConnectFailureException" connection failed and "MassTransit.RequestTimeoutException" timeout waiting for response. The consumer does receive the message and send it back. It's like the web app isn't listening, or unable to accept the connection.
We're running an ASP.NET web application ( not MVC ) on .NET Framework 4.6.2 on Windows Server 2019 on IIS. We're using MassTransit 7.0.4. In production, under load, we can get some exceptions dealing with sockets on RabbitMQ or timeouts from masstransit. It's difficult to reproduce them in Dev. RabbitMQ is in a mirror, it seems to happen once we turn on a high-load service that bumps from 140 message/sec to 250 message/sec.
I have a few questions about the code architecture, and then if anyone else is running into these kinds of timeout issues.
Questions:
Should I have static scope for the IBusControl? IE, should it be static inside Global asax? And does it matter at all if it's a singleton underneath?
Should I create a new IBusControl and start it per request ( maybe stick it in Application BeginRequest ). Would that make a difference?
Would adding another worker process affect the total number of open connections I'm able to make -- If this is a resource issue ( exhausting threads, connections or some resource ).
Exceptions:
MassTransit.RequestTimeoutException
Timeout Waiting for response
Stacktrace:
System.Runtime.ExceptionServices.ExceptionDispathInfo.Throw
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
MassTransit.Clients.ResponseHandlerConnectionHandle`1+<GetTask>d_11.MoveNext
System.Threading.ExecutionContext.RunInternal
RabbitMQ.Client.Exceptions.ConnectFailureException
Connection failed
Statcktrace:
RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail
RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingAddressFamily
RabbitMQ.Client.Impl.SocketFrameHandler..ctor
RabbitMQ.Client.ConnectionFactory.CreateFrameHandler
RabbitMQ.Client.EndPointResolverExtensions.SelectOne
RabbitMQ.Client.ConnectionFactory.CreateConnection
How Our Code Works ( overview )
Static IBusControl that is instantiated the first time someone tries to produce a message. The whole connection and send code is a little large to put in here ( connection factory and other metric classes, but below are the interesting parts ).
Static IBusControl B;
B = Bus.Factory.CreateUsingRabbitMq(x =>
{
hostAddress = host.HostAddress;
x.Host(new Uri(host.HostAddress), h =>
{
h.Username(host.UserName);
h.Password(host.Password);
});
x.Durable = false;
x.SetQueueArgument("x-message-ttl", 600000);
});
B.Start(new TimeSpan(0, 0, 10));
// Then send the Actual Messages
// Generic with TRequest and TResponse : class BaseMessage
// Pulling the code out of a few different classes
string serviceAddressString = string.Format("{0}/{1}?durable={2}", HostAddress, ChkMassTransit.QueueName(typeof(TRequest), typeof(TResponse)), false ? "true" : "false");
Uri serviceAddress = new Uri(serviceAddressString);
RequestTimeout rt = RequestTimeout.After(0, 0, 0, 0, timeout.Value);
IRequestClient<TRequest> reqClient = B.CreateRequestClient<TRequest>(serviceAddress, rt);
var v = reqClient.GetResponse<TResponse>(request, sendInfo.CT, sendInfo.RT);
if ( v.Wait(timeoutMS) ) { /*do some stuff*/ }
First, I find your lack of async disturbing. Using Wait or anything like it on TPL-based code is a recipe for death and destruction, pain and suffering, dogs and cats living together, etc.
Yes, you should have a single bus instance that is started when the application starts. Since you're doing request/response, set AutoStart = true on the bus configurator to make sure it's all warmed up and ready.
Never, no, one bus only!
Each bus instance only has a single connection, so you shouldn't see any resource issues related to capacity on RabbitMQ.
MassTransit 7.0.4 is really old, you might consider the easy upgrade 7.3.1 and see if that improves things for you. It's the last version of the v7 codebase available.
I have to implement a chat application using websocket, users will chat via groups, there can be thousands of groups and a user can be in multiple groups. I'm thinking about 2 solutions:
[1] for each group chat, I create a websocket endpoint (using camel-atmosphere-websocket), users in the same group can subscribe to the group endpoint and send/receive message over that endpoint. it means there can be thousands of websocket endpoints. Client side (let's say iPhone) has to subscribes to multiple wbesocket endpoints. is this a good practice?
[2] I just create one websocket endpoint for all groups. Client side just subscribes to this endpoint and I manage the messages distribution myself on server: get group members, pick the websocket of each member from list of connected websockets then write the message to each member via websocket.
Which solution is better in term of performance and easy to implement on both client and server?
Thanks.
EDIT 2015-10-06
I chose the second approach and did a test with jetty websocket client, I use camel atmosphere websocket on server side. On client side, I create websocket connections to server in threads. There was a problem with jetty that I can just create around 160 websocket connections (it means around 160 threads). The result is that I almost see no difference when the number of clients increases from 1 to 160.
Yes, 160 is not a big number, but I think I will do more test when I actually see the performance problem, for now, I'm ok with second approach.
If you are interested in the test code, here it is:
http://www.eclipse.org/jetty/documentation/current/jetty-websocket-client-api.html#d0e22545
I think second approach will be better to use for performance. I am using the same for my application, but it is still in testing phase so can't comment about the real time performance. Now its running for 10-15 groups and working fine. In my app, there is similar condition like you in which user can chat based on group. I am handling the the group creation on server side using node.js. Here is the code to create group, but it is for my app specific condition. Just pasting here for the reference. Getting homeState and userId from front-end. Creating group based on the homeState. This code is only for example, it won't work for you. To improve performance you can use clustering.
this.ConnectionObject = function(homeState, userId, ws) {
this.homeState = homeState;
this.userId = userId;
this.wsConnection = ws;
},
this.createConnectionEntry = function(homeState, userId,
ws) {
var connObject = new ws.thisRefer.ConnectionObject(homeState, userId,
ws);
var connectionEntryList = null;
if (ws.thisRefer.connectionMap[homeState] != undefined) {
connectionEntryList = ws.thisRefer.connectionMap[homeState];
} else {
connectionEntryList = new Array();
}
connectionEntryList.push(connObject);
console.log(connectionEntryList.length);
ws.thisRefer.connectionMap[homeState] = connectionEntryList;
ws.thisRefer.connecteduserIdMap[userId] = "";
}
Browsers implement a restriction on the numbers of websocket that can be opened by the same tab. You can't rely on being able to create as many connection as possible. Go for solution #2
I have a requirement to find response time for API (Rest API calls to external instances) calls going out of an amazon EC2 instance (there is an application running on EC2 making these calls). It will be great if I could also filter the calls based on a regex or complete urls. We have been thinking of logging the calls and analyzing the data or using tools like Dynatrace, Nagios so that code changes are not required.
If someone has implemented such a solution, please let us know.
This may not be a complete answer, but following the idea of starting with logs, I would recommend looking into using Cloudwatch: https://aws.amazon.com/cloudwatch/details/#log-monitoring
Depending on the level of access you have, you use something like https://www.wireshark.org/ to monitor all the traffic and do url/protocol filtering. You might try a log aggregator like http://papertrailapp.com/ which has filtering.
As you stated you could also use some form of APM like dynatrace, statsd, instrumental[1], newrelic, datadog, etc. to do monitoring inside your application.
[1] I work for instrumental.
You can setup a simple proxy that would measure web requests time and write metrics for this measures.
There are some some nice tools, take a look at http://datadoghq.com/product or http://devmetrics.io/logslib
For example, simple nodejs proxy with devmetrics lib:
var httpProxy = require('http-proxy');
var logger = require('devmetrics-core');
var http = require('http');
var proxy = new httpProxy.createProxyServer({});
var proxyServer = http.createServer(function (req, res) {
// now
var start_time = new Date().getTime();
proxy.web(req, res, { target: 'http://localhost:80' });
res.on('finish', function() {
var latency = new Date().getTime() - start_time;
console.log("The request was proxied in " + latency + "ms");
logger.appGauge('web_request', latency);
});
});
proxyServer.listen(3000);
Google Chrome 38 introduced the new "Device Mode & Mobile Emulation" functionality in devtools. In addition to choosing a device for emulation, it is also possible to emulate different network conditions:
Optimizing your site's performance under varying network conditions is
a key aspect of developing for a mobile audience.
Device mode's network conditioning allows you to test your site on a
variety of network connections, including Edge, 3G, and even offline.
Select a connection from the preset dropdown to apply network
throttling and latency manipulation.
For example, we can set it to be like in good old days - GPRS 50 Kbps:
Now we have a good use case for it - we have an internal application for network speed testing. And this new emulation functionality is very helpful for manual testing. But, we'd like to automate it.
Question is:
Is it possible to start chrome via selenium with specified network conditions? Is it something that can be controlled through chrome preferences or command-line arguments?
There are certainly multiple options to simulate slow internet connection, but the question is specifically about chrome+selenium.
The API to control network emulation were added to ChromeDriver. And should be available for quite a while now. According to comment in the linked issue you should use version at least 2.26 because of some bugfix.
According to Selenium changelog bindings are available for these languages:
JavaScript as of version 3.4.0 (commit)
Python as of version 3.5.0 (commit)
Ruby as of version 3.11.0 (commit)
C# as of version 4 (commit)
If you need these binding in other languages you should probably open issue/contribute implementation similar to one of the above.
Example usage from Python is below:
driver.set_network_conditions(
offline=False,
latency=5, # additional latency (ms)
download_throughput=500 * 1024, # maximal throughput
upload_throughput=500 * 1024) # maximal throughput
No, it is not possible to control Network Connectivity Emulation through Chrome preferences or command-line arguments. Network Connectivity Emulation is part of the build-in Chrome debugger. One way way in solving this is to control the debugger. This can be done via an extension or by directly controlling the debugger, see explanation. However, this will not work with WebDriver. The reason for this is that there can only be one "debug" session and WebDriver is already using it, see explanation. Since there is no public interface, there is also no way to control it via WebDriver.
For Device Mode & Mobile Emulation which is also part of the build-in debugger, there is a public interface (details), and thus can be controlled. This can be done through WebDriver Capabilities. Two options 1) Specify a device name 2) Enter your own parameters (limited).
You can use this method to run your test case in specified network conditions
protected void networkThrotting() throws IOException {
Map map = new HashMap();
map.put("offline", false);
map.put("latency", 5);
map.put("download_throughput", 500);
map.put("upload_throughput", 1024);
CommandExecutor executor = ((ChromeDriver)driver).getCommandExecutor();
Response response = executor.execute(
new Command(((ChromeDriver)driver).getSessionId(), "setNetworkConditions", ImmutableMap.of("network_conditions", ImmutableMap.copyOf(map)))
);
}
Indeed C# Selenium latest (3.11) has NetworkConditions added. Now you can use it like this:
var driver = new ChromeDriver(pathToDriver);
driver.NetworkConditions = new ChromeNetworkConditions()
{ DownloadThroughput = 5000, UploadThroughput = 5000, Latency = TimeSpan.FromMilliseconds(5) };
The problem is it's not yet usable because of the bug
https://github.com/SeleniumHQ/selenium/issues/5693
So .Net guys will have to wait until 3.12 Selenium Release.
While this is a very welcome and useful bit of functionality, for serious testing I think the conventional methods of network simulation are still the way to go.
I am aware of 2 solutions in addition to those already linked - the Charles web proxy (very useful tool - commercial) and implementing your own recipe using Linux Traffic Control (e.g. see chapter 6 of LAMPe2e).
By interfering with the network connections rather than the browser, you then get a proper measure of the impact independently of the browser in use.
Why do you just want to use the Chrome functionality?
Let's consider two different approaches,
one where we can throttle the entire network and one where we can specify which network requests to throttle specifically.
Approach 1: throttle the entire network
const { Builder } = require("selenium-webdriver")
async function throttleNetwork() {
let driver = await new Builder().forBrowser("chrome").build();
await driver.setNetworkConditions({
offline: false,
latency: 5000, // Additional latency (ms).
download_throughput: 50 * 1024, // Maximal aggregated download throughput.
upload_throughput: 50 * 1024, // Maximal aggregated upload throughput.
});
driver.get("http://www.google.com/");
}
thanks to Yaroslav for pointing out the commit.
This has a downside where we can't specify a specific network request to throttle and the rest to go unthrottled.
Let's fix this downside in our next approach.
Approach 2: throttle a specific network request
Here we'd be using an npm package from requestly called Requestly for Selenium.
We need to create a rule first in their client application and get the link by creating a shared list.
For example, let's throttle network request to google.com
require("chromedriver");
const { Builder } = require("selenium-webdriver");
const chrome = require("selenium-webdriver/chrome");
const {
getRequestlyExtension,
importRequestlySharedList,
} = require("#requestly/selenium");
const sharedListUrl = "YOUR_SHARED_LIST_LINK_HERE" // For example, use "https://app.requestly.io/rules/#sharedList/1631611216670-delay"
async function throttleGoogle() {
const options = new chrome.Options().addExtensions(
getRequestlyExtension("chrome") // This installs requestly chrome extension in your testing instance
);
const driver = new Builder()
.forBrowser("chrome")
.setChromeOptions(options)
.build();
await importRequestlySharedList(driver, sharedListUrl); // Here we import the shared list we created some time back
driver.get("http://www.google.com/");
}
This was a high-level overview of how we can overcome the downsides of the selenium-only approach. I've written a blog on the same where I go into depth on how to create a rule, shared list, and so on. You can read it here.
The below issue has now been fixed in this commit
For anyone like me in the C# world wondering why the upload/download throughput does not work as expected, it seems the tooltips for these properties are mislabelled. The tooltip states the data rate is measured in kb/s but in my own experience it is actually bytes per second so if you want to use a more familiar measurement like Mbps you will have to multiply by 125,000:
int latencyInMilliseconds = 20;
long downloadLimitMbps = 20;
long uploadLimitMbps = 5;
_driver.NetworkConditions = new ChromeNetworkConditions()
{
Latency = new TimeSpan(0, 0, 0, 0, latencyInMilliseconds),
DownloadThroughput = downloadLimitMbps * 125000, // Mbps to bytes per second
UploadThroughput = uploadLimitMbps * 125000, // Mbps to bytes per second
IsOffline = false,
};
Using these settings and looking at network traffic while my tests are running I can see they result in exactly 20Mbps down and 5Mbps up.
It looks like it's coming soon to Selenium (C#). The commit was on 01/28/2018:
https://github.com/SeleniumHQ/selenium/blob/ef156067a583fe84b66ec338d969aeff6504595d/dotnet/src/webdriver/Chrome/ChromeNetworkConditions.cs
I know this is an old question, but I recently had to solve for this problem and this page came up at the top of my Google search. Here are the main bits from how I did it in C#. Hope this helps someone in the future.
var networkConditions = new ChromeNetworkConditions();
networkConditions.Latency = new TimeSpan(150);
networkConditions.IsOffline = false;
networkConditions.DownloadThroughput = 120 * 1024;
networkConditions.UploadThroughput = 150 * 1024;
Driver.NetworkConditions = networkConditions;
Inspired by the answer from TridentTrue, here is an updated version for Selenium 4.0.0 in C#. If anyone knows how to use it for alpha7 and upwards without beeing version specific feel free to update this. :)
public void LimitNetwork(int latencyInMilliseconds, long downloadLimitMbps, long uploadLimitMbps)
{
IDevTools devTools = driver as IDevTools;
session = devTools.CreateDevToolsSession();
session.Network.Enable(new EnableCommandSettings());
EmulateNetworkConditionsCommandSettings command = new EmulateNetworkConditionsCommandSettings();
command.Latency = latencyInMilliseconds;
command.DownloadThroughput = downloadLimitMbps * 125000; // Mbps to bytes per second
command.UploadThroughput = uploadLimitMbps * 125000; // Mbps to bytes per second
command.Offline = false;
session.Network.EmulateNetworkConditions(command);
}
Update: After I had implemented this for my own, I found a really good article to get an overview in Selenium 4.0, also in Emulating network conditions.
Update 2: My issue was that I forgot to add the Network.Enable command, so don't forget to call it before you do the other stuff.
I have updated the code. :)
I'm not a node.js master, so I'd like to have more points of view about this.
I'm creating an HTTP node.js web server that must handle not only lots of concurrent connections but also long running jobs. By default node.js runs on one process, and if there's a piece of code that takes a long time to execute any subsequent connection must wait until the code ends what it's doing on the previous connection.
For example:
var http = require('http');
http.createServer(function (req, res) {
doSomething(); // This takes a long time to execute
// Return a response
}).listen(1337, "127.0.0.1");
So I was thinking to run all the long running jobs in separate threads using the node-webworker library:
var http = require('http');
var sys = require('sys');
var Worker = require('webworker');
http.createServer(function (req, res) {
var w = new Worker('doSomething.js'); // This takes a long time to execute
// Return a response
}).listen(1337, "127.0.0.1");
And to make the whole thing more performant, I thought to also use cluster to create a new node process for each CPU core.
In this way I expect to balance the client connections through different processes with cluster (let's say 4 node processes if I run it on a quad-core), and then execute the long running job on separate threads with node-webworker.
Is there something wrong with this configuration?
I see that this post is a few months old, but I wanted to provide a comment to this in the event that someone comes along.
"By default node.js runs on one process, and if there's a piece of code that takes a long time to execute any subsequent connection must wait until the code ends what it's doing on the previous connection."
^-- This is not entirely true. If doSomething(); is required to complete before you send back the response, then yes, but if it isn't, you can make use of the Asynchronous functionality available to you in the core of Node.js, and return immediately, while this item processes in the background.
A quick example of what I'm explaining can be seen by adding the following code in your server:
setTimeout(function(){
console.log("Done with 5 second item");
}, 5000);
If you hit the server a few times, you will get an immediate response on the client side, and eventually see the console fill with the messages seconds after the response was sent.
Why don't you just copy and paste your code into a file and run it over JXcore like
$ jx mt-keep:4 mysourcefile.js
and see how it performs. If you need a real multithreading without leaving the safety of single threading try JX. its 100% node.JS 0.12+ compatible. You can spawn the threads and run a whole node.js app inside each of them separately.
You might want to check out Q-Oper8 instead as it should provide a more flexible architecture for this kind of thing. Full info at:
https://github.com/robtweed/Q-Oper8