How to run load test Scenario once every hour? - visual-studio

I have a loadTest with several scenarios, running for 12 hours.
I want to add another scenario, that will run once a hour, by 10 virtual users.
The ugly solution I'm using is to have 12 additional scenarios, each one has its own "delayed start", with 1 hour interval.
This is an ugly solution.
How can I tell a specific scenario to run once a hour.
Note: For this case I don't need it to run sharp each hour. The main idea is to have a task that run +/- every hour.

I suggest having a load test with two scenarios, one for the main user load, the other for the hourly 10-user case. Then arrange that the number of virtual users (VUs) for the 10-user is set to 10 at the start of every hour and reduced as appropriate. The question does not state how long the 10-user tests runs each hour.
The basic way of achieving this is to modify m_loadTest.Scenarios[N].CurrentLoad, for a suitable N, in a load test heartbeat plugin. The heartbeat is called, as it name suggests, frequently during the test. So arrange that it checks the run time of the test and at the start of each hour assign m_loadTest.Scenarios[N].CurrentLoad = 10 and a short time later set it back to 0 (i.e. zero). I believe that setting the value to a smaller value than its previous value allows the individual test executions by a VU to run to their natural end but the VUs will not start new tests that would exceed the value.
The plugin code then could look similar to the following (untested):
public class TenUserLoadtPlugin : ILoadTestPlugin
{
const int durationOf10UserTestInSeconds = ...; // Not specified in question.
const int scenarioNumber = ...; // Experiment to determine this.
public void Initialize(LoadTest loadTest)
{
m_loadTest = loadTest;
// Register to listen for the heartbeat event
loadTest.Heartbeat += new EventHandler<HeartbeatEventArgs>(loadTest_Heartbeat);
}
void loadTest_Heartbeat(object sender, HeartbeatEventArgs e)
{
int secondsWithinCurrentHour = e.ElapsedSeconds % (60*60);
int loadWanted = secondsWithinCurrentHour > durationOf10UserTestInSeconds ? 0 : 10;
m_loadTest.Scenarios[scenarioNumber].CurrentLoad = loadWanted;
}
LoadTest m_loadTest;
}
There are several web pages about variations on this topic. Searching for terms such as "Visual Studio custom load patterns". See this page for one example.

Related

How to synchronise AUTOSAR RTE write and read values with different task runnable?

I am new to AUTOSAR and I would like to call a RTE function to get a specific value from one SWC to another SWC. The RTE_Write is performing by one SWC 1 runnable with 10 msec task and RTE_read is performing by another SWC 2 with 15 msec task. I am using sender receiver interface to implement this concept.
SWC 1 :
Task_10msec()
{
eg:
int val = 0;
val = val +1 ;
Rte_write_test_port(val);
}
SWC 2 :
Task_15msec()
{
eg:
int val = 0;
Rte_read_test_port(&val);
}
Now here i am facing the problem is that RTE_read value is not sync with RTE_Write value because of the runnable timing (SWC 1 is 10 msec and SWC 2 is 15 msec). I would like to know, is there any way to design the interface/ any approach to get the exact read value in SWC 2 after writing from SWC 1?
You could try to add a QueuedReceiverComSpec on the receiver port and set the queueLength to e.g. 2. You should then use Rte_Receive instead of Rte_Read API and read until it returns RTE_E_NO_DATA in order to get all values provided by the other component.
What do you want to achieve?
Shall the receiver only get the latest value written by the sender? Then your solution is already sufficient.
Shall the receiver get all values that the sender wrote to the interface? Then you need to introduce a queue on the receiver port. In the receiver runnable, you can then read all elements in the queue until it is empty.
For more details, check the AUTOSAR RTE specification (chapter 4.3.1).

Cancel ajax calls?

I'm using the Select2 select boxes in my Django project. The ajax calls it makes can be fairly time-consuming if you've only entered a character or two in the query box, but go quicker if you've entered several characters. So what I'm seeing is you'll start typing a query, and it will make 4 or 5 ajax calls, but the final one returns and the results display. It looks fine on the screen, but meanwhile, the server is still churning away on the earlier queries. I've increased the "delay" parameter to 500 ms, but it's still a bit of a problem.
Is there a way to have the AJAX handler on the server detect that this is a new request from the same client as one that is currently processing, and tell the older one to exit immediately? It appears from reading other answers here that merely calling .abort() on the client side doesn't stop the query running on the server side.
If they are DB queries that are taking up time, then basically nothing will stop them besides stopping the database server, which is of course not tangible. If it is computation in nested loops for example, then you could use cache to detect whether another request has been submitted from the same user. Basically:
from django.core.cache import cache
def view(request):
start_time = timestamp # timezone.now() etc.
cache.set(request.session.session_key + 'some_identifier', start_time)
for q in werty:
# Very expensive computation with millions of loops
if start_time != cache.get(request.session.session_key + 'some_identifier'):
break
else:
# Continue the nasty computations
else:
cache.delete(request.session.session_key + 'some_identifier')
But the Django part aside - what I would do: in JS add a condition that when the search word is less than 3 chars, then it waits 0.5s (or less, whatever you like) before searching. And if another char is added then search right away.
I.e.
var timeout;
function srch(param) {
timeout = false;
if (param.length < 3) {
timeout = true;
setTimeout(function () {
if (timeout) {
$.ajax({blah: blah});
}
}, 500);
} else {
$.ajax({blah: blah});
}
}

ZeroMQ (clrzmq4) polling issue

What I'm trying to accomplish is to implement reading a message from one of two sockets, wherever it arrives first. As far as I understand polling (zmq_poll) is the right thing to do (as demonstrated in mspoller in guide). Here I'll provide small pseudo-code snippet:
TimeSpan timeout = TimeSpan.FromMilliseconds(50);
using (var receiver1 = new ZSocket(ZContext.Current, ZSocketType.DEALER))
using (var receiver2 = new ZSocket(ZContext.Current, ZSocketType.PAIR))
{
receiver1.Bind("tcp://someaddress");
// Note that PAIR socket is inproc:
receiver2.Connect("inproc://otheraddress");
var poll = ZPollItem.CreateReceiver();
ZError error;
ZMessage msg;
while (true)
{
if (receiver1.PollIn(poll, out msg, out error, timeout))
{
// ...
}
if (receiver2.PollIn(poll, out msg, out error, timeout))
{
// ...
}
}
}
As you can see it is actually the same exact implementation as in mspoller in guide.
In my case receiver2 (PAIR socket) should receive a large number of messages. In fact I've created a test in which number of messages sent to it is always greater than the number of messages it is capable to receive (at least in demonstrated implementation).
I've run the test for 2 seconds, and I was very surprised with results:
Number of messages sent to receiver2: 180 (by "sent" I mean that they are handed out to another PAIR socket not shown in the previous snippet);
Number of messages received by receiver2: 21 ??? Only 21 messages in 2 seconds??? 10 messages per second???
Then I've tried to play with different timeout values and I've found out that it significantly influences the number of messages received. Duration (2 seconds) and number of messages sent (180) remain the same. The results are:
timeout value of 200 milliseconds - number of messages received drops to 10 (5 per second);
timeout value of 10 milliseconds - number of messages received rises to 120 (60 per second).
The results are telling me that polling simply does not work. If polling were working properly, as far as I understand the mechanism, timeout should not have any influence in this scenario. No matter if we set timeout to 1 hour or 5 milliseconds - since there are always messages to receive there's nothing to wait for, so the loop should work with the same speed.
My another big concern is the fact that even with very small timeout value receiver2 is not capable to receive all 180 messages. I'm struggling here to accomplish receiving rate of 100 messages per second, although I've selected ZeroMQ which should be very fast (benchmarks are mentioning numbers as 6 million messages per second).
So my question is obvious: am I doing something wrong here? Is there a better way to implement polling?
By browsing clrzmq4 code I've noticed that there's also possibility to call pollIn method on enumeration of sockets ZPollItems.cs, line 151, but I haven't found any example anywhere!
Can this be the right approach? Any documentation anywhere?
Thanks
I've found the problem / solution for this. Instead using PollIn method on each socket separately we should use PollIn method on array of sockets. Obviously the example from the guide is HUGELY MISLEADING. Here's the correct approach:
TimeSpan timeout = TimeSpan.FromMilliseconds(50);
using (var receiver1 = new ZSocket(ZContext.Current, ZSocketType.DEALER))
using (var receiver2 = new ZSocket(ZContext.Current, ZSocketType.PAIR))
{
receiver1.Bind("tcp://someaddress");
receiver2.Connect("inproc://otheraddress");
// We should "remember" the order of sockets within the array
// because order of messages in the received array will correspond to it.
ZSocket[] sockets = { receiver1, receiver2 };
// Note that we should use two ZPollItem instances:
ZPollItem[] pollItems = { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZError error;
ZMessage[] msg;
while (true)
{
if (sockets.PollIn(pollItems, out msg, out error, timeout))
{
if (msg[0] != null)
{
// The first message gotten from receiver1
}
if (msg[1] != null)
{
// The second message gotten from receiver2
}
}
}
}
Now receiver2 reaches 15,000 received messages per second, no matter timeout value, and no matter number of messages received by receiver1.
UPDATE: Folks from clrzmq4 have acknowledged this issue, so probably the example will be corrected soon.

Unit Test Only Passes in Debug Mode, Fails in Run Mode

I have the following UnitTest:
[TestMethod]
public void NewGamesHaveDifferentSecretCodesTothePreviousGame()
{
var theGame = new BullsAndCows();
List<int> firstCode = new List<int>(theGame.SecretCode);
theGame.NewGame();
List<int> secondCode = new List<int>(theGame.SecretCode);
theGame.NewGame();
List<int> thirdCode = new List<int>(theGame.SecretCode);
CollectionAssert.AreNotEqual(firstCode, secondCode);
CollectionAssert.AreNotEqual(secondCode, thirdCode);
}
When I run it in Debug mode, my code passes the test, but when I run the test as normal (run mode) it does not pass. The exception thrown is:
CollectionAssert.AreNotEqual failed. (Both collection contain same elements).
Here is my code:
// constructor
public BullsAndCows()
{
Gueses = new List<Guess>();
SecretCode = generateRequiredSecretCode();
previousCodes = new Dictionary<int, List<int>>();
}
public void NewGame()
{
var theCode = generateRequiredSecretCode();
if (previousCodes.Count != 0)
{
if(!isPreviouslySeen(theCode))
{
SecretCode = theCode;
previousCodes.Add(previousCodes.Last().Key + 1, SecretCode);
}
}
else
{
SecretCode = theCode;
previousCodes.Add(0, theCode);
}
}
previousCodes is a property on the class, and its Data type is Dictionary key integer, value List of integers. SecretCode is also a property on the class, and its Data type is a List of integers
If I were to make a guess, I would say the reason is the NewGame() method is called again, whilst the first call hasn't really finished what it needs to do. As you can see, there are other methods being called from within the NewGame() method (e.g. generateRequiredSecretCode()).
When running in Debug mode, the slow pace of my pressing F10 gives sufficient time for processes to end.
But I am not really sure how to fix that, assuming I am right in my identification of the cause.
What happens to SecretCode when generateRequiredSecretCode generates a duplicate? It appears to be unhandled.
One possibility is that you are getting a duplicate, so SecretCode remain the same as its previous value. How does the generator work?
Also, you didn't show how the BullsAndCows constructor is initializing SecretCode? Is it calling NewGame?
I doubt the speed of keypresses has anything to do with it, since your test method calls the functions in turn without waiting for input. And unless generateReq... is spawning a thread, it will complete whatever it is doing before it returns.
--after update--
I see 2 bugs.
1) The very first SecretCode generated in the constructor is not added to the list of previousCodes. So the duplicate checking won't catch if the 2nd game has the same code.
2) after previousCodes is populated, you don't handle the case where you generate a duplicate. a duplicate is previouslySeen, so you don't add it to the previousCodes list, but you don't update SecretCode either, so it keeps the old value.
I'm not exactly sure why this is only showing up in release mode - but it could be a difference in the way debug mode handles the random number generator. See How to randomize in WPF. Release mode is faster, so it uses the same timestamp as seed, so it does in fact generate exactly the same sequence of digits.
If that's the case, you can fix it by making random a class property instead of creating a new one for each call to generator.

Timing of parallel actions using the Task Parallel Library in C#

I am running some experiments, timing them and comparing the times to find the best "algorithm". The question that came up was if running the tasks in parallel would make the relative runningtimes of the experiments wrong and if I would get more representative results by running them sequentially. Here is a (simplified) version of the code:
public static void RunExperient(IEnumerable<Action> experiments)
{
Parallel.ForEach(experiments, experiment =>
{
var sw = Stopwatch.StartNew(); //line 1
experiment(); //line 2
sw.Stop(); //line 3
Console.WriteLine(#"Time was {0}", sw.ElapsedMilliseconds);
});
}
My questions are about what is happening "behind the scenes":
When a task has started, is it possible that the OS or the framework can suspend the task during its execution and continue on later making the running time of the experiment all wrong?
Would I get more representative results by running the experiments sequentially?
That depends on the machine that you are running on and what the experiments do, but generally the answer is yes, they may affect one another. Mainly through resource starvation. Here's an example:
public class Piggy {
public void GreedyExperiment() {
Thread.Priority = ThreadPriority.Highest;
for (var i=0;i<1000000000;i++) {
var j = Math.Sqrt(i / 5);
}
}
}
That's going to do a tight loop on a high priority thread, which will basically consume one processor until it is done. If you only have one processor in the machine and TPL decides to schedule two experiments on it, the other one is going to be starved for CPU time.

Resources