I need to generate some delays to make a load test more realistic. In Visual Studio I have found 2 methods that delay execution of a transaction in a loadtest:
Convert to code
or
Set "Think Time" to first request in the transaction
What would be the best practice here. I am not completely sure that Visual Studio executes these test in sequencional manner, so 2nd option might return false results.
On the other hand, I would loose the ability to change settings from Visual Studios UI, if I generate code from a recorded test. And that increases time spent editing the test quite a bit.
Is there a better way than any of my methods?
I believe that the second way is more safier.
Just have in mind that the Think Time is aplied to each Test Run not to each contained Test. So, if you have a Load Test which contains more than one Tests in the Test Mix the Think Time will apllied at the end of the test run (when all included tests were completed).
Also, if you have more than one virtual user, since it's a load test, Think Time is applied to each user seperately. If, for example, you have 100 user then they will concurently start the tests but they may not finish them at the same time. Each user will wait according to the Think Time before starting the second test without waiting for the others to complete their first test.
Related
We have a legacy code base with some unit tests still coupled by global state (static variables etc.). To find them, I need to know the exact execution order the tests ran when I ran them via VS test explorer.
Is there a way to log execution order in VS test explorer?
I know that vstest.console does output/log the execution order but then I need to narrow down the subset of tests which is very inconvenient with vstest. In VS test runner, i can just run subsets.
I also know that there are other tools (like resharper test runner) but this is also not an option.
Not out of the box that I am aware of, but I guess you can find ways to log it, like have a global counter that you increment in a before method.
But, since the order of the tests are non deterministic by design I am not sure how valuable that information is.
If you know what test are coupled then run those as Ordered Tests or use a different test traits to control how you execute those test. The best option is of course to break their external dependencies.
Have just started using Visual Studio to load test a web application. I have set up a number of load tests successfully but I am having an issue with one particular test. It happens to be the first time I am including a File Upload, so possibly that has something to do with it.
The LoadTest includes one WebTest that performs the File Upload. After completion the Test Details table shows most of the WebTest's taking ~100-120 seconds to complete. When drilling into the Test Log for each WebTest execution and adding up the Request Time components the result is the expected ~1-3 seconds.
Running the WebTest by itself also gets the expected ~1-3 seconds.
Think times are turned off.
I expect there is a parameter/config value that I have overlooked. It would be greatly appreciated if someone could shed any light on this.
Finally found it. In the properties for the load test scenario there is a "Think Profile" option. It was set to "Normal Distribution". For what I am trying to achieve it needed to set it to "Off".
All -- I am using Visual studio 2013 ultimate edition to run my load tests. I have more than one scenario configured on my load test. Each of the scenario run a different set of unit tests. Is is there a way to selectively run the scenarios ? In other words if there are more than one scenario configured, can I just run one of them and not run the rest ?
If the answer is 'NO', any ideas/suggestions would be very helpful.
Thanks in advance.
Two ways I have found:
One way is to set the user load to a constant zero in the unwanted scenarios.
Another way is to set the "Delay start time" of the unwanted scenarios to longer that the total test duration. This option only makes sense when "User test iterations" is false.
This question may be a bit nebulous, so please bear with me.
I am using Visual Studio, and I am new to the entire realm of unit testing. One thing I do a lot though is use Unit Testing as a quick and dirty ad-hoc "administration ui" at times when I need to just TRY things, but don't have time to make an actual admin system.
What I mean is ... sometimes I just want to get some data thrown into my database to see how it looks on a page. So I'll make a dirty unit test...
[Fact]
public void install_some_test_data(){
using(var database = RavenDocumentStore()){
using(var session = database.OpenSession()){
// create some objects
// add some objects
// save some objects
}
}
}
Nowhere in here have I really cared about "testing", I just like the fact that I can right click and say "Run it" and it'll go, without launching up the program, without having to have a lot of interaction, etc. etc.
So my question is this;
Is this okay to do? I realize this isn't how a program should be long term managed, but I was scolded for doing this by another developer simply because I wanted to quickly show them how something they wanted to see worked. Is there really a problem with using these convenient tools as simpler ways of running commands against my database that can give me immediate feedback on whether it passed or failed?
My followup question to that is this; I have had to actually search pretty hard to find a unit test runner that lets me run individual methods instead of running every single unit test in a class. Is this normal? This whole thing kind of confuses me in general. Even when I do use tests for actually testing, why would I always want to run EVERY test every time? Isn't that extremely wasteful of time and resources? How do you control the order that they run in? This seems even more imperative when tests have to depend on some data to exist before they can run appropriately.
At the moment, I am using xunit and then the unit test runner in ReSharper, but out of the box, Visual Studio doesn't seem to let me run unit tests individually, just as huge batches in a single class.
Am I the only person confused by this?
Sure, it's actually very easy to execute a single unit test (without an external test runner required).
Go to TEST -> Windows -> Test Explorer
Now you'll see a window like this
Now you can rightclick on the test you want to execute and select 'run selected methods'.
As to your other question: it's hard to distinguish what you are asking. Are you demonstrating a proof of concept with a unit test? How does this test replace your admin panel?
Just put your cursor on a Test function name. Then go to Test -> Run Selected Scope (or similar - the name changes by context). That test should execute and also create a test list for you automatically.
I'm using MSTest in Visual Studio 2010 and have the need to restore my database after all tests have run.
What I did was decorate a method with the AssemblyCleanupAttribute attribute.
<AssemblyCleanupAttribute()>
Shared Sub AssemblyCleanup()
' Restore my databases which takes a long time...
End Sub
Problem is the clean up takes a reasonable amount of time, so much so that the timeout is reached.
The only reason I started realizing that a timeout occurred is that in debug mode the Output window reports "...QTAgent32.exe, AgentObject: Cleanup: Timeout reached in cleaning up the agent.". Hence it fails very quietly and I would have loved if MSTest reported a Test Run Error.
What is the best way to detect and report the timeout? My ideal solution would be to report the timeout as a test run error.
In short, you cannot cause MSTest to report an error if AssemblyCleanup times out.
If you are encountering this issue, then at this point you need to consider if this limitation of MSTest is too great for you. There are other, and imho better, test frameworks out there.
If you decide to stick with MSTest and just want to ensure, at least, that the code/script in AssemblyCleanup runs to completion then you can choose to either run the clean up code as a Process. That way even if AssemblyCleanup internally calls a Thread.Abort then your Process runs to completion. It's messy though...
Why not wrap the contents of each test in a transaction, and rollback the transaction at the end of the test? See here for more information: http://msdn.microsoft.com/en-us/library/bb381703(v=vs.80).aspx#dtbunttsttedp_topic7