How does one configure VSTS specific load test context parameters for own azure agents intelligently - performance

Recently moved from utilising AWS to Azure for the location of our load test agents, thus making the transition to making full use of VSTS.
It was described that, for the moment, to get a load test file working with VSTS to using our own VMs for testing, we need to provide two context parameters, UseStaticLoadAgents and StaticAgentsGroupName in each loadtest file.
Our load test solution is getting very large, and we have multiple loadtest files where we have to set these two values each time. This leads us into the situation where, if we were to change our agents group name for example, we would have to update each individual load test file with the new information.
Im looking at a way to centralise this until a nicer way is implemented by Microsoft. The idea was to use a load test plugin, to add these context parameters with the plugin drawing the needed values from a centralised config file.
However, it seems that none of the hooks in the load test plugin or simply using the initialise method to manually set these values is working. Likely because they are set after full initialisation.
Has anyone got a nice, code focused solution to manage this and stop us depending on adding brittle values in the editor? Or even gotten the above approach to work?

The loadtest file is the XML file, so you can update it programmatically, for example:
string filePath = #"XXX\LoadTest1.loadtest";
XmlDocument doc = new XmlDocument();
doc.Load(filePath);
XmlNamespaceManager nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("ns", "http://microsoft.com/schemas/VisualStudio/TeamTest/2010");
XmlNode root = doc.DocumentElement;
XmlNode nodeParameters = root.SelectSingleNode("//ns:RunConfigurations/ns:RunConfiguration[#Name='Run Settings1']/ns:ContextParameters", nsmgr);
if(nodeParameters!=null)
{
//nodeParameters.SelectSingleNode("//ns:ContextParameter[#Name='UseStaticLoadAgents']").Value = "agent1";
foreach (XmlNode n in nodeParameters.ChildNodes)
{
switch (n.Attributes["Name"].Value)
{
case "Parameter1":
n.Attributes["Value"].Value = "testUpdate";
break;
case "UseStaticLoadAgents":
n.Attributes["Value"].Value = "agent1";
break;
case "StaticAgentsGroupName":
n.Attributes["Value"].Value = "group1";
break;
}
}
}
doc.Save(filePath);

Related

How to export a GMF diagram outside eclipse?

One of the features of Papyrus that I find really useful is the ability to programmatically interrogate the UML models that it creates by using the UML2 runtime outside the Eclipse UI. This is great for running simple tools to, e.g., produce documentation using POI or write model driven configuration for the Talend MDM tool. However, while traversing and processing the model tree is easily achieved by loading up the resources in a resource set, manipulating the diagrams in the .notation files has proven to be more of a challenge.
I have got to the point (by examining the source for org.eclipse.papyrus.infra.export.ExportAllDiagrams) where I can load all the resources and find the Diagram elements from the .notation file thus:
File uml = new File(model + ".uml");
File di = new File(model + ".di");
File notation = new File(model + ".notation");
URI umlUri = URI.createFileURI(uml.getAbsolutePath());
URI diUri = URI.createFileURI(di.getAbsolutePath());
URI notationUri = URI.createFileURI(notation.getAbsolutePath());
final ModelSet resourceSet = new ModelSet();
resourceSet.getPackageRegistry().put(UMLPackage.eNS_URI, UMLPackage.eINSTANCE);
resourceSet.getPackageRegistry().put(NotationPackage.eNS_URI, NotationPackage.eINSTANCE);
resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put(Resource.Factory.Registry.DEFAULT_EXTENSION, new XMIResourceFactoryImpl());
resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put("notation", new XMIResourceFactoryImpl());
resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put(UMLResource.FILE_EXTENSION, UMLResource.Factory.INSTANCE);
try {
resourceSet.getLoadOptions().put(XMLResource.OPTION_DEFER_IDREF_RESOLUTION, true);
resourceSet.getLoadOptions().put(XMLResource.OPTION_DEFER_ATTACHMENT, true);
resourceSet.getResource(diUri, true);
resourceSet.getResource(umlUri, true);
resourceSet.getResource(notationUri, true);
List<Diagram> diagrams = new ArrayList<Diagram>();
for (Iterator<Notifier> i = resourceSet.getAllContents(); i.hasNext();) {
Notifier n = i.next();
if (n instanceof Diagram) {
diagrams.add((Diagram) n);
}
}
//export(diagrams);
} finally {
// Unload the resource set so that we don't leak loads of UML content in the CacheAdapter
unload(resourceSet);
}
However, the ExportAllDiagrams class ultimately uses org.eclipse.gmf.runtime.diagram.ui.render.util.CopyToImageUtil to render the diagram, at which point it fails because it relies on the DiagramUIRenderPlugin and DiagramUIRenderPlugin.getInstance() returns null.
I then had a look at org.eclipse.gmf.runtime.diagram.ui.render.clipboard.DiagramSVGGenerator but had similar problems with the need for various eclipse plugins to be initialised.
I have no experience of the Eclipse plugin system but I am assuming that that the platform loads and initialises plugins and, therefore, the approaches tried so far need to be running within Eclipse GUI environment in order to work. Is there any other method that could be used to easily render the diagrams to SVG without relying on the whole of the Eclipse runtime?

Trying to generate JMeter Test Plan (jmx) With JMeter API : Not able to save CSVDataSet element

I am creating a JMeter jmx file dynamically by using JMeter APIs. I am able to add a ThreadGroup within a TestPlan and a JavaSampler within the ThreadGroup. But when I add a CSVDataSet element within the Java Sampler, it does not get saved properly.
The following code is used to create a new CSVDataSet element
CSVDataSet csvDataSet = new CSVDataSet();
csvDataSet.setName("CSV Data Set");
csvDataSet.setComment("Sample CSV Data Set");
csvDataSet.setDelimiter(",");
csvDataSet.setFileEncoding("");
csvDataSet.setFilename("d:\\jmeter\\data.csv"); // variable
csvDataSet.setQuotedData(true);
csvDataSet.setRecycle(true);
csvDataSet.setShareMode(shareMode.all);
csvDataSet.setStopThread(false);
csvDataSet.setVariableNames("firstname, lastname, email"); // variable
csvDataSet.setEnabled(true);
When this is saved using SaveService.saveTree, the final jmx does not contain all the values which were set.
<org.apache.jorphan.collections.HashTree>
<CSVDataSet testname="CSV Data Set Config" enabled="true">
<stringProp name="TestPlan.comments">Sample CSV Data Set Config</stringProp>
</CSVDataSet>
<org.apache.jorphan.collections.HashTree/>
As seen above, only the test name, enabled, and comments are added. The rest of the variables are completely ignored.
Is there something that needs to be set in order to get all the values as expected?
or is this a bug in JMeter? I am using version 2.11
The basic code is as per section 4.3 from following link
http://blazemeter.com/blog/5-ways-launch-jmeter-test-without-using-jmeter-gui
To that I add the code shown above. The way it is added is,
testPlanTree.add("testPlan", testPlan);
testPlanTree.add("loopController", loopController);
testPlanTree.add("threadGroup", threadGroup);
testPlanTree.add("httpSampler", httpSampler);
testPlanTree.add("csvDataSet", csvDataSet);
SaveService
.saveTree(testPlanTree, new FileOutputStream("d:\\test.jmx"));
output of CSVDataSet block is as shown above.
After looking into the JMeter source code, it seems all the properties are set using the setProperty function rather than the individual setter functions. So putting the following code does the job of creating the CSVDataSet element properly.
csvDataSet.setProperty("delimiter", ",");
csvDataSet.setProperty("fileEncoding", "");
csvDataSet.setProperty("filename", "d:\\data.csv");
csvDataSet.setProperty("quotedData", true);
csvDataSet.setProperty("recycle", true);
csvDataSet.setProperty("shareMode", "shareMode.all");
csvDataSet.setProperty("stopThread", false);
csvDataSet.setProperty("variableNames", "var1, var2, var3");
Not sure why setters are not used in the code, but this seems to be the way to go for now
It is clearly not a bug in JMeter otherwise CSV Data Set could not be saved.
It is probably an issue in the way you build the HashTree, but unless you show the full code, you cannot get help.
By the way, as I said in a previous answer, what you are trying to do to build different tests based on input parameter is not good idea IMHO, the approach will be very fragile towards upcoming versions of JMeter.
JMeter provides ways to do it that you should follow.

Coded UI error: The following element is not longer availabe

I recorded some test cases with CUIT in VS2010. Everything worked fine the day before. So, today I run again, all the test failed, with the warning: The following element is no longer available ... and I got the exception : Can't perform "Click" on the hidden control, which is not true because all the controls are not hidden. I tried on the other machine, and they failed as well.
Does anyone know why it happens? Is it because of the web application for something else? Please help, thanks.
PS: So I tried to record a new test with the same controls that said "hidden controls", and the new test worked!? I don't understand why.
EDIT
The warning "The following element blah blah ..." appears when I tried to capture an element or a control while recording. The source code of the button is said 'hidden'
public HtmlImage UIAbmeldenImage
{
get
{
if ((this.mUIAbmeldenImage == null))
{
this.mUIAbmeldenImage = new HtmlImage(this);
#region Search Criteria
this.mUIAbmeldenImage.SearchProperties[HtmlImage.PropertyNames.Id] = null;
this.mUIAbmeldenImage.SearchProperties[HtmlImage.PropertyNames.Name] = null;
this.mUIAbmeldenImage.SearchProperties[HtmlImage.PropertyNames.Alt] = "abmelden";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.AbsolutePath] = "/webakte-vnext/content/apps/Ordner/images/logOut.png";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.Src] = "http://localhost/webakte-vnext/content/apps/Ordner/images/logOut.png";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.LinkAbsolutePath] = "/webakte-vnext/e.consult.9999/webakte/logout/index";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.Href] = "http://localhost/webakte-vnext/e.consult.9999/webakte/logout/index";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.Class] = null;
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.ControlDefinition] = "alt=\"abmelden\" src=\"http://localhost/web";
this.mUIAbmeldenImage.FilterProperties[HtmlImage.PropertyNames.TagInstance] = "1";
this.mUIAbmeldenImage.WindowTitles.Add("Akte - Test Akte Coded UI VS2010");
#endregion
}
return this.mUIAbmeldenImage;
}
}
Although I am running Visual Studio 2012, I find it odd that we started experiencing the same problem on the same day, I can not see any difference in the DOM for the Coded UI Tests I have for my web page, but for some reason VS is saying the control is hidden and specifies the correct ID of the element it is looking for (I verified that the ID is still the same one). I even tried to re-record the action, because I assumed that something must have changed, but I get the same error.
Since this sounds like the same problem, occurring at the same time I am thinking this might be related to some automatic update? That's my best guess at the moment, I am going to look into it, I will update my post if I figure anything out.
EDIT
I removed update KB2870699, which removes some voulnerability in IE, this fixed the problems I was having with my tests. This update was added on the 12. september, so it fits. Hope this helps you. :)
https://connect.microsoft.com/VisualStudio/feedback/details/800953/security-update-kb2870699-for-ie-breaks-existing-coded-ui-tests#tabs
Official link to get around the problem :
http://blogs.msdn.com/b/visualstudioalm/archive/2013/09/17/coded-ui-mtm-issues-on-internet-explorer-with-kb2870699.aspx
The problem is more serious than that! In my case I can't even record new Coded UI Tests. After I click in any Hyper Link of any web page of my application the coded UI test builder cannot record that click "The following element is no longer available....".
Apparently removing the updates, as said by AdrianHHH do the trick!
Shut down VS2010, launch it again "Run as administrator".
There may be a field in the SearchProperties (or possible the FilterProperties) that has a value set by the web site, or that represents some kind of window ID on your desktop. Another possibility is that the web page title changes from day to day or visit to visit. Different executions of the browser or different visits to the web page(s) create different values. Removing these values from the SearchProperties (or FilterProperties) or changing the check for the title from an equals to a contains for a constant part of the title should fix the problem. Coded UI often searches for more values than the minimum set needed.
Compare the search properties etc for the same control in the two recorded tests.
Update based extra detail given in the comments:
I solved a similar problem as follows. I copied property code similar to that shown in your question into a method that called FindMatchingControls. I checked how many controls were returned, in my case up to 3. I examined various properties of the controls found, by writing lots of text to a debug file. In my case I found that the Left and Top properties were negative for the unwanted, ie hidden, controls.
For your code rather than just using the UIAbmeldenImage property, you might call the method below. Change an expression such as
HtmlImage im = UIMap.abc.def.UIAbmeldenImage;
to be
HtmlImage im = FindHtmlHyperLink(UIMap.abc.def);
Where the method is:
public HtmlImage FindHtmlHyperLink(HtmlDocument doc)
{
HtmlImage myImage = new HtmlImage(doc);
myImage.SearchProperties[HtmlImage.PropertyNames.Id] = null;
myImage.SearchProperties[HtmlImage.PropertyNames.Name] = null;
myImage.SearchProperties[HtmlImage.PropertyNames.Alt] = "abmelden";
myImage.FilterProperties[HtmlImage.PropertyNames.AbsolutePath] = "/webakte-vnext/content/apps/Ordner/images/logOut.png";
myImage.FilterProperties[HtmlImage.PropertyNames.Src] = "http://localhost/webakte-vnext/content/apps/Ordner/images/logOut.png";
myImage.FilterProperties[HtmlImage.PropertyNames.LinkAbsolutePath] = "/webakte-vnext/e.consult.9999/webakte/logout/index";
myImage.FilterProperties[HtmlImage.PropertyNames.Href] = "http://localhost/webakte-vnext/e.consult.9999/webakte/logout/index";
myImage.FilterProperties[HtmlImage.PropertyNames.Class] = null;
myImage.FilterProperties[HtmlImage.PropertyNames.ControlDefinition] = "alt=\"abmelden\" src=\"http://localhost/web";
myImage.FilterProperties[HtmlImage.PropertyNames.TagInstance] = "1";
myImage.WindowTitles.Add("Akte - Test Akte Coded UI VS2010");
UITestControlCollection controls = myImage.FindMatchingControls();
if (controls.Count > 1)
{
foreach (UITestControl con in controls)
{
if ( con.Left < 0 || con.Top < 0 )
{
// Not on display, ignore it.
}
else
{
// Select this one and break out of the loop.
myImage = con as HtmlImage;
break;
}
}
}
return myImage;
}
Note that the above code has not been compiled or tested, it should be taken as ideas not as the final code.
I had the same problem on VS 2012. As a workaround, you can remove that step, and re-record it again. That usually works.
One of the biggest problem while analyzing the Coded UI test failures is that the error stack trace indicates the line of code which might be completely unrelated to the actual cause of failure.
I would suggest you to enable HTML logging in your tests - this will display step by step details of how Coded UI tried to execute the tests - with screenshots of your application. It will also highlight the control in red which Coded UI is trying to search/operate upon.This is very beneficial in troubleshooting the actual cause of test failures.
To enable tracing you can just add the below code to your app.config file --

Reload rb scripts from different locations in JRuby

BACKGROUND:
I am using JRuby in an eclipse plugin for my product. I have a bunch of scripts that define a DSL and perform operations for me. I want to be able to dynamically reload these scripts whenever required. The scripts could change themselves on file system and moreover the location of the scripts could also change. I could even have multiple copies on file system of slightly modified/changed scripts. Each time I want scripts from a specific location to be utilized.
As I have understood so far, using "load" instead of "require" should do the job. So now if before calling any Ruby methods/functions I use "load 'XXX.rb'", it will reload the XXX.rb utilizing the new changes.
PROBLEM:
In my code I am using ScriptingContainer to run scriplets to access ruby functions. I set load paths on this scripting container to indicate from which locations the scripts should be loaded. However, the problem is that on subsequent calls and even with different instances of ScriptingContainer, I have noticed that the scripts that were loaded the first time are utilized every time. "load" reloads them, but after loading those scripts once, the next time I might need to load similar scripts from a different location but its not happening.
My assumption was that utilizing a different scripting container instance should have done the job but it seems that the load paths are globally set somewhere and calling "setLoadPath" on new ScriptingContainer instances either does not modify existing paths or only appends. If the latter case is true then probably when searching for scripts they are always found on oldest paths set and newer load paths get ignored.
Any ideas???
The solution is to specify scope for a ScriptingContainer instance when creating it. One of the ScriptingContainer constructors takes in a parameter of type LocalContextScope, use one of the constants to define the scope. See LocalContextScope.java
To test this defect and solution I have written a small snippet. You may try it out:
public class LoadPathProblem {
public static void main(String[] args) {
// Create the first container
ScriptingContainer c1 = new ScriptingContainer();
// FIX ScriptingContainer c1 = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
// Setting a load path for scripts
String path1[] = new String[] { ".\\scripts\\one" };
c1.getProvider().setLoadPaths(Arrays.asList(path1));
// Run a script that requires loading scripts in the load path above
EvalUnit unit1 = c1.parse("load 'test.rb'\n" + "testCall");
IRubyObject ret1 = unit1.run();
System.out.println(JavaEmbedUtils.rubyToJava(ret1));
// Create the second container, completely independent of the first one
ScriptingContainer c2 = new ScriptingContainer();
// FIX ScriptingContainer c2 = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
// Setting a different load path for this container as compared to the first container
String path2[] = new String[] { ".\\Scripts\\two" };
c2.getProvider().setLoadPaths(Arrays.asList(path2));
// Run a script that requires loading scripts in the different path
EvalUnit unit2 = c2.parse("load 'test.rb'\n" + "testCall");
IRubyObject ret2 = unit2.run();
/*
* PROBLEM: Expected here that the function testCall will be called from
* the .\scripts\two\test.rb, but as you can see the output says that
* the function was still called from .\scripts\one\test.rb.
*/
System.out.println(JavaEmbedUtils.rubyToJava(ret2));
}
}
Test scripts to try out the above code can be in different folders but with the same filename ("test.rb" for the above example:
./scripts/one/test.rb
def testCall
"Called one"
end
./scripts/two/test.rb
def testCall
"Called two"
end

how to generate MSTEST results in Static folders

Is there a way to control the name of the MSTEST video recoding file names or the folder names with the test name. It seems to generate different guid everytime and thus very difficult to map the test with its corresponding video recording files.
The only solution I can see is to read the TRX file and map the guid to Test Name.
Any suggestions ??
If you're not opposed to doing it by hand, it's pretty easy. I encountered the same problem, and needed them to be somewhere predictable so I could email links to the videos. In the end my solution just ended up being to code in the functionality by hand. It's a bit involved, but not too difficult.
First, you'll need to have Expression Encoder 4 installed.
Then you'll need to add these references to your project:
Microsoft.Expression.Encoder
Microsoft.Expression.Encoder.Api2
Microsoft.Expression.Encoder.Types
Microsoft.Expression.Encoder.Utilities
Next, you need to add the following inclusion statements:
using Microsoft.Expression.Encoder.Profiles;
using Microsoft.Expression.Encoder.ScreenCapture;
Then you can use [TestInitialize] and [TestCleanup] to define the correct behavior. These methods will run at the beginning and end of each test respectively. This can be done something like this:
[TestInitialize]
public void startVideoCapture()
{
screenCapJob.CaptureRectangle = RectangleSelectionUtilities.GetScreenRect(0);
screenCapJob.CaptureMouseCursor = true;
screenCapJob.ShowFlashingBoundary = false;
screenCapJob.OutputScreenCaptureFileName = "path you want to save to";
screenCapJob.Start();
}
[TestCleanup]
public void stopVideoCapture()
{
screenCapJob.Stop();
}
Obviously this code needs some error and edge case handling, but it should get you started.
You should also know that the free version of Expression Encoder 4 limits you to 10 minutes per video file, so you may want to make a timer that will start a new video for you when it hits 10 minutes.

Resources