Function testing on Netbeans 6.8 - user-interface

While not a torrent, but some articles can be found on the net about function testing (particularly http://blogs.oracle.com/geertjan/entry/gui_testing_on_the_netbeans). However the tools mentioned by them do not seem to be maintained, or don't have a plugin working with the most recent version of Netbeans (6.8).
Do you have any function test setup for GUI? What is your level of integration into the development process (IDE integration, ant, etc).
Additional candy is that Netbeans is not only the IDE, but the GUI app is also developed for Netbeans 6.8 Platform (so I'm mainly interested in GUI testing NB-platform apps, but tips for any Swing apps in general would be a help too).

The NetBeans developers do a lot of functional testing and that testing is supported as part of the NetBeans module project.
One of the modules that I work with that has functional tests is here: http://hg.netbeans.org/web-main/file/tip/j2ee.sun.appsrv81
If you create an nbm module project, there are not functional tests defined by default, so you need to create some directories and the like 'by hand' on the Files explorer:
test/qa-functional/src
an initial test
This is a minimal test to get you started.
package a;
import junit.framework.Test;
import org.netbeans.junit.NbTestCase;
import org.netbeans.junit.NbModuleSuite;
public class SampleTest extends NbTestCase {
private final int SLEEP = 10000;
public SampleTest(String testName) {
super(testName);
}
public void testBogus() {
}
public static Test suite() {
return NbModuleSuite.create(
NbModuleSuite.createConfiguration(SampleTest.class).
addTest(SampleTest.class, new String[] { "testBogus"}).
enableModules(".*").clusters(".*"));
}
}
After these things are in place, you should be able to do the following:
Switch to the Files explorer (if you
aren't there already)
Right click on the node for the
build.xml file
Select the Run Target->Advanced...
item. A dialog will open.
Select test-qa-functional from the
combobox entry field labeled 'Select
targets to run:'
Press the Run button to dismiss the
dialog and execute the test.
Once you get the minimal test case running, you can start to examine the qa-functional test that have been written for the NetBeans IDE to learn more.

Related

Visual Studio unit test not working and not showing tests

I am following MSDocs guide to put up some unit testing. I've never used them before and I've never been using them because to me they are totally useless, but I see it's a common practice, so I wanted to learn more.
Obviously, this is not working.
I've put up some .NET Standard 2.0 libraries with little to no code to do some log4net logging. I've addet a "Unit test project" as the guide suggests and added a method decorated with the "TestMethod" attribute and all the other attributes.
Test explorer windows does not show any test and I'm not able to even run them in any way. What am I missing?
Here's the code
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MrLogger_netstandard;
namespace UnitTests
{
[TestClass]
public class UnitTest1
{
[TestMethod]
public void ErrorLogOnFile_Default() => MrLogger.Instance.FileErrorLog("I'M AN ERROR!!! :)");
}
}
The MrLogger class it's just a class that does logger.Debug("Write message"), a classical log4net logging call.
I've tried this too and it changes nothing.
Jeez... help
For anyone wondering, I had to start visual studio as administrator and then it worked.... UGH! :(

How to create a Qt-Quick Test

I have to create a Unit-Test.
But first, I´ve to get clear what to do.
There is a QtQuick2-App written and now I would like to do Unit-Tests with the GUI. What are the steps for Unit-Tests with GUI? After reading the Qt-documents, I could not create any ideas for starting with the test.
Hope somebody can help me.
Edit: I was able to run some tests, after adding tst_button.qml and tst_test.cpp to my Project (main.cpp is in comments now). Is this the right way, or should I create a new project just for the Tests? If yes, what kind of project is needed?
And the last question: Do I need to build up my MainForm for pressing buttons for example?
tst_button.qml
import QtQuick 2.4
import QtTest 1.0
Rectangle{
id: myRec
property var myMainForm: null
TestCase{
name:"ButtonClick"
when:windowShown
function test_init(){
var createMyWindow = "import QtQuick 2.0; MainForm{id:myForm}"
var myMainForm = Qt.createQmlObject(createMyWindow,myRec)
myRec.myMainForm = myMainForm
}
}
}
tst_test.cpp
#include <QtQuickTest/quicktest.h>
QUICK_TEST_MAIN(test)
Testing and Debugging lists two ways:
Qt Test (also known as testlib) - a framework for unit tests of C++ code
Qt Quick Test - a framework for unit tests of QML code
You can use Qt Test for testing Qt Quick applications, but that's generally better for when you need access to C++ API that isn't available in QML.
Do I just add a *.qml file to my project and fill it with my code? If yes, what do I have to do to start the test?
You'll first need to make the tests a separate project, unless you're planning on using qmltestrunner (I have no idea why that tool isn't documented by Qt itself).
The Running Tests section of Qt Quick Test's documentation details how to get a test up and running.
I was able to run some tests, after adding tst_button.qml and tst_test.cpp to my Project (main.cpp is in comments now). Is this the right way, or should I create a new project just for the Tests?
If your application is pure QML and only intended to be run with qmlscene, for example, then doing it that way is fine. However, if you intend to deploy/ship your application, you'll probably need to have an executable, which means making separate projects for the application and the tests.
If yes, what kind of project is needed?
You could have a SUBDIRS project, so that your tests and the application itself can all be opened at once in Qt Creator. Something like this:
myapp.pro
app/
main.cpp
app.pro
resources.qrc
main.qml
tests/
tests.pro
data/
tst_stuff.qml
And the last question: Do I need to build up my MainForm for pressing buttons for example?
No. The .ui feature is just a format that allows Qt Creator to enforce certain constraints to make it easier to design Qt Quick UIs with Qt Quick Designer. MainForm.ui.qml is therefore just a convenience. If you already have an existing component in QML, you can create instances of that and test it.

VS2008->VS2013 upgraded solution can no longer run tests

We've upgraded a VS2008 solution to VS2010 and then to VS2013 Pro. In VS2013 it all builds, but the unit tests all fail with the same error:
The unit test adapter failed to connect to the data source or read the
data.
The folder structure has not changed, and the test data files are all there in the correct location.
Here's a typical test:
[DeploymentItem(#"Autoscribe.Utility.Test\TestData\GetParameters1_RetrievesCorrectParameters.xml")]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", #"|DataDirectory|\Autoscribe.Utility.Test\TestData\GetParameters1_RetrievesCorrectParameters.xml", "test-record", DataAccessMethod.Sequential)]
[TestMethod(), Description("String extensions")]
public void GetParameters1_RetrievesCorrectParameters()
{
// Get test data.
string input = TestContext.DataRow["input"].ToString();
string[] outputs = TestContext.DataRow["outputs"].ToString().Split(new char[] { ',' }, StringSplitOptions.None);
// etc
}
In VS2008 there was a wizard to set the test properties to connect the test data file to the test, and using this often fixed this class of problem,but this doesn't seem available in VS2013 Pro. Can someone tell me:
a) how to connect a test to a data file in VS2013 Pro without having to explicitly type the DataSource attribute, and
b) any suggestions on how to fix this problem?
The trick was to explicitly select the original testrun config. It seems the upgrade does not carry this forward.

Coded UI Tests: Start application under test only if it is not already running

When using "Coded UI tests" in Visual Studio 2010, is there an easy way to start the application under test (AUT) only if the AUT is not already running?
I know that I can implement such a piece of startup code from scratch, but I wonder whether the Visual Studio test framework offers something out of the box.
It seems as if this can be done using this code:
[TestInitialize]
public void MyTestInitialize()
{
if (_application != null)
{
return;
}
Process[] processes = Process.GetProcessesByName("ApplicationUnderTest");
if (processes.Length > 0)
{
_application = ApplicationUnderTest.FromProcess(processes[0]);
}
else
{
_application = ApplicationUnderTest.Launch(#"C:\Path\To\ApplicationUnderTest.exe");
}
}
The AUT is launched only if it is not already running.
There isn't a built in method to start an application if it isn't currently running that I am aware of.
It would be trivial to write something custom to do this just be sure that you are able to get your application under test into a known state so a previous test failure doesn't cause all tests run after it to fail. For web applications getting to a known state is likely as a simple as setting the URL but for complicated desktop applications it could be exceptionally complicated to get back to a known state.

Debugging Nunit Tests inside VS2010 Express

I have written a bunch of unit tests inside VS2010 Express and tests being tests they sometimes fail. Since the express editions of VS don't allow plugins to run I can't simply spin up TestDriven.Net or an equivalent and debug the tests. To try and work around this I've converted my test assembly into a console app and made the main method look like this:
class CrappyHackToDebugUnitTestInVSExpress
{
public static void Main()
{
AppDomain.CurrentDomain.ExecuteAssemblyByName(
#"C:\Program Files\NUnit 2.5.5\bin\net-2.0\nunit-console.exe",
new [] { Assembly.GetExecutingAssembly().Location, "/framework:4.0" });
}
}
In theory I should be able to run this up, set break points in my test. If it worked it would be an acceptable work around, but I keep getting the following:
FileLoadException
Could not load file or assembly 'C:\\Program Files\\NUnit 2.5.5\\bin\\net-2.0\\nunit-console.exe'
or one of its dependencies. The given assembly name or codebase was invalid.
(Exception from HRESULT: 0x80131047)
Now the file exists and when run manually nunit-console runs fine. What might be my problem?
Basically you need to convert your assembly to Windows Forms app, add reference to the nunit-gui-runner.dll assembly and change your Main method to look like this:
[STAThread]
static void Main()
{
NUnit.Gui.AppEntry.Main(new string[] { Assembly.GetExecutingAssembly().Location });
}
here is another example:
...
using NUnit.Gui;
namespace __libs
{
class Program
{
[STAThread]
static void Main(string[] args)
{
NUnit.Gui.AppEntry.Main(new string[] { #"C:\test\bin\Debug\test.exe" });
}
}
}
This will allow you to step into certain tests but is not very good for a red green cycle, so you will want to use this only when debugging and not in other circumstances.
I played with your concept and it appears the issue isn't directly from loading the file, but from dependencies.
I used the following modified code:
And the error was actually a failure to locate nunit.core.dll, which is in the /lib directory.
try
{
String NUnitPath = #"C:\Program Files\NUnit 2.5.7\bin\net-2.0\nunit-console.exe";
AssemblyName asmName = System.Reflection.AssemblyName.GetAssemblyName(NUnitPath);
AppDomain.CurrentDomain.ExecuteAssemblyByName(asmName, new[] { Assembly.GetExecutingAssembly().Location, "/framework:4.0" });
}
catch (Exception ex)
{
Trace.WriteLine(ex.Message);
Trace.WriteLine(ex.StackTrace);
}
(I like getting System.Reflection.AssemblyName because you can inspect and see that everything's in order verses the raw file path.)
A quick bulk copy (xcopy nunit.*.dll) into my test projects' debug directory and it ran just fine. (It should be trivial to discover the minimal dependencies required)
Tested in VC# 2010 Express with NUnit 2.5.7 (breakpoints work, but I didn't really play with any other options.) Although I'm sure you could make a passable build option from it.
Cheers!
PS - First post here so I'm a bit untested as to getting the 'code' blocks formatted. Sorry in advance..
I had a similar problem trying to debug unit tests in VS C# express. Had a hard time getting it to work properly but then I found out about this project template. Works perfectly in C# Express!
http://visualstudiogallery.msdn.microsoft.com/b8a7a8fa-9f5a-4b9b-8e8b-8839a4364f26?SRC=VSIDE
C# Project Template
Integrated tests with Visual Studio, including Visual C# Express
version
Self contained NUnit console runner. Allow to write test fixtures and
test, running from Visual Studio simply by pressing F5 (support test
debugging), or Ctrl-F5 free run with results in console window. In
case of test failure indicate by beep sound.
Contains essential NUnit modules to start test project. No external
dependencies. Simply create new project, using NUnit Test Application
template.

Resources