This is probably an anti-pattern, but I'm new to e2e testing and I'm not sure there's a way around my requirements.
I have the need to test a scenario in my system that is several steps long (maybe 50-60 steps). This means going to about 15-20 pages, clicking and entering various details, and seeing the results. The reason the requirement is so long is that the system processes orders from creating an order, creating line items with various details, and running it through a long production process. My most valuable test would be to run through the whole process and verify the results.
I can't find a good way of isolating, say, steps 40-42 to verify that this process alone works well, because to get an order in that state I'd have to run through the first 39 steps.
Is there a good way to write tests to cover this scenario?
This question is about shared steps management via the user interface in Microsoft test manager VSTS2013 and/or online via the visualstudio.com version.
Say we have a bunch of test cases with some shared steps and we clone (/deep copy, not just 'copy') the test plan. The result is that all shared steps in the test cases are also cloned, which I would expect -- so no question so far.
But, when I then go to a test case and I want to add shared steps to it, the query to locate the shared steps will show all instances of a shared step (assume I am searching by Title). How can I now make the distinction which instance of the shared steps is the right one to choose for the Test plan that my test case is in?
If I choose just any one of them I'll for sure end up with test cases from one test plan linked to shared steps in another test plan, and that sounds like something we would not want (right?).
Any light on how to properly manage test cases versus their shared steps is welcome!
Thank you,
Bill.
If your (cloned) testplan is defined under a certain iteration, e.g., Project Y\Release X, and you insert a shared step, a query is opened. There you need to add an extra clause (with an And) with as Field: "Iteration Path" and Operator "Under" The Value is for this example "Release X".
Hope this helps.
I have a graph generator plugin. I want to create graphs after I input the users in GUI mode. Do I have to run the script in advance and then run it again in order to see the graphs? I'm asking because it wants the 'JMeter Results File' which if I don't run, would not be there.
There are two ways to make graphs: at run-time, or from old results. If you want to do the former, put it in your test and make sure you follow the instructions here:
http://jmeter-plugins.org/wiki/GraphsGeneratorListener/#Generate-CSV-PNG-for-current-test-results
Note that, like many listeners, this has a fairly high performance cost, so it suggests you avoid using this while in GUI mode.
Alternatively, you can run your normal test without this listener, then run a second 'fake' test with it to generate your graphs:
http://jmeter-plugins.org/wiki/GraphsGeneratorListener/#Generate-CSV-PNG-for-existing-previous-test-results
We use Vs2008/2010 with TFS 2010 for our source control, because it also lets us create custom work item types that we can use for project management, such as product backlog items and sprint backlog items.
One item thats not tracked (by machine) is build regression test tasks for release candidates. Our regression testing is part automated, part manual, and the manual part can take several days. Currently we use an excel spreadsheet with a list of all the test cases, and then the testers just fill in results and notes.
I've been proposing creating a build regression test template that contains each test case, default owner, and then when we want to do regression testing on a build, we can automatically create work items for every test in the template.
My argument is that if the regression test work is mandatory for the project, and the results should be tracked, then writing additional TFS work items make sense, especially since the work items can hold estimates, giving managers an idea of how much re-test time remains.
The argument against this is that we already have high level work items to capture the overall project test requirements, and the regression testing is basically a "re-test", so new work items would be duplicate.
My question: Is anyone else doing anything like this? Is it reasonable to use TFS to track outstanding re-test tasks?
Note: we don't own Visual Studio Test Professional
I think it's reasonable to go with your suggested solution. You should have another work item type for the "test tasks", that can be linked as children to the test requirement work items. Doing that, like you said, would allow you to track results, progress, reporting, etc. You can also add other fields like build number, tested by, tested date, etc. to the work item type for history, something that cannot be done with just one test requirement work item type.
Essentially, what you proposed is done in the ITestResult object in the Microsoft.TeamFoundation.TestManagement.Client.dll.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have complex project without comments. The project is programed in Java but have more than one main class, use several .txt files like a template and use several .bat files. I don't know where to start and how to start discovering the project, because I need to make some changes in that project.
As with others I say this is a slow process.
However having done this in the past many times, this is my methodology:
Identify as many requirements that the code fulfils. This may give you the some reasons as to why certain things are the way they are when you look deeper. A common way of finding these is look for any tests that be available. The automated ones are best, but usually they're as missing as the comments.
Find the entry points to the code. These will give you places where you can poke the code to see how different inputs affect the flow. Common entry points are Code Loading 'Main' type functions, service interfaces, web page post backs etc..
Diagram the code. Look for tools that can build black/white box pictures of the code. For me this invaluable. I have on occasion printed out large listings and attacked then with markers and rulers. You're aim to create your own flow chart (mental or other wise) of the code flow.
Using the above (iteratively) build a set of outputs to the code that you think should occur, and add to these the outputs you may already know about such as logs, data files, database writes etc..
Finally if you have time, create some manual tests though preferably in automated test harnesses to verify the above. This where I start to involve the debugger to see detail in the code.
This methodology usually gives me confidence to make changes.
Note this is iterative process and can be done with portions of the code or overall as you see fit. I usually favour a top down approach to start with and then as I gain some insight I drill down till details become overwhelming and then I repeat. However this is just because my mind works in this way - you may be different. Good luck.
Find the main Main class. The starting point.
Start drawing a picture of the classes and the objects they own and the external entities they reference.
Follow all the branches until you can find a logical ending.
I've used UML reverse engineering tools in the past and while a visual picture is good, stepping through the code has always been the hardest and yet best methodology for me.
And, as you step through each piece you can add in your own comments..
I usually start with doxygen, turning on every extracting option (especially EXTRACT_ALL and EXTRACT_PRIVATE), and enable the SOURCE_BROWSER, HAVE_DOT, CALL_GRAPH and CALLER_GRAPH options (you also need to have dot installed). This gives good view of the software. For every function the calls are displayed and linked in a graph, also the sources are linked from there.
While doxygen is intended for C and C++, it also works with Java sources (set the OPTIMIZE_OUTPUT_JAVA option).
Auch. I'm afraid there is no speedy way to do this. Comment out a line (or two) -> test -> see what breaks. You could also put break statements here and there and run the debugger. That should give you some indication how you got there (ie. what the hierarchy between the classes is).
Hopefully the original developers used some patterns that you can recognize and make notes. Make lots of notes of everything. Start by trying to understand the high level structure and work down from there.
Be prepared to spend endless hours not understanding what the thing is doing.
Speak to the client and try to understand what the project is for, and what are all the things that it does. Someone somewhere had to put in some requirements for the stuff that's in there, if only in an email.
I would try to find the first entry point in the code closest to where you suspect you'll need to start making your changes, set a breakpoint, and start debugging. Check out the contents of local variables and work your way deeper as you get to become familiar with whats going on. Then, when you have a basic understanding of the area of code you're going to be working with, start fiddling with some small changes. Test your understanding of it. Try diagramming what you see happening. If you can do that confidently, you'll be able to decide if you need to go back and continue learning more about the code, or if you know enough to get done what you need to get done.
A start is to use an automated uml modeling tool (if you use eclipse you can use a plug-gin), and start creating UML diagrams of the various classes to see how they are related in a high level and visualize the code. This has helped many times
If there are log files being generated, have a look at it to understand the flow from the starting point (main class). Otherwise, put debug statements to understand the flow.
Ya, that sounds like a pretty bad spot to be in.
I would say that the best way is to just walk through the program line for line. Try to grasp the big picture in the code, and write alot of notes, both on paper and in comments in the code.
I would say, a good approach would be to generate documentation using javadoc or doxygen's class diagram feature, then as you run the code traverse through the class diagrams generated using doxygen and see who calls what. This works wonderfully for me everytime i am working on such a project.
I completely agree to most of the answers posted.
I can add to use a development tool that reverse engineering the code and create a class diagram, to have an overall picture of what is involved.
Then you need patience. But you will be a stronger and smarter developer when you'll get through...
Good luck!
One of the best and first things to do is to try to build and run the code. It might sound a bit simplistic but the problem when you take over undocumented code is that you can't even build it and run it. When have no clue were to start.