I am beginning to adopt Test Driven Design (TDD) behaviors and workflow for my iOS development projects. There is at least one impediment though in the context of legacy software. I will often have to add features to a pre-existing code base that I am new to. I will typically want to refactor at the beginning of working with the code base which will often have no tests available to ensure that my refactor-ings are not altering code functionality or worse, adding bugs.
My question is how do TDD folks bootstrap the whole process when the code is not written from scratch but rather legacy code that they are brought in to work on?
Thanks,
Doug
UPDATE
For a concrete example, I am using the example from Martin Fowler's Refactoring re-coded in Objective-C as a training device for TDD (and AppCode) >>
I built the code from tests. I found I needed to add instance variables to the Customer class to ensure I didn't screw up the cost calculations in the statement method as I grew the code. This is the fundamental issue I need insight into.
To begin with, if you don't understand the legacy code you're working with, you need to fix that before you do things which you're concerned may change behavior.
In your situation, after understanding the legacy code, I would write tests that will run against that legacy code. Once you're satisfied that these tests function as you expect, you're in a much better position to test your refactored code to ensure it functions as the old code did.
Related
My goal is to be able to write core testing that I can use within a unit testing framework as well as UI testing with selenium.
For simple test like:
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120
I would create both unit tests to prove that my core API would pass as well as a Selenium test that would prove my UI is doing the correct thing as well.
I briefly tried to find anyone doing something similar through Google, but couldn't find any examples. So I guess my question is, has anyone here done anything similar?
On approach I had thought of was simple adding the feature files to a project or directory and using the add existing item as link as the solution.
Update: Adding feature files to a common directory and adding them as a link appears to be working great. The feature bindings regenerates for each project the feature file was included in so I can run unit tests in one and Selenium UI tests in the other.
First, lets start with why you might want to do this. Its laziness of the good kind.
The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer. Also hence, this book. See also impatience and hubris. (p.609)
Larry Wall, Programming Perl
Except it isn't, because we aren't going to reduce our overall energy expenditure.
When you are using SpecFlow, the easy part to keep up to date is the plain text. You will find yourself refactoring the [Binding]s again and again, but the scenarios tend to be quite easy to work with, and need very little revision once they have been agreed.
In addition the [Binding]s are global. Load them in from any assembly and they are available to the SpecFlow runner. In respect of what you are trying this actually makes things harder as you need to put effort in to keep the UI bindings from being mixed up with the non-UI bindings.
Also consider the way that SpecFlow actually runs the tests from feature files. It's a two stage process.
When you save the .feature file the SpecFlow VS plugin generates a .feature.cs file.
When you run your test engine (e.g. NUnit) it ignores the plain text and uses compiled code from .feature.cs
So if you start using linked .features I have no idea if the SpecFlow plugin will generate .feature.cs for both instances of the file. (If you try this please let us know)
Second lets consider the features themselves. I think you will constantly finding yourself compromising your tests to make them fit the other place they are used. Already in the example you have given you have on the screen. If you are working with just the core API then there won't be a screen, so do we change this to fit better in a non-UI scenario?
Finally you have another thing to consider, just how useful will your tests be. If you have already got a test that tests the Core API, then what will it mean to run same test via Selenium. All you will really test is the UI layer. In my current employment we have a great number of regression tests that perform this very kind of testing, running up a client that connects to a server and manipulating the UI to get the desired scenarios enacted. These are the most fragile tests we have due to their scale. They constantly break and we basically have to check our entire codebase to find the line that broke them. Often something like 10-100 of them break just for a one line change. If these tests weren't so important to the regression cycle then the effort in maintaining them would just be too much. In my own personal projects I tend to remove these tests completely and instead with UIs, I avoid testing the View layer. With WPF MVVM, I execute Commands and test for results in ViewModels. If somebody then decides the TextBox should be a ComboBox or that it will work better in mauve, then my testing is isolated.
In short, there is a reason you can't find anything about this on Google :-)
In general (see http://martinfowler.com/bliki/TestPyramid.html), one should limit the number of automated tests that test the UI directly, and prefer tests that start at the presentation layer (just below the view layer), or below.
SpecFlow is agnostic; the tests can be implemented using e.g. Selenium at the UI layer or just MSTest or NUnit at any of the layers below.
However having said that I appreciate that you will have situations where you are doing ATDD and want to implement SpecFlow scenarios to match each of the acceptance criteria. Some of the criteria will be perfectly fine to test at a lower architectural level, but one or two of them may be specific to the GUI-- for example testing Login and ensuring that the user is redirected to the home page after successful login. If using Angular2 or React routing (see https://en.wikipedia.org/wiki/Single-page_application), that redirect is likely done in the GUI layer itself.
I don't have a perfect answer yet, but as a certified SpecFlow trainer, I have a vested interest in this! The way I am currently leaning is to use a complementary tool like CucumberJS for the front-end specific tests (such as testing React router redirects) and SpecFlow for tests at lower architectural layers. Our front-end uses Node.JS/Express and our backend is .NET Core. The idea is that the front-end tests mostly use the front-end only with mocked out AJAX calls to the backend (see sinonjs), and the back-end tests use EF Core with the in-memory option (see docs.efproject.net/en/latest/providers/in-memory/. So the tests all run fast.
Of course, you still need a few tests that actually go all the way through, but those are different-- we should call those integration tests. I do not believe that acceptance tests need to be integration tests. That way, you have a suite of acceptance tests from doing ATDD, plus a relatively small set of integration tests that test all the way front-to-back. The integration tests run more slowly and require more maintenance, so you separate them out into a different part of the CI/CD build chain.
I hope this makes sense. It is not so much solving the problem as avoiding the problem.
I am at a company that does not understand the concept of using frameworks and the benefits of them. I have tried to explain that it provides structure and organization but the people I am trying to explain to are still a little fuzzy about it. In your opinion, what is the best way to describe a framework in the most simplest terms and how it could overall benefit a company to transition their code from procedural and spaghetti code to a nice organized framework?
Thank you for your time.
I guess the best explanation I can think of for using a framework are to standardize your design process and save yourself a lot of effort as your code-base grows. Not to mention that a lot of work can be taken care of for you by the framework (which could save hours of coding). A framework can give you all the parts you need to build your application, you just have to assemble them.
The best reasons I can think of for using a framework are:
Code reuse -- If you try and follow the design of the framework you can save yourself a lot of coding time. However, some frameworks do require a time investment to master.
Encapsulation -- You can change the underlying implementation of different parts of the framework in a way that doesn't require a lot of code rewriting.
Extendability -- You can extend the code of the framework to add features you need and if you are careful about your design, you can reuse these features too.
I'm sure there are many other good reasons, but I'm sleepy.
EDIT: A good example of the benefits of a framework can be replacing the database adapter with another ie. switching from mysql to postgresql. This could be awful with functional programming but a framework could make this transition very easy.
Your coworkers most likely already use libraries, which one could define as code that exists outside of your project, and is meant to used in many projects.
A framework is like a library, but usually has other featues, such as
It might enforce changes to your code. For example, you wouldn't replace one method of your WebForms project with a call to the ASP.NET MVC framework - the entire project would be written differently to conform to the framework.
It might restrict the universe of applications that you can write. For example, you might be using a CRUD generating framework that lets you make data entry applications, but wouldn't let you make a video editing application.
However, a framework will usually give you a lot of value in return.
Let them do as they like ,first.
then pick up their shortcomings and
finally generalise your framework to avoid procedural code.
I'm going to concentrate on only a part of the question:
In your opinion, what is the best way to describe a framework in the most simplest terms
Framework == Library + Inversion of Control
Could anyone explain the word regression test in an understandable way?
Regression test is a test that is performed to make sure that previously working functionality still works, after changes elsewhere in the system. Wikipedia article is pretty good at explaining what it is.
Your unit tests are automatically regression tests, and that's one of their biggest advantages. Once those tests are written, they will be run in future, whenever you add new functionality or change existing functionality. You don't need to explicitly write regression tests.
Notwithstanding the old joke, "Congress" is not the opposite of "progress;" "regress" is. For your code to regress is for it to "move backward," typically meaning that some bad behavior it once had, which you fixed, has come back. A "regression" is the return of a bug (although there can be other interpretations). A regression test, therefore, is a test that validates that you have fixed the bug, and one that you run periodically to ensure that your fix is still in place, still working.
The word regression as coined by Francis Galton means
Regression: The act of going back
I.e. it is the phenomenon/technique in software testing to check any change / bug fixes hasn't impacted the existing functionality of the system. Thus the intent of regression testing is to ensure that a change, such as a bug fix should not result in another fault being uncovered in the application.
Regression Testing is required when
there is a change in requirements and code is modified according to the requirement
a new feature is added to the software
defects are fixed
a performance issue is fixed
Regression testing can be done both manually and automated.
These are some tools for the automation approach:
QTP
AdventNet QEngine
Regression Tester
vTest
Watir
Selenium
actiWate
Rational Functional Tester
SilkTest
During a regression test, testers run through your application testing features that were known to work in the previous build.
They look specifically for parts of the application that may not have been directly modified, but depend on (and could have residual bugs from) code that was modified.
Those bugs (ones caused by bugs in dependent code even though they were working before) are known as regressions (because the feature was working properly and now has a bug...and therefore, regressed).
Regression testing is a part of testing activity, which can be start after modification has been made to check the reliability of each software released.It's nothing but an impact analysis to check wheather it not affecting critical area of the software.
Do unit test
Do integration test
After (1) and (2) are passed, do regression test
In simple term, regression test is to repeat step (1) and (2) again.
Regression testing basically perform after completing of retesting.
The main purpose of regression testing is to check the impact of modification. Whether still our application is acting stable.
Its necessary to perform a regression testing because sometimes it happened after retesting or while fixing bug developer fixed the bug and missed out something on other code or on dependent code
http://en.wikipedia.org/wiki/Regression_testing
Basically, test the code you've updated to make sure you haven't introduced new bugs and that the functionality still works as before.
Regression test:- IF THERE ANY Changes, delate,modification, up dings or adding in my application . In that case I have to know that my application works as it was working before.
Regression test - Is a type of SW testing where we try to cover or check around the bug Original bug Fix.
The functionality around the bug fix should not get changed or altered due to the Fix provided. Issues found in such process are called as Regression Issues.
In a simple way, Regression test is a test to make sure that the functionality of a system still works after a new code change has been introduced. It doesn't really have to be a thorough testing of the whole functionality (such as functional testing), only the areas that are considered to be impacted by the introduced code changes.
Regression test is a test which enables us to find introduced bug by testing some areas in the software that we are testing. Introduced bug means a bug which is caused by the new changes made by the developer.
The key in the regression test is how we can effectively do the test by wisely deciding some areas which might be impacted by the changes since we can't test all the functionalities due to the time constraint (most of the time). 'Effective' in here means we can find bugs in a relatively short period of time.
Regression testing means testing your software/website repeatedly. The main reason for it is to make sure there aren't any new bugs introduced.
Typically, regression tests will be automated, to reduce the cost of rerunning the test. The more high value test cases you can construct, the better. This is one example of a Play and Record regression testing platform
Definition: - Regression testing is defined as a type of software testing to confirm that a recent program or code change has not harmfully affected existing features.
Regression Testing is Re-Testing to make sure that any modification done in a program will not affect the other functionality.
Regression testing is nothing but a full or partial selection of already executed test cases which are re-executed to ensure existing functionalities work fine.
We can do Regression Testing at all the level of testing like Unit Testing, Integration Testing and System Testing Level.
Need of Regression Testing
Common code changed correctly or not.
Correct or incorrect version control.
Bug fixes perfectly.
Bug fixes completely.
Performance issue fix.
6.Change in requirements and code is modified according to the requirement.
The new feature is added to the software perfectly.
For More Visit LINK
I like this definition of regression testing:
[regression testing] tells you if a previously written and tested code broke after you’ve added an update or a fix
[...] it helps you notice if you’ve unknowingly introduced bugs to your software while adding new code. New bugs of this kind are called regressions.
Basically, a regression is returning to a state where your application has bugs.
Regression testing is an activity performed to ensure the different functionalities of the system are still working as expected and the new functionalities added did not break any of the existing ones.
Secondly, you generally write automated tests or do manual testing to the above mentioned testing. It could be a combination of Unit/API/UI tests that are run on a daily basis. Regression testing can be performed in various phases of the SDLC, it all depends on the context.
Hopefully this gives an idea on what is regression testing.
You have some code you want to remove associated with an obsolete piece of functionality from a ruby project. How do ensure that you get rid of all of the code?
Some guidelines that usually help in refactoring ruby apply, but there are added challenges because having code that isn't being called by anything won't break any unit tests.
Update: Has anyone written anything that allows you to guess based on your version control history if there are commits where you have since deleted most, but not all, of the code and can point out the remaining code?
Current thoughts:
Identify the outermost part of the stack associated with the obsolete functionality: the binary script calling it, or the unit tests calling it.
Look for methods that are only called by methods associated with the obsolete functionality. I often use git grep for this.
In theory, running mutation testing and looking for code that used to be mutation resistant when the old test suite applied, but is now mutation prone might help. It only helps if your code was well-tested in the first place! (Or you can use code coverage tools such as rcov rather than mutation testing)
Running test suites will ensure you haven't removed anything you shouldn't have!
Using autotest can save you time if you're constantly running tests.
If your code was well-structured, it should be easier to find related methods that need to be removed.
Especially in a dynamically typed language, there is no easy way to do this. If you have unittests, thank the developer that wrote them because it will help you remove the code correctly. But you're basically SOL. Remove the code, if it breaks, put it back, figure out where it broke, attempt to work around it, and repeat.
Look at your code coverage. Any code which isn't covered may be part of the code you have left to remove (if any). (Just be sure you have removed you tests. =])
Environment: (C# WinForms application in Visual Studio Professional 2008)
I've been digging around a little for guidance on NUnit best practices. As a solo programmer working in a relatively isolated environment I'm hoping that collective wisdom here can help me.
Scott White has a few good starting points here but I'm not sure I totally agree with everything he's said -- particularly point 2. My instincts tell me that the closer a test is to the code being tested the more likely you are to get complete test coverage. In the comments to Scott's blog posting is a remark that just testing the public interface is considered best practice by some, but I would argue the test framework is not a typical class consumer.
What can you recommend as best practices for NUnit?
If by point 2, you mean the "bin folder per solution" -- I can see your point. Personally, I would simply add the reference to each test project. If, on the other hand, you really mean (1b) "don't put your tests in the same assembly as your code" I heartily agree with him and disagree with you. Your tests should be distinct from your production code in order to enhance code clarity and organization. Keeping your test classes separate helps the next programmer understand it more easily. If you need access to internals in your tests -- and you might since internal methods are "public" to the assembly, you can use the InternalsVisibleTo construct in the Assembly.cs file.
I, too, would recommend that, in general, it is sufficient to unit test only the public interface of the code. Done properly (using TDD), the private methods of your code will simply be refactorings of previous public code and will have sufficient test coverage through the public methods. Of course, this is a guideline not a law so there will be times that you might want to test a private method. In those instances, you can create an accessor and use reflection to invoke the private method.
Another recommendation that I would make is to use unit testing and code coverage in tandem. Code coverage can be a useful heuristic to identify when you need more tests. Lack of coverage should be used as a guide to indicate where more testing may be needed. This isn't to say that you need 100% coverage -- some code may be simple enough not to warrant a unit test (automatic properties, for instance) and they may not be touched by your existing tests.
There were a couple of issues that I had with the article. Probably the biggest is the lack of abstraction away from the database for unit tests. There probably are some integration tests that need to go against the db -- perhaps when testing trigger or constraint functionality if you can't convince yourself of their correctness otherwise. In general, though, I'm of the opinion that you should implement your data access as interfaces, then mock out the actual implementations in your unit tests so that there is no need to actually connect to the database. I find that my tests run faster, and thus I run them more often when I do this. Building up a "fake" database interface might take a little while but can be reused as long as you stick with the same design pattern for your data access.
Lastly, I would recommend using nUnit with TestDriven.Net - a very useful plugin whether you're doing nUnit or MSTest. Makes it very handy to run or debug tests with a right-click context menu.
My instincts tell me that the closer a
test is to the code being tested the
more likely you are to get complete
test coverage. In the comments to
Scott's blog posting is a remark that
just testing the public interface is
considered best practice by some, but
I would argue the test framework is
not a typical class consumer.
If your code cannot be tested using only public entry points, then you have a design problem. You should read more about TDD and SOLID principles (especially single responsibility principle and dependency inversion). Then you will understand that following TDD approach will help you write more testable, flexible and maintainable code, without the need for using such "hacks" as testing classes' private parts.
I also highly recommend reading Google's guide to testability by Miško Hevery, it has plenty of code samples which cover these topics.
I'm in a fairly similar situation and this question describes what I do keep-your-source-close-and-your-unit-tests-closer. There weren't too many others enamoured with my approach but it works perfectly for me.