Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'm looking to enhance our current test suites, and continuous integration builds with full stack integration/acceptance testing.
I'm looking into tools like Culerity and Selenium that can execute front end javascript while running user stories. I'm looking for something that can provide coverage of front-end javascript and high level features without sucking up tons of development time maintaining a complex test environment. We're currently using Rspec, Cucumber, and CruiseControl.rb, so easy integration with those tools would be ideal.
Are any of the headless browsers and js-capable test environments to a point where they are worth the trouble of setting up and maintaining? What are the best option you've come across, and pitfalls to avoid?
Thanks.
You sound like you are way further down this road than I am, but I'll comment anyway.
I am working on a JavaScript project (with a Java + MySQL back end) and decided to use Selenium for testing, and to try to achieve as thorough coverage as I could. I also poked around with a few other testing tools, but I can't say I really got to know any of them. None of them appeared, from their web sites, to be very polished or popular compared to Selenium. I am planning to integrate to CruiseControl eventually, but haven't done so yet.
This has been an interesting project and at the end of the day, I am quite happy with Selenium. Selenium plusses:
Test 'scripts' can all be written in Java, no obscure scripting language involved. Among other things, you can easily do things like manipulate and verify the data in your database before and after tests.
Se also supports Perl, C#, etc. I think, although that is of no interest to me.
Selenium IDE is a great tool for quickly understanding how Se works, how locators work, etc. You don't want to actually run tests long-term using the IDE, but it's great for getting your feet wet, and for ongoing figuring things out.
Se seems to work flawlessly with jUnit. Probably TestNG as well, but have not tried that yet, it's on my todo list.
Excellent documentation and web site.
Minuses:
I spent a LOT of time figuring out how to locate elements in all cases. This is partially the 'fault' of the framework I am using (ExtJS), not Selenium.
It seems no matter what you do, Se has timing dependencies - eg. places where you have to inject artificial pauses to make it work.
There are also monitor-size dependencies in my tests. I think this is extremely undesirable but in some places it seems to be unavoidable. Basically, this is because there are many element types that JS doesn't support you clicking on programatically.
Related to #3, in places I am forced to drive the mouse. That means you have to have a dedicated test PC. Which is no big deal, but doesn't seem right.
Tests are slow - mainly due to the time it takes Se to invoke Firefox. No doubt this is partially my environment, and I suspect I could do lots of things to improve this. However, it is really noticeable and not obvious why. It takes about 10 minutes to run about 40 tests.
Support forum is very spotty. Well, you get what you pay for. But time and again I found someone had posted about my problem, and the post was ignored or else an invalid solution was offered with no follow-up when the OP pointed out that the suggestion was bogus.
HTH, cheers.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions concerning problems with code you've written must describe the specific problem — and include valid code to reproduce it — in the question itself. See SSCCE.org for guidance.
Closed 9 years ago.
Improve this question
I'm writing a gem to wrap the xmpp4r RubyGem. I want to add tests for everything I can, but I'm new to testing and a little stuck here.
Essentially, I want one of my classes to have a 'connect' function, but I have no idea whatsoever how to test such a thing...all the examples I can find online about testing are over-simplified. it "is green broccoli", etc., you get the idea.
How can you test something seemingly more complex? Links to good documentation on this stuff would be nice, as well as more personalized answers from your own experience.
EDIT: Possibly letting the test pass if it doesn't encounter any exceptions? Would I be on the right path by doing this?
Just remember you as a programer, you are the lord of your own code world.
The same way you build applications easy to use for others, you can help yourself and make your code easy to test. One of the main benefits of doing OO programming is that it will encapsulate behaviours into methods, then is easier to test your code using those methods as key testing points. If your code is hard to test, it is becuase you are not seeing yourself or the tester as another user of you application/code.
Testing is not directly related to any framework or technology.
Try to see it as something separated from the code itself, something bigger than that.
If you do not know how to test something is because you most probably do not know why you need it and how it is going to be used.
Try doing some TDD (Test Driven Development) and you will get my point.
Also keep in mind there are different types of testing:
Functional testing / High Level
Unit test / Low Level
UAT / User Acceptance Testing
Performance Testing
Exploratory Testing / Hand testing
Integration testing (you can build some mockups of the pieces that you code will work with to anticipate any problem prior the integration)
Testing is expensive, so it is not a matter to test everything is about test what needs to be tested.
Even the NASA do not test "everything"
Testing is ...deep as programming.
Nice dive!
http://en.wikipedia.org/wiki/Software_testability
First, check if the gem itself is tested. If it is and tests are passing, you should probably trust the gem's code. Then, you can just mock any calls to the gem and return the appropriate calls when testing for success or failure scenarios.
i.e.
xmpp4r.should_receive(:connect).and_return(mock("xmpp4r"))
If the gem is not tested, it's about the same as you didn't have any tests yourself. You should consider using another gem or testing it yourself (or even create one using TDD).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
The other day I was showing one of the testers at my company some tests I had written in cucumber (2 features, 5 scenarios).
Then he asked me question that I could not answer:
How is this better than selenium or any other functionality test recording tool?
I understand that cucumber is a different technology and it's placed at a different level of testing, but I can't understand why I should bother to write and maintain Cucumber/Capybara tests.
Can someone give me a reasonable explanation for using Cucumber/Capybara instead of just Selenium?
This question is borderline asking for an opinion. Your question actually reads to me, "What tool is right for me?" I say this because you don't give a reason for why you chose Cucumber and Capybara. I believe to answer that tester's question, you need to answer a couple more questions first:
1.) What stage in the process are you going to be writing these tests?
Cucumber may not be the right choice for unit tests, depending on the language you're using. But it can be used for any level of testing, from unit to integration to end-user.
2.) Who is going to maintaining your tests? You? Other developers? Testers? Business Analysts? Project Managers?
Automated tests must be maintained, and knowing who will be doing that can help you decide on a tool - as some will be too technical for certain users.
3.) Who is going to be defining new tests?
Cucumber is meant to be used collaboratively between development, QA and business owners. It is the perfect tool for leveraging everyone's knowledge into the automated testing process. It requires the development of an ubiquitous language to be effect however. You can read up on that on James Shore's Art of Agile page.
Once you've answered these questions, you're ready to address the tester's question.
However, there are a couple of points to keep in mind when comparing recording tools (such as Selenium IDE, HP Quick Test Pro, IBM Rational Functional Tester) vs. development tools (nUnit, jUnit, RSpec, Selenium webdriver, Capybara) is that they are targeted towards different audiences. They also have different plusses and minuses.
Recording tools are easy for anyone to use, but the scripts they create are fragile. They break easily and require more maintenance. They are great for one-off automated testing, where you need to get it done quickly and have non-technical manpower.
Development tools have a larger learning curve and require programming (or at the least scripting) experience. The scripts are generally more robust, but require more technical knowledge to maintain. They are a good solution when you want repeatability and plan to use tests for a long time.
I strongly suggest you read The Cucumber Book. It will really help you decide if Cucumber is the right choice for you.
Cucumber isn't only a testing tool. Besides testing Cucumber features also take a role of documentation, a mechanism to collaborate with stakeholders and requirements storage (if you write them in declarative style).
You don't have to use Cucumber with Capybara. You can use selenium directly. But Capybara has the same high-level API for all supporting drivers. It's more high-level than Selenium's and allows to write tests a bit faster. You don't have to change code when you switch from one driver to another.
Tests built using test recording tools are generally much worse. Selenium IDE may produce valid programming code but it's not good-looking and thus quiet difficult to maintain.
Cucumber is tool used to make tests readable to business users. It consists of plain English sentences that are matched using regex to your Capybara steps.
Using recording tools won't do you any good in the long run. They were meant for beginners and aren't that powerful so I recommend you go straight to coding.
You can use Selenium alone for your tests, but I would recommend you continue to use Cucumber for documentation purposes, if you find them useful and easy to work with. After all, Cucumber can use Capybara or the Selenium web driver.
Selenium ide is good for testing features that have mostly visual elements (links, text and etc.). But often web apps have features that don't represent itself as visual elements, like sending emails, queueing jobs, communicating with 3rd party services and etc. You could, for example, test if an 'Email has been sent' message is present after submitting a form. But it doesn't really tell you if an email is actually sent and therefore you aren't really testing the whole feature here.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I recently started a career in software development after graduating a couple of years ago in CS. The current project I'm on is a large ongoing project that has it's origins in the 90s with a mix of C, C++, and Java. There are multiple platforms (UNIX, WIN, etc) being supported, older technologies in use like CVS, and some dated documentation in some areas.
The extent of my software development skills stem from going to university as I've had little real world experience. I felt like I had a decent foundation in CS but I cannot but help feel slightly overwhelmed by it all. I'm excited to be part of something so huge but at the same time I feel like it's a lot of information to absorb.
My coworkers have been great people and answer a lot of questions I. My employer hired me knowing that I am entry level.
I've tried poking around the source code and examining how everything gets built but it's on a scale I've never seen before.
How do more experienced people situate themselves when joining a large ongoing project? What are some common tasks you do when getting yourself up to speed?
Good question. I haven't had your exact experience, but in cases like this I like to think, "how do you eat a whale?" The answer is (predictably) "one bite at a time." Reasonable people won't expect you to grasp the whole thing immediately, but they will want to see progress. Perhaps there are some small areas of the larger project that are not too complex, without too many dependencies. Work toward understanding one of those and you're one 'bite' (and/or 'byte') closer to expertise on the whole project.
Being familiar with all existing documentation I would try to get the big picture. Literally.
generate a TreeMap of the source code
I would use GrandPerspective on Mac or WinDirStat on Windows. It will give you some insights about the structure of the project's files (sometimes it gives some hints about the code structure). Having this, you can ask your colleagues for some of the clusters, what they do, how they relate to each other.
learn how to build the project
This is important to have it compiling all the time if you are about to do any changes. Having tests executed at the build time is always a good thing, so ask for it also. Even better if there is some kind of continuous integration server in place. If there is, look at its configuration - figure out how the build is done. If there was no CI server, but you already got the knowledge how to build the project, create such a server on your local machine, and show it to your fellows - they should fell in love with it.
browse the source code with Structure101 or similar tool
This is useful especially for Java projects. This tool does great job. That will give you more details about the code structure, and sometimes about the system architecture. This experience may be sometimes hard, you may learn from this tool that a code is basically a Big Ball of Mud ;)
look for tests, and explore them
If you will be lucky there may be some JUnit, or CPPUnit tests. This is always good to try to understand what those tests are doing. It may be a good starting point to explore the code further.
My coworkers have been great people
and answer a lot of questions I. My
employer hired me knowing that I am
entry level.
You have little to worry about, you're employer knows what you are capable of and your co-workers seem eager to help you out - to be honest most developers love explaining things to others...
From what I've seen, it take truly 6+ years to become fully knowledgeable in a language, so don't expect to become a guru within a year... and even these so called gurus end up learning something new about their language everyday.
Learning a new system (large) will always take time.... the systems were usually not built in 2 weeks but over many years, so don't expect to understand it fully yet. You'll eventually discover what each part does piece by piece.
I know how you feel, because I felt like that once...
"I took a speed reading course and read 'War and Peace' in twenty minutes. It involves Russia." (Woody Allen)
I agree on what the others said before me. You need some tools that give you an overview on the code. I personally used inFusion (http://www.intooitus.com/inFusion) because it gives also other interesting data beside structure.
The method that has worked best for me is to grab a copy from source control, with the intention of throwing this version away...
Then try and refactor the code. It is even better if you can refactor the code that you know you will be working on at a later stage.
The reason this is effective is because:
refactoring gives you a goal for you to aim towards. Whereas "playing" an "breaking" the code is great - it is unfocused.
To refactor code you really have to understand the code.
Refactored code leaves code that has less concepts to retain in memory. If you don't understand a large codebase its not because you are a graduate - its because nobody can retain more than 7 (give or take a few) concepts at a time.
If you follow correct refactoring guidelines it means you will be writing tests. Although, make sure that you will be working on the modules that you are testing as writing tests can be very time consumning (although very rewarding)
Do invest in buying this book at some point:
http://www.amazon.co.uk/Refactoring-Improving-Design-Existing-Technology/dp/0201485672
But these links should get you started:
Signs that your code needs refactoring and what refacoring to use (From Refactoring - Martin Fowler)
http://industriallogic.com/papers/smellstorefactorings.pdf
A taxonomy of code smells:
http://www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm
Good luck!!!
I agree to the first comment but I also Think that you have to learn and see the big picture in some way. You have to trace the main flow from code at least.
I was in the exact same situation several years ago when I joined a software project with 50+ ClearCase version control vobs, 5 million lines of code, and some of it dating back to the 1980's.
The first thing I did was look through every source controlled directory and made a quick summary of my best guess about what the software in that folder did and what language the code was. You can make a pretty good guess by looking at filenames and any comments or documents in those folders.
I then looked at the build scripts to see if they were readable enough to get an idea of dependencies between different parts of the code.
Finally - and I believe this was the most valuable - throw an IDE like Eclipse or NetBeans on top of the code and start reading through pieces of it. Having the ability to jump to the definition of any functions or classes using the IDE allows you to move around a massive software baseline with relative ease.
Overall, have some confidence - it is unlikely that anyone else on the project knows all of the code, so you don't need to either. Use what other people said to get a good idea of the overall project and interfaces and requirements (if they exist) and poke through the code to get an idea of the most commonly used classes and methods.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
Sometimes one needs to dig into someone else's code, understand it and maybe refactor/fix it. So I'm wondering what tools/practices do you use to do that? It's not about any specific language or platform, rather some advises in general.
I myself find it very useful to run the code in question under debugger (if it is possible of course), set breakpoints and step through the code.
Also I've used a few UML tools with reverse engineering features to see the big picture. Visualization works pretty well for me.
There's one tool, that I've used quite a bit: Understand. It's not a UML tool, but it has a bunch of visualizations for the code. For instance control flow diagram for a function, which is not part of UML, but sometimes it's very useful.
I've heard that sometimes people write tests for the code they want to learn, but it hasn't been working for me. Sometimes the code I want to understand is not very testable.
There are other tools, like profilers, dependency checkers, loggers etc. Does anybody use it with any success? And how?
What the first thing use start with? What's the next?
I have been in this situation before... It's not much fun. The best way I've found is to write unit-tests to cover the areas that are going to change. Even if tests already exist, this gives you experience with both the api and behavior of the code you intend to refactor. Additionally, you gain more tests to verify the replacement implementation.
Typically, I divide these kinds of requests into 2 camps:
Fix the bug, not the whole code. In this case, I'd tend to read the code and see if I can easily put in the fix without making major significant changes, e.g. adding 5 lines of code or less. This is where the changes made are like a surgical strike that does minimal impact to achieve the desired result.
Tear down the code and put up much better stuff. In this case, there isn't a good little way to do the change or there are lots of little changes that want to get bundled up into a mini-project where digesting all the code is needed and this isn't a quick minor change. Taking an ASP site and putting it into the latest .Net framework would be an example here.
Determining which kind of change something is requires a bit of practice and experience understanding what someone else may call, "One more thing if you could..." that may be a huge deal but they don't see it that way,e.g. could we add a little module that records any error and sends me an e-mail when something bad happens that may not be so easy after all the other requirements get worked out like what if the code can't send a mail message what should it do.
Do check out nWire. It is an Eclipse plugin, currently for Java & PHP, which analyzes your code and offers code exploration, visualization and search tools. The tool is about understanding the associations in your code: class hierarchies, invocations, instantiations and pretty much everything else.
I find it very useful for digging into piles of code (BTW, for me, even my own code becomes alienated after a couple of months of not touching it).
Are you looking for high-level structure, like modules, namespaces, and class design? Or are you trying to understand the internals of the classes, e.g. how specific lines of code work? Those are two different kinds of analysis tools.
Sort of platform specific, but ReSharper is great for teasing out the hidden meaning of .NET code. I love the "Find Usages Advanced" feature where you can look for sets, gets, implementations etc...
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
We're currently not applying the automated building and testing of continous integration in our project. We haven't bothered this far as we're only 2 developers working on it, but even with a team of 2 I still think it would be valuable to use continous integration and get a confirmation that our builds don't break or tests start failing.
We're using .Net with C# and WPF. We have created Python-scripts for building the application - using MSbuild - and for running all tests. Our source is in SVN.
What would be the best approach to apply continous integration with this setup? What tool should we get? It should be one which doesn't require alot of setup. Simple procedures to get started and little maintanance is a must.
Have a look at JetBrains' TeamCity. Free for a small team like yours. Easy to install and minimal fuss. And it looks good. Far better than CruiseControl.NET.
CruiseControl.NET is good too, but definitely requires more work to get setup.
I've been using Hudson (open source software) and found it really flexible. It's more popular in the Java community, but there are MSBuild and MSTest plug-ins available. Hudson also makes it easy to schedule builds or run builds when changes are checked into svn. I found this blog very useful as a starting point.
Cruise Control .NET
Try Cruise (http://www.thoughtworks-studios.com/cruise-release-management) (the re-write of CruiseControl.NET) by Thoughtworks. Its very sexy, much easier to get going and very nice to use. Great feedback too. And its free for teams of less than 10.
Even with two its a great tool to have and once you've done it once its much easier to set up other projects. Having it build, fresh, from SVN when you check in and then tell you everything is ok is a really nice feeling thats easy to get used to.
Allow a good two days though for any build system to wire it all up, thats not installing thats just getting everyhing wired up as it should be. Trick is to do baby steps, get it checking out your code and add more and more layers as you go. Once you have a base set up you can add the other bells and whistles when you get time until after a week or two you have the whole thing singing and dancing. Sounds like a lot of work but its well worth it.