I've been set a task of writing a grammar / DSL with XText. It seems reasonably simple. I've been asked to incrementally add rules to the grammar as per a specification. I want to be able to test each new rule that I create.
I have Spring Tool Suite Version: 3.9.2.RELEASE, Build Id: 201712210947, Platform: Eclipse Oxygen.2 (4.7.2).
I've seen textboxes in web browsers that show syntax highlighting based on the validity of the input as per the grammar rules. Also, I'm aware that unit testing is a possibility.
What's the simplest way, minimal fuss, of verifying that the grammar works as expected?
Your help is greatly appreciated.
Thanks in advance,
Ryan
Writing a bunch of unit tests using ParseHelper and ValidationTestHelper should be the most sustainable way to do this.
e.g. https://github.com/eclipse/xtext-eclipse/blob/master/org.eclipse.xtext.xtext.ui.examples/projects/domainmodel/org.eclipse.xtext.example.domainmodel.tests/src/org/eclipse/xtext/example/domainmodel/tests/ValidationTests.xtend
The answer depends on what you mean by 'verifying that the grammar works as expected'. As Christian noted, unit tests using ParseHelper and ValidationTestHelper will tell you if your grammar is being parsed into the correct model and generating the appropriate errors otherwise. If however you want to test things like content assist, syntax higlighting and such, you have to go a little further and write some tests using AbstactEditorTest, AbstractOutlineTest, etc. in the ui.tests package for your Xtext project. After looking at a lot of tutorials I finally went ahead and bought Lorenzo Bettini's book:
Implementing Domain-Specific Languages with Xtext and Xtend - Second Edition
It was a great help in understanding Xtext, and how to test a grammar as you create it.
Related
I want to develop a sonarqube plugin for analyzing a new language. The language's grammar is developed. There is a parser available for it. The parser throws out an AST.
I want to write visitors for this AST and then develop rules. Is there a sample plugin(I saw the github sonar-examples but I didnt find it to be clear) or good documentation on what steps I should take?
Guidance from people who have developed Sonarqube plugins for languages much appreciated!
This Supporting New Languages outline should get you started. Beyond that, go to the Google Group to ask specific questions.
A comprehensive test suite would be a valuable tool to have, especially when evaluating all of the variant parsers out there. Does such a beast exist?
In a perfect world, I imagine it would have different sections for different versions of the YAML spec...
There is currently one in the making, see here.
We also generate a result matrix for YAML implementations which
we know of
are written in a language we know so we can implement adapter code to validate the test suite against it
Full disclosure: I am the author of NimYAML and some of the test cases and adapters.
Are there any useful plugins for source code metrics for Hudson/Jenkins?
I'm looking for total lines of code, total number of tests, classes, etc. with graphing.
Does anything like this exist?
Are you using Java? If so, SONAR should certainly be your first port of call. It does a lot on it's own and also wraps up all the major Java analysis tools, such as:
Out of the box, you'll get metrics on:
Potential Architectural & Design issues
Unit test coverage (uses cobertura)
Lines of code\packages\classes etc
Potential bugs
Code duplication
Adherence to code formatting standards
(plus many more)
It allows you to traverse from the high level analysis through to the source code it relates to. It will be easier if you're using Maven for your build though...
There is a Hudson plugin. And it's free.
Try CCCC (http://sourceforge.net/projects/cccc/). It does code counting, module counting (classes), etc., and the plugin also graphs it for you. (for C, C++)
Incidently, what language are you looking at?
There's also CLOC (Count lines of Code) which will tell you how many lines of each language you have, although I can't seem to find a link for it.
You don't specify which language you are using, but Redsolo's awesome blog post Guide to building .NET projects using Hudson shows you how to use FxCop and NUnit on Hudson to give some of what you are looking for. The Violations plugin used also supports Simian, CPD, PMD and PyLint.
I was wondering what would be a good UI to specify test cases.
Currently we use macros with excel to specify our test cases and generate an xml out of it and export it to the script generator.
Excel is good and really flexible and allows testers to enter their test cases very quickly.
However the xml generated is sometimes not well formed and the system has a huge learning curve.
I want to change the UI from excel to something else that would allow testers to enter test cases quickly and provide flexibility.
A nice TDD tool is SLIM/FitNesse. It is a wiki system which allows to enter special tables and/or commands which trigger test methods. These test methods can be written in Java and .NET (other languages might be supported). Also there are various plug-ins for doing DB testing or Selenium web tests. Here is a first tutorial video.
I've used Test Link for this sort of task. It's an opensource php project.
You might check out Fitnesse, which does a similar thing. http://fitnesse.org/
Recently, I thought about how can I improve the quality of the projects, by using Continuous checking of xHTML source at Continuous Integration machine.
Look, we have a project
http://sourceforge.net/projects/jtidy - jTidy
JTidy is a Java port of HTML Tidy, a HTML syntax checker and pretty printer.
It can validate the xHTML through a command-line interface. Or this tool can be extended in the way we need, because all source code are open.
We can overwrite every Selenium validation method, such as assertTextPresent, or any other, so it will be calling the jTidy(by providing current state's HTML source), and if some errors or warnings will occur - it can be saved to Continuous Integration machine build's logs - so any project's related can see this info.
We can not to rewrite all the Selenium methods, to integrate this call on every step, but to make this calls where we want(after DOM manupulations).
Yes, we can use W3C markup validators for our sites, but there isn't any possibility to validate not initial state of page's source with this validators. After page creation, there might be lots of DOM manipulations which can produce markup errors/warnings - we can find it immediately with this scheme.
One of the benefits of using Continuous integration is that you have quick feedback from code - how it integrates with existing code base, test whether unit and functional tests pass. Why not to get an additional useful info, such as instant xHTML markup validation status. The earlier we identify the problem, the easier to fix it.
I haven't found anything on this theme in google yet.
And want to know, what do you think about this idea?
Seems like a worthwhile idea.
I've done two similar things with CI before:
I've used Ant's XMLValidate task to validate static xhtml files as part of the build process
I've used httpunit to pull pages that I then parsed as xml
But the idea of tying into Selenium to validate the content inherently during a functional test run is novel to me.
I think, that idea is brilliant but it is very hard to implement it from scratch.
But this idea is like evolution of build/quality validation process, so it will be released as ready-to-use tool with documentation someday.
Good idea! - in fact I just had exactly the same idea and was just checking to see if anyone had done it before - argh! Looks like you beat me to it :)
I was thinking along the lines of capturing and auto-submitting each page hit by selenium to the w3c HTML and CSS validtors (by file rather than link so state is held) - failing on any errors. I like the jtidy idea though.
Great in principle, but I'm not quite sure how to call it from Selenium. I'd love to see documentation explaining how to run it from Selenese, or from PHPUnit.