OSGI Testing frameworks - some observations.
I am in the middle of writing a system that makes use of OSGI. However all of the popular testing frameworks (Spring-OSGI, PaxExam) require a lot of artefacts for even the simplest test.
Wishlist / Goals
Ideally i would like a single Test that uses TinyBundle to assemble bundles and gives them to the framework. The framework will then do the heavy lifting of starting the container, deploying, running each test, updating the ui to show results etc.
On the surface it would appear that PAX-EXAM would satisfy this but it has additional requirements, which I have not been able to solve, within Eclipse. My problems are:
Each bundle requires a separate project.
Each project gets a manifest.mf in the $project/meta-inf/.
Ideally i would like to bundle all my manifests and "internal" classes in separate sub packages of the test rather than having them scattered about in their respective projects.
I have found packaging everything into the one project just does not work when the tests
execute even if the bundles provisioned are identical in content. However if i split everything into separate projects stuff just works.
Maven
I wish to avoid maven as this implies that a more complex system that would end up requiring building, deploying into the repo which in the end even when automated just slows things down even more. This would conflict with my use of Infinitest which automagically detects changed classes and just executes the right tests.
Eclipse Project Plugin launch configurations.
This approach requires one to pick the bundles to deploy prior to executing the junit test. This of course only works if one has separate projects with a one to one mapping per bundle. Again this goes against my attempts to consolidate all test dependeny bundles under one project.
How, Can it be done ???
How can i achieve this ?
Is this practically possible ?
Whats the simplest alternative ?
One alternative, which also leverages Pax Exam, is what we did in an OSGi testing framework (that tests OSGi framework implementations). Rather than duplicating the whole explanation of how it works, you can find that here:
http://opensource.luminis.net/wiki/display/OSGITEST/OSGi+testing+framework
It does not use Maven, and writing new tests is explained here:
http://opensource.luminis.net/wiki/display/OSGITEST/Writing+a+framework+test
Maybe some of the solutions can inspire you. All in all, there are many test frameworks out there for OSGi (just like there are for non-OSGi) but so far there has one been any one that "makes the rest obsolete".
Related
This is not related to update the pom dependencies using a maven goal. I already have that sorted out.
So I am responsible for creating, packaging and maintaining common utilities. These common utilities are in turn used by all the teams in the org. Problem is that the teams using these utilities do not update the dependencies unless it is the last resort. We would like them to use the latest release version of our common utilities barring very few.
Now I have come across this Maven Versions Plugin by mojohaus which I think serves my need by using two goals - versions:update-properties and versions:use-latest-releases. It serves my purpose except two things:
I do not see a way to exclude certain groupid:artifactid from the update dependency/property
We really want this to be a compulsory thing (maybe part of the Test execution - this is for test automation utilities mainly) rather than a maven goal. Because if it is a maven goal, it needs to be invoked separately and hence becomes optional for teams.
We know that forcefully updating to latest version might cause some issues with defect re-produciblity, but we are willing to take that risk. Our utilities are really test products.
Any direction/help on this is appreciated.
Edit: We run our tests using maven goals clean install already. So they use existing pom. We want dependencies update to happen before the tests run. Also it is desirable to commit the changes to source control (bitbucket) if possible.
We have our tests setup using Jenkins but teams also run multiple test on local machines.
Edit: Found the answer to #1. The plugin provides to exlude regex for group and artifact id. using tags excludes and excludesList
Sorry for insisting. From the Maven point of view I see two main solutions:
Your projects and your utility jar are tightly coupled and every project always needs to use the latest version. Then you can bundle all projects and your utility in one multi-module project. This makes sure that everything is up to date all the time, but it requires that all projects and your utilities are always build together (not separately).
You distribute your utilities to different projects which build and release at different times. Then it is up to the projects to decide when to update. There is unfortunately no standard way to deprecate jars.
If I understand you correctly, you want something "in the middle". This may be hard to achieve.
I'm fairly unexperienced and all new to the whole world of build tools so here's my situation: I am developing a webapp with JSF, PrimeFaces and Hibernate on wildfly-9.0.2-final. All java files (incl. ManagedBeans, DAOs, Model classes, etc.) are currently in a regular eclipse java project called MyApp-CORE. There is no html or any other resources in that project, but all the third-party libraries like PrimeFaces, commons-xy, etc. Then I got two dynamic web projects with all the .xhtml files and stuff. Both web projects include the CORE in their build path (all done via eclipse built-in tools). Basically I followed Structure for multiple JSF projects with shared code so far. All projects are versioned using Git. I was now asking myself how to mavenize the whole thing and also how to properly include tests. The final result should be:
I want a build file for each web project that includes the CORE dependency and all of it's transitive dependencies, creates a .war file and deploys it either on the production system or locally (Depending on some parameters I want to be able to maintain).
This buildfile could then for instance test and build the CORE and then the .war file.
Since I'm using JSF, mostly the only option for testing is JSFUnit. Should I test each web project individually and put all the test cases there (which would be highly redundant because they're mostly the same, just a few features are different) or should I rather create a separate web project called MyApp-TEST which tests the CORE.jar and also - depending on some configuration - each web project.
I've already created a structure that makes it possible to include the core in the web project but unfortunately I loose the perks of hot deployment in wildfly when just including it as a dependency from my local maven repository.
So, to summarize it:
What would be a best practice for this setup, eventually leading to a continuous integration scenario?
How should I include the test cases (full integration tests that test actual UI behaviour)?
Which Tool (Maven, Gradle, Ant, etc.) would be best for that task?
Keep using hot deployment for smooth development?
Thanks in advance for any comments, hints or shared experience!
We have a very big web application containing many features.Now for maintainability we want to split the application in components so that can remove / add particular components (jars). For that one suggestion is coming is to use OSGI. I think converting jars into bundle will take huge effort. I think same functionality can be achieved by Maven. According to my understanding OSGI is packaging tool. If I can make Maven plug-in for each component then any particular component can be included or removed at compile as opposed to run time as in case of OSGI.
Modularizing the application using Maven will be simpler than OSGI. I have read similar post on this site and it commented that OSGI and Maven are like comparing apple with orange. But I think in one sense both are same as they both meant for packaging difference is one is used at run time and one for compile time
Looking forward for well though answer :)
best wishes
Shailesh
As you already hinted at yourself: you're comparing apple with orange.
OSGi is not a packaging tool.
OSGi bundles are plain JAR files with some OSGi-specific metadata in the Manifest file.
You can create OSGi bundles using Maven e.g. using the Maven Bundle Plugin (I can recommend this approach). So regardless if you're using OSGi or not I strongly recommend using Maven.
Here some use cases for OSGi:
You want to create different versions of your application e.g. for different customers. With OSGi you can just add/ remove bundles without having to touch any other configuration.
You need a plugin system so 3rd parties can provide plugins to your application
You want your application to be truely modular
You want to share some code with other applications but want to hide some internal classes
...
OSGI is much much more than a packaging tool. You could say that OSGI has a packaging tool inside. Maven is a packaging tool and a dependency manager. I'd say that, given the level of complexity and the use you say you'll make of this technology, go with Maven.
I'm a bit surprised at the sheer number of articles / blogs / questions / answers about Clojure mentioning Maven.
In about ten years working as a Java dev, working on both desktop apps and webapps, I've never ever used Maven once (typically --and that's a personal opinion but I know some people do think the same-- the projects I've seen using Maven where including the "kitchen sink" whereas the ones built with a more "controlled" build process where way "cleaner" and producing smaller jars, faster build time, etc.).
Is Maven a requirement when you want to build a Clojure app?
Is Maven mandatory when you want to use Leiningen? For example, can I add external jars as dependencies to a leiningen project "manually", without needing Maven?
I think my question boils down to this: can you, in the Clojure/JVM world get away without ever using Maven, just like you can build, test, package and ship both Java desktop, webapps and Android apps without ever needing Maven?
Short answer: No. Clojure's just a jar and you can use it as "raw" as you like, just like any other Java library.
Longer answer: Maven's not a requirement, but the tooling around Clojure, especially Leiningen, is highly Maven-aligned so your life will be easier if you just submit to Maven's will. But, with a little work, it's not that hard to get along without Maven. At work, I use a mix of Leiningen and our existing Ant/Ivy-based build infrastructure. I use Ant to resolve dependencies (from our curated internal repo) and then use a hack of Leiningen's :resource-paths to get it to pick up the non-Maven jars. At some point I'll make a true plugin to do this stuff, but it's been working fine for me so far.
Also, if you're an Eclipse person and use CounterClockwise, you can treat your project like any other Java project in Eclipse, managing the classpath manually. It just happens that you've got some Clojure code in there as well.
Of course, the drawback in either approach is that if you want to grab something that's available from either Maven central or Clojars, you'll either have to set up some kind of mirror for your infrastructure, or manually pull down the transitive dependencies and add them to your project.
You do not have to use maven, or leiningen, for that matter. You can run the clojure REPL with
java -jar clojure-1.3.0.jar
and it will work just fine. I know where you're coming from, because a year ago, I was in the same boat as you. never used maven, seen a couple of less than great projects that used it poorly, and generally distrusted maven. Ant+ivy works great, who needs maven?
As a result of using leiningen, and starting a new job where we have a very well setup maven config, I've changed my tune completely; I now think maven is great, and far prefer it to ant, which I've used for a long time.
Specifically with clojure, some advantages of leiningen are:
automagic project scaffolding: "lein new" sets up a new project for you. convention is important, it helps other developers understand your project and quickly ramp up on it.
dependency and plugin management. adding a dependency is trivial, and there isn't a great way that I am aware of to do non-maven dependency management that is tightly integrated with clojure.
"lein repl" sets up your classpath and everything correctly, so you don't have to fool with it, you can just start and run a REPL.
artifact creation: building a fat jar (lein uberjar) is straightforward, and already set up for you.
So, while it's definitely possible to use clojure without leiningen, I honestly don't see why you would want to. There are too many niceities for day to day development not to use it, in my opinion.
regarding "Is Maven mandatory when you want to use Leiningen?"
leiningen uses and wraps Maven so you can't truly avoid it, though you don't ever need to touch any XML
Maven is not mandatory for Clojure as others have rightly pointed out.
However, I would like to encourage you to embrace Maven (either directly or via leiningen). It has some big advantages in the Clojure world:
You will be more productive in the long run - once you've got it set up, it is pretty impressive to see a complex build, test and deployment execute in a single command. Do you really want to be manually setting classpaths, downloading dependencies, incrementing version numbers, uploading to FTP sites etc.?
One of the biggest advantages of using Clojure on the JVM is access to the Java library ecosystem. Maven is the tool of choice for automatically managing dependencies on Java libraries. Nearly everything you are likely to need will be in either the Maven Central or Clojars repositories.
Clojure tends to have small focused libraries - this is good because they are "simple" and you can pick and choose the functionality you like. At the same time, it also means that you are likely to have numerically more library dependencies. Hence a build management system is going to be more useful.
You can make Maven builds as "lean" as you like. Obviously you can pull in every dependency under the sun if you like, but you can also create very lean jars by excluding the unnecessary cruft. That's your choice.
It is the de-facto standard in the Clojure world. If nothing else, using it means that you will be able to collaborate with others more easily.
P.S. Like you, I didn't really see the point of Maven 5 years ago. Then I gave it a go and saw how powerful it could be for my workflow. Now I'm a convert :-)
There are two main approaches when developing an OSGi application with Maven: POM-first and MANIFEST first.
I'm looking for an answer that is in a form of a table that shows pros and cons of each method.
To be more specific, I would also like to know how it relates to:
Maturity of toolset
Vendor independence
Development ease (which includes finding people who can do the development on the tooling)
Compatibility
Avoiding ClassNotFound
Avoiding manual work
At present this is what I can come up with
POM-First Pros (using maven-bundle-plugin)
Leverages existing Maven skills, repositories and tooling.
Likely easier to find people who know how to manage pom.xml rather than MANIFEST.MF along with pom.xml
Most of the information in MANIFEST.MF can be obtained from the pom.xml itself.
Can work with other IDEs not just Eclipse based ones.
Less invasive, just add the single plugin and change the packaging type to "bundle"
POM-First Cons
ClassNotFoundException more likely to occur at runtime. However, this can be mitigated using pax-exam (although it is very complicated to set up).
Still need to understand how the MANIFEST is setup to make sure the instructions configuration element is set correctly.
MANIFEST-first Pros (using tycho-maven-plugin)
Seems to be the recommended approach, or at least talked about as the recommended approach, but I can't really see why it has significant benefit. (Hence why this question was asked).
Good for developing Eclipse plugins and integrates well with PDE
Provides tooling for testing thus allowing ClassNotFoundException to appear during JUnit testing rather than runtime.
MANIFEST-first Cons
Seems to only work well on Eclipse based IDEs. You don't have to use Eclipse, but without the PDE would you want to?
Violates DRY principles since I have to do put keep the names and versions from the POM and MANIFEST.MF in sync.
Need to name things in a specific fashion
You cannot mix, meaning existing Maven multi-project installations cannot just tack on OSGi support
A lot more configuration compared to maven-bundle-plugin is needed to get less warnings: http://wiki.eclipse.org/Tycho/Reference_Card#Examplary_parent_POM
Have to make test cases a separate project. It won't run when built in src/test/java.
Seems that it will only test classes that are exposed, in other words those in ".internal." is not testable.
If I were asked for a recommendation for an enterprise that is using Maven already and want to move to OSGi then it would be POM first
If I were asked for a recommendation for someone who is doing Eclipse plugin development, then it is Manifest first -- with tycho
I think you should choose by use case. For server side OSGi projects I favour the pom first style. It nicely matches the maven builds and is much less error prone than Manifest first.
In fact bnd which is behind the maven bundle plugin gets the Manifest right for most cases without any additional config. The trick is to use some naming rules. For example if you name internal package impl or internal the will not be exported. Using this style you can not use the Eclipse plugin perspective (at least without bndtools which I do not like) but I did not yet miss this perspective. I am a developer in the Apache Karaf, CXF and Camel projects where we use this style and it works great. Especially for CXF and Camel it is great that we can support OSGi and non OSGi deployments with the same build and tools.
For Eclipse RCP applications Manifest first is the way to go as you need the plugin perspective and the Eclipse IDE tools. If you want to combine that with maven then tycho is probably the way to go.
MANIFEST first does not lock you to Eclipse (although I'd be surprised if more than a tiny minority would use anything else). The MANIFEST is the file that counts, and needs to be added to a jar, regardless how you do that.
On the other hand, POM first completely locks you to Maven, you lose the advantage that an OSGi bundle is a regular jar you can make any way you want.
I've tried both, I really prefer MANIFEST first. The MANIFEST file is a really important file, I prefer to craft that file over crafting a file that produces that file. If something weird happens, (and it will at some point) the MANIFEST file is the first to check, it's just easier if it's your own file. Besides, you will have to be familiar with it anyway.
So, if Maven is your alpha and omega, POM first will suit you best, but you'll still need to have in-depth understanding of the MANIFEST file.