Overriding openshift maven script (jenkins gear) - shell

I have Jenkins instance on Openshift
I've enabled Jenkins build of my Openshift application
Here part of Jenkins shell script parts and logs what they produce :
# Sync any libraries
rsync $upstream_ssh:~/.m2/ ~/.m2/
# Build/update libs and run user pre_build and build
gear build
command above triggers such action
+ gear build
Found pom.xml... attempting to build with 'mvn -e clean package -Popenshift -DskipTests'
Apache Maven 3.0.3 (r1075437; 2011-06-20 13:22:37-0400)
Maven home: /etc/alternatives/maven-3.0
I want to customize maven options (in this case allow to run test by deleting -DskipTests part )
Further in script I tried to add some steps combining with disabling this gear build command
# Run tests here
mvn -e clean package -Popenshfit
Above run mvn clean package but doesn't update last successful artifact, and logs shows
[WARNING] The requested profile "openshfit" could not be activated because it does not exist.
I have that profile in my pom.xml so I don't know what happening
<profiles>
<profile>
<id>openshift</id>
This is doesn't working to:
mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean package -Popenshift
Probably above is some old staff (taken from https://www.openshift.com/blogs/jenkins-polyglot-persistence-part-2)
Any idea about customizing this gear build or overriding maven options on Openshift are welcome :) I can make changes directly on machines (jenkins, jenkins-build) (as far free openshift account allow me :))
Edit
Some workaround is to run
gear build and then mvn clean package so I get build run twice and all test goals are meet and war is deployed on upstream instance and additionally Cobertura reports work fine with my Jbehave tests but this is inelegant, resources consuming solution.

I found this question while "Doing my homework" with regards to another question about Maven in OpenShift -- as to how OpenShift uses the Maven profile during the build process, broadly, as towards a goal of installing a "customized" third-party .war file providing the Liferay portal, along with some 'module' customizations, in an app using the 'jbossas' cartridge.
Although my portal-gproj app doesn't use Jenkins in its build -- as I expect that it won't have a spare gear available for Jenkins, initially, of the three "free gears" available to start with. It's a scalable app, though. However, I've found a reference item, as a blog entry by Bill DeCoste such that at least mentions two configuration features about Jenkins, as with regards to customization of OpenShift Maven builds in Jenkins: http://blog-judcon.rhcloud.com/?p=16 To summarize, the article mentions, with regards to Jenkins: One "Build/Execute shell section of the Job configuration."
Though I've not used Jenkins, myself, but I've seen it in use, however -- I think it's used in the eXo Platform for instance. I assume that Jenkins would have a sort of web-based interface for job configuration? (I tend to prefer the shell/filesystem interface onto Maven, personally)
Alternately, there is the OpenShift Origin Server source code, and an OpenShift Origin System Architecture Guide corresponding, as in reference to the OpenShift Origin baseline, on which (I assume) OpenShift Online is running, in its layer on the Amazon EC2 cloud. "If all else fails, there's the source code," just like in so many things Red Hat, LOL. It seems that there are both jenkins and jenkins-client cartridges under OpenShift Origin. Certainly, the details of the process used by either of those cartridges would be apparent from its source code.
Hope this helps! Coffee cheers.

Related

Jenkins & Maven - build process

I am learning about Jenkins and I have to explore some existing build jobs that others wrote (in the company that I'm working).
So I am trying to understand a job which uses mvn command.
So under the build part (inside the job), I see these details:
Maven version: 3.0.5
Root POM: pom:xml
Goals and options: clean install -U -Pnotest,docs
I'm trying to understand what this mvn command means?
I tried to google it: "clean install -U"
But I didn't find what the parameter U means.
And I don't know what is "-Pnotest,docs".
can you guide me regarding how I can find what's it? (maybe "-Pnotest,docs" is from a xml file or it's from the artifactory etc..)
Thanks a lot!!!!
-U Forces a check for miss releases and updated snapshots on remote repositories
If Maven is regularly used in your company, and you will have to work with it on a day-to-day basis, I would advise you to find a mentor (any colleague that knows the tool well and is ready to share its knowledge with you) and work with them. Maven, when you first look at it, can be quite of a mouthful and you'll learn it more efficiently with their help.
For the problem at hand, Elarbi Mohamed Aymen's answer already tells you what the -U flag corresponds to. As for -P, it is used to activate profiles (in your case notest and docs). These profiles are usually defined in the pom.xml of the project being build.
See Running Apache Maven for the basic commands, and as advised on that page run mvn -h to have the complete list of flags the command can use.
Maven is one of the mechanism how to handle the build process and check project dependencies, especially for Java.
One of the option can be to have physically included dependencies (artifacts / libs) in the project, but its not so useful- in case of new version, you have to replace the file, sometimes you are using same lib in more apps, ten you have to handle it manually in all projects.
Except this, there is the maven- it has a global repository with shared artifacts / libs , which are common used- ref. https://repo1.maven.org/maven2/.
Except this, you can make your own libs/ artifacts in this case, its a modules / applications which are reusable, then you are storing it in private repository- this is the artifactory.
When you want to build your project, in case of maven project you have pom.xml , which is like manual for maven what to do / how to build.
clean and install are common goals, clean will wipe your local maven repository, install will download them again, with parameter -U it force to download them.
You can define your own goals in pom file, eg. to "tree build"- build some dependent modules, then build parent project.
Eg. with -D you pass parameters to the maven eg.
mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app
- that will generate new project, based on given archetype- "template", with the given groupID and artifactID- groupID can be eg. company name, artifactID is then the name of specific app / component.
-P,--activate-profiles <arg> Comma-delimited list of profiles
to activate
-D,--define <arg> Define a system property

What does maven clean install -U do?

I have eclipse ide with m2e plugin, maven and weblogc app server running from my local box.
I have imported someone else's multiple maven projects from bitbucket to my box. I was told that one of them is main and rest are dependencies in which I never seen anything like that before. I have always dealt with single maven project. Anyhow from the instruction, it says I have to run maven command such as "clean install -U".
In the IDE so I touched run configuration for each mvn project by setting goal as "clean install -U". By reading maven guide, I kind understand what each term means but when you combine together with a passing parameter, what does it do actually? I didn't expect a jar (web app) to be deployed to an application server but it did also.
-U forces maven to check any external dependencies (third party dependencies) that might need to be updated based on your POM files.
clean install are both basic maven lifecycle phases (https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html).
install normally would simply take the artifact that is built and put it in the local repository, i.e. a directory on the box you are building on (.m2 directory most of the time). It would not do a deployment to a server - typically the deploy phase would be used to do that.
However, developers can override and add to what maven does in the various phases, so just like in the days of ant things can easily devolve into chaos no one can understand on complex projects ;-).
sometimes in the integration-test phase, developers will tell maven to start up a container temporarily to run the web app on, so that tests can be run against it, and then that container is shut down when the integration-test phase completes.

Maven. Creating a project offline fails so how do I begin?

First off I'm not on a separate system in order to post this question and have no way of connecting the other system to the internet. This machine I'm on is locked down so I am not able to install or do anything other than email and view the web for research.
Used Reference: http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
Environment: Maven 2.2.1
Yes the Maven build functions fine for the existing project.
Situation:
I'm new to Maven and this project I'm on, the other developers are GONE. There is a .bat file that was used to run the project builds and that's all I know of Maven.
I am attempting to learn Maven on the other machine that has it installed I am using the Apache site mentioned above. The instructions tells me to create a project using the command
mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false but, I get the
'BUILD FAILURE'
Error reading archetype catalog http://repo1.maven.org/maven2... (and other errors)
The desired archetype does not exist (org.apache.maven.archetypes:maven-archetype=quickstart:1.0
So, question is what am I to do in order to create a dummy project so I can see WTF is going on with Maven when I'm in an environment that is not able to reach the 'maven.org' site?
If there are threads on learning Maven offline that someone can point me to that too would be of use. I'm not seeing anything though that helps me get started on this.
Thank you for looking and your time.
The problem seems that maven is looking for the architype on the central repository.
You can create your own pom.xml with an editor:
http://maven.apache.org/guides/introduction/introduction-to-the-pom.html

Maven deploy multi module project only if all modules build successfully

I have a maven multi module project with several modules. I want to deploy them (mvn deploy) only if they all pass a full mvn install (which includes the tests).
Currently, I run a mvn install on the project. If all modules pass, I run mvn deploy to do the deployment. The problem I see is the waste of time calling mvn twice (even if I skip tests on the second run).
Does anyone have an idea on this?
EDIT: I have learned that using Artifactory as a repository manager and the maven-artifactory-plugin with your maven setup will add the atomic deploy behaviour to the mvn deploy command. See the Build Integration section in the Artifactory documentation.
[DISCLOSURE - I'm associated with JFrog. Artifactory creator.]
Take a look at the deployAtEnd parameter of Maven Deployment plugin: http://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html
This is a bit tricky. Maven is not atomic when it executes the build life-cycle. So a broken set of artifacts may end up in a repository.
One solution I know is Nexus Pro: http://www.sonatype.com/Products/Nexus-Professional/Features - it allows you to promote builds or define certain repos as staging. So only verified versions get promoted to be used. Maybe artifactory has something similar - I just don't know.
If that solution is too expensive you probably need to create a cleanup build or profile to remove artifacts that where already uploaded. My first guess would be to write a Maven plugin to use the the proxy remote API or maybe the maven features are already sufficient. But since deploy means update the meta-data xml files too I dont think there is a delete - not sure on this either.

Maven WAR overlay problems, while using Hudson + Artifactory

We have three artifacts:
common.jar : with common classes.
public.war : depending on the common.jar, contains only public site resources.
internal.war : depends on both common.jar and public.war, adding authentication
information and security context resource files. Also contains
few administration site classes.
Currently I have structured these in such way, that internal.war overlays itself with public.war.
Building the project locally, installing the artifacts to local repo, works perfectly.
Problems start when trying to get the Hudson builds working with following sequence:
Build all projects in dependency order.
Modify common.jar (say, add a new class method)
Modify internal.war classes in such way that they are compile-time dependent on changes done in 2. step.
Commit both changes, triggering the Hudson builds.
Internal.war build fails because it can not find the symbols added in step 2.
Somehow the build in step 5. is using an old version of the common.jar, and failing because of it.
The common.jar version number does not change, let's say it's 1.0.0-SNAPSHOT for the purposes of this example.
If I DO change the common.jar version number, the build works. (Supposedly because there is only one release by a release version number).
Now, what could cause this using of old artifacts in Hudson builds?
We are running maven builds on Hudson with command "clean package -e -X -U"
"Deploy artifacts to maven repository" has been checked.
It's hard to definitively answer this without access to the real poms, but here is what I would do:
1) Make sure Hudson is using the exact same version of Maven as you are on your local machine
2) Examine the effective pom.xml of internal.war on the Hudson machine in a terminal via mvn help:effective-pom making sure you are running the same mvn executable as your Hudson job does. You need to verify the version of the common.jar in the effective pom.xml of internal.war. It could be different than what you expect due to profiles or settings.xml differences.
3) Check the settings.xml file for your Hudson install of Maven. In particular you need to verify all is well in your distributionManagement, servers, and repositories stanzas. Another good way to check this is to go to your internal.war project and run mvn help:effective-settings and see if what is there matches what is on your local machine.
Something is awry and it won't take long to find with the right analysis.

Resources