hipi gradle build error - hadoop

root#ubuntu:/usr/local/hipi# gradle
FAILURE: Build failed with an exception.
Where:
Build file '/usr/local/hipi/core/build.gradle' line: 8
What went wrong:
A problem occurred evaluating project ':core'.
Could not find method distributions() for arguments [build_7msgs7jcv6q1ci90h6pi2434k3$_run_closure2#39d1aa64] on project ':core'.
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 7.635 secs
Build.gradle code :
defaultTasks 'install'
allprojects {
apply plugin: 'eclipse'
}
subprojects {
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile 'org.apache.hadoop:hadoop-common:2.7.1'
compile 'org.apache.hadoop:hadoop-mapreduce-client-core:2.7.1'
compile 'com.googlecode.json-simple:json-simple:1.1.1'
compile 'com.drewnoakes:metadata-extractor:2.8.1'
compile 'com.twelvemonkeys.imageio:imageio-jpeg:3.1.1'
compile 'com.twelvemonkeys.imageio:imageio-pnm:3.1.1'
compile 'com.twelvemonkeys.imageio:imageio-tiff:3.1.1'
compile group: 'org.bytedeco', name: 'javacpp', version: '1.0'
compile group: 'org.bytedeco', name: 'javacv', version: '1.0'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.0'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.0', classifier: 'macosx-x86_64'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.0', classifier: 'linux-x86_64'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.0', classifier: 'windows-x86_64'
testCompile "junit:junit:4+"
}
}
task install {
dependsOn(':tools:hibImport:jar')
dependsOn(':tools:hibDownload:jar')
dependsOn(':tools:hibInfo:jar')
dependsOn(':tools:hibToJpeg:jar')
dependsOn(':tools:hibDump:jar')
dependsOn(':tools:covar:jar')
doLast {
println '\nFinished building the HIPI library along with all tools and examples.'
}
}

The problem is in core folder's gradle file. Can you show me that file?
The content of core's directory gradle file should be :
apply plugin: 'java-library-distribution'
jar {
baseName = 'hipi'
version = '2.1.0'
}
distributions {
main{
baseName = 'hipi'
version = '2.1.0'
}
}
javadoc {
title = "HIPI - Hadoop Image Processing Interface"
options.links("http://docs.oracle.com/javase/8/docs/api/",
"http://hadoop.apache.org/docs/stable/api/",
"http://javadoc.metadata-extractor.googlecode.com/git/2.7.0/",
"http://bytedeco.org/javacpp-presets/opencv/apidocs/")
}
test {
testLogging.showStandardStreams = true
}

Related

How to enable Allure history using Gradle and Junit5?

I know I could copy the history directory from allure-reports to a generated allure-results, and then do allure generate to get the history to show up, but I am looking for a way to achieve this with the built-in functionality in allure-gradle.
Currently, I run ./gradlew clean test, then ./gradlew allureReport, and this gives me a fresh html report with no history or reruns. I have noticed that the test task deletes the entire allure-reports directory and then re-makes and re-populates it.
Here is my build.gradle:
buildscript {
repositories {
mavenLocal()
jcenter()
}
}
plugins {
// Apply the java-library plugin to add support for Java Library
id 'java-library'
id 'java'
id 'eclipse'
// Allure reporter
id 'io.qameta.allure' version '2.8.1'
}
repositories {
mavenCentral()
jcenter()
}
def allureVersion = "2.13.1"
allure {
autoconfigure = true
version = allureVersion
useJUnit5 {
version = '2.0-BETA10'
}
downloadLink = "https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/2.13.1/allure-commandline-2.13.1.zip"
}
test {
useJUnitPlatform() {
includeEngines 'junit-jupiter'
}
ignoreFailures = true
}
dependencies {
compile group: 'org.junit.jupiter', name: 'junit-jupiter-api', version: '5.6.0'
compile group: 'org.junit.jupiter', name: 'junit-jupiter-engine', version: '5.6.0'
compile group: 'org.junit.platform', name: 'junit-platform-launcher', version: '1.6.0'
// Needed as a workaround for JUnit 5 not failing tests that stopped due to an error message. The fix is in eclipse 4.10
compile group: 'org.junit.vintage', name: 'junit-vintage-engine', version: '5.6.0'
compile group: 'io.qameta.allure', name: 'allure-junit5', version: '2.13.1'
compile group: 'io.qameta.allure', name: 'allure-java-commons', version: '2.13.1'
// https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java
compile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '3.141.59'
}
I have also tried specifying the results and reports directories to be outside of the build directory, like this:
allure {
autoconfigure = true
version = allureVersion
resultsDir = file('../../allure-results')
reportDir = file('../../allure-report')
...
}
This allows the allure-results folder to keep the previous report, which show up as retries (not quite right), but history still doesn't work.
I have solved this problem by adding at the running test script these lines( I have some tests scuites)
call gradlew test -PsuiteFile=YOUR_TEST_SUITE.xml
mkdir allure-results\history
xcopy build\allure-results\* allure-results\history
allure serve allure-results\history
So I have added folder allure-results\history to keep all results

Gradle + Spring Boot can't handle jars with more than 65535 files

Using zip64 true doesn't create a usable jar file. It can't locate the main class inside that jar, although the file, structure and location of the Manifest.mf are exactly the same as in previous builds that worked.
The real problem is: I don't need sonarqube or gatling in my final build (those are test-related), but AFAIK there is no way to exclude plugins.
The "fatJar"-task is for creating the jar.
Any kind of help is highly appreciated.
plugins {
id "org.sonarqube" version "2.2.1"
id "com.github.lkishalmi.gatling" version "0.4.1"
}
// Run in terminal with "gradle sonarqube"
sonarqube {
properties {
property "sonar.projectName", "asd"
property "sonar.projectKey", "org.sonarqube:java-gradle-simple"
property "sonar.host.url", "http://asd"
property "sonar.login", "asd"
property "sonar.password", "asd"
}
}
// Run in terminal with "gradle gatlingrun", start the application before.
gatling {
logLevel 'ERROR'
simulations = {
include "**/LoginAndSync.scala"
}
}
group 'asd'
version '1.0-SNAPSHOT'
apply plugin: 'java'
sourceCompatibility = 1.8
repositories {
mavenCentral()
}
task fatJar(type: Jar) {
//zip64 true
manifest {
attributes 'Implementation-Title': 'Gradle Jar File Example',
'Implementation-Version': version,
'Main-Class': 'application.Asd'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
dependencies {
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: '1.4.1.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa', version: '1.4.1.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-security', version: '1.4.1.RELEASE'
compile group: 'org.springframework', name: 'spring-orm', version: '4.3.3.RELEASE'
compile group: 'mysql', name: 'mysql-connector-java', version: '6.0.5'
compile group: 'com.amazonaws', name: 'aws-java-sdk', version: '1.11.80'
compile group: 'io.gatling.highcharts', name: 'gatling-charts-highcharts', version: '2.2.3'
testCompile group: 'org.springframework.boot', name: 'spring-boot-starter-test', version: '1.4.1.RELEASE'
testCompile group: 'com.h2database', name: 'h2', version: '1.4.193'
testCompile group: 'junit', name: 'junit', version: '4.12'
testCompile group: 'org.springframework.security', name: 'spring-security-test', version: '4.0.0.RELEASE'
}
This is the Exception I get:
xecution failed for task ':fatJar'.
> archive contains more than 65535 entries.
To build this archive, please enable the zip64 extension.
See: https://docs.gradle.org/3.3/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:zip64
Thanks to M.Deinum, I did a new approach:
I added spring-boot as plugin in build.gradle
plugins {
id 'org.springframework.boot' version '1.5.1.RELEASE'
}
and can now just use gradle build in the console, to get a running jar, which can be found in ./build/libs/.

Create a jar with external dependencies using Gradle

I have added following in my build.gradle file.
jar {
doFirst {
into('lib') {
from configurations.runtime
}
}}
When I run "gradle build" it creates a JAR file containing all the dependent JARs. But, then it puts that JAR into a ZIP file containing the dependent jars in lib folder again. This ZIP file structure looks like this:
-- 4.11-SNAPSHOT.jar
---- lib
----- lib/dependent1.jar
----- lib/dependent2.jar
-- lib
---- dependent1.jar
---- dependent2.jar
I don't want the ZIP to be generated. I only want the single JAR containing all the dependencies to get generated.
-- 4.11-SNAPSHOT.jar
---- lib
----- lib/dependent1.jar
----- lib/dependent2.jar
Complete build.gradle is below:
repositories {
maven {
url 'https://repository.cloudera.com/content/repositories/releases/'
}
maven {
url 'https://repository.cloudera.com/content/repositories/third-party/'
}
}
configurations.all {
// check for updates every build
resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
}
dependencies {
compile group: 'commons-collections', name: 'commons-collections', version: '3.2'
compile group: 'com.company.projectname', name: 'domain', version: pppVersion
compile group: 'com.company.projectname', name: 'model', version: modelVersion
compile group: 'com.company.projectname', name: 'pipeline-util', version: pppVersion
compile group: 'com.cloudera.crunch', name: 'crunch', version: '0.3.0-3-cdh-5.2.1'
compile group: 'org.apache.hadoop', name: 'hadoop-common', version: '2.5.0-cdh5.3.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-common', version: '2.5.0-cdh5.3.3'
compile group: 'ch.hsr', name: 'geohash', version: '1.3.0'
testCompile group: 'junit', name: 'junit', version: '4.+'
}
jar {
manifest {
attributes 'Main-Class': 'com.company.projectname.delta.DeltaClusterer'
}
classifier 'job'
doFirst {
into('lib') {
from configurations.runtime
}
}
}
There is a parent build.gradle as well. And it is as follows:
/**
* Dependencies for build script plugins. Project dependencies should
* be added further in this file.
*/
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'com.netflix.nebula:gradle-ospackage-plugin:2.2.0'
classpath 'org.hidetake:gradle-ssh-plugin:1.0.4'
classpath 'be.xvrt:release-plugin:0.5.1'
}
}
task wrapper( type: Wrapper ) {
gradleVersion = '2.2'
}
// Release script for automated version management.
// Apply at root so it uses/updates the root gradle.properties file.
apply plugin: 'be.xvrt.release'
subprojects {
apply plugin: 'java' // Support for Java.
apply plugin: 'maven-publish' // Upload artifacts to Nexus.
apply plugin: 'be.xvrt.release' // Release script for automated version management.
apply plugin: 'org.hidetake.ssh' // Execute SSH commands.
apply plugin: 'rpm' // Builds RPM artifacts.
apply plugin: 'sonar-runner' // Creates Sonar reports.
apply plugin: 'project-report' // Creates a dependency report, very useful for finding dependency conflicts.
/**
* Global configuration.
*/
sourceCompatibility = 1.6
targetCompatibility = 1.6
compileJava.options.encoding = 'UTF-8'
compileTestJava.options.encoding = 'UTF-8'
/**
* Global dependency settings for all projects. Per project dependencies
* should be added to the build.gradle file in every project.
*/
// Common dependencies for all projects.
dependencies {
// Also see resolutionStrategy below when modifying semantic-TTOM-ddct version!
compile group: 'com.teleatlas.models', name: 'semantic-TTOM-ddct', version: '4.0.2.0'
compile group: 'com.google.guava', name: 'guava', version: '15.0'
compile group: 'commons-collections', name: 'commons-collections', version: '3.2.1'
compile group: 'commons-lang', name: 'commons-lang', version: '2.6'
compile group: 'commons-io', name: 'commons-io', version: '2.4'
compile group: 'args4j', name: 'args4j', version: '2.0.29'
compile group: 'log4j', name: 'log4j', version: '1.2.17'
testCompile group: 'junit', name: 'junit', version: '4.12'
testCompile group: 'org.mockito', name: 'mockito-core', version: '1.10.8'
}
// The install artifact is only required for command line applications.
if ( !tasks.findByPath( 'war' ) ) {
task buildInstallZip( type: Zip ) {
from jar.outputs.files
// Copy all dependencies into lib.
into( 'lib' ) {
from configurations.runtime.allArtifacts.files
from configurations.runtime
}
}
}
/**
* Publish to Nexus.
*/
artifacts {
archives buildSourcesJar
archives buildJavadocJar
if ( !tasks.findByPath( 'war' ) ) {
archives buildInstallZip
}
}
publishing {
publications {
maven( MavenPublication ) {
if ( tasks.findByPath( 'war' ) ) {
from components.web
}
else {
from components.java
}
artifact buildSourcesJar {
classifier 'sources'
}
artifact buildJavadocJar {
classifier 'javadoc'
}
if ( !tasks.findByPath( 'war' ) ) {
artifact buildInstallZip {
classifier 'install'
}
}
}
}
}
}
/**
* Build RPM artifacts for command line applications.
*/
configure( [project( ':project1' ),
project( ':project2' ),
project( ':project3' )] ) {
task rpm( type: Rpm, dependsOn: 'jar' ) {
if ( version.contains( 'SNAPSHOT' ) ) {
logger.info( 'Building SNAPSHOT RPM.' )
version project.version.replace( '-SNAPSHOT', ".${System.currentTimeMillis()}" )
}
else {
logger.info( 'Building RELEASE RPM.' )
version project.version
release '1'
}
vendor 'company'
packageGroup 'company'
permissionGroup "Applications/${project.name}"
arch NOARCH
os LINUX
into '/usr/share/java/'
from( jar.outputs.files ) {
into project.name
}
from( configurations.runtime ) {
into "${project.name}/lib"
}
}
// When publishing artifacts, the rpm is also published (and created).
publish.dependsOn publishRpm
publishRpm.dependsOn rpm
}
What needs to be done for this?
I'm guessing the problem is the buildInstallZip task you have defined in the parent build.gradle file. It looks like that task is duplicating the functionality yo're now adding to the jar task in the subproject.
Did you write the parent build file, or was that provided by someone else? Maybe 2 people are both trying to solve the same problem in 2 different ways?
Anyway, to solve, either disable the buildInstallZip task in that subproject, or remove your modifications from the jar task and instead use the zip produced by buildInstallZip as you application distribution.

Gradle does nothing on distZip task from distribution plugin

I have a Problem with the distribution Plugin of Gradle. I just want to use the plugin to bundle all my files together (jar's, shell-scripts, ...).
Here my build.gradle:
apply plugin: 'eclipse'
apply plugin: 'maven-publish'
apply plugin: 'distribution'
sourceCompatibility = 1.7
targetCompatibility = 1.7
publishing {
...
}
repositories {
mavenCentral()
}
dependencies {
// public libraries
compile group: 'javax', name: 'javaee-api', version: '7.0'
compile group: 'javax.mail', name: 'javax.mail-api', version: '1.5.2'
compile group: 'commons-cli', name: 'commons-cli', version: '1.2'
compile group: 'org.apache.axis', name: 'axis', version: '1.4'
compile group: 'org.apache.axis', name: 'axis-jaxrpc', version: '1.4'
compile group: 'commons-discovery', name: 'commons-discovery', version: '0.4'
compile group: 'commons-logging', name: 'commons-logging', version: '1.1.1'
compile group: 'wsdl4j', name: 'wsdl4j', version: '1.6.2'
compile group: 'javax.xml.soap', name: 'saaj-api', version: '1.3'
compile group: 'org.slf4j', name: 'slf4j-api', version: '1.7.10'
runtime group: 'ch.qos.logback', name: 'logback-classic', version: '1.1.2'
// this is a dirty workaround
// because I don't have deploy rights on artifactory and nobody has time for deploying my artifacts
compile fileTree(dir: 'lib', include: '*.jar')
}
Here the output:
$ ./gradlew distTar
Defaulting memory setting '-Xmx1024M'...
:distZip UP-TO-DATE
BUILD SUCCESSFUL
Total time: 2.44 secs
And nothing happens. No Zip File is created.
I'm using the latest version of Gradle (2.2.1)
Important for me is also, that I can use the installDist option of the plugin.
Any Ideas what's going wrong?
I found out what the Problem was:
The distZip wasn't working, because there were no files or libraries configured to be packed into the archive. By default, only the src/$distribution.name/dist is considered. The rest must be specified.
To ensure that at least the jar-File of the project is contained too, I used the plugin "java-library-distribution" instead of "distribution"
Furthermore I specified more files to consider:
distributions {
main {
contents {
from { 'distrib' }
}
}
}
If your $buildDir is empty, you have to configure a distribution:
distributions {
main {
baseName = 'someName'
contents {
from { 'src/readme' }
}
}
}
Also see https://gradle.org/docs/current/userguide/distribution_plugin.html

How to give dependecies in multiproject in Gradle and what is the structure?

I have a multi project to be built using Gradle which has a java Project and it'll be dependancy for web Project. When I build with eclipse i have given depencies of javaProj and webProj to webprojectEAR and it works(after deployed).
.ear file needs to be created using single build run whcih has the following content.
- lib
javaProj.jar
- META-INF
- webProj.war
my project structure as follows
- javaProj
build.gradle
- webProj
build.gradle
- webProjEAR
build.gradle
- settings.gradle
I'm trying to run the webProjEAR/build.gradle file and given as dependency as follows in it.
project(':webProjEAR') {
dependencies {
compile project(':javaProj')
compile project(':webProj')
}
}
it failed with error "Could not resolve all dependencies for configuration 'webProjEAR:testRuntime' .when I run build files separatly I am able to create jar and war files.
Please can anyone help me how dependencies should be mentioned in the build files. And, where are the madatory locations to have a build.gradle file.
=== More Information added to the original question from here ===
My build files are as below. I have made the changes Igor Popov had mentioned.
// javaProj/build.gradle
apply plugin: 'java'
repositories {
flatDir { dirs "../local-repo" }
}
sourceSets {
main {
java.srcDir "$projectDir/src"
resources.srcDir "$projectDir/XML"
}
}
jar {
from ('classes/com/nyl/xsl') {
into 'com/nyl/xsl'
}
}
dependencies {
compile group: 'slf4j-api' , name: 'slf4j-api' , version: '1.5.6'
compile group: 'slf4j-jdk14' , name: 'slf4j-jdk14' , version: '1.5.6'
compile group: 'com.ibm.xml.thinclient_1.0.0' , name: 'com.ibm.xml.thinclient_1.0.0' , version: '1.0.0'
compile group: 'junit', name: 'junit', version: '4.11'
compile group: 'saxon9' , name: 'saxon9'
compile group: 'saxon9-dom' , name: 'saxon9-dom'
compile group: 'xmlunit' , name: 'xmlunit' , version: '1.4'
}
==============================================
// webProj/build.gradle
apply plugin: 'war'
repositories {
flatDir { dirs "../local-repo" }
}
webAppDirName = 'WebContent'
dependencies {
providedCompile group: 'com.ibm.xml' , name: 'com.ibm.xml'
providedCompile group: 'com.ibm.ws.prereq.xdi2' , name: 'com.ibm.ws.prereq.xdi2'
providedCompile group: 'com.ibm.ws.xml.admin.metadata' , name: 'com.ibm.ws.xml.admin.metadata'
providedCompile group: 'guava' , name: 'guava' , version: '11.0.2'
providedCompile group: 'j2ee' , name: 'j2ee'
providedCompile group: 'slf4j-api' , name: 'slf4j-api' , version: '1.5.6'
providedCompile group: 'slf4j-log4j12' , name: 'slf4j-log4j12' , version: '1.5.6'
}
====================================================
// webProjEAR/build.gradle
apply plugin: 'java'
apply plugin: 'ear'
repositories {
flatDir { dirs "../local-repo" }
}
dependencies {
compile project(':javaProj')
compile project(':webProj')
}
ear {
appDirName 'src/main/app'
libDirName 'APP-INF/lib'
from ('../javaProj/build/libs') {
into 'lib'
}
from ('../webProj/build/libs') {
into '/'
}
}
I would like to know what should contain in the roojProj/build.gradle file. Also like to know anything needs to be changed in the above files. Thanks
The overall structure should be something like:
rootProj
javaProj
build.gradle
webProj
build.gradle
webProjEAR
build.gradle
settings.gradle
build.gradle
The settings.gradle file should contain:
include 'javaProj', 'webProj', 'webProjEAR'
For the webProjEAR/build.gradle you can remove the project(':webProjEAR'). So it should look like:
dependencies {
compile project(':javaProj')
compile project(':webProj')
}
To fix the error you get, you should also post all your dependencies for webProjEAR (if you have others that you didn't include).

Resources