Running Hadoop Wordcount Job Error - hadoop

I am new to using Hadoop, and I am currently trying to run wordcount on a single node cluster on my ubuntu desktop. I am trying to follow this guide to run the hadoop job:
http://javabeginnerstutorial.com/hadoop/your-first-hadoop-map-reduce-job/
Currently I am stuck on step 3, when I try to mvn clean install, this is the error I receive. I am using the exact same pom.xml file that is linked on step 3. Was I supposed to edit something in the pom.xml file in order to successfully compile? Or do the links possibly not work? Any help would be great. Thanks.
hduser#venus:~/development/Hadoop_projects/word_count$ mvn clean install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.impetus.code:hadoop-examples:jar:1.0
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-compiler-plugin is missing. # line 38, column 12
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building hadoop-examples 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # hadoop-examples ---
[INFO]
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) # hadoop-examples ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/hduser/development/Hadoop_projects/word_count/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:compile (default-compile) # hadoop-examples ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.3:testResources (default-testResources) # hadoop-examples ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/hduser/development/Hadoop_projects/word_count/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:testCompile (default-testCompile) # hadoop-examples ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.10:test (default-test) # hadoop-examples ---
[INFO] No tests to run.
[INFO] Surefire report directory: /home/hduser/development/Hadoop_projects/word_count/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.874s
[INFO] Finished at: Wed Jul 15 00:34:04 PDT 2015
[INFO] Final Memory: 7M/240M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project hadoop-examples: Error creating properties files for forking; nested exception is java.io.IOException: No such file or directory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Try to get the pom and maven dependice from this http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-core
Try to check the hadoop version before adding this pom file.

Run the maven command with -DskipTests

Related

Cannot run "mvn jetty:run" in my fhir project

Im new at fhir server.
My code mainly from repo https://github.com/hapifhir/hapi-fhir-jpaserver-starter
I tried to install fhir server on my ubuntu 20.04(lts) and stuck with this problem.
my mvn version
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 17.0.5, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-50-generic", arch: "amd64", family: "unix"
ERROR
[INFO] Scanning for projects...
[WARNING] The project ca.uhn.hapi.fhir:hapi-fhir-jpaserver-starter:war:6.2.1 uses prerequisites which is only intended for maven-plugin projects but not for non maven-plugin projects. For such purposes you should use the maven-enforcer-plugin. See https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html
[INFO]
[INFO] ------------< ca.uhn.hapi.fhir:hapi-fhir-jpaserver-starter >------------
[INFO] Building HAPI FHIR JPA Server - Starter Project 6.2.1
[INFO] --------------------------------[ war ]---------------------------------
[INFO]
[INFO] >>> jetty-maven-plugin:9.4.48.v20220622:run (default-cli) > test-compile # hapi-fhir-jpaserver-starter >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-maven) # hapi-fhir-jpaserver-starter ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.8:prepare-agent (default-prepare-agent) #
hapi-fhir-jpaserver-starter ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.8/org.jacoco.agent-0.8.8-runtime.jar=destfile=/root/hapi-fhir-jpaserver-starter/target/jacoco.exec,excludes=ca/uhn/fhir/model/dstu2/**/*.class:ca/uhn/fhir/jpa/rp/r5/*.class:ca/uhn/fhir/jpa/rp/r4/*.class:ca/uhn/fhir/jpa/rp/r4b/*.class:ca/uhn/fhir/jpa/rp/dstu3/*.class:ca/uhn/fhir/jpa/rp/dstu2/*.class,dumponexit=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # hapi-fhir-jpaserver-starter ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # hapi-fhir-jpaserver-starter ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 31 source files to /root/hapi-fhir-jpaserver-starter/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.158 s
[INFO] Finished at: 2022-11-21T15:39:38Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project hapi-fhir-jpaserver-starter: Compilation failure -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Codename One online building error by terminal

I'm trying to use my CN1 account to compile online my project whit Ubuntu 16.04 LTS terminal command:
$ mvn cn1:build -Dcodename1.platform=android -Dcodename1.buildTarget=android-device
but I got this long error:
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for bo.conectica.carry:carry-common:jar:1.0-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.systemPath' for com.codenameone:codenameone-buildclient:jar refers to a non-existing file /home/synapsido/.codenameone/CodeNameOneBuildClient.jar # bo.conectica.carry:carry:1.0-SNAPSHOT, /home/synapsido/Documentos/carry/pom.xml, line 293, column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective model for bo.conectica.carry:carry-android:jar:1.0-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.systemPath' for com.codenameone:codenameone-buildclient:jar refers to a non-existing file /home/synapsido/.codenameone/CodeNameOneBuildClient.jar # bo.conectica.carry:carry:1.0-SNAPSHOT, /home/synapsido/Documentos/carry/pom.xml, line 293, column 21
[WARNING]
[WARNING] Some problems were encountered while building the effective model for bo.conectica.carry:carry:pom:1.0-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.systemPath' for com.codenameone:codenameone-buildclient:jar refers to a non-existing file /home/synapsido/.codenameone/CodeNameOneBuildClient.jar # line 293, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] carry [pom]
[INFO] carry-common [jar]
[INFO] carry-android [jar]
[INFO]
[INFO] ----------------------< bo.conectica.carry:carry >----------------------
[INFO] Building carry 1.0-SNAPSHOT [1/3]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] >>> codenameone-maven-plugin:7.0.36:build (default-cli) > package # carry >>>
[INFO]
[INFO] <<< codenameone-maven-plugin:7.0.36:build (default-cli) < package # carry <<<
[INFO]
[INFO]
[INFO] --- codenameone-maven-plugin:7.0.36:build (default-cli) # carry ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for carry 1.0-SNAPSHOT:
[INFO]
[INFO] carry .............................................. FAILURE [ 10.155 s]
[INFO] carry-common ....................................... SKIPPED
[INFO] carry-android ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.516 s
[INFO] Finished at: 2022-01-09T21:32:07-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.codenameone:codenameone-maven-plugin:7.0.36:build (default-cli) on project carry: Execution default-cli of goal com.codenameone:codenameone-maven-plugin:7.0.36:build failed: Plugin com.codenameone:codenameone-maven-plugin:7.0.36 or one of its dependencies could not be resolved: Could not find artifact com.codenameone:codenameone-buildclient:jar:7.0.36 at specified path /home/synapsido/.codenameone/CodeNameOneBuildClient.jar -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
maybe my mvn plugin version is deprecated? How to Update?
That particular target doesn't seem to trigger the download of that dependency. If you run mvn verify first, or the "Run In Simulator" option in your IDE, it will cause these dependencies to be downloaded.
e.g.
mvn verify
then
mvn cn1:build -Dcodename1.platform=android -Dcodename1.buildTarget=android-device
will work.
It seems that this is an older version of the maven plugin (7.0.36) where the current is 7.0.55. Make sure to update the version and dependencies. You can use https://start.codenameone.com/ to generate a new reference.
The new version no longer depends on the file in that location.

Issues in exceution while run the test script through Maven

While i am executing my test script through maven test, it shows error. below are the errors at runtime.
[INFO] Scanning for projects... [INFO]
[INFO]
------------------------------------------------------------------------ [INFO] Building Talentrack V1 [INFO]
------------------------------------------------------------------------ [INFO] Downloading:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom
[INFO] Downloaded:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom
(11 KB at 4.5 KB/sec) [INFO] Downloading:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/2.12.4/surefire-2.12.4.pom
[INFO] Downloaded:
https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/2.12.4/surefire-2.12.4.pom
(14 KB at 21.1 KB/sec) [INFO] [INFO] ---
maven-resources-plugin:2.6:resources (default-resources) # Talentrack
--- [WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non
existing resourceDirectory
D:\Talentrack_talent\Talentrack\src\main\resources [INFO] [INFO] ---
maven-compiler-plugin:3.3:compile (default-compile) # Talentrack ---
[INFO] Changes detected - recompiling the module! [WARNING] File
encoding has not been set, using platform encoding Cp1252, i.e. build
is platform dependent! [INFO] Compiling 10 source files to
D:\Talentrack_talent\Talentrack\target\classes [INFO]
------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO]
------------------------------------------------------------- [ERROR] No compiler is provided in this environment. Perhaps you are running
on a JRE rather than a JDK? [INFO] 1 error [INFO]
------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO]
------------------------------------------------------------------------ [INFO] Total time: 4.087 s [INFO] Finished at:
2016-05-17T16:35:05+05:30 [INFO] Final Memory: 9M/121M [INFO]
------------------------------------------------------------------------ [ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.3:compile
(default-compile) on project Talentrack: Compilation failure [ERROR]
No compiler is provided in this environment. Perhaps you are running
on a JRE rather than a JDK? [ERROR] -> [Help 1] [ERROR] [ERROR] To
see the full stack trace of the errors, re-run Maven with the -e
switch. [ERROR] Re-run Maven using the -X switch to enable full debug
logging. [ERROR] [ERROR] For more information about the errors and
possible solutions, please read the following articles: [ERROR] [Help
1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Compilation failure [ERROR] No compiler is provided in this
environment. Perhaps you are running on a JRE rather than a JDK?
You need to specify the JDK path into your system variables / bash profile which can be done by adding a JAVA_HOME variable to it.

StreamingWordCount example maven compile time errors

I am getting the following errors while running compiling the Google dataflow example on the cloud instance. I am trying to run the streaming example by the name of StreamingWordCount using the following command
mvn compile exec:java -Dexec.mainClass=com.google.cloud.dataflow.examples.complete.StreamingWordCount.java
Is there something that I am doing wrong?Any help is appreciated.
Thanks
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Google Cloud Dataflow Java Examples - All manual_build
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- build-helper-maven-plugin:1.10:add-source (add-java8-main-source) # google-cloud-dataflow-java-examples-all ---
[INFO] Source directory: /home/gaurav_bareja/pipeline/DataflowJavaSDK-examples/src/main/java8 added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.10:add-test-source (add-java8-test-source) # google-cloud-dataflow-java-exam
ples-all ---
[INFO] Test Source directory: /home/gaurav_bareja/pipeline/DataflowJavaSDK-examples/src/test/java8 added.
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # google-cloud-dataflow-java-examples-all ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/gaurav_bareja/pipeline/DataflowJavaSDK-examples/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) # google-cloud-dataflow-java-examples-all ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (java8-compile) # google-cloud-dataflow-java-examples-all ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] >>> exec-maven-plugin:1.1:java (default-cli) > validate # google-cloud-dataflow-java-examples-all >>>
[INFO]
[INFO] <<< exec-maven-plugin:1.1:java (default-cli) < validate # google-cloud-dataflow-java-examples-all <<<
[INFO]
[INFO] --- exec-maven-plugin:1.1:java (default-cli) # google-cloud-dataflow-java-examples-all ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.496 s
[INFO] Finished at: 2016-03-19T13:23:56+00:00
[INFO] Final Memory: 15M/56M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.1:java (default-cli) on project google-cloud-d
ataflow-java-examples-all: An exception occured while executing the Java class. com.google.cloud.dataflow.examples.
complete.StreamingWordCount.java -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecution
Exception
You seem to be attempting to run the StreamingWordCount example that comes with the Google Cloud Dataflow SDK for Java. The example under that name doesn't exist -- hence, the exception you are seeing.
You likely want to run WindowedWordCount instead, an example that shows how to run the same pipeline over either unbounded PCollections in the streaming mode or bounded PCollections in the batch mode.
Please follow the instructions here.

Maven gives MojoExecutionException when Maven install

I am trying to install a Maven project, I am getting the following error message.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mail maven webapp 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) # mail ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) # mail ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 13 source files to C:\Users\gk\workspace-juno\mail\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) # mail
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) # mail ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to C:\Users\gk\workspace-juno\mail\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.13:test (default-test) # mail ---
[INFO] Surefire report directory: C:\Users\gk\workspace-juno\mail\target\surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Configuring TestNG with: TestNG652Configurator
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.684 sec
Results :
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-war-plugin:2.3:war (default-war) # mail ---
[INFO] Packaging webapp
[INFO] Assembling webapp [mail] in [C:\Users\gk\workspace-juno\mail\target\mail-1.0-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp resources [C:\Users\gk\workspace-juno\mail\src\main\webapp]
[INFO] Webapp assembled in [895 msecs]
[INFO] Building war: C:\Users\gk\workspace-juno\mail\target\mail.war
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.398s
[INFO] Finished at: Mon May 20 12:28:45 IST 2013
[INFO] Final Memory: 15M/35M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.3:war (default-war) on project mail: Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executing in update mode) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Please help me in resolving this issue.
This error is self explainable:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.3:war (default-war) on project mail: Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executing in update mode) -> [Help 1]
It says that you either need to have a "web.xml" file pre-existing in "WEB-INF" directory, or specify its location explicitly using a "webxml" parameter in the pom.xml or command line.

Resources