UPDATE-2
I have updated the Windows SDK compilers and now have reduced the errors to 2.
c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\intrin.h(26): fatal error C1083: Cannot open include file: 'ammintrin.h': No such file or directory [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
now where ever I have searched, I have found that to get this .h file I need to install the Microsoft Visual Studio Service Pack 1. I don't have Visual Studio, I am using Windows SDK 7.1. Now, where can I find the equivalent Service Pack 1 for Windows SDK 7.1?
UPDATE-2 END
UPDATE
After staring at the stack trace for sometime I see that the error occurs here
Midl:
C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /out"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/" /h "hadoopwinutilsvc_h.h" /tlb "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb" /robust hadoopwinutilsvc.idl
TRACKER : error TRK0002: Failed to execute command: ""C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe" /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /outC:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/ /h hadoopwinutilsvc_h.h /tlb C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb /robust hadoopwinutilsvc.idl". The handle is invalid. [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
So, I tried to run just this command
"C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe" /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /outC:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/ /h hadoopwinutilsvc_h.h /tlb C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb /robust hadoopwinutilsvc.idl"
Now I get the following error
64 bit Processing .\hadoopwinutilsvc.idl
hadoopwinutilsvc.idl
midl : command line error MIDL1001 : cannot open input file oaidl.idl
When I searched for the file oaidl.ild. I found it in folder
C:\Program Files\Microsoft SDKs\Windows\v7.1\Include
I included this in the PATH and ran the command again with the same error. I noticed that the file name is 'OAIdl.Idl' and not 'oaidl.idl'.
The first line in the file hadoopwinutilsvc.idl is 'include oaidl.idl'.
Does this mean that these is an error in hadoop source?
END UPDATE - Original question below.
I am trying to build hadoop on Windows 7 x64.
I keep getting the following error.
I have provided the relevant stack trace below. Any help will be much appreciated.
The command I ran on the command prompt is mvn package -Pdist,native-win -DskipTests -Dtar
[DEBUG] env: PATH=C:\csvn\bin\;C:\csvn\Python25\;C:\Program Files\CollabNet\Subversion Client;C:\Windows\system32;C:\Windows;C:\Windows\system32\wbem;C:\Windows\system32\windowspowershell\v1.0\;c:\program files\ibm\gsk8\lib64;c:\program files (x86)\ibm\gsk8\lib;C:\IBM\WEBSPH~1\APPSER~1\db2\BIN;C:\IBM\WEBSPH~1\APPSER~1\db2\FUNCTION;C:\IBM\WEBSPH~1\APPSER~1\db2\SAMPLES\REPL;C:\Program Files\nodejs\;%JAVA_HOME%\bin;C:\Program Files (x86)\Skype\Phone\;C:\Program Files (x86)\CMake\bin;C:\Program Files (x86)\Git\cmd;C:\Program Files\Microsoft Windows Performance Toolkit\;C:\Users\ajayamrite\AppData\Roaming\npm;C:\apache-maven-3.3.3/bin;C:\cygwin64\bin;C:\protoc-2.5.0-win32;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin;C:\hadoop\bin;c:\hadoop\sbin;C:\Program Files (x86)\CMake\bin;C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\x86_amd64
[DEBUG] env: PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
[DEBUG] env: PLATFORM=x64
[DEBUG] env: PROCESSOR_ARCHITECTURE=AMD64
[DEBUG] env: PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 70 Stepping 1, GenuineIntel
[DEBUG] env: PROCESSOR_LEVEL=6
[DEBUG] env: PROCESSOR_REVISION=4601
[DEBUG] env: PROGRAMDATA=C:\ProgramData
[DEBUG] env: PROGRAMFILES=C:\Program Files
[DEBUG] env: PROGRAMFILES(X86)=C:\Program Files (x86)
[DEBUG] env: PROGRAMW6432=C:\Program Files
[DEBUG] env: PROMPT=$P$G
[DEBUG] env: PSMODULEPATH=C:\Windows\system32\WindowsPowerShell\v1.0\Modules\
[DEBUG] env: PUBLIC=C:\Users\Public
[DEBUG] env: PYTHONHOME=C:\csvn\Python25\
[DEBUG] env: SESSIONNAME=Console
[DEBUG] env: SYSTEMDRIVE=C:
[DEBUG] env: SYSTEMROOT=C:\Windows
[DEBUG] env: TEMP=C:\Users\AJAYAM~1\AppData\Local\Temp
[DEBUG] env: TMP=C:\Users\AJAYAM~1\AppData\Local\Temp
[DEBUG] env: USERDOMAIN=bedouinvm
[DEBUG] env: USERNAME=ajayamrite
[DEBUG] env: USERPROFILE=C:\Users\ajayamrite
[DEBUG] env: WDIR=C:\
[DEBUG] env: WINDIR=C:\Windows
[DEBUG] env: ZLIB_HOME=C:\zlib127-dll
[DEBUG] Executing command line: [msbuild, C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common/src/main/winutils/winutils.sln, /nologo, /p:Configuration=Release, /p:OutDir=C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/bin/, /p:IntermediateOutputPath=C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/, /p:WsceConfigDir=../etc/hadoop, /p:WsceConfigFile=wsce-site.xml]
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
Build started 5/27/2015 3:13:28 PM.
Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
Building solution configuration "Release|x64".
Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln" (1) is building "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.vcxproj.metaproj" (2) on node 1 (default targets).
Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.vcxproj.metaproj" (2) is building "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj" (3) on node 1 (default targets).
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppBuild.targets(297,5): warning MSB8003: Could not find WindowsSDKDir variable from the registry. TargetFrameworkVersion or PlatformToolset may be set to an invalid version number. [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
InitializeBuildStatus:
Touching "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.unsuccessfulbuild".
Midl:
C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /out"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/" /h "hadoopwinutilsvc_h.h" /tlb "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb" /robust hadoopwinutilsvc.idl
TRACKER : error TRK0002: Failed to execute command: ""C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe" /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /outC:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/ /h hadoopwinutilsvc_h.h /tlb C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb /robust hadoopwinutilsvc.idl". The handle is invalid. [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
Done Building Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj" (default targets) -- FAILED.
Done Building Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.vcxproj.metaproj" (default targets) -- FAILED.
Done Building Project "C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln" (default targets) -- FAILED.
Build FAILED.
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln" (default target) (1) ->
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.vcxproj.metaproj" (default target) (2) ->
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj" (default target) (3) ->
(PrepareForBuild target) ->
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppBuild.targets(297,5): warning MSB8003: Could not find WindowsSDKDir variable from the registry. TargetFrameworkVersion or PlatformToolset may be set to an invalid version number. [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln" (default target) (1) ->
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\winutils.vcxproj.metaproj" (default target) (2) ->
"C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj" (default target) (3) ->
(Midl target) ->
TRACKER : error TRK0002: Failed to execute command: ""C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\midl.exe" /W2 /WX /nologo /char signed /env x64 /Oicf /app_config /outC:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/ /h hadoopwinutilsvc_h.h /tlb C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\target/winutils/libwinutils.tlb /robust hadoopwinutilsvc.idl". The handle is invalid. [C:\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
1 Warning(s)
1 Error(s)
Time Elapsed 00:00:01.36
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [ 1.558 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 1.334 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 2.580 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.170 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 2.133 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 2.585 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 2.008 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 3.022 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 2.663 s]
[INFO] Apache Hadoop Common ............................... FAILURE [ 22.818 s]
[INFO] Apache Hadoop NFS .................................. SKIPPED
[INFO] Apache Hadoop KMS .................................. SKIPPED
[INFO] Apache Hadoop Common Project ....................... SKIPPED
[INFO] Apache Hadoop HDFS ................................. SKIPPED
[INFO] Apache Hadoop HttpFS ............................... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED
[INFO] Apache Hadoop HDFS Project ......................... SKIPPED
[INFO] hadoop-yarn ........................................ SKIPPED
[INFO] hadoop-yarn-api .................................... SKIPPED
[INFO] hadoop-yarn-common ................................. SKIPPED
[INFO] hadoop-yarn-server ................................. SKIPPED
[INFO] hadoop-yarn-server-common .......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED
[INFO] hadoop-yarn-server-tests ........................... SKIPPED
[INFO] hadoop-yarn-client ................................. SKIPPED
[INFO] hadoop-yarn-server-sharedcachemanager .............. SKIPPED
[INFO] hadoop-yarn-applications ........................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
[INFO] hadoop-yarn-site ................................... SKIPPED
[INFO] hadoop-yarn-registry ............................... SKIPPED
[INFO] hadoop-yarn-project ................................ SKIPPED
[INFO] hadoop-mapreduce-client ............................ SKIPPED
[INFO] hadoop-mapreduce-client-core ....................... SKIPPED
[INFO] hadoop-mapreduce-client-common ..................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
[INFO] hadoop-mapreduce-client-app ........................ SKIPPED
[INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
[INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] hadoop-mapreduce ................................... SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Azure support ........................ SKIPPED
[INFO] Apache Hadoop Client ............................... SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 43.747 s
[INFO] Finished at: 2015-05-27T15:13:30+01:00
[INFO] Final Memory: 68M/306M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (compile-ms-winutils) on project hadoop-common: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (compile-ms-winutils) on project hadoop-common: Command execution failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed.
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:303)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 20 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:402)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:164)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:750)
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:292)
... 22 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
I'm also trying to build hadoop 2.7.1 on Windows 7 x64.
For the question at UPDATE-2:
I have downloaded the file "ammintrin.h" and puted it in: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include" (I don't have Visual Studio, only installed Windows SDK 7.1)
It worked for me without installing Microsoft Visual Studio Service Pack 1.
Here there is the file: http://www.mathworks.com/matlabcentral/answers/uploaded_files/735/ammintrin.m
(Notice that you have to rename with the .h extension)
Related
I am trying to build Spark 2.2.0-rc2 release using mvn but unable to do so.
$ uname -a
Linux knoldus-Vostro-15-3568 4.4.0-46-generic #67-Ubuntu SMP Thu Oct 20 15:05:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
openjdk version "1.8.0_131"
Below is the error stack that I am getting:
$ ./build/mvn -Phadoop-2.7,yarn,mesos,hive,hive-thriftserver -DskipTests clean install
...
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) # spark-tags_2.11 ---
[INFO] Using zinc server for incremental compilation
java.lang.NoClassDefFoundError: Could not initialize class sun.util.calendar.ZoneInfoFile
at sun.util.calendar.ZoneInfo.getTimeZone(ZoneInfo.java:589)
at java.util.TimeZone.getTimeZone(TimeZone.java:560)
at java.util.TimeZone.setDefaultZone(TimeZone.java:666)
at java.util.TimeZone.getDefaultRef(TimeZone.java:636)
at java.util.Date.<init>(Date.java:254)
at java.util.zip.ZipUtils.dosToJavaTime(ZipUtils.java:71)
at java.util.zip.ZipUtils.extendedDosToJavaTime(ZipUtils.java:88)
at java.util.zip.ZipEntry.getTime(ZipEntry.java:194)
at sbt.IO$.next$1(IO.scala:278)
at sbt.IO$.sbt$IO$$extract(IO.scala:286)
at sbt.IO$$anonfun$unzipStream$1.apply(IO.scala:255)
at sbt.IO$$anonfun$unzipStream$1.apply(IO.scala:255)
at sbt.Using.apply(Using.scala:24)
at sbt.IO$.unzipStream(IO.scala:255)
at sbt.IO$$anonfun$unzip$1.apply(IO.scala:249)
at sbt.IO$$anonfun$unzip$1.apply(IO.scala:249)
at sbt.Using.apply(Using.scala:24)
at sbt.IO$.unzip(IO.scala:249)
at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1$$anonfun$5.apply(AnalyzingCompiler.scala:140)
at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1$$anonfun$5.apply(AnalyzingCompiler.scala:140)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)
at scala.collection.immutable.List.foldLeft(List.scala:84)
at scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:138)
at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:105)
at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1.apply(AnalyzingCompiler.scala:140)
at sbt.compiler.AnalyzingCompiler$$anonfun$compileSources$1.apply(AnalyzingCompiler.scala:139)
at sbt.IO$.withTemporaryDirectory(IO.scala:344)
at sbt.compiler.AnalyzingCompiler$.compileSources(AnalyzingCompiler.scala:139)
at sbt.compiler.IC$.compileInterfaceJar(IncrementalCompiler.scala:58)
at com.typesafe.zinc.Compiler$.compilerInterface(Compiler.scala:148)
at com.typesafe.zinc.Compiler$.create(Compiler.scala:53)
at com.typesafe.zinc.Compiler$$anonfun$apply$1.apply(Compiler.scala:40)
at com.typesafe.zinc.Compiler$$anonfun$apply$1.apply(Compiler.scala:40)
at com.typesafe.zinc.Cache.get(Cache.scala:41)
at com.typesafe.zinc.Compiler$.apply(Compiler.scala:40)
at com.typesafe.zinc.Main$.run(Main.scala:96)
at com.typesafe.zinc.Nailgun$.zinc(Nailgun.scala:93)
at com.typesafe.zinc.Nailgun$.nailMain(Nailgun.scala:82)
at com.typesafe.zinc.Nailgun.nailMain(Nailgun.scala)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.martiansoftware.nailgun.NGSession.run(NGSession.java:280)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 5.657 s]
[INFO] Spark Project Tags ................................. FAILURE [ 0.371 s]
[INFO] Spark Project Sketch ............................... SKIPPED
[INFO] Spark Project Networking ........................... SKIPPED
[INFO] Spark Project Shuffle Streaming Service ............ SKIPPED
[INFO] Spark Project Unsafe ............................... SKIPPED
[INFO] Spark Project Launcher ............................. SKIPPED
[INFO] Spark Project Core ................................. SKIPPED
[INFO] Spark Project ML Local Library ..................... SKIPPED
[INFO] Spark Project GraphX ............................... SKIPPED
[INFO] Spark Project Streaming ............................ SKIPPED
[INFO] Spark Project Catalyst ............................. SKIPPED
[INFO] Spark Project SQL .................................. SKIPPED
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] Spark Project YARN ................................. SKIPPED
[INFO] Spark Project Mesos ................................ SKIPPED
[INFO] Spark Project Hive Thrift Server ................... SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Integration for Kafka 0.8 .................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
[INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.855 s
[INFO] Finished at: 2017-05-30T13:47:02+05:30
[INFO] Final Memory: 50M/605M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-tags_2.11: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.: CompileFailed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-tags_2.11
I am not able to understand this error.
scala> java.util.TimeZone.getDefault
res0: java.util.TimeZone = sun.util.calendar.ZoneInfo[id="Asia/Kolkata",offset=19800000,dstSavings=0,useDaylight=false,transitions=6,lastRule=null]
locale is LANG=en_IN LANGUAGE=en_IN:en LC_CTYPE="en_IN" LC_NUMERIC="en_IN" LC_TIME="en_IN" LC_COLLATE="en_IN" LC_MONETARY="en_IN" LC_MESSAGES="en_IN" LC_PAPER="en_IN" LC_NAME="en_IN" LC_ADDRESS="en_IN" LC_TELEPHONE="en_IN" LC_MEASUREMENT="en_IN" LC_IDENTIFICATION="en_IN" LC_ALL=
I guess the issue is with your time zone. Export LC_ALL=en_US.UTF-8 and start over. Make sure to have en_US.UTF-8 in locale for all entries.
$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
I had similar issue and the reason was missing timezone info file in java installation (after updating java 8 to newer version).
When I looked for tzdb.dat file - there was only link pointing to missing target
I was able to solve it on RedHat just by:
yum update tzdata-java
and this site was helpful here:
https://ekuric.wordpress.com/tag/tzdb-dat/
for other OS-es solution might be similar (there is even timezone updater tool from Oracle: https://www.oracle.com/technetwork/java/javase/documentation/tzupdater-readme-136440.html)
I am building spark using cygwin on a windows system. I can build fine when I do this and it works after
git clone https://github.com/apache/spark
dos2unix spark/build/mvn
build/mvn -DskipTests clean package
but when I try to use the -Psparkr profile
build/mvn -DskipTests -Psparkr clean package
I get an error, I tried with the -X and -e option but nothing seems to give me any indication why it fails.
[DEBUG] Executing command line: [cmd, /c, ..\R\install-dev.bat]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 5.613 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 27.617 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 16.601 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 12.118 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 22.492 s]
[INFO] Spark Project Core ................................. FAILURE [03:19 min]
[INFO] Spark Project Bagel ................................ SKIPPED
[INFO] Spark Project GraphX ............................... SKIPPED
[INFO] Spark Project Streaming ............................ SKIPPED
[INFO] Spark Project Catalyst ............................. SKIPPED
[INFO] Spark Project SQL .................................. SKIPPED
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Twitter ..................... SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Project External MQTT ........................ SKIPPED
[INFO] Spark Project External MQTT Assembly ............... SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:44 min
[INFO] Finished at: 2015-09-24T09:37:59-04:00
[INFO] Final Memory: 46M/490M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.10: Command execution failed. Process exited with an error: -1 (Exit value: -1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.10: Command execution failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed.
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:276)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 20 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: -1 (Exit value: -1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:660)
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:265)
... 22 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-core_2.10
Any clues to why it would be failing?
Likely it is because of missing dependencies on R language which are required by -Psparkr compilation option.
On Debian GNU/Linux (in particular 8/jessie) you would need to run
apt-get install r-base r-base-dev
or
aptitude install r-base r-base-dev
Prefer aptitude version if you have it installed.
read the README.md of SparkR at https://github.com/apache/spark/tree/master/R.
cd the source code dir (spark-1.6.1/R) and run ./install-dev.sh
then, run build/mvn -DskipTests -Psparkr package will be fine
I'm following the tutorial to build and install hadoop.
http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os
However, when I give the below command from the VS2010 command prompt:
mvn package -Pdist,native-win -DskipTests -Dtar
I get the below error:
main:
[mkdir] Skipping C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native because it already exists.
[exec] Current OS is Windows 8.1
[exec] Executing 'cmake' with arguments:
[exec] 'C:\hdfs\hadoop-hdfs-project\hadoop-hdfs/src/'
[exec] '-DGENERATED_JAVAH=C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target/native/javah'
[exec] '-DJVM_ARCH_DATA_MODEL=64'
[exec] '-DREQUIRE_LIBWEBHDFS=false'
[exec] '-DREQUIRE_FUSE=false'
[exec] '-G'
[exec] 'Visual Studio 10 Win64'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
'C:\hdfs\hadoop-hdfs-project\hadoop-hdfs/src/'
'-DGENERATED_JAVAH=C:\hdfs\hadoop-hdfs-project\hadoophdfs\target/native/javah'
'-DJVM_ARCH_DATA_MODEL=64'
'-DREQUIRE_LIBWEBHDFS=false'
'-DREQUIRE_FUSE=false'
'-G'
'Visual Studio 10 Win64'
The ' characters around the executable and arguments are not part of the command.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [ 1.781s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 1.333s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 1.030s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.375s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 2.104s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 6.628s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 1.047s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 1.173s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 1.594s]
[INFO] Apache Hadoop Common ............................... SUCCESS [ 59.046s]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 1.905s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 6.491s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.150s]
[INFO] Apache Hadoop HDFS ................................. FAILURE [ 19.351s]
[INFO] Apache Hadoop HttpFS ............................... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED
[INFO] Apache Hadoop HDFS Project ......................... SKIPPED
[INFO] hadoop-yarn ........................................ SKIPPED
[INFO] hadoop-yarn-api .................................... SKIPPED
[INFO] hadoop-yarn-common ................................. SKIPPED
[INFO] hadoop-yarn-server ................................. SKIPPED
[INFO] hadoop-yarn-server-common .......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED
[INFO] hadoop-yarn-server-tests ........................... SKIPPED
[INFO] hadoop-yarn-client ................................. SKIPPED
[INFO] hadoop-yarn-applications ........................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
[INFO] hadoop-yarn-site ................................... SKIPPED
[INFO] hadoop-yarn-registry ............................... SKIPPED
[INFO] hadoop-yarn-project ................................ SKIPPED
[INFO] hadoop-mapreduce-client ............................ SKIPPED
[INFO] hadoop-mapreduce-client-core ....................... SKIPPED
[INFO] hadoop-mapreduce-client-common ..................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
[INFO] hadoop-mapreduce-client-app ........................ SKIPPED
[INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
[INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] hadoop-mapreduce ................................... SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Client ............................... SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] -----------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] -----------------------------------------------------------------------
[INFO] Total time: 01:47 min
[INFO] Finished at: 2015-03-28T21:18:11+00:00
[INFO] Final Memory: 78M/363M
[INFO] -----------------------------------------------------------------------
[WARNING] The requested profile "native-bin" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native"): CreateProcess error=2, The system cannot find the file specified [ERROR] around Ant part ...<exec failonerror="true" dir="C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake">... # 5:107 in C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native"): CreateProcess error=2, The system cannot find the file specifiedaround Ant part ...<exec failonerror="true" dir="C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake">... # 5:107 in C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native"): CreateProcess error=2, The system cannot find the file specifiedaround Ant part ...<exec failonerror="true" dir="C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake">... # 5:107 in C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 20 more
Caused by: C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml:5: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native"): CreateProcess error=2, The system cannot find the file specified
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:675)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 22 more
Caused by: java.io.IOException: Cannot run program "cmake" (in directory "C:\hdfs\hadoop-hdfs-project\hadoop-hdfs\target\native"): CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at java.lang.Runtime.exec(Runtime.java:620)
at org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:862)
at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:481)
at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:495)
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:631)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
... 34 more
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
at java.lang.ProcessImpl.start(ProcessImpl.java:137)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 40 more
I don't understand the error at all. I have cmake installed and C:\cmake\bin added to the path as well.
Hmm...this error makes me think you had a typo in your Maven command:
[WARNING] The requested profile "native-bin" could not be activated because it does not exist.
Check your command; make sure it's native-win, not native-bin
I also wrote a guide recently for building Hadoop on Server 2012 R2 (should be identical for 8.1). Let me know.
Hello I'm trying to build hadoop 2.6.0 on Windows 8.1. Unfortunately without luck so far.
I have installed:
jdk1.7.0_71 (added Variable JAVA_HOME with value C:\Program Files\Java\jdk1.7.0_71 to the User Variables)
cygwin64 (added its installation-directory as value D:\cygwin64\bin to the PATH Variable under System Variables)
Maven 3.2.5 (added its installation-directory as value D:\maven\bin to the PATH Variable under System Variables)
Protocol Buffer 2.5 (added its installation-directory as value D:\protobuf to the PATH Variable under System Variables)
Visual Studio 2010
In the Visual Studio 2010 Command Prompt (started as Administrator) I have changed the drive to d: and used the folder of the hadoop src as a starting point ("D:\hdp").
I do again set the JAVA_HOME in short notation and do set the Platform
set JAVA_HOME=C:\PROGRA~1\Java\jdk1.7.0_71
set Platform=x64
Afterwards I try to build hadoop using the following Maven command:
mvn -e -X package -Pdist -DskipTests -Dtar
After Building fails at "Apache Hadoop Common" stating the following:
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.6.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: protoc failure -> [Help 1]
I have slightly changed the file "ProtocMojo.java" under D:\hdp\hadoop-maven-plugins\src\main\java\org\apache\hadoop\maven\plugin\protoc by inserting the following afer line 56 in the file:
protocCommand = "D:\\protobuf\\protoc.exe";
This helps to build further to "Apache Hadoop HDFS" where it fails again stating as follows:
[INFO] Executing tasks
Build sequence for target(s) `main' is [main]
Complete build sequence is [main, ]
main:
[mkdir] Skipping D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target\native because it already exists.
[exec] Current OS is Windows 8.1
[exec] Executing 'cmake' with arguments:
[exec] 'D:\hdp\hadoop-hdfs-project\hadoop-hdfs/src/'
[exec] '-DGENERATED_JAVAH=D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native/javah'
[exec] '-DJVM_ARCH_DATA_MODEL=64'
[exec] '-DREQUIRE_LIBWEBHDFS=false'
[exec] '-DREQUIRE_FUSE=false'
[exec] '-G'
[exec] 'Visual Studio 10 Win64'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
'D:\hdp\hadoop-hdfs-project\hadoop-hdfs/src/'
'-DGENERATED_JAVAH=D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native/javah'
'-DJVM_ARCH_DATA_MODEL=64'
'-DREQUIRE_LIBWEBHDFS=false'
'-DREQUIRE_FUSE=false'
'-G'
'Visual Studio 10 Win64'
The ' characters around the executable and arguments are
not part of the command.
[exec] CMake Error: Could not create named generator Visual Studio 10 Win64
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [ 0.906 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 0.719 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 1.469 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.265 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 1.766 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 5.516 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 1.431 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 2.119 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 1.969 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:11 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 4.087 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 11.742 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.110 s]
[INFO] Apache Hadoop HDFS ................................. FAILURE [ 11.782 s]
[INFO] Apache Hadoop HttpFS ............................... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED
[INFO] Apache Hadoop HDFS Project ......................... SKIPPED
[INFO] hadoop-yarn ........................................ SKIPPED
[INFO] hadoop-yarn-api .................................... SKIPPED
[INFO] hadoop-yarn-common ................................. SKIPPED
[INFO] hadoop-yarn-server ................................. SKIPPED
[INFO] hadoop-yarn-server-common .......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED
[INFO] hadoop-yarn-server-tests ........................... SKIPPED
[INFO] hadoop-yarn-client ................................. SKIPPED
[INFO] hadoop-yarn-applications ........................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
[INFO] hadoop-yarn-site ................................... SKIPPED
[INFO] hadoop-yarn-registry ............................... SKIPPED
[INFO] hadoop-yarn-project ................................ SKIPPED
[INFO] hadoop-mapreduce-client ............................ SKIPPED
[INFO] hadoop-mapreduce-client-core ....................... SKIPPED
[INFO] hadoop-mapreduce-client-common ..................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
[INFO] hadoop-mapreduce-client-app ........................ SKIPPED
[INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
[INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] hadoop-mapreduce ................................... SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Client ............................... SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:57 min
[INFO] Finished at: 2015-03-09T17:08:10+01:00
[INFO] Final Memory: 88M/1092M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... # 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-h
dfs\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildExcep
tion has occured: exec returned: 1
around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... # 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-hdfs\targ
et\antrun\build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... # 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-hdfs\targ
et\antrun\build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml:5: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-hdfs
This is where I'm stucked and do not know how to solve this problem. Help is appreciated as I am new to hadoop and building in general.
Thanks!
elro
I had the same error. Fixed it by making sure the cmake wasn't the one that comes with cygwin. For some reason, that cmake doesn't recognize Visual Studio.
Did you forget the native-win profile? Try mvn package -Pdist,native-win -DskipTests -Dtar
I've built hadoop on Windows before and didn't have to change any source code. Try changing the source back, run the above command, and post more of the protoc error you got with maven.
I need help as i am trying to figure this out from last 2-3 days..
I am setting up Hadoop on Windows-7 (64-bit) machine. This is to try out the integration of R with Hadoop.
I followed instructions for Hadoop installation as given in the URL - http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os
Environment Variables are as below
JAVE_HOME : C:\Program Files\Java\jdk1.6.0_45
M2_HOME : C:\Hadoop\apache-maven-3.1.1
PATH : C:\cygwin64\bin;C:\Hadoop\apache-maven-3.1.1\bin;C:\Hadoop\protoc-2.5.0-win32
mvn package -Pdist,native-win -DskipTests -Dtar
i ran above command from Windows SDK 7.1 command prompt and getting error below... Early help is hugely appreciated..
****************************************************************************************
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [8.893s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [4.782s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [9.500s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [2.145s]
[INFO] Apache Hadoop Project Dist POM .................... FAILURE [4.141s]
[INFO] Apache Hadoop Maven Plugins ....................... SKIPPED
[INFO] Apache Hadoop Auth ................................ SKIPPED
[INFO] Apache Hadoop Auth Examples ....................... SKIPPED
[INFO] Apache Hadoop Common .............................. SKIPPED
[INFO] Apache Hadoop NFS ................................. SKIPPED
[INFO] Apache Hadoop Common Project ...................... SKIPPED
[INFO] Apache Hadoop HDFS ................................ SKIPPED
[INFO] Apache Hadoop HttpFS .............................. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................ SKIPPED
[INFO] Apache Hadoop HDFS Project ........................ SKIPPED
[INFO] hadoop-yarn ....................................... SKIPPED
[INFO] hadoop-yarn-api ................................... SKIPPED
[INFO] hadoop-yarn-common ................................ SKIPPED
[INFO] hadoop-yarn-server ................................ SKIPPED
[INFO] hadoop-yarn-server-common ......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager .................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-yarn-client ................................ SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-yarn-project ............................... SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Gridmix ............................. SKIPPED
[INFO] Apache Hadoop Data Join ........................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Pipes ............................... SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 32.981s
[INFO] Finished at: Thu Feb 13 14:06:51 IST 2014
[INFO] Final Memory: 32M/190M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (pre-dist) on project hadoop-project-dist: An Ant BuildException has occured : exec returned: 2 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (pre-dist) on project hadoop-project-dist: An Ant BuildException has occured: exec returned: 2
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException
has occured: exec returned: 2
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:283)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: C:\Hadoop\hdfs\hadoop-project-dist\target\antrun\build-main.xml:31: exec returned: 2
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:650)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:676)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:502)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:270)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-project-dist
Thanks in advance
Gopal
Have you solved the problem? I had the similar problem when I was installing Hadoop. I found that all the problems in building the source file phase were related to Path variable setting.
1. Make sure you have JDK installed not only JRE available
2. One additional path setting needed, if you are following instruction on this link: http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os :
Add .NET MsBuild.exe home directory to Path, it should be similar to this C:\Windows\Microsoft.NET\Framework64\v4.030319
3. Check all the path values are corrected. During my installation, I had a problem which was related to Cygwin path, cygwin was installed in the folder named cygwin64, but at the beginning I just add cygwin as path value, which took me some time to figure it out.
I use the following command, it succeeds.
mvn clean package -DskipTests -Dtar
"mvn package -Pdist,native-win -DskipTests -Dtar" According to this command you are trying to compile and package "native libraries"(native-win keyword) and to do this whatever packages given in your given link is not sufficient.
To solve this you have two alternatives:
1) No need of native libraries, hadoop will use "builtin-java classes" instead of native libraries. use this command : "mvn package -Pdist -DskipTests -Dtar" instead of yours
2)if you want to go for native libraries- Follow Native Libraries Guide. Here all needed packages and other important information for building native libraries is given.
My suggestion is use 1st option because i read somewhere that building native libraries on windows have lot of issues and not preferred on windows.
Read "Building.txt" in Hadoop src folder it may help you for using variety of commands for packaging.
Above may be the solution for your problem.Try it
I had the same error. When I ran mvn package -Pdist,native-win -DskipTests -Dtar with -X, maven showed me that there are some troubles with executing dist-copynativelibs.sh script.
The problem was that Windows command prompt can't run sh commands, but only bash commands. Don't know why - Cygwin was installed correctly and added to path. So I have changed executable="sh" to executable="bash" in pom file D:\hdfs\hadoop-project-dist\pom.xml:
<exec executable="bash" dir="${project.build.directory}" failonerror="true">
<arg line="./dist-copynativelibs.sh"/>
</exec>
and build was success.