Spring XD sqoop job failure - spring-xd

Question

HortonWorks version is 2.4
SpringXd version - 1.3.1
I am trying to run a sqoop job for import from Spring XD. It is giving following stack trace. Can you let me know what is the issue here.
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main tool.CodeGenTool - Beginning code generation
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:54+0530 1.3.1.RELEASE INFO main orm.CompilationManager - HADOOP_MAPRED_HOME is /Users/nxjain/Documents/spring-xd-1.3.1.RELEASE/xd/lib/hdp22
2016-05-16T11:29:56+0530 1.3.1.RELEASE INFO main orm.CompilationManager - Writing jar file: /tmp/sqoop-nxjain/compile/855df3a42e72c92f2a5d87ffc3827e48/Events.jar
2016-05-16T11:29:56+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Beginning import of Events
2016-05-16T11:29:57+0530 1.3.1.RELEASE WARN main util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-16T11:29:57+0530 1.3.1.RELEASE INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM Events AS t WHERE 1=0
2016-05-16T11:29:57+0530 1.3.1.RELEASE WARN main mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
2016-05-16T11:29:57+0530 1.3.1.RELEASE INFO main client.RMProxy - Connecting to ResourceManager at /172.16.128.128:8050
2016-05-16T11:29:58+0530 1.3.1.RELEASE INFO main db.DBInputFormat - Using read commited transaction isolation
2016-05-16T11:29:58+0530 1.3.1.RELEASE INFO main mapreduce.JobSubmitter - number of splits:1
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.JobSubmitter - Submitting tokens for job: job_1463375311370_0002
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main impl.YarnClientImpl - Submitted application application_1463375311370_0002
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.Job - The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1463375311370_0002/
2016-05-16T11:29:59+0530 1.3.1.RELEASE INFO main mapreduce.Job - Running job: job_1463375311370_0002
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1463375311370_0002 running in uber mode : false
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - map 0% reduce 0%
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1463375311370_0002 failed with state FAILED due to: Application application_1463375311370_0002 failed 2 times due to AM Container for appattempt_1463375311370_0002_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://sandbox.hortonworks.com:8088/cluster/app/application_1463375311370_0002Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e09_1463375311370_0002_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
at org.apache.hadoop.util.Shell.run(Shell.java:487)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.Job - Counters: 0
2016-05-16T11:30:03+0530 1.3.1.RELEASE WARN main mapreduce.Counters - Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Transferred 0 bytes in 5.7269 seconds (0 bytes/sec)
2016-05-16T11:30:03+0530 1.3.1.RELEASE WARN main mapreduce.Counters - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2016-05-16T11:30:03+0530 1.3.1.RELEASE INFO main mapreduce.ImportJobBase - Retrieved 0 records.
2016-05-16T11:30:03+0530 1.3.1.RELEASE ERROR main tool.ImportTool - Error during import: Import job failed!

Answer 1

This is most likely due to a classpath issue - you could check the logs for the YARN container that failed.
Take a look at this comment: https://github.com/spring-projects/spring-xd/pull/1451#issue-56883941
There we added some specific config settings for the HDP version being used. You would have to adjust based on the version and build you are using. Just be aware that we haven't tested XD with HDP 2.4. It might work though, so let us know how you make out.