Reading data from csv file using pig - hadoop

Question

I'm trying to read a csv file on pig shell on mac. All I'm doing is load a file into a variable and dump the variable. Here is how I'm doing it:
movies = LOAD '/user/myhome/movies_data.csv' USING PigStorage(',') as (id,name,year,rating,duration);
DUMP movies;
The data I'm using is downloaded from github provided here
This file is available in locally installed hdfs on my mac. When I do dump I get an error:
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias movies
at org.apache.pig.PigServer.openIterator(PigServer.java:935) at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:754)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:376)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66) at
org.apache.pig.Main.run(Main.java:565) at
org.apache.pig.Main.main(Main.java:177) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497) at
org.apache.hadoop.util.RunJar.run(RunJar.java:221) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by:
java.io.IOException: Job terminated with anomalous status FAILED at
org.apache.pig.PigServer.openIterator(PigServer.java:927) ... 13 more
When I hit the app cluster link when this job is run, I get the following exception:
Diagnostics: Exception from container-launch.
Container id: container_1443887668938_0007_02_000001 Exit code: 127
Stack trace: ExitCodeException exitCode=127: at
org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at
org.apache.hadoop.util.Shell.run(Shell.java:455) at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) Container exited with a
non-zero exit code 127 Failing this attempt. Failing the application.
Pig version is 0.15.0 and hadoop is 2.6.1. Am I missing something here?

Answer 1

You could use CSVLoader from piggybank. Get the piggybank jar if not available and register it and use the CSVLoader. Something like this.
register '/your/path/to/piggybank/jar' ;
define CSVLoader org.apache.pig.piggybank.storage.CSVLoader();
movies = LOAD '/user/myhome/movies_data.csv' USING CSVLoader as (id,name,year,rating,duration);