debugging a mahout logistic regression - hadoop

Question

I am new to mahout.. And I am trying out the standard "donut" example listed here:
http://imiloainf.wordpress.com/2011/11/02/mahout-logistic-regression/
So this example works like a charm.
But when I try to implement it on my dataset (whcih is a huge dataset) it doesnt works.
The dataset is in one csv file.. everything is same except it has a lot of features (~100) and is 1TB file.
I am getting this error.
bin/mahout trainlogistic --input /path/mahout_input/complete/input.csv \
--output mahoutmodel --target default --categories 2 --predictors O1 E1 I1 \
--types numeric --features 30 --passes 100 --rate 50
Running on hadoop, using HADOOP_HOME=/opt/mapr/hadoop/hadoop-0.20.2
No HADOOP_CONF_DIR set, using /opt/mapr/hadoop/hadoop-0.20.2/conf
Exception in thread "main" java.lang.NullPointerException
at org.apache.mahout.classifier.sgd.CsvRecordFactory.firstLine(CsvRecordFactory.java:167)
at org.apache.mahout.classifier.sgd.TrainLogistic.main(TrainLogistic.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
What am i doing wrong?
How do you debug this.. what is the error??
Thanks

Answer 1

My guess is your input doesn't exist or is empty. I'd check that /path/mahout_input/complete/input.csv is really what you mean.

Answer 2

Either check your input path or make sure your first line of input path having values in "" only like "x1","x2","x3","lablel"..so on

Answer 3

Happened to me as well.
My fault was to bypass an incorrect --target parameter, which does not exists in columns. Specifically my header line was
myColumn1,myColumn2,myColumn3
and my command line was
mahout trainlogistic --input ./input.csv --output ./logistic_model
--target myMisTypedColumn1 --predictors myColumn2 myColumn3 --types w w w --features 2 --passes 100 --rate 50 --categories 2
One another tip is: Dont use " ( quotes) or long column names, so you should avoid the headache of "Does mahout did not like my column name?" etc.
And as a feedback to MAHOUT: the error message is terrible. We should never see a "NullPointerException" in such a promising framework.