Method not supported in spark - jdbc

Question

First, I start thrift server in spark. /sbin/start-thriftserver.sh
and the deamon is started.
hadoop 13015 1 99 13:52 pts/1 00:00:09 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java -cp /home/hadoop/spark/lib/hive-jdbc-0.13.0.jar:/home/hadoop/spark-1.4.1-bin-hadoop2.6/sbin/../conf/:/home/hadoop/spark-1.4.1-bin-hadoop2.6/lib/spark-assembly-1.4.1-hadoop2.6.0.jar:/home/hadoop/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/home/hadoop/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/home/hadoop/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar
-Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal
After, I start /bin/pyspark
my hive version is 0.13.1,
spark version is 1.4.1,
hadoop version is 2.7
spark classpath is below.
SPARK_CLASSPATH= /home/account/spark/lib/hive-jdbc-0.13.0.jar:
/home/account/spark/lib/hive-exec-0.13.0.jar:
/home/account/spark/lib/hive-metastore-0.13.0.jar:
/home/account/spark/lib/hive-service-0.13.0.jar:
/home/account/spark/lib/libfb303-0.9.0.jar:
/home/account/spark/lib/log4j-1.2.16.jar
In pyspark(python-shell), I wrote this code.
>>> df = sqlContext.load(source="jdbc",driver="org.apache.hive.jdbc.HiveDriver", url="jdbc:hive2://IP:10000/default", dbtable="default.test")
but It's not working, I get this error. How can I resolve this error?
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/dev/user/ja/spark/python/pyspark/sql/context.py", line 458, in load
return self.read.load(path, source, schema, **options)
File "/home/dev/user/ja/spark/python/pyspark/sql/readwriter.py", line 112, in load
return self._df(self._jreader.load())
File "/home/dev/user/ja/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py", line 538, in __call__
File "/home/dev/user/ja/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/protocol.py", line 300, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o29.load.
: java.sql.SQLException: Method not supported
at org.apache.hive.jdbc.HiveResultSetMetaData.isSigned(HiveResultSetMetaData.java:141)
at org.apache.spark.sql.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:132)
at org.apache.spark.sql.jdbc.JDBCRelation.<init>(JDBCRelation.scala:128)
at org.apache.spark.sql.jdbc.DefaultSource.createRelation(JDBCRelation.scala:113)
at org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:269)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:207)
at java.lang.Thread.run(Thread.java:722)
I think HiveResultSetMetaData.isSigned method is not supported in hive.
but I don't know how can I resolve this error. please help..
thank you

Answer 1

It's Uncertain. but I answer to my question.
I think that It caused by version.
when I execute command below, I get "Method not support" error.
but when I this command at spark-1.3.1, It's worked.
>>> df = sqlContext.load(source="jdbc",driver="org.apache.hive.jdbc.HiveDriver", url="jdbc:hive2://IP:10000/default", dbtable="default.test")
so I think problem is version.
but It's my guess.
This page maybe help to you.
http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.2.4/ds_Hive/jdbc-hs2.html