Apache Drill - Slow Queries - hadoop

Question

I have the following storage plugin set-up in Drill:
{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "thrift://hivemetastore.hostname.com:9083",
"hive.metastore.sasl.enabled": "false"
}
}
however, a simple
SELECT * FROM hive.table LIMIT 5;
...
5 rows selected (35.383 seconds)
0: jdbc:drill:>
is taking over 30 seconds to respond. What am I missing / where should I begin the troubleshoot?
The Hive metastore server is the same as Drill right now. And there are less than 20,000 records in the table.

Answer 1

Only MapR Drill on the MapR sandbox should use the sparse storage plugin config you are using. In the sandbox, things are configured under the covers.
EMBEDDED METASTORE SERVICE
Assuming you are using a Drill installation, not the sandbox, and you're using the embedded metastore service (the default), configProps needs to look something like this (per the docs):
"configProps": {
"hive.metastore.uris": "",
"javax.jdo.option.ConnectionURL": "jdbc:<database>://<host:port>/<metastore database>",
"hive.metastore.warehouse.dir": "/tmp/drill_hive_wh",
"fs.default.name": "file:///",
"hive.metastore.sasl.enabled": "false"
}
Remove the "hive.metastore.uris": "thrift://:" from your storage plugin config. That is for use with a remote hive metastore service.
The "javax.jdo.option.ConnectionURL" might be a MySQL database. The Hive metastore service provides access to the physical DB like MySQL. MySQL stores the metadata. The "fs.default.name" is the file system location where the data is located.
The embedded metastore configuration is for testing only, and not for use in production systems per the docs. For performance improvement, please configure the remote metastore. Also, check the compatibility of the version of Hive you are using. Open source Apache Drill 1.0 supports Hive 0.13. Drill 1.1 and later supports Hive 1.0.
REMOTE METASTORE SERVICE
If you're using the remote metastore, the "fs.default.name" should point to the main control node. Point to a NameNode, for example. If you're using MapR Drill, "fs.default.name" should be maprfs:///. The MapR FileClient figures out CLDB locations from mapr-clusters.conf. Start the metastore service, which is installed on top of Hive as a separate package:
hive --service metastore
The remote metastore config should look something like this if you're using open source Apache Drill:
{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "thrift://mfs41.mystore:9083",
"hive.metastore.sasl.enabled": "false",
"fs.default.name": "maprfs://10.10.10.41/"
}
}