Why Can't I Create a Table After Successfully Creating a Schema in Hive Standalone Metastore? - trino

Question

Problem & Question
I am working on a docker-compose stack with Minio for object storage, a Hive Standalone Metastore, and Trino as the query engine. My repo is store here - https://github.com/rylativity/trino_hive-meta_minio and steps to reproduce the issue are included at the end of this post.
When I attempt to use the Hive Standalone Metastore, I get an error when attempting to create a table. I am able to create the schema with CREATE SCHEMA hive.taxi;, and I can see the taxi.db folder created in the /opt/warehouse folder in my Hive Metastore container when I run the CREATE SCHEMA command. However, when I run CREATE TABLE hive.taxi.trips (...<col_type_info>...) WITH (external_location = 's3a://test/trips, format = 'PARQUET'), I immediately see an NoSuchObjectException error thrown by the ThriftMetastoreClient in the Trino container's logs that says the hive.taxi.trips table cannot be found and then a timeout occurs 60s later:
trino_1 | 2022-06-07T15:09:41.496Z DEBUG dispatcher-query-13 io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:taxi, tblName:trips, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) took 29.53ms and failed with NoSuchObjectException(message:hive.taxi.trips table not found)
However, if I use a FileHiveMetastore (stored within the Trino container), I am able to create and query the table successfully, which leads me initailly to believe that the issue is with my Hive Standalone Metastore setup and not with the other services.
Does anyone know why I might be receiving a NoSuchObjectException - hive.taxi.trips table not found when attempting to create a table and after I am able to successfully create the hive.taxi schema?
Steps to Reproduce
Clone the repo - https://github.com/rylativity/trino_hive-meta_minio
Run docker-compose up -d to bring up the containers. The minio-init container will create a 'test' bucket and an access-key/secret to access the bucket.
Run ./dataload_scripts/load_taxidata_to_minio.sh. This will download parquet files containing yellow-cab trip data from the public NYC-TLC bucket and load it into the 'test' bucket in Minio.
Run docker-compose exec trino trino to open a Trino shell inside the Trino container.
Inside the Trino shell, run CREATE SCHEMA hive.taxi; and then run
CREATE TABLE IF NOT EXISTS filehive.taxi.trips(
VendorID BIGINT,
tpep_pickup_datetime TIMESTAMP,
tpep_dropoff_datetime TIMESTAMP,
passenger_count DOUBLE,
trip_distance DOUBLE,
PULocationID BIGINT,
DOLocationID BIGINT,
RatecodeID DOUBLE,
store_and_fwd_flag VARCHAR,
dropoff_longitude DOUBLE,
dropoff_latitude DOUBLE,
payment_type BIGINT,
fare_amount DOUBLE,
improvement_surcharge DOUBLE,
congestion_surcharge DOUBLE,
mta_tax DOUBLE,
tip_amount DOUBLE,
tolls_amount DOUBLE,
extra DOUBLE,
airport_fee DOUBLE,
total_amount DOUBLE
)
WITH (
external_location = 's3a://test/taxi',
format = 'PARQUET'
);
If you run docker-compose exec metastore ls /opt/warehouse/, you will see that the taxi.db folder gets created after you run the CREATE SCHEMA... command. If you look at the Trino logs after running the CREATE TABLE... command, you will see the "NoSuchObjectException - hive.taxi.trips does not exist" error.
If you replace "hive" with "filehive" in the CREATE SCHEMA... and CREATE TABLE... commands, you will see that there is no issue when using a Trino FileHiveMetastore.
At any point, to start over, you can run docker-compose down -v and then return to step 1