I have been given access to HUE Hive platform by my client. I have also raised all the access requests for the database and all of them are approved also. But I can't see any databases or tables in the Hive interface. Is there any procedure to connect to a database or it should reflect on the Hive interface automatically?
Related
I am trying to configure an external hive metastore for my azure synapse spark pool. The rational behind using external meta store is to share table definitions across databricks and synapse workspaces.
However, I am wondering if its possible to access backend data via the metastore. For example, can clients like PowerBI,tableau connect to external metastore and retrieve not just the metadata, but also the business data in the underlying tables?
Also what additional value does an external metastore provides ?
You can configure the external Hive Metadata in Synapse by creating a Linked Service for that external source and then query it in Synapse Serverless Pool.
Follow the below steps to connect with External Hive Metastore.
In Synapse Portal, go to the Manage symbol on the left side of the of the page. Click on it and then click on Linked Services. To create the new Linked Service, click on + New.
Search for Azure SQL Database or Azure Database for MySQL for the external Hive Metastore. Synapse supports these two Hive external metastore. Select and Continue.
Fill in all the required details like Name, Subscription, Server name, Database name, Username and Password and Test the connection.
You can test the connection with the Hive metadata using below code.
%%spark
import java.sql.DriverManager
/** this JDBC url could be copied from Azure portal > Azure SQL database > Connection strings > JDBC **/
val url = s"jdbc:sqlserver://<servername>.database.windows.net:1433;database=<databasename>;user=utkarsh;password=<password>;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;"
try {
val connection = DriverManager.getConnection(url)
val result = connection.createStatement().executeQuery("select * from dbo.persons")
result.next();
println(s"Successful to test connection. Hive Metastore version is ${result.getString(1)}")
} catch {
case ex: Throwable => println(s"Failed to establish connection:\n $ex")
}
Check the same in below snippet for your reference.
can clients like PowerBI,tableau connect to external metastore and retrieve not just the metadata, but also the business data in the underlying tables?
Yes, Power BI allows us to connect with Azure SQL Database using in-built connector.
In Power BI Desktop, go to Get Data, click on Azure and select Azure SQL Database. Click connect.
In the next step, go give the Server name in this format <utkarsh.database.windows.net>, database name, Username and Password and you can now access data in Power BI. Refer below image.
I'm starting to study Apache Kafka and Kafka Connect.
I'm trying to get data from a remote Oracle Database that my user only have read privilegies and can't list tables (i don't have permission to change that). To every query, i have to pass a dblink, but in the JDBC Connector, i didn't find a option to pass a dblink.
I can do the query if i pass a specific query on the connector configuration, but i want to fetch allot of tables and speficifying the query on the connector, would make me create allot of connectors.
There's a way to pass the dblink on the connector configuration or to the JDBC URL?
I Have a presto Database.
I want to create a new user which will be used by creating a connection via JDBC connection
I was searching this link
for creating user syntax
this is the only create options it has
8.5. CREATE SCHEMA
8.6. CREATE TABLE
8.7. CREATE TABLE AS
8.8. CREATE VIEW
The CREATE SCHEMA doesn't appear to have the ability for it - it also doesn't appear to have IDENTIFIED BY syntax.
How can I create a user with a password for Presto?
Presto does not have the notion of users or storage of its own. It relies on the underlying data source. Which connectors are you using? If you're using the Hive connector, then depending on how it's configured you could connect as a particular Hive user.
I have a PostgreSql database and I need to connect it to read data from oracle view and store that data in custom table
The PostgreSql database will connect to oracle everyday automatically to read the latest updates from oracle view
How to create it?
It sounds like you probably want a SQL/MED foreign data wrapper. Check out oracle_fdw. You could also use the generic odbc_fdw or jdbc_fdw wrappers via Oracle's ODBC or JDBC drivers.
Another option is DBI-Link.
Combine these with a cron job if you want to copy to a local view.
I have an Oracle view containing very large amount of data in it and I want to migrate this data in a table in Greenplum database. Is there any way I can write any query in Postgresql to fetch that Oracle view's data?
If not possible by query in Postgresql, kindly suggest me some way to access Oracle view from Linux server, so that I can create data file from that Oracle view to my Linux server and load that file via gpfdist to a Greenplum table.
NOTE: an Oracle view is from third party, I only have an access to view that data (I have all the connection info) I can access that view via SQL Developer
NOTE: Exporting data from SQL Developer to my local machine is not feasible here as the data is very large
Thanks,
Sunny
The last time I used Greenplum (3 years ago) I don't think there were any untrusted languages like plperlu, so fetching directly from Oracle from within Greenplum might not be possible. If the data has a primary key, are you able to fetch in batches, compress it, then ship it to Greenplum?
Do you have a Greenplum support contract? If so, you could also try them if you haven't already: https://sso.emc.com/sso/login.htm
I recall that gpfdist can be configured to fetch from remote servers with a bit of fiddling, so if you are able to copy out the Oracle data to disk, you could fetch it using gpfdist without any intermediary steps.