We have two different Azure cloud resource groups, RG1 and RG2, where RG1 hosts the ADB_source of the data source, and RG2 hosts the ADB_sink & ADLS_sink(gen2) of the data sink.
Use Case:
We have a few delta tables in ADB_source (ACL enabled) where a list of users has Read access.
In the ADB_source workspace, we need to read the delta tables and write them into ADLS_sink as parquet for further processing at the sink.
What's Available:
We have a high concurrency cluster created in ADB_Source workspace, which -
Allows only Python & SQL (dbutils.fs also restricted).
Credential Passthrough is disabled.
Has ACLs Enabled in spark config.
Has mount point created to a container in ADLS_sink.
Has no Admin Access to the cluster.
Errors Observed:
We could read the delta tables as expected and run action commands as long as they are in the ADB_source workspace. However, when we write that data into the ADLS_sink with .save(), we get the below error.
Py4JJavaError: An error occurred while calling o410.save. : java.lang.SecurityException: User does not have permission SELECT on any file. User does not have permission MODIFY on any file.
I would appreciate it if anyone could explain this and recommend additional security checks/accesses needed to implement the use case successfully.
This is happening because ACL is enabled, please refer to the documentation below:
https://learn.microsoft.com/en-us/azure/databricks/kb/security/table-create-security-exception
Related
I have delete task on Azure Data Factory which I want delete files on VM with FileSystem as Linkedservice.
but i try debug this activity have error and denied permission like this:
FYI, i can delete this file from VM with delete manual with press key del on keyboard. big many thanks to answer my question.
Make sure you have followed Prerequisites and the username and password provided at FileSystem linked service has enough privilege (write permission) to delete the source file. (latest version of self-hosted integration runtime, If the access is restricted to IPs that are approved in the firewall rules, you can add Azure Integration Runtime IPs to the allow list)
You can verify access by using a lookup activity with same source file.
In FileSystem linked service you would have to specify the path to a Folder, i.e. "the root path of the folder or the Azure File Storage endpoint"
e.g. C:\[Folder]
Next in the Dataset you can further specify the relative directory or the File name.
I am trying to create 30 databases (oci_database_database resource) under 5 existing db_homes. All of these resources are under a single DB System :
When applying my code, a first database is successfully created then when terraform attempts to create the second one I get the following error message : "Error: Service error:IncorrectState. The existing Db System with ID has a conflicting state of UPDATING", which causes the execution to stop.
If I re-apply my code, the second database is created then I get the same previous error when terraform attempts to create the third one.
I am assuming I get this message because terraform starts creating the following database as soon as the first one is created, but the DB System status is not up to date yet (still 'UPDATING' instead of 'AVAILABLE').
A good way for the OCI provider to avoid this issue would be to consider a database creation as completed when the creation is indeed completed AND the associated db home and db system's status are back to 'AVAILABLE'.
Any suggestion on how to adress the issue I am encountering ?
Feel free to ask if you need any additional information.
Thank you.
As mentioned above, it looks like you have opened a ticket regarding this via github. What you are experiencing should not happen, as terraform should retry after seeing the error. As per your github post, the person helping you is in need of your log with timestamp so they can better troubleshoot. At this stage I would recommend following up there and sharing the requested info.
I'm looking for the correct way to give users anonymous access to a Kibana dashboard, but at the same time preventing them from having access to other Kibana features.
I read that the role kibana_dashboard_only_user is now deprecated (I'm on 7.6.2), and that I have to give anonymous users the cluster monitor privilege, so I tried creating a role that has:
cluster monitor privilege
read and view-metadata access to the index concerned
dashbaord read privilege on global spaces
but is doesn't work: I get the following error:
[security_exception] action [indices:data/read/search] is unauthorized
for user [anonymous_user]
then I tried adding the read rights on the indices .kibana*. It worked, but all the Kibana features are available, not only the dashboards.
How can I solve this?
Thanks.
I installed IBM Filenet Content Engine 5.2,on my machine.I am getting problem while configuring GCD datasources for new profile.
Let me first explain the setps I did,then I would mention the problem that I am getting.
First,I created GCD database in DB2,then I created datasources required for configuration of profile in WAS Admin Console.I created J2C Authentication Alias,for user which has access to GCD database and configured it with datasources.I am getting test database connection as successful but when I run task of configuring GCD datasources,it fails with the following error:-
Starting to run Configure GCD JDBC Data Sources
Configure GCD JDBC Data Sources ******
Finished running Configure GCD JDBC Data Sources
An error occurred while running Configure GCD JDBC Data Sources
Running the task failed with the following message: The data source configuration failed:
WASX7209I: Connected to process "server1" on node Poonam-PcNode01 using SOAP connector; The type of process is: UnManagedProcess
testing Database connection
DSRA8040I: Failed to connect to the DataSource. Encountered java.sql.SQLException: [jcc][t4][2013][11249][3.62.56] Connection authorization failure occurred. Reason: User ID or Password invalid. ERRORCODE=-4214, SQLSTATE=28000 DSRA0010E: SQL State = 28000, Error Code = -4,214.
It looks like simple error of user id and password not valid.I am using same alias for other datasources as well and they are working fine.so not sure,why I am getting error.I have also tried changing scope of datasources,but no success.Can somebody please help?
running "FileNet Configuration Manager" task of configuring GCD datasources will create all the needs things in WAS (including Alias), do not created it before manually.
I suspect it had an issue with exciting JDBC data sources/different names Alias
Seems from your message that you are running it from Filene configuration manager. Could you please double check from your database whether user id is authorised to execute query in GCD database. It is definitely do it with permission issue.
I'm trying to access to Hive by the command window.
I just run "Hive" in the appropiate directory but I get an error "Login denied".
I've read that log4j is used to log in, but I don't know whether I have to create an account and write my user data there or not.
Thank you very much
The Hive service should be working right now. From a FI-LAB VM of your own, you simply have to log into the Head Node using your Cosmos credentials (if you have no Cosmos credentials, get them by registering here):
[root#your_filab_vm]$ssh cosmos.lab.fi-ware.org
Once logged in the Head Node, type the following command:
[your_cosmos_username#cosmosmaster-gi]$ hive
Logging initialized using configuration in jar:file:/usr/local/hive-0.9.0-shark-0.8.0-bin/lib/hive-common-0.9.0-shark-0.8.0.jar!/hive-log4j.properties
Hive history file=/tmp/<your_cosmos_username>/hive_job_log_<your_cosmos_username>_201407212017_1797291774.txt
hive>
As you can see, in this example your Hive history will be written within:
/tmp/<your_cosmos_username>/hive_job_log_<your_cosmos_username>_201407212017_1797291774.txt