I'm trying to implement JDBCRealm in tomcat (as described in http://tomcat.apache.org/tomcat-4.1-doc/realm-howto.html#Configuring%20a%20Realm) to check credential agains Oracle Database. The thing is that I want database to communicate over SSL. So I configured listener to use TCPS. Like that:
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=<hostname>)(PORT=1521)))<br>
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=<hostname>)(PORT=1512)))
Generated certificates, etc. I have no problem connecting to database using tcps from sqlplus or from WLS (I can use connection property oracle.net.ssl_cipher_suites=(SSL_DH_anon_WITH_3DES_EDE_CBC_SHA,SSL_DH_anon_WITH_RC4_128_MD5,SSL_DH_anon_WITH_DES_CBC_SHA) there without any problem).
However, I cant get JDBCRealm to work with the database over TCPS. If I configure realm like this:
<Realm className="org.apache.catalina.realm.JDBCRealm" debug="99" driverName="oracle.jdbc.driver.OracleDriver" connectionURL="jdbc:oracle:thin:#(DESCRIPTION = (ADDRESS = (PROTOCOL = TCPS)(HOST = <hostname>)(PORT = 1512)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = <service name>)))" connectionName="<login>" connectionPassword="<password>" userTable="users" userNameCol="user_name" userCredCol="user_pass" userRoleTable="user_roles" roleNameCol="role_name" />
I'm getting following error:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I imported certificates in JKS store which I configured in Tomcat like this:
With no success.
I'm not a professional in Tomcat (more in databases). I'll appreciate any help or pointing me in right direction. Thanks in advance!
Error seems to indicate that SSL certificate is not trusted.
I suggest to verify that you indeed have certificate in your trust store (there is a command line tool in JDK to list trust store content, you can Google it) and then add following parameter in Tomcat startup script:
javax.net.ssl.trustStore=<path to trust store>
Related
I am developing a Spark Streaming application which would listen to a folder (partitioned as yyyyMMdd) and aggregate the number of records written per minutes then persist the results to an Oracle table.
I have developed a JDBCSink (ForeachWriter) and in the open method I'm trying to open a connection to Oracle but I am getting "oracle.net.ns.NetException: could not resolve the connect identifier" exception while creating the Oracle connection. I am using Oracle wallet (SSO) and I'm able to connect over sqlplus using this wallet by setting TNS_ADMIN environment variable.
I am pushing the tnsnames.ora, sqlnet.ora, cwallet.sso and ewallet.p12 with the spark-submit --files option, and I have verified the files are pushed to the executors with the SparkFiles.get method in the sink class. I have also added third party Oracle dependencies for Oracle wallet with spark-submit --jars option (namely ojdbc7.jar,oraclepki.jar,osdt_cert.jar,osdt_core.jar)
The code piece for opening the connection is as follows:
Class.forName("oracle.jdbc.driver.OracleDriver")
System.setProperty("oracle.net.tns_admin", new Path(SparkFiles.get("tnsnames.ora")).getParent.getName)
val ds = new OracleDataSource()
val props = new Properties()
props.setProperty(OracleConnection.CONNECTION_PROPERTY_WALLET_LOCATION,
new Path(SparkFiles.get("cwallet.sso")).getParent.getName)
ds.setConnectionProperties(props)
ds.setURL("jdbc:oracle:thin:#xe")
I have tried to isolate the problem
The Oracle version is 12.1.0.2 (I am using a Docker image)
spark-submit2 ^
--master local ^
--files "%CWD%\wlt\tnsnames.ora,%CWD%\wlt\sqlnet.ora,%CWD%\wlt\cwallet.sso,%CWD%\wlt\ewallet.p12" ^
--jars "%CWD%\lib\ojdbc7.jar,%CWD%\lib\oraclepki.jar,%CWD%\lib\osdt_cert.jar,%CWD%\lib\osdt_core.jar" ^
--class OJDBCSinkMain ^
.\target\spark-streaming-ojdbc-sink-1.0-SNAPSHOT-jar-with-dependencies.jar
My sqlnet.ora file is as follows:
NAMES.DIRECTORY_PATH=(TNSNAMES, EZCONNECT)
SQLNET.WALLET_OVERRIDE=TRUE
SSL_CLIENT_AUTHENTICATION=FALSE
SSL_VERSION=0
and my tnsnames.ora file is:
xe =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = xe)
)
)
I also verified the credentials for Oracle service xe exists in my wallet:
comment mkstore -wrl . -listCredential
List credential (index: connect_string username)
1: xe system
Do you have any comments? Thanks in advance.
My mistake, I needed a rubber dock to spot!!! SparkFiles.get("tnsnames.ora")).getParent.getName returns relative path, not the absolute. My problem is solved now.
I have Windows Server 2008 r2 with myora.local domain and AD. There is server Oracle. And I have client on Win7.
On server:
-C:\krb\krb.conf
MYORA.LOCAL
MYORA.LOCAL myora.local admin server
-C:\krb\krb5.realms
[libdefaults]
default_realm=MYORA.LOCAL
[realms]
MYORA.LOCAL= {
kdc=DomainController.myora.local:88
}
[domain_realm]
.local.myora=MYORA.LOCAL
-sqlnet.ora
SQLNET.KERBEROS5_CONF= c:\krb\krb.conf
SQLNET.KERBEROS5_REALMS = c:\krb\krb5.realms
SQLNET.KERBEROS5_CC_NAME = C:\krb\v5srvtab
SQLNET.AUTHENTICATION_SERVICES= (BEQ, TCPS, NTS, KERBEROS5)
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
SQLNET.KERBEROS5_CONF_MIT = TRUE
SQLNET.AUTHENTICATION_KERBEROS5_SERVICE = krbtgt
okinit work for Kefir#MYORA.LOCAL (user name on computer-server)
I do all the same steps on client (except sqlnet.ora), but okinit trows error:
From local name user1 it can't find, from computer name It has credential problems.
What should I change or add?
The error may be telling you that for your database service principle (the account you issued the keytab from ) you haven't selected "pre-authentication not required" in account properties.
For your client you should remove beq from the list of authentication services in sqlnet.ora. If you aren't using them also remove NTS (windows native) and TCPS (certificate).
In your krb5.conf file add upper-case to your domain realm, as well as an alias not prefixed by a period - like this:
[domain_realm]
.local.myora=MYORA.LOCAL
local.myora=MYORA.LOCAL
.MYORA.LOCAL=MYORA.LOCAL
MYORA.LOCAL=MYORA.LOCAL
You might want to take a look at this video - there's a chance it will answer other questions you are likely to run into
https://www.youtube.com/watch?v=d_d0j9ssQys&ab_channel=OracleDevelopers
I am using Azure databricks and I ran the following Python code:
sas_token = "<my sas key>"
dbutils.fs.mount(
source = "wasbs://<container>#<storageaccount>.blob.core.windows.net",
mount_point = "/mnt/gl",
extra_configs = {"fs.azure.sas.<container>.<storageaccount>.blob.core.windows.net": sas_token})
This seemed to run fine. So I then ran:
df = spark.read.text("/mnt/gl/glAgg_LE.csv")
Which gave me the error:
shaded.databricks.org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Not sure what I'm doing wrong though. I'm pretty sure my sas key is correct.
Ok if you are getting this error - double check both the SAS key and the container name.
Turned out I had pointed it to the wrong container!
i want to create an origin source from oracle. so I choos as origin oracle cdc. then I configured each parameter:
Schema Name
Table
Username
Password
JDBC Connection String
but when I run the process, i find into my log:
2017-08-22 11:07:22,447 test/testb156f588-dbd7-4e4c-8896-caf658d14d77 ERROR Error while connecting to DB
com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: java.lang.RuntimeException: Unable to get driver instance for jdbcUrl=jdbc:oracle:thin:#(DESCRIPTION =
(ENABLE=BROKEN)
(ADDRESS = (PROTOCOL = TCP)(HOST = myhost)(PORT = myport))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = orcl.WORLD)))
at com.streamsets.pipeline.lib.jdbc.JdbcUtil.createDataSourceForRead(JdbcUtil.java:638)
at com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.OracleCDCSource.init(OracleCDCSource.java:643)
at com.streamsets.pipeline.api.base.BaseStage.init(BaseStage.java:52)
at com.streamsets.pipeline.configurablestage.DStage.init(DStage.java:40)
at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:156)
at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:105)
at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:53)
at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:299)
at com.streamsets.datacollector.runner.Pipeline.init(Pipeline.java:214)
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:96)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:79)
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:646)
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:143)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:233)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Unable to get driver instance for jdbcUrl=jdbc:oracle:thin:#(DESCRIPTION =
(ENABLE=BROKEN)
(ADDRESS = (PROTOCOL = TCP)(HOST = myhost)(PORT = myport))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = orcl.WORLD)))
at com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:88)
at com.zaxxer.hikari.pool.PoolElf.initializeDataSource(PoolElf.java:157)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:113)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:73)
at com.streamsets.pipeline.lib.jdbc.JdbcUtil.createDataSourceForRead(JdbcUtil.java:630)
... 19 more
Caused by: java.sql.SQLException: No suitable driver
at java.sql.DriverManager.getDriver(DriverManager.java:315)
at com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:81)
... 23 more
Do you have any idea?
I had to add the ojdbc jar under the repository
/opt/streamsets-extra/streamsets-datacollector-jdbc-lib/lib
Seems that it's missing to set environment variable STREAMSETS_LIBRARIES_EXTRA_DIR which's a part of installing external library installation process.
There are three types of installation depending on initialization style of Data Collector :
Manually start
Using SysV Init ( supported by CentOS 6, Oracle Linux 6, Red Hat
Enterprise Linux 6 or Ubuntu 14.04 LTS )
Using Systemd Init ( supported by CentOS 7, Oracle Linux 7, Red Hat
Enterprise Linux 7 or Ubuntu 16.04 LTS )
If you're installing SDC
being started manually, then variable STREAMSETS_LIBRARIES_EXTRA_DIR is expected to be set from the command line by
export STREAMSETS_LIBRARIES_EXTRA_DIR="/opt/streamsets-data-collector/streamsets-data-collector-3.15.0/streamsets-libs-extras/"
starting as a service, then this parameter already would already exist
within the $SDC_DIST/libexec/_sdc file as
STREAMSETS_LIBRARIES_EXTRA_DIR="${STREAMSETS_LIBRARIES_EXTRA_DIR:=${SDC_DIST}/streamsets-libs-extras}"
where $SDC_DIST variable is the extraction directory for SDC installation file(tarball or RPM).
and the same path should be added to the file $SDC_CONF/sdc-security.policy such as
grant codebase "file:///opt/streamsets-data-collector/streamsets-data-collector-3.15.0/streamsets-libs-extras/-" {
permission java.security.AllPermission;
};
where $SDC_CONF variable is typically defined by the path /etc/sdc
Now, we're ready to log in Data Collector Console in order to add JDBC External library following these steps :
In Data Collector, in the top right toolbar, click the Package
Manager icon:
In the navigation panel, click External Libraries :
Data Collector lists any currently installed external libraries.
Immediately under the top right toolbar, click the Install
External Libraries icon:
In the Install External Libraries dialog box, select the stage
library JDBC from the file system (assuming you were already registered and downloaded streamsets-datacollector-jdbc-lib);
and the choose the .jar file such as ojdbc8.jar which can be downloaded from JDBC and UCP Downloads page ( in my case I've chosen the link named Oracle Database 12c Release 2 (12.2.0.1) drivers due to my remote DB version ).
As the last step, Don't forget to click Cancel in the Install External Libraries window, and then run the following command :
service sdc restart ( for SysV Init)
or
systemctl restart sdc ( for Systemd Init )
( You can click Restart Data Collector in the Install External Libraries window provided that you started the Data Collector manually from the command line. )
Under the Legacy configuration tab try specifying the JDBC Driver Class name as oracle.jdbc.driver.OracleDriver
I'm trying to connect to IBM WAS from local machine. I'm trying to connect to server using following code:
Properties Props = new Properties()
Props.setProperty(AdminClient.CONNECTOR_TYPE,
AdminClient.CONNECTOR_TYPE_SOAP)
Props.setProperty(AdminClient.CONNECTOR_SECURITY_ENABLED, "true")
Props.setProperty(AdminClient.CACHE_DISABLED, "false")
Props.setProperty("javax.net.ssl.trustStore", "WebAS")
Props.setProperty("javax.net.ssl.trustStorePassword", "WebAS")
Props.setProperty(AdminClient.CONNECTOR_HOST, "127.0.0.1")
Props.setProperty(AdminClient.CONNECTOR_PORT, "9060")
Props.setProperty(AdminClient.USERNAME, "user")
Props.setProperty(AdminClient.PASSWORD, "password")
System.setProperty("com.ibm.SSL.ConfigURL", /specified location of
file/ "ssl.client.props") Props.setProperty("com.ibm.SSL.ConfigURL",
/specified location of file/ "ssl.client.props") //I generated
keystore.jks `
Props.setProperty("javax.net.ssl.trustStore",directory.toURI().toURL()+"keystore.jks"
); Props.setProperty("javax.net.ssl.keyStore",
directory.toURI().toURL()+"keystore.jks");
this.adminClient=AdminClientFactory.createAdminClient(Props)
When run the following code, I get following error
com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: The system cannot create a SOAP connector to connect to host 127.0.0.1 at port 9043.
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.NoClassDefFoundError: com/ibm/security/certclient/util/PkSsCertFactory
Caused by: java.lang.ClassNotFoundException: com.ibm.security.certclient.util.PkSsCertFactory
I looked up for classname: com.ibm.security.certclient.util.PkSsCertFactory to include. I cannot find neither the IBM jar file or groupId, artifact id, version. Please help me out
Export the dmgr's CA certificate to nodes truststore and give a try.
As I can see your node agent is taking port 9043 to connect to dmgr, if all configurations made are default, then this should not happen.
please check your nodes and dmgrs serverindex.xml file.