Need to create kafka-connect JdbcSinkConnector for Oracle DB using Datasource details which are externalized in LDAP.
Following are the connector details kafka-connect-jdbc/sink-quickstart-oracle.properties where connection details like connection.url, connection.username, connection.password etc. should come from LDAP.
name=test-sink
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
tasks.max=1
topics=orders
connection.url= ???? #should be sourced from LDAP
connection.username= ???? #should be sourced from LDAP
connection.password= ???? #should be sourced from LDAP
auto.create=true
Out of the box, Kafka does not provide such a feature or configuration.
As part of KIP-297, you would have to write/find your own implementation of a ConfigProvider from which you could do an LDAP lookup, and add that built JAR to your Connect workers' classpaths.
Related
I'm trying to access a SAP Advantage DB with kafka connect using JDBC.
I'm using the docker container, and I have added the jdbc driver Jar
FROM: http://devzone.advantagedatabase.com/dz/content.aspx?Key=20&Release=19&Product=12&Platform=11
When I try to use it I get a bad URL error
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
mode=bulk
topic.prefix=adv-
connection.password=password
tasks.max=1
connection.user=admin
name=JdbcSourceConnector2
connection.url=jdbc:extendedsystems:advantage://localhost:6262/mydb
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter=org.apache.kafka.connect.json.JsonConverter
You need to register the Database in Dialect class of JDBC connector. Only then "advantage" URL will be identified
Please add advantage in the class and rebuild the connector . Then you will not face the above issue of Bad URL error
I'm using Spring Boot 1.3.1 and I can auto-configure my JMS components through application.properties.
As I'm using Artemis as my message broker, I'm using spring-boot-starter-artemis, and as states the documentation in this link, all I have to do is replace the spring.hornetq.* properties and place spring.artemis.* instead.
The problem is that it doesn't show how to configure the username, password and SSL configuration for HornetQ/Artemis. I configured Artemis to use SSL and user/pass authentication like it's shown in here.
Does anyone know what properties can I state to add SSL and user/pass?
Thanks!
Okay, currently the Artemis Spring Boot starter is very basic, it really doesn't support clusters, SSL or client authentication.
Doing a quick look at the starter's source code in GitHub, in the ArtemisConnectionFactoryFactory.java file, at lines 127 and onwards, I will have to add the TransportConstants necessary to configure SSL, authentication, etc.
An example of a more complete connection configuration is in Artemis' test source code, in CoreClientOverTwoWaySSLTest.java, on lines 178 and onwards, there is a really complete connection setup, so to contribute, I have to change the Artemis auto-configuration properties to take the new property options, and add them all to ArtemisConnectionFactoryFactory.java following the example in CoreClientOverTwoWaySSLTest.java
I'll do a fork on the starter, make the modifications, and figure out what bureaucracy is needed to submit a pull request and get it accepted on spring-boot.
Spring Boot does not support such arrangement. Not everything should happen via properties if you ask me but in any case a good way to find out is asking your IDE to auto-complete the keys for that namespace (you'll quickly find out there is nothing related to SSL and security in there). If you don't use an IDE, this appendix should help
I've been searching extensively for a description of how to set up JMS access from a remote client to a file based JNDI MQ Series provider without success.
My JMS client works Ok on the same Linux machine as my MQSeries 7.5 server using file based JNDI.
How does one set up a remote client to use file based JNDI? Is it even possible or must one use LDAP?
I've seen hints that one should be able to have a remote client but nothing very clear.
I'm using Spring JMSTemplate which uses a provider url. On the same machine my Tomcat context.xml file uses a file: fileName url which, as I say, works ok collocated with the MQSeries server.
Thanks
Not a problem. If you are using a File based JNDI then you just need to add a QCF that contains the appropriate information for the remote queue manager. i.e. hostname, port # and channel name
DEFINE QCF(myQCF) QMANAGER(MQWT1) CHANNEL(TEST.CHL) HOSTNAME(22.22.22.22) PORT(1414) TRANSPORT(CLIENT) FAILIFQUIESCE(YES)
I was assuming that there was more than there is to file based JNDI. All it is is reading a property file. Using the "file"" url format allows you to read remote files.
Anyone have any experience using JMeter and Oracle AQ? I'm trying to publish messages to the Oracle AQ queue using JMS Publisher but the only information I have is from a bindings file.
First, read this documentation (Oracle® Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server, Chapter 8: Interoperating with Oracle AQ JMS), it provides you what you need to know about AQ user permission grants, AQ JMS Connection Factories names, and AQ Remote JNDI JMS Destination names. It does say WebLogic, but they use the very same config we do.
Put Oracle JDBC jar (ojdbc6.jar) and Oracle AQ jar (aqapi.jar, found in Weblogic's $WL_HOME/server/lib) in JMeter's lib dir.
Create a jndi.properties file with the following entries:
java.naming.factory.initial=oracle.jms.AQjmsInitialContextFactory
db_url=Your Oracle DB instance JDBC URL
java.naming.security.principal=username_with_aq_permission_grants
java.naming.security.credentials=password
Put the jndi.properties into a jar with
jar cf my-jndi-properties.jar jndi.properties
and put it in that same JMeter lib dir.
Create a JMS Publisher in JMeter, and fill these:
Check "Use jndi.properties file"
Connection Factory: See Table 8-1 and put a AQ JMS Prefix Value. Don't use the generic factories.
Destination: Either Queues/Your_Queue_Name or Topics/Your_Topic_Name
Uncheck "Use Authorization?"
Fill out what else you need.
Have fun.
Thanks David L for the jndi.properties info.
I know that application servers can typically provide a connection to your database via JNDI. Is it possible for hibernate running in Java SE to provide a JNDI url/datasource like an application container such as Tomcat/Glassfish/Jetty would normally?
From the documentation it seems like it should be possible. You just have to provide a few parameters:
Important Hibernate properties for JNDI datasources
hibernate.connection.datasource (required)
hibernate.jndi.url
hibernate.jndi.class
hibernate.connection.username
hibernate.connection.password