Is it possible to consume "EXPERIMENTAL CHANGEFEED FOR" (core) type queries over JDBC?
Is it possible to consume "CREATE CHANGEFEED FOR" (enterprise) type queries over JDBC?
thanks for your interest in CockroachDB changefeeds. Enterprise changefeeds should work fine with JDBC or any other SQL driver: the CREATE CHANGEFEED statement sets up the changefeed to deliver data to a Kafka or cloud storage target, and immediately returns a job ID that you can use to monitor the health of the changefeed via the SHOW JOBS statement or the web UI.
Core changefeeds work a little differently from other SQL statements: when you issue a CHANGEFEED FOR statement, CockroachDB streams results back indefinitely and never returns unless something goes wrong or the query is canceled. Currently, this streaming behavior isn't implemented in the way that the Postgres JDBC driver expects (see #4035 and the linked work-in-progress PRs), so consuming results using Postgres JDBC cursors won't work. We're working on adding support for this.
Related
I am trying to use Kafka jdbc connector to only pull in rows from my database that have changed since the last pull.
The database is controlled by another team and they have a habit of reloading the entire database twice a day even if no information have changed. They also update the field :load-time, so the kafka connector, it will always look like a change.
Is there a way to tell kafka jdbc connector to only look in the relevant columns to detect a change?
I’m trying to learn about streaming services and reading kafka doc’s :
https://kafka.apache.org/quickstart
https://kafka.apache.org/24/documentation/streams/quickstart
To take a simple example I’m attempting to refactor a Spring web services GET request which accepts an ID parameter and returns a list of attributes associated with that ID. The DB backend is Oracle.
What is the approach for loading a single Oracle DB table which can be served by Kafka ? The above docs don't contain information for this. Do I need to replicate the Oracle DB to a NoSql DB such as MongoDB ? (Why we require Apache Kafka with NoSQL databases?)
Kafka is an event streaming platform. It is not a database. Instead of thinking about "loading a single Oracle DB table which can be served by Kafka", you need to think in terms of what events are you looking for that will trigger processing?
Change Data Capture (CDC) products like Oracle Golden Gate (there are other products too) will detect changes to rows and send messages into Kafka each time a row changes.
Alternatively you could configure a Kafka JDBC Source Connector to execute a query and pull data into Kafka.
I am developing java applications which connect to Oracle databases. Application loads some amount of data on startup. Because of slow connection to TEST environment, applications start up take some time.
I am looking if there is some proxy/cache tool, which would locally store results for every query. So it could load result from memory if query was already called, instead of calling DB again. This could save a lot of time.
I guess ProxySQL does something similar but it is targeted for MySQL. Is there something for Oracle DB ?
Check out the Oracle Client Result Cache. It works with the JDBC OCI driver.
https://docs.oracle.com/database/121/TGDBA/tune_result_cache.htm#TGDBA626
I am trying to submit XQuery queries to an Oracle 11g database through their XQJ API.
When I instantiate an oracle.xquery.xqj.OXQDataSource as explained in http://www.oracle.com/technetwork/articles/oem/xquery-jdbc-325944.html, I can submit queries fine except that I haven't found how I can set up the server connection (server name, port, username, password, ...) info:
This datasource claims that it doesn't support setting any property.
It doesn't implement the data source constructor which takes a JDBC connection.
I don't see any non standard method to set such info.
When I try to access some random collection like collection("oradb:/foo") I just get an empty result set even when no server is running, suggesting that the driver doesn't even try to connect.
What have I missed and how can I set the server connection info?
Thanks,
Eric
Thanks to Charles Foster I can answer to my own question: the XQJ implementation from Oracle is an old standalone version from January 2010 that is pretty useless and doesn't interact with Oracle databases.
Despite all the Oracle statements about XQJ, I haven't been able to find any client/server XQJ implementation (except one from DataDirect of course) and the way to submit XQuery queries to Oracle databases appears to be through JDBC, embedded in PL-SQL statements.
It is possible in 12.
XQJ to run queries in Java:
http://docs.oracle.com/database/121/ADXDK/adx_j_xqj.htm#ADXDK99930
XQJ to run queries against the database:
http://docs.oracle.com/database/121/ADXDK/adx_j_xqjxdb.htm#ADXDK136
I need to see the query being sent to Oracle from a Java program. In the PostgreSQL JDBC driver, toString() does the job, but the same does not apply to prepared statements from Oracle JDBC implementation. Any ideas how to achieve that?
Check out Log4Jdbc. This sits between your JDBC driver and the application, logging all DB traffic that goes back and forth. It's driver-agnostic, so need for driver-specific logging code.
Extremely handy, and would be even handier if it supported DataSources, but sadly it doesn't.
I think the getOriginalSql() method returns the String being sent to Oracle.