I am using Snowflake JDBC to execute multi-statement scripts on Snowflake. My application is started by a Oozie job on a hadoop cluster(in migration phase). The requirement here is, when Oozie job is killed and there by killing the running application instance, the query that was submitted using JDBC should get cancelled by Snowflake.
I have added ABORT_DETACHED_QUERY=true to the JDBC connection url which looks like jdbc:snowflake://<account>.snowflakecomputing.com/?warehouse=<WH>&db=<DB>&schema=<SCHEMA>&ABORT_DETACHED_QUERY=true.
Even after 25 mins, the script execution is not cancelled by Snowflake. I tried to find out the underlying problems. I tried to query the session on SESSIONS view using session-id but it was not there. I also tried to query for active connections but could not find a way to do it.
So I have two queries,
Is it the right way to configure ABORT_DETACHED_QUERY parameter?
How do you check for active JDBC connections on Snowflake, because SHOW CONNECTIONS didn't return any connection to my application?
Also,
I am using commons-dbcp BasicDataSource as datasource manager,
commons-dbutils to submit query using int QueryRunner.execute(String) method.
This is a session parameter not a connection string parameter therefore the proper way to set it is by using an ALTER command:
ALTER SESSION SET ABORT_DETACHED_QUERY=TRUE;
After 5 minutes your queries should be aborted if the connectivity is lost due to abrupt termination of a session.
Related
I have to deal with the following scenario for spring application with Oracle database:
Spring application uses the primary database. In the meantime the secondary database stores data for disaster recovery (from primary).
The first step is currently provided. At this moment I have to implement:
When the primary database gets offline application should change the connection to the secondary database).
The implementation should be programmatically. How can I achieve that without changing the code that currently exists? Is there any working solution (library)?
I think about AbstractRoutingDataSource and ping databases (e.g. every 5 seconds) but I'm not sure about this solution.
So, let's to summarize the issue. I was unable to use Oracle RAC (Real Application Cluster). If the implementation should be programmatically you can try AbstractRoutingDataSource approche.
I have implemented timer that pings current database every 1 second (you can use validation query and check if you can read from database... if no we assume there is no connection and we can switch a datasource).
Thanks to that I was able to change datasource on runtime when current datasource is offline. What is more important it was automatic.
On the other hand, there are disadvantages:
For short time user can see the errors if the database is not
switched yet.
Some part of application may stop working if it is not properly
secured against the lack of connection to the database.
I use datagrip to move some data from a mysql installation to another postresql-database.
That worked for 3 other tables like a charm. The next one, over 500.000 rows big, could not be imported.
I use the function "Copy Table To... (F5)".
This is the log.
16:28 Connected
16:30 user#localhost: tmp_post imported to forum_post: 1999 rows (1m
58s 206ms)
16:30 Can't save current transaction state. Check connection and
database settings and try again.
For other errors like wrong data types, null data on not null columns, a very helpful log is created. But not now.
The problem is also relevant when using the database plugin for IntelliJ-based IDEs, not only DataGrip
The simplest way to solve the issue is just to add "prepareThreshold=0" to your connection string as in this answer:
jdbc:postgresql://ip:port/db_name?prepareThreshold=0
Or, for example, if you a using several settings in the connection string:
jdbc:postgresql://hostmaster.com:6432,hostsecond.com:6432/dbName?&targetServerType=master&prepareThreshold=0
It's a well-known problem when connecting to the PostgreSQL server via PgBouncer rather than a problem with IntelliJ itself. When loading massive data to the database IntelliJ splits data into chunks and loads them sequentially, each time executing the query and committing the data. By default, PostgreSQL starts using server-side prepared statements after 5 execution of a query.
The driver uses server side prepared statements by default when
PreparedStatement API is used. In order to get to server-side prepare,
you need to execute the query 5 times (that can be configured via
prepareThreshold connection property). An internal counter keeps track
of how many times the statement has been executed and when it reaches
the threshold it will start to use server side prepared statements.
Probably your PgBouncer runs with transaction pooling and the latest version of PbBouncer doesn't support prepared statements with transaction pooling.
How to use prepared statements with transaction pooling?
To make prepared statements work in this mode would need PgBouncer to
keep track of them internally, which it does not do. So the only way
to keep using PgBouncer in this mode is to disable prepared statements
in the client
You can verify that the issue is indeed because of the incorrect use of prepared statements with the pgbouncer via viewing IntelliJ log files. For that go to Help -> Show Log in Explorer, and search for "org.postgresql.util.PSQLException: ERROR: prepared statement" exception.
2022-04-08 12:32:56,484 [693272684] WARN - j.database.dbimport.ImportHead - ERROR: prepared statement "S_3649" does not exist
java.sql.SQLException: ERROR: prepared statement "S_3649" does not exist
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgConnection.executeTransactionCommand(PgConnection.java:755)
at org.postgresql.jdbc.PgConnection.commit(PgConnection.java:777)
Before I start with my question, I would like to clarify that I am java/j2ee developer and have limited understanding of things on oracle side.
I am using glassfish server with JDBC connection Pooling and in back side, oracle database.
Also i am using global temporary table of oracle to execute some work flow.
i am storing session specific data in global temp table.
Now my issue is most of the time i am getting the same sessionId for each connection.
Does that means i can't use glboal temporary table with glassfish JDBC connection Pooling.
Another interesting thing is if i removed connection pooling then i am getting different sessionID for each connection.
Please provide your suggestions.
When using Connection Pooling it's always best to not leave states in the database session when the connection is released into the pool. That's because there is no guarantee that you'll get back the same connection the next time you need one. A global temp table (GTT) is an example of such a state and belongs to one Database session, or to one JDBC connection (there is a 1-1 mapping between DB session and JDBC connection). It won't be visible if you use another JDBC connection.
So if your business logic requires that you use a GTT then you should not release the connection back to the pool until you're dong using this GTT. Note that this goes against the best practices which recommend to release the connection back to the pool as soon as possible. As an alternative you may use a normal table and commit your temporary results into it so that they can be accessed through any other connection.
I have a web service which processes inserts/updated data to DB. When client calls this webservice, UserId(currently logged in user to portal) will be sent in Request. I need to pass this userId to Db connection or set it in sys context for Audit purpose. we have existing audit tables and triggers to inserts/updates to Audit table after insert/update on actual table. So to track these changes I need to pass this UserId somehow to connection so that it can be retrieved from DB from Sys Context or $session and inserts in Audit table. I am currently using Spring and Hibernate transactions to process data with DB.
I tried to Set client info on Connection but it's not working. I tried below:
Session session=sessionFactory.getCurrentSession();
SessionImpl sImpl=(SessionImpl) session;
Connection connection=sImpl.connection();
connection.setClientInfo("ClientUser", "ABC");
And also I am trying to set client info by calling Stored procedure DBMS_APPLICATION_INFO.SET_CLIENT_INFO before performing operation on DB every time from application code.but I am not sue if it's a correct way to handle it.
I am trying it with both OCI and thin JDBC drivers but not able find a way to set this user id.
Can someone let me know if there is any efficient way to pass user id on sys context or with Connection. I am currently using hibernate4, Spring, Websphere Server, Oracle DB.
I am using Spring #Transactional to handle hibernate Connections and transactions to perform operation on DB.Connections are from Connection pool and I am using org.springframework.jndi.JndiObjectFactoryBean for dataSource.
is there any way to have interceptor or wrapper around connection to set it when we get the connection from connection pool.
Has anyone done this before?
This is described in spring data JDBC Extensions for the Oracle Database
Chapter
8.2 Configuration of a Custom DataSource Connection Preparer
...but you could implement a ConnectionPreparer that would use the current users login id. That way you can capture user login information even if your data source is configured with a shared user name.
This is a solution for oracle, which I think you are using. It should be also possible to adapt that to another database.
is there a way to set the lock timeout in jdbc? It should work for PostgreSQL, Oracle, SQL Server and MySQL.
I found the method setQueryTimout in the statement class. Is this the right thing? Or is this a general timeout, so when a long update takes longer than the query timout, does an exception occur? Even if the query does not wait for a lock?
What is the best way the set lock timeout in jdbc?
There is no standard JDBC option to configure a lock timeout. This is database specific and not supported by the JDBC standard. You will need to find out how each database supports lock timeout and how this is configured in their driver, and then handle the differences between each driver.
The query timeout is not a lock timeout. It specifies the time a query is allowed to run (if supported by the driver and database), this is intended to kill/prevent long-running queries.