Exception with Spring Data Cassandra - spring-boot

I am getting exception while querying cassandra using Spring-Data-Cassandra . Please help
2021-06-01 12:09:48.594 INFO 9568 --- [nio-8080-exec-3] c.e.demo2.Controller.DemoController : Error : org.springframework.data.cassandra.CassandraUncategorizedException: Query; CQL [select * from summary_data where proj_id = ? and category = ? and name = ? and time >= ?]; Query timed out after PT10S; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT10S
private final AsyncCassandraOperations asyncCassandraTemplate;
public List<Data1> getData(String convProjectId, List<String> stageNames,
String eventCategory, List<String> distinctDateHour) {
final String cql = "select * from summary_data where proj_id = ? and category = ? and name = ? and time >= ?";
List<BanEventLifecycle> bList = new ArrayList<>();
ArrayList<ListenableFuture<List<Data1>>> bFutureList = new ArrayList<>();
distinctDateHour.forEach(dateHr-> stageNames.forEach(stageName->{
ListenableFuture<List<Data1>> futureBData = asyncCassandraTemplate.getAsyncCqlOperations().query(cql,
ps -> ps.bind().setString(0, projectId.toLowerCase()).setString(1,dateHr)
.setString(2, category.toLowerCase()).setString(3, name),
(row, rowNum) -> Data1.getData1(row));
bFutureList.add(futureBData);
}));
bFutureList.forEach(future -> {
try {
banList.addAll(future.get());
} catch (Exception exception) {
}
});
return bList;
}
Spring-Data-Cassandra : 3.0.9.RELEASE

The Java driver for Cassandra throws a DriverTimeoutException when it doesn't get a response back from the coordinator within the driver request timeout which is your case is 10 seconds (PT10S).
You need to review the Cassandra logs to determine why the coordinator didn't respond to driver's request. Either you were executing an expensive query or the cluster was overloaded at the time.
Note that you shouldn't confuse DriverTimeoutException with a ReadTimeoutException which is the error thrown by the coordinator when it hasn't received responses from the replica(s) within the read request timeout. Cheers!

Related

How to use RemoteFileTemplate<SmbFile> in Spring integration?

I've got a Spring #Component where a SmbSessionFactory is injected to create a RemoteFileTemplate<SmbFile>. When my application runs, this piece of code is called multiple times:
public void process(Message myMessage, String filename) {
StopWatch stopWatch = StopWatch.createStarted();
byte[] bytes = marshallMessage(myMessage);
String destination = smbConfig.getDir() + filename + ".xml";
if (log.isDebugEnabled()) {
log.debug("Result: {}", new String(bytes));
}
Optional<IOException> optionalEx =
remoteFileTemplate.execute(
session -> {
try (InputStream inputStream = new ByteArrayInputStream(bytes)) {
session.write(inputStream, destination);
} catch (IOException e1) {
return Optional.of(e1);
}
return Optional.empty();
});
log.info("processed Message in {}", stopWatch.formatTime());
optionalEx.ifPresent(
ioe -> {
throw new UncheckedIOException(ioe);
});
}
this works (i.e. the file is written) and all is fine. Except that I see warnings appearing in my log:
DEBUG my.package.MyClass Result: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>....
INFO org.springframework.integration.smb.session.SmbSessionFactory SMB share init: XXX
WARN jcifs.smb.SmbResourceLocatorImpl Path consumed out of range 15
WARN jcifs.smb.SmbTreeImpl Disconnected tree while still in use SmbTree[share=XXX,service=null,tid=1,inDfs=true,inDomainDfs=true,connectionState=3,usage=2]
INFO org.springframework.integration.smb.session.SmbSession Successfully wrote remote file [path\to\myfile.xml].
WARN jcifs.smb.SmbSessionImpl Logging off session while still in use SmbSession[credentials=XXX,targetHost=XXX,targetDomain=XXX,uid=0,connectionState=3,usage=1]:[SmbTree[share=XXX,service=null,tid=1,inDfs=false,inDomainDfs=false,connectionState=0,usage=1], SmbTree[share=XXX,service=null,tid=5,inDfs=false,inDomainDfs=false,connectionState=2,usage=0]]
jcifs.smb.SmbTransportImpl Disconnecting transport while still in use Transport746[XXX/999.999.999.999:445,state=5,signingEnforced=false,usage=1]: [SmbSession[credentials=XXX,targetHost=XXX,targetDomain=XXX,uid=0,connectionState=2,usage=1], SmbSession[credentials=XXX,targetHost=XXX,targetDomain=null,uid=0,connectionState=2,usage=0]]
INFO my.package.MyClass processed Message in 00:00:00.268
The process method is called from a Rest method, which does little else.
What am I doing wrong here?

In SQL Server Profiler shows - Query Execution takes 1ms but in Spring App takes 30ms, where is the delay?

I have a simple Table created in Azure SQL Database
CREATE TABLE DeleteMe(
[ID] [int] NOT NULL,
[LastName] [char](255) NOT NULL,
[FirstName] [char](255) NOT NULL,
[Age] [int] NOT NULL,
[DOJ] [datetime2](0) NOT NULL,
[Role] [varchar](255) NULL
) ON [PRIMARY]
GO
CREATE CLUSTERED INDEX [clusteredindexdeleteme] ON DeleteMe
(
[ID] ASC,
[LastName] ASC,
[FirstName] ASC,
[Age] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, DROP_EXISTING = OFF, ONLINE = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO
In SQL Server Profiler shows - Query Execution takes less than a 1ms
but in Spring App takes 30ms
INFO : 01.05.2022:1754 (40.419) [[]main] HikariDataSource: springHikariCP - Starting...
INFO : 01.05.2022:1754 (41.941) [[]main] HikariDataSource: springHikariCP - Start completed.
Execution Started: 2022/01/05 17:54:41.948
INFO : 01.05.2022:1754 (41.950) [[]main] HikariDataSource: springHikariCP - Starting...
INFO : 01.05.2022:1754 (42.139) [[]main] HikariDataSource: springHikariCP - Start completed.
Select Query Execution Started: 2022/01/05 17:54:42.225
Select Query Execution Completed: 2022/01/05 17:54:42.251
Java Application:
public class JDBCSample {
public static void main(String[] args) {
...
try {
Connection conn = dataSource().getConnection();
conn.setAutoCommit(false);
Statement statement = conn.createStatement();
ResultSet rs ;
try {
LocalDateTime executionStartTime = LocalDateTime.now();
System.out.println("Select Query Execution Started: " + dtf.format(executionStartTime));
rs = statement.executeQuery("SELECT ID from Delete Where ID = 50");
executionEndTime = LocalDateTime.now();
System.out.println("Select Query Execution Completed: " + dtf.format(executionEndTime));
}
catch (SQLException ex)
{
System.out.println("Error message: " + ex.getMessage());
return; // Exit if there was an error
}
} catch (Exception e) {
e.printStackTrace();
}
}
#Bean(destroyMethod = "close")
public static DataSource dataSource(){
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setDriverClassName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
hikariConfig.setJdbcUrl("jdbc:sqlserver://....;encrypt=true;trustServerCertificate=false;");
hikariConfig.setUsername("...");
hikariConfig.setPassword("...");
hikariConfig.setMaximumPoolSize(1);
hikariConfig.setConnectionTestQuery("SELECT 1");
hikariConfig.setPoolName("springHikariCP");
HikariDataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource;
}
}
Note: When I execute the same application against the local server installed on the Dev laptop, it is taking only 6ms.
Note: I am executing this application from Azure Kubernetes.
Where is the delay? How do I fix this so that the Spring application will not take more than 2ms.
From what you have shared here I assume the delay is in network latency.
Dev Java App to Dev Server takes 6 seconds.
Azure Kubernetes App to Azure SQL Database takes 30 ms.
Which I think is reasonable if you include the network latency between app and server. I dont think you will get result in 2 ms. If you get, please let me know ;)
Try to check network latency in some ways like doing on SELECT 1 query and calculate turnaround time.
interesting read:
https://azure.microsoft.com/en-in/blog/testing-client-latency-to-sql-azure/
https://learn.microsoft.com/en-us/archive/blogs/igorpag/azure-network-latency-sql-server-optimization
https://github.com/RicardoNiepel/azure-mysql-in-aks-sample
There is a way to measure latency network in java
We opened a case with Microsoft support and figured that there is an issue with coreDNS at AKS cluster and requested Microsoft Product group to increase the resources for coreDNS Pods.

Error executing query in Java code to connect to Presto

We are trying to connect to Presto using Java code and execute some queries. Catalog we are using is MySQL.
Presto is installed on the Linux server. Presto CLI is working fine on Linux. Started Presto in Linux.
MySQL is also installed on the Linux machine. We are able to access MySQL in windows using DbVisualizer.
I created a MySQL connector catalog for Presto. I'm successful in querying data of MySQL using Presto CLI as presto --server localhost:8080 --catalog mysql --schema tutorials.
Executing the Java code on the Windows machine, I'm able to access MySQL and execute queries, but we are unable to query data. When we try to run a query from Presto, it is giving us Error Executing Query. In the below example, I have used a jar from Trinosql
package testdbPresto;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
public class PrestoJdbc {
public static void main(String args[]) throws SQLException, ClassNotFoundException {
try{
//connect mysql server tutorials database here
Class.forName("com.facebook.presto.jdbc.PrestoDriver");
String url = "jdbc:trino://35.173.241.37:8080/mysql/tutorials";
Properties properties = new Properties();
properties.setProperty("user", "root");
properties.setProperty("password", "Redcar88!");
properties.setProperty("SSL", "true");
Connection connection = DriverManager.getConnection(url, properties);
Statement statement = null;
statement = connection.createStatement();
//select mysql table author table two columns
String sql;
sql = "select auth_id, auth_name from mysql.tutorials.author";
ResultSet resultSet = statement.executeQuery(sql);
//Extract data from result set
while (resultSet.next()) {
//Retrieve by column name
String name = resultSet.getString("auth_name");
//Display values
System.out.println("name : " + name);
}
//Clean-up environment
resultSet.close();
statement.close();
connection.close();
}catch(Exception e){ e.printStackTrace();}
}
}
Output:
java.sql.SQLException: Error executing query
at io.trino.jdbc.TrinoStatement.internalExecute(TrinoStatement.java:274)
at io.trino.jdbc.TrinoStatement.execute(TrinoStatement.java:227)
at io.trino.jdbc.TrinoStatement.executeQuery(TrinoStatement.java:76)
at testdbPresto.PrestoJdbc.main(PrestoJdbc.java:29)
Caused by: java.io.UncheckedIOException: javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
at io.trino.jdbc.$internal.client.JsonResponse.execute(JsonResponse.java:154)
at io.trino.jdbc.$internal.client.StatementClientV1.<init>(StatementClientV1.java:110)
at io.trino.jdbc.$internal.client.StatementClientFactory.newStatementClient(StatementClientFactory.java:24)
at io.trino.jdbc.QueryExecutor.startQuery(QueryExecutor.java:46)
at io.trino.jdbc.TrinoConnection.startQuery(TrinoConnection.java:728)
at io.trino.jdbc.TrinoStatement.internalExecute(TrinoStatement.java:239)
... 3 more
Caused by: javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
at sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:448)
at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:174)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1279)
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1188)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:401)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:373)
at io.trino.jdbc.$internal.okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:299)
at io.trino.jdbc.$internal.okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:268)
at io.trino.jdbc.$internal.okhttp3.internal.connection.RealConnection.connect(RealConnection.java:160)
at io.trino.jdbc.$internal.okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
at io.trino.jdbc.$internal.okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
at io.trino.jdbc.$internal.okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
at io.trino.jdbc.$internal.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at io.trino.jdbc.$internal.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at io.trino.jdbc.$internal.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at io.trino.jdbc.$internal.client.OkHttpUtil.lambda$basicAuth$1(OkHttpUtil.java:85)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at io.trino.jdbc.$internal.client.OkHttpUtil.lambda$userAgent$0(OkHttpUtil.java:71)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at io.trino.jdbc.$internal.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at io.trino.jdbc.$internal.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
at io.trino.jdbc.$internal.okhttp3.RealCall.execute(RealCall.java:77)
at io.trino.jdbc.$internal.client.JsonResponse.execute(JsonResponse.java:131)
... 8 more
It is quite old question but it might be still relevant.
You are trying connect to trino with presto jdbc driver. PrestoSQL was rebranded as Trino . So in order to access trino via jdb, you should use trino jdbc driver.
Add trino dependency in your classpath.
If you use maven, add this dependency in the pom.
<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-jdbc</artifactId>
<version>${trino-jdbc.version}</version>
</dependency>
Then use the following driver
Class.forName("io.trino.jdbc.TrinoDriver");
Here is a code that works with Trino.
fun main() {
val trinoUrl = "jdbc:trino://myDomain:443"
val properties = Properties()
properties.setProperty("user", "noUserS")
// properties.setProperty("password", "noPass")
properties.setProperty("SSL", "true")
DriverManager.getConnection(trinoUrl, properties).use { trinoConn ->
trinoConn.createStatement().use { statement ->
statement.connection.catalog = "catalog1"
statement.connection.schema = "default"
println("Executing query...")
statement.executeQuery("""
select
restaurantId,
type,
time
from table1
where time > CURRENT_TIMESTAMP - INTERVAL '1' hour
""".trimIndent()
).use { resultSet ->
val list = mutableListOf<Map<String, String>>()
while (resultSet.next()) {
val data = mapOf(
"restaurantId" to resultSet.getString("restaurantId"),
"type" to resultSet.getString("type"),
"time" to resultSet.getString("time")
)
list.add(data)
}
println("Records returned: ${list.size}")
println(list)
}
}
}
exitProcess(0)
}
It is Kotlin, but it's easy to understand.
The .use {..} it's try-with-resources in Java.
Hope this helps.

No server chosen by com.mongodb.async.client.ClientSessionHelpe from cluster description ClusterDescription

I am trying to connect to aws DocumentDB with async mongoClient.
I created a DocumentDB cluster in aws and success connect via ssh command line.
I went over here and created MongoClient and success connected and insert events.
But when I tried create com.mongodb.async.client.MongoClient, connection failed with folowing error:
No server chosen by WritableServerSelector from cluster description
ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE,
serverDescriptions=[ServerDescription{address=aws-cluster:27017,
type=UNKNOWN, state=CONNECTING,
exception={com.mongodb.MongoSocketReadTimeoutException: Timeout while
receiving message}, caused by
{io.netty.handler.timeout.ReadTimeoutException}}]}. Waiting for 30000
ms before timing out.
ClusterSettings clusterSettings = ClusterSettings.builder()
.applyConnectionString(new ConnectionString(connectionString)).build();
List<MongoCredential> credentials = new ArrayList<>();
credentials.add(
MongoCredential.createCredential(
mongoUserName,
mongoDBName,
mongoPassword));
MongoClientSettings settings = MongoClientSettings.builder()
.credentialList(credentials)
.clusterSettings(clusterSettings)
.streamFactoryFactory(new NettyStreamFactoryFactory())
.writeConcern(WriteConcern.ACKNOWLEDGED)
.build();
com.mongodb.async.client.MongoClient mongoClient = MongoClients.create(settings);
MongoDatabase testDB = mongoClient.getDatabase("myDB");
MongoCollection<Document> collection = testDB.getCollection("test");
Document doc = new Document("name", "MongoDB").append("type", "database");
//**trying insert document => here I got an error**
collection.insertOne(doc, new SingleResultCallback<Void>() {
#Override
public void onResult(final Void result, final Throwable t) {
System.out.println("Inserted!");
}
});
Do you have any ideas, why does it happen?
I solved it by using uri:
String uri = "mongodb://<username>:<Password>#<hostname>:27017/?ssl=true&ssl_ca_certs=cert";
MongoClientSettings settings = MongoClientSettings.builder()
.streamFactoryFactory(new NettyStreamFactoryFactory())
.applyConnectionString(new ConnectionString(uri))
.build();
com.mongodb.async.client.MongoClient mongoClient = MongoClients.create(settings);
I encountered a similar error , for me it was related to the TLS configs.
I disabled the TLS in documentDB https://docs.aws.amazon.com/documentdb/latest/developerguide/security.encryption.ssl.html
In my case I had to restart the cluster after disabling the TLS. (TLS was not needed for the use case). After the restart the connection was established successfully.

h2 mixed mode connection problem

I start h2 database in a servlet context listener:
public void contextInitialized(ServletContextEvent sce) {
org.h2.Driver.load();
String apprealPath = sce.getServletContext().getRealPath("\\");
String h2Url = "jdbc:h2:file:" + apprealPath + "DB\\cdb;AUTO_SERVER=true";
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
StatusPrinter.print(lc);
logger.debug("h2 url : " + h2Url);
try {
conn = DriverManager.getConnection(h2Url, "sa", "sa");
} catch (SQLException e) {
e.printStackTrace();
}
logger.debug("h2 database started in embedded mode");
sce.getServletContext().setAttribute("connection", conn);
}
then I try to use dbvisualizer to connect to h2 using following url :
jdbc:h2:tcp://localhost/~/cdb
but get these error messages:
An error occurred while establishing the connection:
Type: org.h2.jdbc.JdbcSQLException Error Code: 90067 SQL State: 90067
Message:
Connection is broken: "Connection refused: connect" [90067-148]
I tried to replace localhost with "172.17.33.181:58524" (I found it in cdb.lock.db)
reconnect with user "sa" password "sa" ,then server response changed to :
wrong username or password !
In the Automatic Mixed Mode, you don't need to (and you can't) use jdbc:h2:tcp://localhost. Just use the same URL everywhere, that means jdbc:h2:file:...DB\\cdb;AUTO_SERVER=true.
You can use the same database URL independent of whether the database is already open or not. Explicit client/server connections (using jdbc:h2:tcp:// or ssl://) are not supported.

Resources