Tracing SQL Queries with X-Ray and Spring Boot 2 - spring-boot

The current X-Ray SQL tracing interceptor uses Tomcat JDBC Pool but Spring Boot 2 uses HikariCP as default pool, is it possible to configure the jdbc tracing in HikariCP instead?
Here (https://forums.aws.amazon.com/thread.jspa?threadID=254847) they suggest to use both Datasources:
DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource();
HikariDataSource hikariDataSource = new HikariDataSource();
... // data source configuration
dataSource.setJdbcInterceptors("com.amazonaws.xray.sql.postgres.TracingInterceptor;");
hikariDataSource.setDataSource(dataSource);
But if I have the HikariCP library in the classpath spring will configure that as datasource.
I've tried with a DatasourceBuilder and also forcing the type using the parameter spring.datasource.type
Any hint?

In Spring boot , you can use still use Tomcat over HikariCP as connection pool:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
<exclusions>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
</dependency>

We are currently investigating a solution that will work with both Tomcat JDBC and HikariCP. We are aware that there are currently no work arounds without having Tomcat JDBC as a dependency. Please stay tuned.

Resolved the same using TracingDataSource, while still using HikariCP as the connection pool
Reference https://github.com/aws/aws-xray-sdk-java/issues/88#issuecomment-570328275
Code: (Note, I am using AWS Secrets Manager JDBC Library aws-secretsmanager-jdbc to connect to the database using secrets stored in AWS Secrets Manager)
import com.amazonaws.xray.sql.TracingDataSource;
...
...
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return TracingDataSource
.decorate(DataSourceBuilder.create()
.driverClassName("com.amazonaws.secretsmanager.sql.AWSSecretsManagerPostgreSQLDriver")
.url("jdbc-secretsmanager:postgresql://" + System.getenv("PGHOST") + ":"
+ System.getenv("PGPORT") + "/" + System.getenv("PGDATABASE"))
.username(System.getenv("SECRET_NAME")).build());
}
Dependency:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-sql</artifactId>
</dependency>

Related

Where to define SSL connection with Hibernate reactive

I'm running spring webflux app with Hibernate reactive defined like wise:
EntityManagerFactory emf = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME, hibernateReactiveProperties());
Mutiny.SessionFactory reactiveSessionFactory = emf.unwrap(Mutiny.SessionFactory.class);
Version:
<dependency>
<groupId>org.hibernate.reactive</groupId>
<artifactId>hibernate-reactive-core</artifactId>
<version>1.1.1.Final</version>
</dependency>
I'm using Postgresql, javax.persistence.jdbc.driver : io.vertx.pgclient.spi.PgDriver12
How do I configure an SSL connection?

ElasticSearchHighLevelClient springboot Autoconfiguration not working

I am trying to autoconfigure elasticsearch high-level rest client using this documentation
https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-elasticsearch
I added the following properties to my application.properties:
spring.elasticsearch.rest.uris=https://hostname.com
spring.elasticsearch.rest.read-timeout=10s
spring.elasticsearch.rest.username= user
spring.elasticsearch.rest.password= password
I added these dependencies in my pom.xml
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
and a property
<elasticsearch.version>7.2.1</elasticsearch.version>
In my application I did this
#Configuration
#EnableAutoConfiguration
#SpringBootApplication
#ComponentScan
public class MainClass {
public static void main(String[] args) {
SpringApplication.run(MainClass.class, args);
}
}
But when I try to use the HighLevelClient it gives me NullPointerException
#Autowired
private RestHighLevelClient client;
Am I missing something?
You should add the stacktrace for the NullPointerException.
You did not write which versions you use (Spring Boot, Spring Data Elasticsearch). But the current Spring Boot 2.2.2 targets Spring Data Elasticsearch 3.2.x which uses Elasticsearch 6.8.5. So this won't work with an Elasticsearch 7 cluster. Support for Elasticsearch 7 is coming with the next version (Spring Data Elasticsearch 4.0)
I hope you are using spring-boot-autoconfigure dependency because RestHighLevelClient bean is initialized by this library based on the presence of RestHighLevelClient in the classpath. Check the dependency tree for the presence of spring-boot-autoconfigure.

How to replace default hikari cp to tomcat pool on spring boot 2.0

I have migrated spring boot application to 2.0 and found out some problems with hikari connection pool. When I am fetching database data this results to hikari cp timeout ie. connection is not available. I don't know why when in the previous version this worked correctly.
Therefore I tried to use tomcat pool with this config in application.yml but it did not work (in correct YAML formatting).
spring.datasource.type=org.apache.tomcat.jdbc.pool.DataSource
My pom.xml has these dependencies related to DB things:
spring-boot-jpa
spring-boot-jdbc
jdbc7
How to exclude hikari and use tomcat connection pool?
I have found out the solution.
This can be resolved in pom.xml by modifying like that:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
<exclusions>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
</dependency>
However the hikari problem was probably with default small size of connection pool. So this problem could be resolved also with this change but not verified by myself. Just note for others. Something like that:
spring.datasource.hikari.connection-timeout=60000
spring.datasource.hikari.maximum-pool-size=5
Since Spring Boot 2.0 release, spring-boot-starter-jdbc and spring-boot-starter-data-jpa resolve HikariCP dependency by default and spring.datasource.type property has HikariDataSource as default value.So if u have both dependency in your application you should exclude it from both like below.
implementation('org.springframework.boot:spring-boot-starter-data-jpa') {
exclude group: 'com.zaxxer', module: 'HikariCP'
}
implementation('org.springframework.boot:spring-boot-starter-jdbc') {
exclude group: 'com.zaxxer', module: 'HikariCP'
}
After that you can configure other pooling technologies that u likes to use, like below
.
In your application.yml file :
spring:
datasource:
type: org.apache.tomcat.jdbc.pool.DataSource
In dependency :
implementation('org.apache.tomcat:tomcat-jdbc')
Also:
spring:
datasource:
type: org.apache.tomcat.jdbc.pool.DataSource
works with
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
</dependency>

using embedded amqp swith spring boot and camel

I want to integration test a custom camel component and therefor need an embedded/in memory messanging I can easily use to test from/to endpoints.
I am hoping that I can achieve this via spring-boot-amqp-starter.
I used this example for a start, which has the dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-broker</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
</dependency>
and the config:
spring:
activemq:
broker-url: vm://embedded?broker.persistent=false,useShutdownHook=false
// ...
This is working, when I use regular Listener annotations in spring, I have a sender using the template and a consumer logging the messages.
Now I go one step further and use camel, but it does not recognize the vm:embedded broker but tries to connect to tcp://localhost, which is not running.
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("activemq:foo").to("log:sample");
from("timer:bar").setBody(constant("Hello from Camel")).to("activemq:foo");
}
};
How can I configure activemq-camel to use the embedded broker?
Update:
I use dependency management imports for spring-boot-dependencies (1.5.9)
and camel-spring-boot-dependencies (2.20.1).
This has been fixed in newer versions of activemq-camel when you use that with Spring Boot. Now the activemq-camel component will honor the spring boot configuration of the spring.activemq.* settings.

Configuring embedded Derby in Spring Boot app

Could you please help me set up connection to the embedded Derby database in Spring Boot application?
I searched the web but can only find solutions for server-type Derby, not for embedded Derby.
spring.jpa.database = ?
spring.jpa.hibernate.ddl-auto = create-drop
Derby as an in-memory database
If you want to Configure in-memory Derby database with spring boot, the minimal thing you need is to mention the runtime dependency (along with spring-boot-starter-data-jpa) as
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<scope>runtime</scope>
</dependency>
This bare minimum configuration should work for any embedded datasource in Spring Boot. But as of current Spring Boot version (2.0.4.RELEASE) this lead to an error only for Derby
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing
DDL via JDBC Statement
Caused by: java.sql.SQLSyntaxErrorException: Schema 'SA' does not exist
This happens because of default configuration ofspring.jpa.hibernate.ddl-auto=create-drop see this spring-boot issue for details.
So you need to override that property in your application.properties as
spring.jpa.hibernate.ddl-auto=update
Derby as a persistent database
If you want to use Derby as your persistent database. Add these application properties
spring.datasource.url=jdbc:derby:mydb;create=true
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.DerbyTenSevenDialect
spring.jpa.hibernate.ddl-auto=update
You don't need connection properties if you are using Spring Boot. Just add these to your POM:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
</dependency>
Then add the usual controller, service, and repository classes.

Resources