I have setup multiple data sources and I want to show in the logs the specific connection being used for some requests for debugging purposes, is there a way to log DB connection details (username or connection string) for every request made to the DB?
You should use a datasource proxy for logging.
First you have to add a new dependency
<dependency>
<groupId>com.github.gavlyukovskiy</groupId>
<artifactId>datasource-proxy-spring-boot-starter</artifactId>
<version>1.8.1</version>
</dependency>
Then you have to configure logging in the application.properties:
logging.level.net.ttddyy.dsproxy.listener=debug
Now you'll see the datasource name in the logs
2022-12-07 08:14:56.294 DEBUG 36832 --- [main] n.t.d.l.l.SLF4JQueryLoggingListener :
Name:dataSource, Connection:5, Time:0, Success:True
Type:Prepared, Batch:False, QuerySize:1, BatchSize:0
Query:["insert into person (name, id) values (?, ?)"]
Params:[(Peter Muster,1)]
To fully answer your question: It's not possible to log username, password, and URL. This wouldn't be secure anyway. The datasource name should be good enough.
Source: https://vladmihalcea.com/log-sql-spring-boot/
Related
Using Springboot 2.7.2 and Hibernate 5.6 with Oracle 12.2 to write a web application. I use the repository model to do an insert and test with mockmvc. With SQL Debug turned on I get an error ORA 32575 at the point where it executes the insert statement. In the debug log it has INSERT INTO TABLE (COL1, COL2, ID) VALUES ('X','Y',DEFAULT). The Oracle error 32575 follows this. The ID field in question is part of a Hibernate pojo and is a primary key and uses GenerationType.SEQUENCE. It is an Entity that points to a Table.
The DataSource is "thin" driver using the ojdbc8.jar. The Datasource is set up using a #Configuration" annotation in the application during Tomcat startup. If you take all of this by itself I dont get the error above.
However, I have a requirement to connect to each database user through a PROXY USER account because we use Oracle Label Security. It looks something like GRANT CONNECT TO userX THROUGH proxyuser. Using the database driver it would be something like
Properties proxyProps = new Properties()
proxyProps.set(Connection.PROXY_USER_NAME, user)
oraCon.openProxySession(Connection.PROXYTYPE_USER_NAME)
This is being done inside of an application class called ProxyDelegatingDatasourceThin which extends DelegatingDataSource which is a Spring class that I believe gets called when a new connection attempt is made.
Again, queries work fine, updates seem to work, its only the INSERTS. The ID column itself is set to NUMBER and is flagged as a Primary Key. It is not set as any kind of an IDENTITY column.
The error seems to want the ID column to be omitted from the INSERT Statement all together but Hibernate or Spring is generating it with the DEFAULT in the VALUES that is associated to ID.
Im hoping someone can help. Spent days on this.
Set the #ID column to be nullable, insertable, updatable all set to false
Tried using merge and persist from the entity manager (EntityManager) instead of using the Spring Repository save() method.
The implicit caching property is set to true
The error seems to want the ID column to be omitted from the INSERT Statement all together but Hibernate or Spring is generating it with the DEFAULT in the VALUES that is associated to ID.
Adding more info...
When I remove the code above which opens the proxy session, I don't get the error. I also printed some info from the database context while using the proxy session and the PROXY USER is to the PROXY ACCOUNT and the SESSION user is to the user that is connecting through the PROXY ACCOUNT.
Whether I use the Oracle Thin Driver or UCP I get the same result.
I have a spring boot app which uses spring data and hikaricp for db connection pooling. I noticed the following behaviour that looks strange to me:
I have one method which is not transactional and in that method several db queries are executed using spring data repositories
public void testMethod(final Long firstRepositoryId, final Long secondRepositoryId) {
final DomainObject result = firstRepository.findById(firstRepositoryId);
// here there's some code that is processing the result without db queries
secondRepository.findById(secondRepositoryId);
// some more logic here without additional db queries
}
So as expected when there's no transaction on the method then the spring data methods opens a transaction for executing the query and complete it after the methods returns. I have enabled transaction logging so there's the following log output:
2021-06-03 15:34:30.961 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.966 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.967 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.SecondRepository.findById]
2021-06-03 15:34:30.972 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.SecondRepository.findById]
Everything seems to be exactly how I expects to be. The thing I can't understand is the hikari behaviour. This method is invoked within a http request. A connection is taken from hikari cp right after the execution of the firstRepository.findById but this connection is returned in the pool only after the http controller returns response. What I expect is that a connection is taken after a transaction is opened and returned back after the transaction is completed. Is there something that I miss or maybe I have some wrong configuration?
P.S. I'm monitoring the active hikari connections through the spring boot actuator prometheus data. And to be able to reproduce the behavior I explained above I'm suspending the connection thread with several debug breakpoints.
I found out what causes this behaviour- it's Spring functionality to maintain the hibernate session in view in order to be able to retrieve lazy loaded data in the view. In order to disable this you need the following property:
spring.jpa.open-in-view=false
Here's another SO post where is explained what it does:
What is this spring.jpa.open-in-view=true property in Spring Boot?
From Checkmarx report:
Application contains Hardcoded connection details. This can expose database password.
I'm using jasypt-spring-boot for password encryption. But in Checkmarx, it marks it as medium vulnerability.
Dependency used:
<dependency>
<groupId>com.github.ulisesbocchio</groupId>
<artifactId>jasypt-spring-boot</artifactId>
<version>2.0.0</version>
</dependency>
In PropertFile:
projectname.password=ENC(encrypted password)
#Value("${projectname.password}")
private String dBpassword;
and using the above dBpassword to connect to the database.
Checkmarx is going to have a certain amount of false positives. It isn't possible for it to know every single possibility. So in this case, it doesn't realize that the #Value annotation is actually reading it from a config file. In that case you need to follow your organizations process for handling a false positive and marking the finiding as such in Checkmarx.
I have a spring-boot project where the database configuration should be saved in the first time the user execute the application, after ask for the connection data (server url, username and password). the application.properties file embbed in the WAR file of the application should be something like that:
security.basic.enabled=false
# THYMELEAF (ThymeleafAutoConfiguration)
spring.thymeleaf.prefix=classpath:/templates/
spring.thymeleaf.suffix=.html
spring.thymeleaf.mode=HTML5
spring.thymeleaf.encoding=UTF-8
spring.thymeleaf.content-type=text/html
spring.thymeleaf.cache=false
# Multipart Configuration
spring.servlet.multipart.maxFileSize = 150MB
spring.servlet.multipart.maxRequestSize = 150MB
spring.servlet.multipart.fileSizeThreshold = 5MB
# Setting max size of post requests to 6MB (default: 2MB)
server.tomcat.max-http-post-size=157286400
After the first run, a second application.properties should be created with the database configuration. But right now, when I try run the project with the configuration above, it do not start, showing the following error message:
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class
Action:
Consider the following:
If you want an embedded database (H2, HSQL or Derby), please put it on the classpath.
If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).
without give the application the change to generate the second configuration file, with the database properties.
How I could force the application to run this first time, even without the database stuff?
update
Ok, then I get rid of this error message by keeping this 2 dependencies on my pom.xml:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>LATEST</version>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.4.1</version>
</dependency>
and now the applications runs without error, and I am able to open the installation page to create the database on the postgresql server and import the initial data (user data included).
But I notice that this initial data is not being stored on the postgresql server (despite the datababse and all the tables are being created normally). I suspect they are being created on the hyperSql database on memory, what do not help me at all.
My guess is that's happen because I autowire some Dao classes for this entities on my InstallService class, and the code only create the data on the postgresql where I directly make the connection with this specific server.
Considering the method responsible by populate de initial data on my database is something like that:
public void criarUsuario(String user, String pass, String nome, String sobrenome, String email) {
Usuario novo = new Usuario();
novo.setUsername(user);
novo.setPassword(pass);
novo.setFirstName(nome);
novo.setLastName(sobrenome);
novo.setEmail(email);
novo.setEnabled(true);
novo.setLocked(false);
novo.setCredenciais(new ArrayList<Credencial>());
novo.getCredenciais().add( credencialDao.findBy("nome", "admin") );
usuarioDao.insert(novo);
...
}
What I could change here to persist this data on the postgresql database, instead of the hyperSql database?
update 2
Now I managed to run the application even with only the postgresql dependency om my pom.xml. I did that by adding a initial configuration on my first application.properties (locatd on /src/main/resources):
spring.datasource.driverClassName=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:5432/appdata
spring.datasource.username=...
spring.datasource.password=..
spring.datasource.continue-on-error=true
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.show-sql=false
spring.jpa.hibernate.ddl-auto=none
spring.jpa.generate-ddl=false
But when I try access the application on the browser, I got an error saying the database not exist (of course not, when I run the application the first time I have a specific flow to create database, tables and populate initial data).
is there any way to supress this "database not exist" error when I run the project?
Hi I am using below maven dependency in my Spring boot application.
<dependency>
<groupId>com.github.ulisesbocchio</groupId>
<artifactId>jasypt-spring-boot-starter</artifactId>
<version>1.16</version>
</dependency>
I am using below command to encrypt my DB password.
encrypt input="test" password=test algorithm=PBEWithMD5AndDES
In my application.properties, i have below properties,
spring.datasource.url=jdbc:sqlserver://localhost:1234;database=TEST_DB
spring.datasource.username=test
spring.datasource.password=ENC(OPdJ9jOw7tbJR+MlptpCHg==)
spring.datasource.driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.poolName=SpringBootHikariCP
spring.datasource.maximumPoolSize=50
spring.datasource.minimumIdle=30
spring.datasource.maxLifetime=2000000
spring.datasource.connectionTimeout=30000
spring.datasource.idleTimeout=30000
spring.datasource.pool-prepared-statements=true
spring.datasource.max-open-prepared-statements=250
In my application code, i am able to get decrypted value of spring.datasource.password property. But when i use JDBCTemplate, i am getting below exception.
Edit :
Mistakenly added wrong stacktrace. Below is correct one. Due to 3 unsuccessful attempts, the password got expired.
[ERROR] 2018-09-03 04:50:45.578 [https-jsse-nio-8092-exec-9] util - LOG : Class :Controller|| Method :process || org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'test'. ClientConnectionId:ba0abe4d-014a-42d3-b39e-ec1efc0e7131
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:394)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:484)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:494)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:500)
at com.test.dao.AppDaoImpl.generateQuery(AppDaoImpl.java:77)
at com.test.dao.AppDaoImpl$$FastClassBySpringCGLIB$$5d4e5540.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at com.test.dao.AppDaoImpl$$EnhancerBySpringCGLIB$$eb86c18b.generateQuery(<generated>)
It is not able to login to database. Below is my AppDaoImpl.java
#Repository
public class AppDaoImpl implements AppDao {
#Autowired
JdbcTemplate jdbcTemplate;
#Override
public Integer generateQuery(String id) throws ImpsException {
return jdbcTemplate.update(QueryConst.SQL_INSERT_QUERY, id);
}
}
Before jasypt, i was just autowiring my JDBCTemplate and everything was working fine. Now it is not able to login.
PS: I am running my application with
-Djasypt.encryptor.password=test -Djasypt.encryptor.algorithm=PBEWithMD5AndDES
From the message Reason: The password of the account has expired. its clear that password has expired.
Log in to SQL server as a system administrator. Change the password to something else. Make sure enforce password policy is off and then change the password back to the original password.
OR
Right click 'Username' and go to Properties
You can find 'Enforce password expiration' on login properties window, which needs to be unchecked.