I am using spring boot2 with Prometheus. we are using Postgres as db. The Prometheus url is not fetching db metrics.
Any references would be very helpful.
I have tried
#Configuration
#AutoConfigureAfter({DataSourceAutoConfiguration.class})
public class MyConfiguration {
#Autowired
HikariDataSource dataSource;
#Bean
PostgreSQLDatabaseMetrics dbMeterics() {
return new PostgreSQLDatabaseMetrics(dataSource, "database-name");
}
}
When I hit the end point /prometheus, I get this error
java.lang.NullPointerException: null
at io.micrometer.core.instrument.binder.db.PostgreSQLDatabaseMetrics.lambda$bindTo$1(PostgreSQLDatabaseMetrics.java:101)
at io.micrometer.core.instrument.internal.DefaultGauge.value(DefaultGauge.java:40)
at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$newGauge$3(PrometheusMeterRegistry.java:235)
at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:69)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)
at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)
at org.springframework.boot.actuate.metrics.export.prometheus.PrometheusScrapeEndpoint.scrape(PrometheusScrapeEndpoint.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76)
In line 101 the registry received via DI is used to build the pgMetrics object. Since you are getting the null pointer at this line, you might be missing the import of Prometheus registry as a dependency.
Related
I'm using a Spring Boot Scheduler to run a query on the DB daily to find some records based on a condition and update the records returned. Fetching the records using JPA works fine, but when I loop through them, update them, and try to save each updated record I get the following error:
Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Error while committing the transaction
Caused by: javax.persistence.RollbackException: Error while committing the transaction at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:81) at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:562) ... 30 more Caused by: java.lang.NullPointerException at com.xxx.yyy.config.JpaAuditingConfiguration.auditorProvider$lambda-0(JpaAuditingConfiguration.kt:15) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) at com.sun.proxy.$Proxy168.getCurrentAuditor(Unknown Source) at java.base/java.util.Optional.map(Optional.java:265) at org.springframework.data.auditing.AuditingHandler.getAuditor(AuditingHandler.java:109) at org.springframework.data.auditing.AuditingHandler.markModified(AuditingHandler.java:104) at org.springframework.data.jpa.domain.support.AuditingEntityListener.touchForUpdate(AuditingEntityListener.java:112).
Here is the scheduler code I have. If I run the same code inside my service and call it using an endpoint everything works fine:
#Component
class Scheduler(
private val repository: Repository
) {
#Scheduled(cron = "0 0 2 * * *")
fun expire() {
val records = repository.findRecords()
for (record in records) {
try {
// Call some external API using record.id but this part is commented out for now until the saving works
record.active = false
repository.save(record)
} catch (ex: Exception) {
logger.error("Error expiring record " + record.id)
logger.error("Exception: ${ex.printStackTrace()}")
continue
}
}
}
}
the null pointer exception happens in the JpaAuditingConfiguration config I use for storing the created_at and last_modified_at dates. Here is the code I have for that class:
#Configuration
#EnableJpaAuditing(auditorAwareRef = "auditorProvider")
class JpaAuditingConfiguration {
#Bean
fun auditorProvider(): AuditorAware<String> {
return AuditorAware { Optional.of(SecurityContextHolder.getContext().authentication.name) }
}
}
Your JpaAuditingConfiguration requires the security context to be non null when you make modifications. When you're running your task in a scheduler there is no active request, so no active session, and therefore your authentication is null.
Usually, this is solved by making a special app user and manually authenticating them in your scheduled task.
I've hit a brick wall in a Spring/Kotlin/JDBC/Postgres project and am hoping the community can help.
What I'm Trying To Do
Use Spring JDBC API (JdbcTemplate) to create a temporary table (temporary_pokemon) in a Postgres database.
Unwrap the JDBC connection from the JdbcTemplate so I can use the same connection to load a CSV into the database using the Postgres JDBC driver's copyIn method (https://jdbc.postgresql.org/documentation/publicapi/org/postgresql/copy/CopyIn.html)
What Is Going Wrong
The copyIn method errors with the message relation "temporary_pokemon" does not exist, and my assumption here is that the unwrapped connection is somehow separate/different to the db.execute command which creates the table.
Ideally there's a way to re-use the same connection while still being able to rely largely on Spring Boot's autoconfiguration and things like automatic connection pooling, etc.
What I've Tried So Far
Adding the #Transactional annotation
Creating a DataSource manually using Spring Boot's DataSourceBuilder (this seems to work, I am assuming that it only creates a single connection which gets reused)
The error message
Caused by: org.postgresql.util.PSQLException: ERROR: relation "temporary_pokemon" does not exist
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2553) ~[postgresql-42.2.18.jar:42.2.18]
at org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1212) ~[postgresql-42.2.18.jar:42.2.18]
at org.postgresql.core.v3.QueryExecutorImpl.startCopy(QueryExecutorImpl.java:894) ~[postgresql-42.2.18.jar:42.2.18]
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:45) ~[postgresql-42.2.18.jar:42.2.18]
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:177) ~[postgresql-42.2.18.jar:42.2.18]
at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:160) ~[postgresql-42.2.18.jar:42.2.18]
at com.example.demo.IngestPostgres.ingest(IngestPostgres.kt:32) ~[main/:na]
at com.example.demo.IngestPostgres$$FastClassBySpringCGLIB$$f1321c17.invoke(<generated>) ~[main/:na]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.2.jar:5.3.2]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.3.2.jar:5.3.2]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.2.jar:5.3.2]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.3.2.jar:5.3.2]
... 14 common frames omitted
Code Snippets
My #Component for the CommandLineRunner:
#Component
class Seed : CommandLineRunner {
#Autowired
lateinit var ingester : IngestPostgres
override fun run(vararg args: String?) {
val buffer = BufferedReader(FileReader(File("src/main/resources/ingest.csv")))
ingester.ingest(buffer)
}
}
The IngestPostgres #Component:
#Component
class IngestPostgres {
#Autowired
private lateinit var db: JdbcTemplate
#Transactional
fun ingest(bufferedReader: BufferedReader) {
db.execute("""
DROP TABLE IF EXISTS temporary_pokemon;
CREATE TABLE temporary_pokemon (
pokemon_id INT,
pokemon_name VARCHAR,
pokemon_type VARCHAR
);
""".trimIndent())
val pgConnection = db.dataSource?.connection?.unwrap(PgConnection::class.java)!!
CopyManager(pgConnection).copyIn(
"COPY temporary_pokemon FROM stdin DELIMITER ',' CSV HEADER",
bufferedReader
)
// snipped - later code INSERTS contents of temporary_pokemon into main pokemon table
}
}
My dependencies in build.gradle.kts:
dependencies {
implementation("org.springframework.boot:spring-boot-starter-jdbc")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.postgresql:postgresql")
testImplementation("org.springframework.boot:spring-boot-starter-test")
}
Thanks for any help. I'm not an expert at Spring/JDBC by any means, so apologies in advance if I've missed something that's common knowledge. I've tried searching on SO and Google but to no avail.
Seems like your queries use different connections. So results of first query are not committed when the second query starts. You can get a single connection and use it for execution of both queries or use JdbcTemplate api for importing csv data to your table
I am trying to create a RedisTemplate bean which will have the updated value serializer to serialize an object in JSON format in redis.
#Configuration
class RedisConfig {
#Bean(name = ["redisTemplate"])
#Primary
fun template(factory: RedisConnectionFactory): RedisTemplate<Any, Any> {
val template = RedisTemplate<Any, Any>()
template.connectionFactory = factory
template.valueSerializer = Jackson2JsonRedisSerializer(Object::class.java)
template.afterPropertiesSet()
return template
}
}
As per my understanding, spring should use the JSON serializer to serialize the object returned by the methods marked with Cacheable annotation. Despite this configuration, spring seems to be using the default Java serializer as this exception confirms this fact.
java.io.NotSerializableException: en.prateekj.vds.dto.Task
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at java.util.ArrayList.writeObject(ArrayList.java:766)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1128)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.springframework.core.serializer.DefaultSerializer.serialize(DefaultSerializer.java:46)
at org.springframework.core.serializer.support.SerializingConverter.convert(SerializingConverter.java:63)
at org.springframework.core.serializer.support.SerializingConverter.convert(SerializingConverter.java:35)
at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.serialize(JdkSerializationRedisSerializer.java:94)
at org.springframework.data.redis.serializer.DefaultRedisElementWriter.write(DefaultRedisElementWriter.java:43)
at org.springframework.data.redis.serializer.RedisSerializationContext$SerializationPair.write(RedisSerializationContext.java:219)
at org.springframework.data.redis.cache.RedisCache.serializeCacheValue(RedisCache.java:238)
at org.springframework.data.redis.cache.RedisCache.put(RedisCache.java:144)
at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:87)
at org.springframework.cache.interceptor.CacheAspectSupport$CachePutRequest.apply(CacheAspectSupport.java:770)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:398)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:314)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
Am I missing any configuration or something by which spring is not able to determine what RedisTemplate to use?
You have probably solved it meanwhile, but for further answer seekers.
According to spring data redis reference:
By default, RedisCache and RedisTemplate are configured to use Java native serialization.
From stacktrace I can see that you are actually using Redis for caching, so you need to configure RedisCache and not RedisTemplate. RedisCache is not picking up your #Bean because it is not using RedisTemplate internally.
Example how you can do it in Java:
#EnableCaching
#Configuration
public class CacheConfig {
#Bean
#Primary
public RedisCacheConfiguration defaultCacheConfig(ObjectMapper objectMapper) {
return RedisCacheConfiguration.defaultCacheConfig()
.serializeKeysWith(SerializationPair.fromSerializer(new StringRedisSerializer()))
.serializeValuesWith(SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer(objectMapper)));
}
}
Tomcat start throw java.lang.StackOverflowError when use spring mybatis. Besides, this error occur randomly, it's very weird.
ERROR org.mybatis.spring.mapper.MapperFactoryBean.checkDaoConfig(MapperFactoryBean.java:97) - Error while adding the mapper 'interface com.myWeb.dao.MyClassMapper' to configuration.
java.lang.StackOverflowError
at java.lang.String.getChars(String.java:783)
at java.lang.String.concat(String.java:1976)
at java.net.URLClassLoader$1.run(URLClassLoader.java:357)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1603)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1533)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$ForEachHandler.handleNode(XMLScriptBuilder.java:160)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.parseDynamicTags(XMLScriptBuilder.java:83)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.access$800(XMLScriptBuilder.java:35)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$IfHandler.handleNode(XMLScriptBuilder.java:167)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$ChooseHandler.handleWhenOtherwiseNodes(XMLScriptBuilder.java:199)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$ChooseHandler.handleNode(XMLScriptBuilder.java:187)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.parseDynamicTags(XMLScriptBuilder.java:83)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.access$800(XMLScriptBuilder.java:35)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$ForEachHandler.handleNode(XMLScriptBuilder.java:152)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.parseDynamicTags(XMLScriptBuilder.java:83)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.access$800(XMLScriptBuilder.java:35)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder$TrimHandler.handleNode(XMLScriptBuilder.java:121)
at org.apache.ibatis.scripting.xmltags.XMLScriptBuilder.parseDynamicTags(XMLScriptBuilder.java:83)
OMG, after a few days researching, i finally found the problem.it's because Mybatis initiate multilayer recursion when spring creating mapper instance and cause the stack overflow. i trace to the SqlSessionDaoSupport(MapperFactoryBean
inherit it) in org.mybatis.spring.support and find this:
#Autowired(required = false)
public final void setSqlSessionFactory(SqlSessionFactory sqlSessionFactory) {
if (!this.externalSqlSession) {
this.sqlSession = new SqlSessionTemplate(sqlSessionFactory);
}
}
#Autowired(required = false)
public final void setSqlSessionTemplate(SqlSessionTemplate sqlSessionTemplate) {
this.sqlSession = sqlSessionTemplate;
this.externalSqlSession = true;
}
When MapperFactoryBean instance is created, Spring will inject SqlSessionTemplate for it. When injecting SqlSessionTemplate it will gain all the Dao from the container. Dao has been created in the container for the corresponding bean can capture, but if the bean has not been created, then Spring will create the Dao of MapperFactoryBean. when create MapperFactoryBean it will be injection SqlSessionTemplate again. This will be continuing until all of the Dao have been created.So it cause the final stack overflow error casually if you are unlucky.
In short：it's because my mybatis-spring version is too low(i use the version 1.1.1). And this kind of autowired was removed from setSqlSessionTemplate and setSqlSessionFactory in version 1.2.0.
So: by changing mybatis-spring version to higher than 1.2.0, this problem was solved.
Recently we upgraded from hibernate 3.5 to 4.1.7 as well as spring from 3.0.5 to 3.1.3. Hibernate is configured via jpa in spring so no changes is made.
After the upgrade, most of the stuff works fine but one function that uses stored procedure is broken with the following exception:
java.lang.ClassCastException: $Proxy188 cannot be cast to oracle.jdbc.OracleConnection
at oracle.sql.TypeDescriptor.setPhysicalConnectionOf(TypeDescriptor.java:829)
at oracle.sql.TypeDescriptor.(TypeDescriptor.java:583)
at oracle.sql.ArrayDescriptor.(ArrayDescriptor.java:224)
at org.springframework.data.jdbc.support.oracle.SqlArrayValue.createTypeValue(SqlArrayValue.java:71)
at org.springframework.jdbc.core.support.AbstractSqlTypeValue.setTypeValue(AbstractSqlTypeValue.java:58)
at org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:281)
at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:217)
at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:128)
at org.springframework.jdbc.core.CallableStatementCreatorFactory$CallableStatementCeatorImpl.createCallableStatement(CallableStatementCreatorFactory.java:212)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1008)
at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064)
at org.springframework.jdbc.object.StoredProcedure.execute(StoredProcedure.java:144)
In debug mode, I found the AbsructSqlTypeValue.setTypeValue() method has the following implementation:
public final void setTypeValue(PreparedStatement ps, int paramIndex, int sqlType, String typeName)
throws SQLException {
Object value = createTypeValue(ps.getConnection(), sqlType, typeName);
if (sqlType == TYPE_UNKNOWN) {
ps.setObject(paramIndex, value);
}
else {
ps.setObject(paramIndex, value, sqlType);
}
}
The ps.getConnection() method here actually returns a new Hibernate 4 LogicalConnectionImpl which wraps around the real OracleConnection. And that's why the it throws the ClassCastException in Oracle driver.
The reason why it calls to oracle.SqlArrayValue is because the stored procedure takes list of longs as input parameter. When the input parameter is defined, we uses OracleTypes.ARRAY then while binding the values, we create a new SqlArrayValue object to wrap around the Long[]. I tried to use the generic Types.Array and Long[] directly but it didn't work either with the following exception:
Caused by: java.sql.SQLException: Fail to convert to internal representation: [Ljava.lang.Long;#337f5afe
at oracle.sql.ARRAY.toARRAY(ARRAY.java:187)
at oracle.jdbc.driver.OraclePreparedStatement.setObjectCritical(OraclePreparedStatement.java:8782)
at oracle.jdbc.driver.OraclePreparedStatement.setObjectInternal(OraclePreparedStatement.java:8278)
at oracle.jdbc.driver.OraclePreparedStatement.setObjectInternal(OraclePreparedStatement.java:8877)
at oracle.jdbc.driver.OracleCallableStatement.setObject(OracleCallableStatement.java:4992)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.setObject(OraclePreparedStatementWrapper.java:240)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at oracle.ucp.jdbc.proxy.StatementProxyFactory.invoke(StatementProxyFactory.java:230)
at oracle.ucp.jdbc.proxy.PreparedStatementProxyFactory.invoke(PreparedStatementProxyFactory.java:124)
at oracle.ucp.jdbc.proxy.CallableStatementProxyFactory.invoke(CallableStatementProxyFactory.java:101)
at $Proxy214.setObject(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:122)
I don't understand why the jdbcTemplate somehow uses the hibernate connection instead of the native OracleConnection, maybe there is some configuration somewhere can fix it magically?
Found the root cause of it. The class that extends StoredProcedure didn't define the jdbcTemplate property so the default one is used which doesn't have nativeJdbcExtractor defined. After adding the jdbcTemplate dependency to refer to the one defined with org.springframework.jdbc.support.nativejdbc.CommonsDbcpNativeJdbcExtractor as nativeJdbcExtractor resolve the issue. I guess hibernate 3.5 with spring 3.0 doesn't have this issue since at that time the returned jdbc connection is already the OracleConnection.