When starting my application the keyspace is always created and possibly one or two tables before the PT2S error message . Somehow the spring.data.cassandra.request.timeout property is not honored, or maybe there is something wrong with my configuration? The "DriverConfigLoaderBuilderCustomizer" bean does not make any difference.
pom.xml
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.5.RELEASE</version>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Hoxton.SR9</version>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-cassandra</artifactId>
<spring.framework.version>5.3.1</spring.framework.version>
application.yml
spring:
data:
cassandra:
port: 9042
keyspace-name: abc
contact-points: localhost
local-datacenter: datacenter1
replication-factor: 1
request:
timeout: 15s
connection:
init-query-timeout: 15s
CassandraConfig.java
#Configuration
#EnableReactiveCassandraRepositories(basePackages = "a.b.c.repository")
public class CassandraConfig extends AbstractReactiveCassandraConfiguration {
#Value("${spring.data.cassandra.contactpoints}")
.
.
#Override
protected String getKeyspaceName() {
return keyspace;
}
#Override protected String getContactPoints() {
return contactPoints;
}
#Override protected int getPort() {
return port;
}
#Override
protected String getLocalDataCenter() {
return datacenter;
}
#Override
public SchemaAction getSchemaAction() {
return SchemaAction.NONE;
}
#Override
protected List<CreateKeyspaceSpecification> getKeyspaceCreations() {
return Collections.singletonList(CreateKeyspaceSpecification.createKeyspace(getKeyspaceName())
.ifNotExists()
.with(KeyspaceOption.DURABLE_WRITES, true)
.withNetworkReplication(DataCenterReplication.of(getLocalDataCenter(), getReplicationFactor())));
}
#Override
protected KeyspacePopulator keyspacePopulator() {
ResourceKeyspacePopulator keyspacePopulate = new ResourceKeyspacePopulator();
keyspacePopulate.addScript(new ClassPathResource("table-schema.cql"));
return keyspacePopulate;
}
private long getReplicationFactor() {
return replicationFactor;
}
//#Bean
//public DriverConfigLoaderBuilderCustomizer driverConfigLoaderBuilderCustomizer() {
//return loaderBuilder -> loaderBuilder
//.withDuration(DefaultDriverOption.REQUEST_TIMEOUT, Duration.ofSeconds(15))
//.withDuration(DefaultDriverOption.CONNECTION_INIT_QUERY_TIMEOUT, Duration.ofSeconds(15));
//}
}
Trimmed error log
BeanCreationException: Error creating bean with name 'cassandraSessionFactory'
Invocation of init method failed; nested exception is ScriptStatementFailedException: Failed to execute CQL script statement #2 of class path resource [table-schema.cql]: CREATE TABLE IF NOT EXISTS mytable...
Caused by: org.springframework.data.cassandra.core.cql.session.init.ScriptStatementFailedException: Failed to execute CQL script statement #2 of class path resource [table-schema.cql]: CREATE TABLE IF NOT EXISTS mytanble... nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S
at org.springframework.data.cassandra.core.cql.session.init.ScriptUtils.executeCqlScript(ScriptUtils.java:555) ~[spring-data-cassandra-3.0.5.RELEASE.jar:3.0.5.RELEASE]
at org.springframework.data.cassandra.core.cql.session.init.ResourceKeyspacePopulator.populate
Caused by: com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S
at com.datastax.oss.driver.api.core.DriverTimeoutException.copy(DriverTimeoutException.java:34)
We had a similar problem and tried your exact steps first.
The driver examples code than stirred us in the right direction.
We have a custom cassandra config with excluded auto config:
#SpringBootApplication(exclude = {CassandraAutoConfiguration.class,CassandraDataAutoConfiguration.class})
Running spring-boot-starter-parent version 2.3.0.RELEASE.
CassandraConfig:
#Configuration
#EnableCassandraRepositories(basePackages = {"com.example.model.cassandra"})
public class CassandraConfig extends AbstractCassandraConfiguration {
#Override
protected SessionBuilderConfigurer getSessionBuilderConfigurer() {
return new SessionBuilderConfigurer() {
#Override
public CqlSessionBuilder configure(CqlSessionBuilder cqlSessionBuilder) {
return cqlSessionBuilder
.withConfigLoader(DriverConfigLoader.programmaticBuilder().withDuration(DefaultDriverOption.REQUEST_TIMEOUT, Duration.ofMillis(15000)).build());
}
};
}
}
Related
I'm using Redission and embedded-redis in unit tests. According to the documentation i start the redis server in the test class like this:
private RedisServer redisServer;
#PostConstruct
public void postConstruct() {
if (redisServer == null || !redisServer.isActive()) {
redisServer = RedisServer.builder()
.port(6900)
.setting("maxmemory 128M")
.build();
redisServer.start();
}
}
#PreDestroy
public void preDestroy() {
redisServer.stop();
}
and the redission client looks like this:
#Configuration
#ConditionalOnClass(RedissonClient.class)
public class RedissonConfiguration {
#Bean(destroyMethod = "shutdown")
public RedissonClient redissonClient() {
Config config = new Config();
config.useSingleServer()
.setAddress("redis://localhost:6900");
return Redisson.create(config);
}
}
now i got the the following error in the logs:
[InstanceCleaner] r.e.AbstractRedisInstance Stopping redis server...
[InstanceCleaner] r.e.AbstractRedisInstance : Redis
exited [isson-netty-7-5] o.r.c.h.ErrorsLoggingHandler :
Exception occured. Channel: [id: 0xa9eeeee5, L:/127.0.0.1:49429 -
R:localhost/127.0.0.1:6699] java.io.IOException: Eine vorhandene
Verbindung wurde vom Remotehost geschlossen
The problem is that preDestroy is called before the destroyMethod (shutdown) of the bean.
Is there an other way to stop the server at the end?
I can't use #DependsOn at RedissonConfiguration because it would depend on a TestConfiguration class.
I found a solution. This interface is part of the normal prod classes. Now i can annotate the RedissonClient bean with dependsOn("EmbeddedRedis").
Not quite sure if i'm happy with it but it works....
interface EmbeddedRedis {
#Component(value = "EmbeddedRedis")
#Profile("!local")
class EmptyRedis implements EmbeddedRedis {
}
#Component(value = "EmbeddedRedis")
#Profile("local")
class Redis implements EmbeddedRedis, DisposableBean {
private final int port;
private RedisServer redisServer;
public Redis(#Value("${spring.redis.port}")final int port) {
this.port = port;
}
#PostConstruct
public void postConstruct() {
if (redisServer == null || !redisServer.isActive()) {
redisServer = RedisServer.builder()
.port(port)
.setting("maxmemory 128M")
.build();
redisServer.start();
}
}
#Override
public void destroy() {
redisServer.stop();
}
}
}
I use Spring Boot and Spring Data with Cassandra. On application startup spring establishes a connection to the database to setup the schema and initialize spring data repositories. If the database is not available, the application won't start.
I want, that the application just logs an error and starts. Of course, I can't use the repositories anymore, but other services (rest controllers etc), which are independent from the database should work. It would also be nice to see in actuator healthcheck, that cassandra is down.
For JDBC, there is a spring.datasource.continue-on-error property. I couldn't find something similar for Cassandra.
I also tried to create a custom cassandra configuration and trying to catch Exception on CqlSession creation, but I couldn't achieve the desired behavior.
EDIT: As suggested by #adutra, I tried to set advanced.reconnect-on-init, Application tries to establish the connection, but the application is not fully initialized (e.g. REST controller are not reachable)
#Configuration
public class CustomCassandraConfiguration extends CassandraAutoConfiguration {
#Bean
public DriverConfigLoaderBuilderCustomizer driverConfigLoaderBuilderCustomizer() {
return builder -> builder.withBoolean(DefaultDriverOption.RECONNECT_ON_INIT, true);
}
}
EDIT2: I have now working example (application starts, custom health check for cassandra), but if feels pretty ugly:
CustomCassandraAutoConfiguration
#Configuration
public class CustomCassandraAutoConfiguration extends CassandraAutoConfiguration {
private final Logger logger = LoggerFactory.getLogger(getClass());
#Override
#Bean
public CqlSession cassandraSession(CqlSessionBuilder cqlSessionBuilder) {
try {
return super.cassandraSession(cqlSessionBuilder);
} catch (AllNodesFailedException e) {
logger.error("Failed to establish the database connection", e);
}
return new DatabaseNotConnectedFakeCqlSession();
}
#Bean
public CassandraReactiveHealthIndicator cassandraHealthIndicator(ReactiveCassandraOperations r, CqlSession session) {
if (session instanceof DatabaseNotConnectedFakeCqlSession) {
return new CassandraReactiveHealthIndicator(r) {
#Override
protected Mono<Health> doHealthCheck(Health.Builder builder) {
return Mono.just(builder.down().withDetail("connection", "was not available on startup").build());
}
};
}
return new CassandraReactiveHealthIndicator(r);
}
}
CustomCassandraDataAutoConfiguration
#Configuration
public class CustomCassandraDataAutoConfiguration extends CassandraDataAutoConfiguration {
public CustomCassandraDataAutoConfiguration(CqlSession session) {
super(session);
}
#Bean
public SessionFactoryFactoryBean cassandraSessionFactory(CqlSession session, Environment environment, CassandraConverter converter) {
SessionFactoryFactoryBean sessionFactoryFactoryBean = super.cassandraSessionFactory(environment, converter);
// Disable schema action if database is not available
if (session instanceof DatabaseNotConnectedFakeCqlSession) {
sessionFactoryFactoryBean.setSchemaAction(SchemaAction.NONE);
}
return sessionFactoryFactoryBean;
}
}
DatabaseNotConnectedFakeCqlSession (Fake session implementation)
public class DatabaseNotConnectedFakeCqlSession implements CqlSession {
#Override
public String getName() {
return null;
}
#Override
public Metadata getMetadata() {
return null;
}
#Override
public boolean isSchemaMetadataEnabled() {
return false;
}
#Override
public CompletionStage<Metadata> setSchemaMetadataEnabled( Boolean newValue) {
return null;
}
#Override
public CompletionStage<Metadata> refreshSchemaAsync() {
return null;
}
#Override
public CompletionStage<Boolean> checkSchemaAgreementAsync() {
return null;
}
#Override
public DriverContext getContext() {
return new DefaultDriverContext(new DefaultDriverConfigLoader(), ProgrammaticArguments.builder().build());
}
#Override
public Optional<CqlIdentifier> getKeyspace() {
return Optional.empty();
}
#Override
public Optional<Metrics> getMetrics() {
return Optional.empty();
}
#Override
public <RequestT extends Request, ResultT> ResultT execute( RequestT request, GenericType<ResultT> resultType) {
return null;
}
#Override
public CompletionStage<Void> closeFuture() {
return null;
}
#Override
public CompletionStage<Void> closeAsync() {
return null;
}
#Override
public CompletionStage<Void> forceCloseAsync() {
return null;
}
#Override
public Metadata refreshSchema() {
return null;
}
}
Any suggestions?
You can set the option datastax-java-driver.advanced.reconnect-on-init to true to achieve the effect you want. Its usage is explained in the configuration reference page in the driver docs:
Whether to schedule reconnection attempts if all contact points are unreachable on the first initialization attempt.
If this is true, the driver will retry according to the reconnection policy. The SessionBuilder.build() call - or the future returned by SessionBuilder.buildAsync() - won't complete until a contact point has been reached. If this is false and no contact points are available, the driver will fail with an AllNodesFailedException.
However be careful: with this option set to true, as stated above, any component trying to access a CqlSession bean, even if the session bean is lazy, will block until the driver is able to connect, and might block forever if the contact points are wrong.
If that's not acceptable for you, I would suggest that you wrap the CqlSession bean in another bean that will check if the future returned by SessionBuilder.buildAsync() is done or not, and either block, throw or return null, depending on the caller's expectations.
[EDIT] I've reached out internally to the DataStax Drivers team last night and adutra has responded so I'm withdrawing my response.
I have simple spring cloud kafka stream application. The application terminates each time there is an exception and I'm unable to overwrite this behaviour. The desired outcome is incremental backoff when there are certain types of exceptions or to continue on other type of exceptions. I use springCloudVersion - Hoxton.SR3 and spring boot: 2.2.6.RELEASE
application.yaml
spring:
cloud:
stream:
binders.process-in-0:
destination: test
kafka:
streams:
binder:
deserializationExceptionHandler: logAndContinue
configuration:
default.key.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
default.value.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
Beans
#Bean
public java.util.function.Consumer<KStream<String, String>> process() {
return input -> input.process(() -> new EventProcessor());
}
#Bean
public StreamsBuilderFactoryBeanCustomizer customizer() {
return fb -> {
fb.getStreamsConfiguration().put(StreamsConfig.DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG,
ContinueOnErrorHandler.class);
};
}
EventProcessor
public class EventProcessor implements Processor<String, String>, ProcessorSupplier<String, String> {
private ProcessorContext context;
#Override
public void init(ProcessorContext context) {
this.context = context;
}
#Override
public void process(String key, String value) {
throw new RuntimeException("Some exception");
}
#Override
public void close() {
}
#Override
public Processor<String, String> get() {
return this;
}
}
ContinueOnErrorHandler
public class ContinueOnErrorHandler implements ProductionExceptionHandler {
#Override
public ProductionExceptionHandlerResponse handle(ProducerRecord<byte[], byte[]> record, Exception exception) {
return ProductionExceptionHandlerResponse.CONTINUE;
}
#Override
public void configure(Map<String, ?> configs) {
//ignore
}
}
The custom processor you are using from the consumer is throwing a RuntimeException in the process method. It is not caught by anything. When that exception is thrown, the application simply exits.
The production exception handler that you are using does not have any effect here since you are not producing anything here. Consumer does not produce anything. If you have a use case of producing something, you should switch to java.util.funciton.Function instead.
In order to fix the issue here, as you are processing the record in the custom processor (EventProcessor), if you get an exception, you should catch it and take appropriate actions. For e.g, here is a template:
#Override
public void init(ProcessorContext context) {
this.context = context;
}
#Override
public void process(String key, String value) {
try {
// start processing
// exception thrown
}
catch (Exception e){
// Take the appropriate action
}
}
This way, the application won't be terminated when the exception is thrown in the processor.
I create an application with spring boot that use the data grid redHat (Infinispan 'Estrella Galicia' 9.3.6.Final in standalone) to management of java bean in memory chache.
When i get the cache configured on datagrid, i show the following error:
*java.net.SocketTimeoutException: FaultTolerantPingOperation{default, flags=0} timed out after 60000 ms
at org.infinispan.client.hotrod.impl.operations.HotRodOperation.run(HotRodOperation.java:172) ~[infinispan-client-hotrod-9.4.0.Final.jar:9.4.0.Final]"
The configuration of my application is the following:
Console RedHat Datagrid:
Maven dependencies:
Configuration Spring:
#Primary
#Bean(name = "cacheManager")
public CacheManager cacheManager() throws Exception {
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.addServer().host("127.0.0.1").port(11222);
RemoteCacheManager remoteCacheManager = new RemoteCacheManager(builder.build());
return new SpringRemoteCacheManager(remoteCacheManager);
}
#Bean
public CacheResolver cacheResolver() throws Exception {
NamedCacheResolver resolver = new NamedCacheResolver();
resolver.setCacheManager(cacheManager());
resolver.setCacheNames(Collections.singleton("default"));
return resolver;
}
This is the Java Bean
public class UdmOutput {
private String entityCode;
private Udm unitaPrimaria;
private List<Udm> udmList;
public UdmOutput() {
}
public String getEntityCode() {
return entityCode;
}
public void setEntityCode(String entityCode) {
this.entityCode = entityCode;
}
public List<Udm> getUdmList() {
return udmList;
}
public void setUdmList(List<Udm> udmList) {
this.udmList = udmList;
}
public Udm getUnitaPrimaria() {
return unitaPrimaria;
}
public void setUnitaPrimaria(Udm unitaPrimaria) {
this.unitaPrimaria = unitaPrimaria;
}
}
This is the code used to insert java bean in cache:
#CachePut(cacheResolver="cacheResolver", key="#keyLogic")
public UdmOutput saveObjectInCache(UdmOutput msg, String keyLogic) {
return msg;
}
When run of the method ends i get the exception SocketTimeoutException
Can you help me ?
Thanks a lot,
Giuseppe.
I'm trying to support a multi-tenant by schema in my spring boot (1.4) application. I have the following in my config:
hibernate:
format_sql: true
default_schema: corrto
multiTenancy: SCHEMA
tenant_identifier_resolver: com.config.HeaderTenantIdentifierResolver
multi_tenant_connection_provider: com.config.SchemaPerTenantConnectionProvider
My MultiTenantConnectionProvider implementation is as follows:
public class SchemaPerTenantConnectionProvider implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return this.dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
final Connection connection = this.getAnyConnection();
// need to do stuff here
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
}
It is failing because dataSource is null. I'm assuming it hasn't been created yet but I'm having a hard time finding solutions via Google.
I met the same problem.It seems that in the yml ,HeaderTenantIdentifierResolver and SchemaPerTenantConnectionProvider is managed by hibernate.See here.