Our application encoutner timeout exception while indexing(store) document in elasticsearch server. It do not frequently happen but approximately once a day. Here are details.
pom.xml
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.0.0</version>
</dependency>
initialization
#Bean
public RestHighLevelClient buildHighLevelClient() {
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(httplist.toArray(new HttpHost[]{})));
return client;
}
exception information
java.lang.RuntimeException: error while performing request
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:682)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:220)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:192)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:428)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:414)
at org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:229)
......
Caused by: java.util.concurrent.TimeoutException
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:364)
at org.apache.http.nio.pool.AbstractNIOConnPool.processNextPendingRequest(AbstractNIOConnPool.java:344)
at org.apache.http.nio.pool.AbstractNIOConnPool.release(AbstractNIOConnPool.java:318)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.releaseConnection(PoolingNHttpClientConnectionManager.java:303)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.releaseConnection(AbstractClientExchangeHandler.java:239)
I worked around the issue with the following setting:
RestClientBuilder builder = RestClient.builder(
new HttpHost("localhost", 9200, "http")); builder.setRequestConfigCallback(
new RestClientBuilder.RequestConfigCallback() {
#Override
public RequestConfig.Builder customizeRequestConfig(
RequestConfig.Builder requestConfigBuilder) {
return requestConfigBuilder.setConnectionRequestTimeout(-1);
}
});
The underlying Apache RestClient has a limitation of AFAIK 10 concurrent connections in its connection pool. So if there are more then 10 concurrent requests, additional requests get queued and might hit the default DEFAULT_CONNECTION_REQUEST_TIMEOUT_MILLIS = 500; set in org.elasticsearch.client.RestClientBuilder
Related
I'm getting JTA exception while running an example from the Pro Spring 5 book (12. Using Spring Remote, boot-jms project). Here is the entirety of the code (exclude the imports). It only has one file Application.java:
#SpringBootApplication
public class Application {
private static Logger logger = LoggerFactory.getLogger(Application.class);
#Bean
public JmsListenerContainerFactory<DefaultMessageListenerContainer> connectionFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
// You could still override some of Boot's default if necessary.
return factory;
}
public static void main(String... args) throws Exception {
ConfigurableApplicationContext ctx = SpringApplication.run(Application.class, args);
JmsTemplate jmsTemplate = ctx.getBean(JmsTemplate.class);
jmsTemplate.setDeliveryDelay(5000L);
for (int i = 0; i < 10; ++i) {
logger.info(">>> Sending: Test message: " + i);
jmsTemplate.convertAndSend("prospring5", "Test message: " + i);
}
System.in.read();
ctx.close();
}
#JmsListener(destination = "prospring5", containerFactory = "connectionFactory")
public void onMessage(Message message) {
TextMessage textMessage = (TextMessage) message;
try {
logger.info(">>> Received: " + textMessage.getText());
} catch (JMSException ex) {
logger.error("JMS error", ex);
}
}
}
First, I got the following error:
Exception in thread "main" java.lang.AbstractMethodError:
com.atomikos.jms.AtomikosJmsMessageProducerProxy.setDeliveryDelay(J)V
at
org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:628)
at
org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:608)
at
org.springframework.jms.core.JmsTemplate.lambda$send$3(JmsTemplate.java:586)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:504)
at
org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584)
at
org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661)
at com.apress.prospring5.ch12.Application.main(Application.java:54)
Then I commented out "jmsTemplate.setDeliveryDelay(5000L);". So I was able to get pass this error but only end up with the JTA exception.
Exception in thread "main"
org.springframework.jms.UncategorizedJmsException: Uncategorized
exception occurred during JMS processing; nested exception is
com.atomikos.jms.AtomikosTransactionRequiredJMSException: The JMS
session you are using requires a JTA transaction context for the
calling thread and none was found. Please correct your code to do one
of the following:
start a JTA transaction if you want your JMS operations to be subject to JTA commit/rollback, or
increase the maxPoolSize of the AtomikosConnectionFactoryBean to avoid transaction timeout while waiting for a connection, or
create a non-transacted session and do session acknowledgment yourself, or
set localTransactionMode to true so connection-level commit/rollback are enabled. at
org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:311)
at
org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:185)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:507)
at
org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584)
at
org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661)
at com.apress.prospring5.ch12.Application.main(Application.java:54)
Caused by: com.atomikos.jms.AtomikosTransactionRequiredJMSException:
The JMS session you are using requires a JTA transaction context for
the calling thread and none was found. Please correct your code to do
one of the following:
start a JTA transaction if you want your JMS operations to be subject to JTA commit/rollback, or
increase the maxPoolSize of the AtomikosConnectionFactoryBean to avoid transaction timeout while waiting for a connection, or
create a non-transacted session and do session acknowledgment yourself, or
set localTransactionMode to true so connection-level commit/rollback are enabled. at
com.atomikos.jms.AtomikosTransactionRequiredJMSException.throwAtomikosTransactionRequiredJMSException(AtomikosTransactionRequiredJMSException.java:23)
at
com.atomikos.jms.ConsumerProducerSupport.enlist(ConsumerProducerSupport.java:90)
at
com.atomikos.jms.AtomikosJmsMessageProducerProxy.send(AtomikosJmsMessageProducerProxy.java:34)
at
org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:634)
at
org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:608)
at
org.springframework.jms.core.JmsTemplate.lambda$send$3(JmsTemplate.java:586)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:504)
... 3 more
My project includes the dependency of spring-boot-starter-artemis and spring-boot-starter-jta-atomikos:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-artemis</artifactId>
<version>2.0.6.RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>artemis-jms-server</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jta-atomikos</artifactId>
<version>2.0.6.RELEASE</version>
</dependency>
<dependency>
<groupId>com.atomikos</groupId>
<artifactId>transactions-hibernate4</artifactId>
<version>4.0.4</version>
</dependency>
The error message seems suggesting the Transaction is missing, so I had added #Transactional annotation at the listener and jmsTemplate, but the same problem still persists.
The problem seems is that I had included spring-boot-starter-jta-atomikos and transactions-hibernate4 in the dependencies. Once I removed these 2 dependencies, the code works. The intention of this example is not to do transaction but demonstrate spring-boot doing JMS over artemis. But since I have these 2 transaction related files in classpath, spring-boot automatically asks for transaction.
I implemented websockets into my application. I copied the configuration and dependencies from jHipster generated app, but I am getting the following errors:
java.lang.IllegalArgumentException: No 'javax.websocket.server.ServerContainer' ServletContext attribute. Are you running in a Servlet container that supports JSR-356?
and
org.apache.catalina.connector.ClientAbortException: java.io.IOException: An established connection was aborted by the software in your host machine
I believe these errors are the reason for the socket connection not being consistent and the therefore the client is not able to send and/or receive any messages.
I searched for a solution but other post didn't help (ie. adding glassfish dependencies).
These are my ws dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-messaging</artifactId>
</dependency>
Do I need to include some other dependencies or is the problem elsewhere?
I found a solution here.
I added these 2 beans:
#Bean
public TomcatServletWebServerFactory tomcatContainerFactory() {
TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();;
factory.setTomcatContextCustomizers(Collections.singletonList(tomcatContextCustomizer()));
return factory;
}
#Bean
public TomcatContextCustomizer tomcatContextCustomizer() {
return new TomcatContextCustomizer() {
#Override
public void customize(Context context) {
context.addServletContainerInitializer(new WsSci(), null);
}
};
}
My requirements would be I need fetch the data from ElasticsticSearch DB through Java Rest client API using spring boot. So I have written spring boot code as below.
Dependency in pom.xml:
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.7.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.7.0</version>
</dependency>
and Client class as below:
private RestHighLevelClient buildClient() {
if (restHighLevelClient == null) {
synchronized (this) {
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(username, password));
try {
HttpClientConfigCallback httpClientConfigCallback = null;
httpClientConfigCallback = new HttpClientConfigCallback() {
#Override
public HttpAsyncClientBuilder customizeHttpClient(
HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder
.setDefaultCredentialsProvider(
credentialsProvider);
}
};
restHighLevelClient = new RestHighLevelClient(
RestClient.builder(
new HttpHost(elasticSearchHost, port,
scheme),
new HttpHost(elasticSearchHost2,
port2, scheme))
.setHttpClientConfigCallback(
httpClientConfigCallback));
} catch (Exception e) {
logger.error("RestHighLevel client create field. Exception: {}",
e.getMessage());
}
}
}
We initiate above client during application startup and close it only when the application is getting shut down.
Then I have written respecitve API using(SerchResponse, multiserachResponse, CountResponse) to fetch the data from ELS.
This code working fine in non PROD environment because of less data.
However in PROD environment we have huge data, so when we call these API's we were getting below error.
2020-05-28T15:24:52.96+0530 [RTR/0] OUT 2020-05-28T15:24:53.72+0530 [APP/PROC/WEB/0] ERR Exception in thread "I/O dispatcher 1"
java.lang.OutOfMemoryError: Direct buffer memory
2020-05-28T15:24:53.72+0530 [APP/PROC/WEB/0] ERR at
java.nio.Bits.reserveMemory(Bits.java:694) 2020-05-28T15:24:53.72+0530
[APP/PROC/WEB/0] ERR at
java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
2020-05-28T13:37:03.99+0530 [RTR/4] OUT 2020-05-28T13:37:04.25+0530
[APP/PROC/WEB/0] OUT 2020-05-28 08:07:04.256 ERROR 14 ---
[io-8080-exec-10] c.d.d.e.ElasticSearchRepositoryImpl : Exception ::
java.lang.IllegalStateException: Request cannot be executed; I/O
reactor status: STOPPED 2020-05-28T13:37:04.25+0530 [APP/PROC/WEB/0]
OUT at org.apache.http.util.Asserts.check(Asserts.java:46)
Can any one faced such issue, please let me know how to resolve this issue.
I have a Redis database with an RLEC (RedisLabs Enterprise Cluster) UI which has been set up for SSL connections.
I have a java app which is able to connect to the redis database using Jedis.
This works:
Jedis jedis = new Jedis(redisInfo.getHost(), redisInfo.getPort(), useSsl);
// make the connection
jedis.connect();
// authorize with our password
jedis.auth(redisInfo.getPassword());
Env vars:
"-Djavax.net.ssl.keyStoreType=PKCS12 -Djavax.net.ssl.keyStorePassword=iloveredis -Djavax.net.ssl.keyStore=$PWD/META-INF/clientKeyStore.p12 -Djavax.net.ssl.trustStoreType=JKS -Djavax.net.ssl.trustStorePassword=iloveredis -Djavax.net.ssl.trustStore=$PWD/META-INF/clientTrustStore.jks"
I also have a Spring Boot app in which I'm trying to connect to the Redis db using JedisConnectionFactory, and I'm not able to. (Using the same app, I am able to connect to a Redis db which does not have SSL enabled).
In my pom.xml:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<version>2.1.4.RELEASE</version>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>
In my redis configuration file:
#Configuration
#EnableRedisRepositories
public class RedisConfig {
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisConfig = new RedisStandaloneConfiguration();
redisConfig.setHostName(redisInfo.getHost());
redisConfig.setPort(redisInfo.getPort());
redisConfig.setPassword(RedisPassword.of(redisInfo.getPassword()));
boolean useSsl = env.getProperty("spring.redis.ssl", Boolean.class);
JedisClientConfiguration jedisConfig;
if (useSsl) {
jedisConfig = JedisClientConfiguration
.builder()
.useSsl()
.build();
} else {
jedisConfig = JedisClientConfiguration
.builder()
.build();
}
JedisConnectionFactory jcf = new JedisConnectionFactory(redisConfig, jedisConfig);
return jcf;
}
#Bean
public RedisTemplate<?, ?> redisTemplateJedis() {
final RedisTemplate<byte[], byte[]> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory());
template.setValueSerializer(new Jackson2JsonRedisSerializer<Object>(Object.class));
return template;
}
This is the error I’m getting:
org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisDataException: ERR unencrypted connection is prohibited
One other point is that for testing purposes, both the server and app are using self-signed certificates (which work with Jedis).
I do not know how to configure JedisConfigurationFactory so that I don't get this error.
Edited.
To recap, I could connect to Redis with SSL enabled with Jedis the library, but not the Spring library JedisConnectionFactory.
I was trying this in Pivotal Cloud Foundry (PCF).
I wrote to Mark Paluch, author of spring-data-redis, and he suggested I turn off auto-reconfiguration to get it to work in PCF.
I found this page on turning off auto-reconfiguration:
https://docs.cloudfoundry.org/buildpacks/java/configuring-service-connections/spring-service-bindings.html#manual
Cloud Foundry will automatically create a RedisConnectionFactory bean for you, so my JedisConnectionFactory was not getting used.
I had to turn off auto-reconfiguration. Or rather turn on manual configuration.
My JedisConnectionFactory bean (with SSL enabled) then started getting instantiated (along with the cloud service connector’s RedisConnectionFactory bean).
And I had to set my JedisConnectionFactory bean to Primary as there were now two connection factory beans.
I was also getting exceptions about unexpected end of stream.
I had to turn on usePoolingin JedisClientConfiguration.
This is where I posted to jira about the issue (now moved to github):
https://github.com/spring-projects/spring-data-redis/issues/1542
I have test with JMS Inbound Gateway in Spring Integration 5.1.3
But I got error as following:
Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:138) ~[spring-integration-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:105) ~[spring-integration-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73) ~[spring-integration-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
POM:
<dependencies>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-jms</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
</dependencies>
I configure the Inbound Gateway as following:
#Bean
JmsInboundGateway jmsInboundGateway(
MessageChannel errorChannel,
ConnectionFactory connectionFactory,
DetailJmsProperties properties) {
final Listener listener = properties.getListener();
return Jms
.inboundGateway(connectionFactory)
.destination("request-queue")
.requestChannel("inputChannel")
.replyChannel("outputChannel")
.defaultReplyQueueName("response-queue")
.get();
}
And, the Service Activator:
#ServiceActivator(inputChannel = "inputChannel", outputChannel = "outputChannel")
public String process(String request) {
String response = null;
try {
LOGGER.info("Received message content: [{}]", request);
response = request + " was processed";
}
catch (Exception e) {
LOGGER.error("Error", e);
}
return response;
}
By the way, it works only if I remove outputChannel = "outputChannel" in Service Activator.
Is there any explanation for this issue, do I have any misunderstanding ?
You can't use DSL factories (Jms) like that, they are intended for use in a DSL flow
#Bean
IntegrationFLow flow()
return IntegrationFlows.from(jmsInboundGateway())
.handle("service", "process")
.get();
The DSL processing does all the wiring.
It works without the channel because a component without an output channel routes the reply to the replyChannel header.
If you don't want to use the DSL, you must wire up the inbound gateway directly as a bean instead of using the Jms factory.