I have a multi-module spring gradle app. 2 modules are executable, both modules use vaadin 14.1.17. I use a remote server to deploy two jar at the same time, but every time I run the second application, the first breaks. help to understand what the problem is. stacktrace broken application:
com.vaadin.flow.server.BootstrapException: Unable to read webpack stats file.
at com.vaadin.flow.server.BootstrapHandler$BootstrapPageBuilder.setupFrameworkLibraries(BootstrapHandler.java:872) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.BootstrapHandler$BootstrapPageBuilder.setupDocumentHead(BootstrapHandler.java:749) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.BootstrapHandler$BootstrapPageBuilder.getBootstrapPage(BootstrapHandler.java:524) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.BootstrapHandler.synchronizedHandleRequest(BootstrapHandler.java:465) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.SynchronizedRequestHandler.handleRequest(SynchronizedRequestHandler.java:40) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.VaadinService.handleRequest(VaadinService.java:1540) ~[flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.server.VaadinServlet.service(VaadinServlet.java:247) [flow-server-2.1.5.jar!/:2.1.5]
at com.vaadin.flow.spring.SpringServlet.service(SpringServlet.java:95) [vaadin-spring-12.1.2.jar!/:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:459) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:352) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.web.servlet.mvc.ServletForwardingController.handleRequestInternal(ServletForwardingController.java:141) [spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
each application runs on a separate port. In addition, before the first call through a web browser, the first application does not break. I launch applications on a Linux server as a systemd service.
I repeat again. Before I access through the browser, both services are running and do not contain errors. But after I access the FIRST running application, it crashes, but the second continues to work
Related
I have deployed a container in Google Cloud Run. It's an app built in Spring boot and Java.
I have exposed an endpoint(rest controller) to Cloud run container which further invokes a REST api to hit an external endpoint and at that point it fails. I am hitting the exposed endpoint from Postman.
Container is able to listen to incoming requests and even able to connect to Cloud SQL. Problem is - it cant hit an external API.
I even added an outbound rule in firewall to allow all the traffic but no success. Later, tried to turn the logs ON for firewall but no logs were generated.
Would anyone have any idea what could be the issue here ?
Please see the logs below and exception
2021-09-30T14:33:06.955652Z2021-09-30 14:33:06.955 WARN 1 --- [nio-8080-exec-3] io.netty.util.internal.MacAddressUtil :
Failed to find a usable hardware address from the network interfaces; using random bytes: 61:71:ba:a4:d5:85:ee:11
021-09-30T14:33:08.094246Z2021-09-30 14:33:08.093 DEBUG 1 --- [or-http-epoll-2] r.netty.http.client.HttpClientConnect : [id:93c6dc9b-1, L:/169.... - R:www.homedepot.ca/23.211.51.167:443] Handler is being applied: {uri=https://www.homedepot.ca/ method=GET}
2021-09-30T15:13:41.887524Z org.springframework.web.reactive.function.client.WebClientRequestException: handshake timed out after 10000ms;
nested exception is io.netty.handler.ssl.SslHandshakeTimeoutException: handshake timed out after 10000ms at
org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
2021-09-30T15:13:41.887759Z at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:414)
2021-09-30T15:13:41.887776Z at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:251)
2021-09-30T15:13:41.887805Z at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491)
2021-09-30T15:13:41.887813Z at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299)
2021-09-30T15:13:41.887822Z at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
2021-09-30T15:13:41.887859Z at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27)
2021-09-30T15:13:41.887873Z at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:190)
2021-09-30T15:13:41.887893Z at reactor.core.publisher.MonoCreate$MonoSink.error(MonoCreate.java:189)
2021-09-30T15:13:41.887903Z at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect$ClientTransportSubscriber.onError(HttpClientConnect.java:304)
2021-09-30T15:13:41.887916Z at reactor.core.publisher.MonoCreate$MonoSink.error(MonoCreate.java:189)
2021-09-30T15:13:41.887934Z at reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.onUncaughtException(PooledConnectionProvider.java:218)
2021-09-30T15:13:41.887958Z at reactor.netty.resources.PooledConnectionProvider$PooledConnection.onUncaughtException(PooledConnectionProvider.java:467)
2021-09-30T15:13:41.887979Z at reactor.netty.channel.ChannelOperationsHandler.exceptionCaught(ChannelOperationsHandler.java:129)
2021-09-30T15:13:41.887993Z at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
2021-09-30T15:13:41.888011Z at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
2021-09-30T15:13:41.888032Z at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
2021-09-30T15:13:41.888049Z at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:424)
2021-09-30T15:13:41.982305Z at io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:92)
2021-09-30T15:13:41.982894Z2021-09-30 15:13:41.281 WARN 1 --- [or-http-epoll-3] r.netty.http.client.HttpClientConnect : [id:c351aec9, L:/169.... - R:www.homedepot.ca/96.7.86.161:443] The connection observed an error
Code in Spring boot that is making the call
WebClient webClient =
WebClient.builder()
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.build();
Mono<HDCanadaProdResp> webClientResponse =
webClient.get().uri(stockApiurl).retrieve().bodyToMono(HDCanadaProdResp.class);
webClientResponse.block();
All the code works in local system in a Docker Container, not sure what's the issue on Cloud run.
Here is the Dockerfile
FROM maven:3.6.3-openjdk-11-slim as builder
WORKDIR /app
COPY pom.xml .
RUN mvn dependency:go-offline
COPY src/ /app/src/
RUN mvn package -DskipTests
FROM openjdk:11-jre-slim
COPY --from=builder /app/target/*.jar /app.jar
#PORT for cloud run, coz cloud maps to this port
ENV PORT 8080
ENV HOST 0.0.0.0
CMD ["java", "-jar", "/app.jar"]
Update on Nov 7, 2021
I changed my spring Boot code where I was using WebClient API to make rest call and now the error has changed.
old code
WebClient webClient =
WebClient.builder()
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate, br")
.defaultHeader(HttpHeaders.ACCEPT_LANGUAGE, "en-CA,en-US;q=0.7,en;q=0.3")
.defaultHeader(HttpHeaders.HOST, "www.homedepot.ca")
.build();
Mono<HDCanadaRecomProdResp> webClientResponse =
webClient.get().uri(prodApiurl).retrieve().bodyToMono(HDCanadaRecomProdResp.class);
return webClientResponse.block();
New code
HttpComponentsClientHttpRequestFactory requestFactory =
new HttpComponentsClientHttpRequestFactory();
requestFactory.setConnectTimeout(60 * 1000);
requestFactory.setReadTimeout(60 * 1000);
RestTemplate restTemplate = new RestTemplate(requestFactory);
ResponseEntity<HDCanadaRecomProdResp> response = null;
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.set(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate, br");
headers.set(HttpHeaders.ACCEPT_LANGUAGE, "en-CA,en-US;q=0.7,en;q=0.3");
headers.set(HttpHeaders.HOST, "www.homedepot.ca");
headers.setAccept(Arrays.asList(MediaType.ALL));
HttpEntity<Map<String, String>> entity = new HttpEntity<>(headers);
response =
restTemplate.exchange(prodApiurl, HttpMethod.GET, entity, HDCanadaRecomProdResp.class);
}
And now the error has changed to
org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://www.homedepot.ca/api/rec/v1/recommendations/fbt/products/1001200972/store/7301":
Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:785) at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:711) at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:602) at
com.pricetracker.productfetcher.client.HomeDepotCAClient.getProductDetails(HomeDepotCAClient.java:68) at
com.pricetracker.productfetcher.client.HomeDepotCAClient.getProductDetails(HomeDepotCAClient.java:24) at
com.pricetracker.productfetcher.service.ProductSearchServiceImpl.getProductFromWS(ProductSearchServiceImpl.java:116) at
com.pricetracker.productfetcher.service.ProductSearchServiceImpl.getProduct(ProductSearchServiceImpl.java:82) at
com.pricetracker.productfetcher.service.ProductSearchServiceImpl.productSearch(ProductSearchServiceImpl.java:61) at
com.pricetracker.productfetcher.controller.ProductsController.searchProduct(ProductsController.java:51) at
com.pricetracker.productfetcher.controller.ProductsController$$FastClassBySpringCGLIB$$37eced0e.invoke(<generated>) at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at
org.springframework.validation.beanvalidation.MethodValidationInterceptor.invoke(MethodValidationInterceptor.java:123) at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at
com.pricetracker.productfetcher.controller.ProductsController$$EnhancerBySpringCGLIB$$ec439be9.searchProduct(<generated>) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197) at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141) at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1064) at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) at
Use strace for this kind of Cloud Run related issues. Also view the logs in the Cloud Logging section of the GCP Console (not in the "Logs'' tab of the Cloud Run section), you can look for "Container Sandbox" with a "DEBUG" severity in the varlog/system logs. Also try running your application locally on Docker using these instructions and verify if your applications boots fine locally. The container must listen for requests on 0.0.0.0 on the port to which requests are sent. By default, requests are sent to 8080. Add the --min-instances in cloudbuild.yaml file and give it some value for now and then try.
If you see this log (which was not visible earlier, as it was coming during startup of container) Debug 2021-11-07T12:39:14.876726806ZContainer Sandbox: Unsupported syscall setsockopt(0xa,0x11,0x67,0x3ed0ac62bffc,0x4,0x1a) and you are able to run the setup locally, that means there is no issue with the docker image or your configurations. Unsupported syscall- This message indicates that system calls may not be supported in this Cloud Run (fully managed) as container instances are sandboxed using the gVisor container runtime sandbox.
See socket(7) - Linux manual page 8 for details. So gvisor does not support this socket option. Have a look at this Github issue which talks about the same error message you received.
It's a known issue in Cloud Run fully managed. You can try Cloud Run for Anthos with the same configurations and container setup (with less allocated resources).
for Integration with IBM MQ, I am using Spring Boot to create a REST API with services that will attempt to put messages in a queue as well as to retrieve messages from a queue.
I am using the following versions:
Spring Boot: 2.3.1
IBM MQ: 9.1.4.0
My SpringBoot application has the dependencies:
1. mq-jms-spring-boot-starter (dependency): 2.0.9
2. spring-boot-starter-web
I have install IBM MQ using the recommended installation, and chose NOT to configure a domain user id for IBM MQ to run under.
As for the Spring Boot application, I have built it according to the following:
com.mi.bootibmmqdemo
BootIbmMqDemoApplication.java ----> Spring boot runnable class
com.mi.bootibmmqdemo.controllers.mq
BootIbmMqDemoController.java ----> Rest Controller
com.mi.bootibmmqdemo.model.mq
MqRequestMessage.java ----> POJO request object
MqResponseMessage.java ----> POJO response object
I constructed the REST controller like so:
package com.mi.bootibmmqdemo.controllers.mq;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.JmsException;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.mi.bootibmmqdemo.model.mq.MqRequestMessage;
import com.mi.bootibmmqdemo.model.mq.MqResponseMessage;
#RestController
#RequestMapping("/boot-ibm-mq")
public class BootIbmMqDemoController {
Logger logger = LoggerFactory.getLogger(BootIbmMqDemoController.class);
#Autowired
JmsTemplate jmsTemplate;
#PostMapping("/sendMq")
public MqResponseMessage sendToQueue(#RequestBody MqRequestMessage mqRequestMessage) {
try {
jmsTemplate.convertAndSend(mqRequestMessage.getQueueName(), mqRequestMessage.getMessage());
return new MqResponseMessage("OK");
} catch (JmsException ex) {
ex.printStackTrace();
return new MqResponseMessage("FAIL");
}
}
#GetMapping("/receiveMq")
String recv(){
try{
return jmsTemplate.receive("DEV.QUEUE.1").toString();
}catch(JmsException ex){
ex.printStackTrace();
return "FAIL";
}
}
}
My application.properties file contains the following configuration:
ibm.mq.queueManager=QM1
ibm.mq.channel=DEV.ADMIN.SVRCONN
ibm.mq.connName=localhost(1414)
ibm.mq.user=admin
ibm.mq.password=passw0rd
Testing with Postman using the below JSON request:
{
"queueName":"DEV.QUEUE.1",
"message":"Hello World!"
}
I am receiving the 'FAIL' outcome, and in the logs I am getting the following error:
org.springframework.jms.JmsSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'QM1' with connection mode 'Client' and host name 'localhost(1414)'.; nested exception is com.ibm.msg.client.jms.DetailedJMSSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'QM1' with connection mode 'Client' and host name 'localhost(1414)'.
Please check if the supplied username and password are correct on the QueueManager to which you are connecting.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2035' ('MQRC_NOT_AUTHORIZED').
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:286)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:185)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:507)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584)
at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661)
at com.mi.bootibmmqdemo.controllers.mq.BootIbmMqDemoController.sendToQueue(BootIbmMqDemoController.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.ibm.msg.client.jms.DetailedJMSSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'QM1' with connection mode 'Client' and host name 'localhost(1414)'.
Please check if the supplied username and password are correct on the QueueManager to which you are connecting.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:531)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:424)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8475)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:7815)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:303)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)
at com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6016)
at com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6041)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
... 53 more
Caused by: com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2035' ('MQRC_NOT_AUTHORIZED').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203)
While searching online for the potential cause and starting from the error code, I have found the suggestion indicating that the channel authority rule for whole queue manager needs to be disabled, by using command prompt (windows) navigate to the MQ installation directory, in my case it is: D:\IBM\MQ\bin
and running the following:
D:\IBM\MQ\bin>runmqsc QM1
5724-H72 (C) Copyright IBM Corp. 1994, 2019.
Starting MQSC for queue manager QM1.
alter qmgr chlauth(disabled)
1 : alter qmgr chlauth(disabled)
AMQ8005I: IBM MQ queue manager changed.
end
2 : end
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.
D:\IBM\MQ\bin>runmqsc QM1
5724-H72 (C) Copyright IBM Corp. 1994, 2019.
Starting MQSC for queue manager QM1.
ALTER QMGR CHLAUTH(DISABLED)
1 : ALTER QMGR CHLAUTH(DISABLED)
AMQ8005I: IBM MQ queue manager changed.
end
2 : end
One MQSC command read.
No commands have a syntax error.
All valid MQSC commands were processed.
D:\IBM\MQ\bin>
I restarted the queue manager and restarted the Spring Boot application and re-tested with postman, same error is repeated.
UPDATE: adding MQ logs:
6/17/2020 08:17:06 - Process(24384.13) User(MUSR_MQADMIN) Program(amqzlaa0.exe)
Host(E107) Installation(Installation1)
VRMF(9.1.4.0) QMgr(QM1)
Time(2020-06-17T05:17:06.756Z)
CommentInsert1(admin)
CommentInsert2(BootIbmMqDemoApplication)
CommentInsert3(N/A)
AMQ5534E: User ID 'admin' authentication failed
EXPLANATION:
The user ID and password supplied by the 'BootIbmMqDemoApplication' program
could not be authenticated.
Additional information: 'N/A'.
ACTION:
Ensure that the correct user ID and password are provided by the application.
Ensure that the authentication repository is correctly configured. Look at
previous error messages for any additional information.
----- amqzfuca.c : 4716 -------------------------------------------------------
6/17/2020 08:17:06 - Process(24384.13) User(MUSR_MQADMIN) Program(amqzlaa0.exe)
Host(E107) Installation(Installation1)
VRMF(9.1.4.0) QMgr(QM1)
Time(2020-06-17T05:17:06.756Z)
CommentInsert1(admin)
CommentInsert2(DEV.AUTHINFO)
CommentInsert3(CHCKCLNT(REQDADM))
AMQ5542I: The failed authentication check was caused by the queue manager
CONNAUTH CHCKCLNT(REQDADM) configuration.
EXPLANATION:
The user ID 'admin' and its password were checked because the queue manager
connection authority (CONNAUTH) configuration refers to an authentication
information (AUTHINFO) object named 'DEV.AUTHINFO' with CHCKCLNT(REQDADM).
This message accompanies a previous error to clarify the reason for the user ID
and password check.
ACTION:
Refer to the previous error for more information.
Ensure that a password is specified by the client application and that the
password is correct for the user ID. The authentication configuration of the
queue manager connection determines the user ID repository. For example, the
local operating system user database or an LDAP server.
If the CHCKCLNT setting is OPTIONAL, the authentication check can be avoided by
not passing a user ID across the channel. For example, by omitting the MQCSP
structure from the client MQCONNX API call.
To avoid the authentication check, you can amend the authentication
configuration of the queue manager connection, but you should generally not
allow unauthenticated remote access.
----- amqzfuca.c : 4739 -------------------------------------------------------
Kind Regards,
After upgrading to Spring Boot 2.0.1, I am experiencing inexplicable problems with my applications.
There are two applications involved. Requests to App 1 make a back-end call to App 2 to fetch some data. The call to App 2 goes through an AWS elastic load balancer (ELB). The client used to make the call is Apache Commons HttpClient wrapped by my utility.
After App 1 was upgraded to Boot 2.0.1, I am seeing a small minority of the calls from App 1 to App 2 hang for a long time (15 minutes). When I use JConsole to get a thread dump of the hung threads, I see this stack trace:
Stack trace:
java.net.PlainSocketImpl.socketClose0(Native Method)
java.net.AbstractPlainSocketImpl.socketPreClose(AbstractPlainSocketImpl.java:693)
java.net.AbstractPlainSocketImpl.close(AbstractPlainSocketImpl.java:530)
- locked java.lang.Object#29a91ad3
java.net.SocksSocketImpl.close(SocksSocketImpl.java:1075)
java.net.Socket.close(Socket.java:1495)
- locked java.lang.Object#71e238ff
- locked java.net.Socket#73f58606
sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624)
- locked sun.security.ssl.SSLSocketImpl#22264f18
sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1585)
sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1723)
sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2020)
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1135)
- locked sun.security.ssl.SSLSocketImpl#22264f18
- locked java.lang.Object#4338a60d
sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
- locked sun.security.ssl.AppInputStream#23fd5b55
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(BHttpConnectionBase.java:344)
org.apache.http.impl.BHttpConnectionBase.isStale(BHttpConnectionBase.java:364)
org.apache.http.impl.conn.CPool.validate(CPool.java:71)
org.apache.http.impl.conn.CPool.validate(CPool.java:45)
org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:249)
- locked org.apache.http.pool.AbstractConnPool$2#7c672c9a
org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:193)
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:282)
org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:269)
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:191)
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
com.kendelong.util.http.HttpConnectionService.doExecuteAndGetResponse(HttpConnectionService.java:243)
com.kendelong.util.http.HttpConnectionService.getResult(HttpConnectionService.java:189)
com.kendelong.util.http.IHttpConnectionService$getResult$0.call(Unknown Source)
com.hatchbaby.sub.util.MainSiteHttpProxyService.getMemberData(MainSiteHttpProxyService.groovy:68)
com.hatchbaby.sub.util.MainSiteHttpProxyService$$FastClassBySpringCGLIB$$3cc5cf68.invoke(<generated>)
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
com.kendelong.util.performance.PerformanceMonitoringAspect.monitorInvocation(PerformanceMonitoringAspect.java:108)
com.kendelong.util.performance.PerformanceMonitoringAspect.monitorAnnotatedClasses(PerformanceMonitoringAspect.java:83)
sun.reflect.GeneratedMethodAccessor177.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
com.hatchbaby.sub.util.MainSiteHttpProxyService$$EnhancerBySpringCGLIB$$edc027d4.getMemberData(<generated>)
com.hatchbaby.sub.service.security.HatchBabyAuthenticationUserDetailsService.loadUserDetails(HatchBabyAuthenticationUserDetailsService.java:39)
com.hatchbaby.sub.service.security.HatchBabyAuthenticationUserDetailsService.loadUserDetails(HatchBabyAuthenticationUserDetailsService.java:20)
org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationProvider.authenticate(PreAuthenticatedAuthenticationProvider.java:103)
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doAuthenticate(AbstractPreAuthenticatedProcessingFilter.java:184)
org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:118)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:245)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
com.hatchbaby.sub.web.RequestInfoFilter.doFilter(RequestInfoFilter.java:99)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
com.hatchbaby.sub.web.SpecialRedirectFilter.doFilter(SpecialRedirectFilter.java:43)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.ebaysf.web.cors.CORSFilter.handleSimpleCORS(CORSFilter.java:302)
org.ebaysf.web.cors.CORSFilter.doFilter(CORSFilter.java:160)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper#6ca9f281
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:748)
The thread hangs in this state for roughly 15 minutes (actally 15 min 44 seconds plus/minus 5 seconds), or a rough integer multiple thereof. Eventually the request succeeds.
What is even odder is that when I compare the logs, I see the request enter App 1 and make the call to App 2 (say, at 8:00). The thread hangs in the socketClose0() method as seen in the thread dump. When the request finally completes 15 minutes later (at 8:15), the timestamps in the logs of App 2 and its ELB show that the request arrived the later time (8:15): it's as if the socket is trying to close for 15 minutes, it finally closes (without having contacted the App 2 server), and immediately retries. Usually it succeeds, but sometimes it gets stuck again for another 15 minutes (hence the multiples of 15 minutes). But the request doesn't even register at App 2's ELB until 15 minutes later (8:15).
I have seen this behavior in my QA environment (one server each) and in my production environment (2 servers each). If I roll the code back to the version with Spring Boot 1.5.9, the problem disappears. However, I have reverted most of the suspect libraries one-by-one to the Boot 1.5.9 version, and the problem persists. I have reverted httpclient (obviously), groovy, tomcat, and commons-codec. I was unable to revert Spring (reset spring.version property) as the app would not start. As part of the upgrade to Boot 2.0.1, I switched from EhCache distributed by JGroups to Hazelcast as the Hibernate second-level cache, but I have completely removed Hazelcast from the app and the problem persists. Reverting Hibernate or Spring Data JPA is more difficult because of code changes.
There are no timeouts set at 15 minutes in my app, of course. Both connection and socket timeouts for httpclient are 5 seconds.
Does anyone have any clue, or any troubleshooting suggestions?
UPDATE
Setting -Djavax.net.debug=all yields this output:
http-nio-8080-exec-6, setSoTimeout(1) called
[Raw read]: length = 5
0000: 15 03 03 00 1A .....
[Raw read]: length = 26
0000: 98 1C 7C C4 27 0B 2B 40 E5 AC D7 38 A9 6F F7 73 ....'.+#...8.o.s
0010: 83 18 E8 15 14 7C 52 10 47 5E ......R.G^
http-nio-8080-exec-6, READ: TLSv1.2 Alert, length = 26
Padded plaintext after DECRYPTION: len = 2
0000: 01 00 ..
http-nio-8080-exec-6, RECV TLSv1.2 ALERT: warning, close_notify
http-nio-8080-exec-6, called closeInternal(false)
http-nio-8080-exec-6, SEND TLSv1.2 ALERT: warning, description = close_notify
Padded plaintext before ENCRYPTION: len = 2
0000: 01 00 ..
http-nio-8080-exec-6, WRITE: TLSv1.2 Alert, length = 26
[Raw write]: length = 31
0000: 15 03 03 00 1A 00 00 00 00 00 00 00 02 1F A3 99 ................
0010: 4C CD 61 EB 02 2C 14 0D 00 27 03 51 05 F7 1F L.a..,...'.Q...
http-nio-8080-exec-6, called closeSocket(false)
At this point everything hangs. Looking with netstat on the client, I see the socket is in LAST_ACK state, meaning it's waiting for a final ACK packet.
I removed App 2's ELB from the mix and connected directly to the server for App 2. The problem persists. When client connections are in LAST_ACK state, there are no corresponding connections showing up in netstat on the server side. If the server had not sent the final ACK, it would be in FIN_WAIT_2, but since the connection is already gone, it must have sent the final ACK but it never arrived at the client.
How could a Spring Boot upgrade lead to blocked ACK packets on socket close operations?
All of this is happening on Amazon Linux 2017.9.
UPDATE 2
Five months later, same thing. I upgraded App2 to Spring Boot 2.0.5, no problems.
App1, when upgraded to 2.0.5, starts hanging it's threads. I turned on HttpClient wire logging, and what I see just where it's about to make the network call, there is no outgoing data on the wire logger, and 15m33s later I get an "end of stream" coming in and the request finally closes.
Halleuja, I figure it out. My HTTP library was setting socketLinger = socketTimeout + 500. However, socketLinger and socketTimeout are quite different! My socketTimeout is set to 5000 ms. I released version 1.7.0 which stopped setting socketLinger and presto the problem went away.
I still don't know why I never saw problems with this before, or why the Spring Boot 2 upgrade caused the bug to manifest, but at least it's gone now!
I am migrating an app to WAS 7.0 (fix pack 13). The app uses a number of jars which are located in the Appserver\lib\ext dir. If struts.jar (1.2.7) is present when invoking the WAS admin console it crashes with an error 404 screen (below). If I remove struts.jar from lib\ext the admin console works, naturally my app needs it. WAS jar loader is set to parent first, my app is parent last.
Thanks in advance for any advice.
Error screens and dump, note the login screen appears normally, this is after clicking "Log in":
Error404
An error occurred while processing request:%2Fibm%2Fconsole%2Findex.do
Message:org.apache.struts.tiles.DefinitionsFactoryException: IO Error while parsing file '/WEB-INF/com.ibm.ws.console.core/console-defs.xml'. Connection timed out: connect
Details:
javax.servlet.UnavailableException: org.apache.struts.tiles.DefinitionsFactoryException: IO Error while parsing file '/WEB-INF/com.ibm.ws.console.core/console-defs.xml'. Connection timed out: connect
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:534)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:502)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:179)
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java:341)
at com.ibm.ws.webcontainer.servlet.FilterProxyServlet.dispatch(FilterProxyServlet.java:88)
at com.ibm.ws.webcontainer.servlet.FilterProxyServlet.service(FilterProxyServlet.java:62)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1657)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1597)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:131)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.setUpCommandAssistence(WSCUrlFilter.java:933)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.continueStoringTaskState(WSCUrlFilter.java:500)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.doFilter(WSCUrlFilter.java:321)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:188)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:116)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:77)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:997)
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.invokeFilters(DefaultExtensionProcessor.java:985)
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:682)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3826)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:276)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:931)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1583)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:186)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:445)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:504)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:301)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:275)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1016)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:639)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1772)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1563)
The WAS console uses it's own version of Struts.
Since you have placed a different version of struts (high up in the classloader - lib/ext), the WAS console has trouble.
It is in 99.99 % of the cases a bad idea to dump in jars like these to lib/ext.
You can use application specific classpath or use shared library approach if you need these jars to be shared across multiple applications.
Reconsider the usage of placing jars in lib/ext.
HTH
Manglu
I have integrated a simple service call in Kony, When I run my App on iOS simulator i cannot see any errors, instead in the middleware.log files I can see below error, Can anyone please help.
Logs
X-Forwarded-For=null] 12:15:55,270 DEBUG factory.KonyAppFactory - Memcache is Enabled and Memcache Session Management instance is created
[appID=ProjectBestBuy01
requestID=61c164cc-1743-437f-8288-8e6ec0903d86
UA=BestBuy/1.0 CFNetwork/711.1.12 Darwin/15.0.0
rcid=NA
referer=NA
node.no=1
REMOTEADDRESS=127.0.0.1
REMOTEADDRESS=127.0.0.1
X-Forwarded-For=null] 11:39:51,112 ERROR cache.MemCacheWrapper - Unable to store object in Memcache node /127.0.0.1:21201 for Key: 10e6351c7-fe0d-44e4-bbc5-3e448d0af7a6 in 3 attempt from Server Node Num: 1
java.util.concurrent.TimeoutException: Timed out waiting for operation
at net.spy.memcached.MemcachedClient$OperationFuture.get(MemcachedClient.java:1656)
at com.konylabs.middleware.cache.MemCacheWrapper.storebyNoofattempts(MemCacheWrapper.java:277)
at com.konylabs.middleware.cache.MemCacheWrapper.store(MemCacheWrapper.java:235)
at com.konylabs.middleware.common.AbstractCacheSessionManager.store2Cache(AbstractCacheSessionManager.java:280)
at com.konylabs.middleware.common.AbstractCacheSessionManager.exit(AbstractCacheSessionManager.java:212)
at com.konylabs.middleware.common.MemCacheDCFilterAction.doChainDCFilter(MemCacheDCFilterAction.java:115)
at com.konylabs.middleware.common.MiddlewareMemCacheDCFilter.doFilter(MiddlewareMemCacheDCFilter.java:42)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at com.konylabs.middleware.common.XSSFilter.doFilter(XSSFilter.java:169)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at com.konylabs.middleware.common.AddAdditionalResponseHeaderAttribute.doFilter(AddAdditionalResponseHeaderAttribute.java:94)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[appID=ProjectBestBuy01
requestID=3dc52d1e-9b5a-45d5-8b64-92f8cf1f0cb7
UA=BestBuy/1.0 CFNetwork/711.1.12 Darwin/15.0.0
rcid=NA
You question don't have enough information.
1.Are you trying to save any information on middle ware meme cache ?
2.if yes , how you are trying to save preprocessor,postprocessor or javaservice ?
3.Are you set scope as session in service def. file ?