Error when starting Mule-ESB Mx4j agent in WebSphere 7 - spring

I'm trying to use an Mx4j agent (and Spring Framework 3.0.5) to expose some POJOs in my Mule-ESB (Mule 3.1.2) application as an HTTP service. The agent is configured in mule-config.xml as follows:
<management:jmx-mx4j-adaptor jmxAdaptorUrl="http://0.0.0.0:9990" />
Also, I use the Spring MBeanExporter to expose the desired POJOs:
<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
<property name="assembler" ref="assembler" />
<property name="namingStrategy" ref="namingStrategy" />
<property name="autodetect" value="true" />
</bean>
<bean id="jmxAttributeSource" class="org.springframework.jmx.export.annotation.AnnotationJmxAttributeSource" />
<bean id="assembler" class="org.springframework.jmx.export.assembler.MetadataMBeanInfoAssembler">
<property name="attributeSource" ref="jmxAttributeSource" />
</bean>
<bean id="namingStrategy" class="org.springframework.jmx.export.naming.MetadataNamingStrategy">
<property name="attributeSource" ref="jmxAttributeSource" />
</bean>
Everything works fine in my desktop environment, which runs in Jetty. When I deploy the EAR to our WebSphere 7 Server, however, the application doesn't start, raising the following exception:
[3/30/12 16:33:58:858 BRT] 00000038 webapp I com.ibm.ws.webcontainer.webapp.WebApp log SRVE0296E: [BaseApp#BaseApp.war][/context][Servlet.LOG]:.Failed to invoke lifecycle phase "start" on object: org.mule.module.management.agent.Mx4jAgent#13ef13ef:.org.mule.api.lifecycle.LifecycleException: Failed to invoke lifecycle phase "start" on object: org.mule.module.management.agent.Mx4jAgent#13ef13ef
at org.mule.lifecycle.phases.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:236)
at org.mule.lifecycle.RegistryLifecycleManager$RegistryLifecycleCallback.onTransition(RegistryLifecycleManager.java:276)
...
...
Caused by: org.mule.module.management.agent.JmxManagementException: Failed to start Mx4j agent
at org.mule.module.management.agent.Mx4jAgent.start(Mx4jAgent.java:205)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.mule.lifecycle.phases.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:225)
... 67 more
Caused by: javax.management.InstanceNotFoundException: Mule.BaseApp.6:name=Mx4jHttpAdapter
at java.lang.Throwable.<init>(Throwable.java:67)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1384)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:892)
at com.ibm.ws.management.AdminServiceImpl.getMBeanInfo(AdminServiceImpl.java:1524)
at com.ibm.ws.management.AdminServiceImpl.checkForOpDeprecation(AdminServiceImpl.java:2662)
at com.ibm.ws.management.AdminServiceImpl.preInvoke(AdminServiceImpl.java:2284)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1309)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1225)
at com.ibm.ws.management.PlatformMBeanServer.invoke(PlatformMBeanServer.java:743)
at org.mule.module.management.agent.Mx4jAgent.start(Mx4jAgent.java:201)
Looking at the second (and last) stack trace cause, there seem to be some confusion or assumption about the JMX Server Instance Name. However, I have no control on that (and I can't figure why it works fine in Jetty and not in WAS 7).
Has anyone stepped through that before? Am I doing something wrong? Am I missing something here?
Thanks a lot for any response!

For WAS using JMX is not that straightforward. Complications are:
Security - one can't access JMX endpoint by default without further security configuration (out of scope for this discussion).
Websphere's JMX server implementation actually modifies MBean domain names to be prefixed with WAS Node and Cell. There is a WAS-specific API that can get you those, but the takeaway is this can't be done without further coding.
Andrew

Related

Mail Service Task in Flowable throws api.FlowableException - java.net.ConnectException: Connection refused

In our project we have a task of sending notification to some user after some task is completed.
Following is the code written in the BPMN2.0 xml -
<serviceTask id="notifyPartner" name="Notify Partner" flowable:type="mail">
<documentation>Notify Partner about the assignment.</documentation>
<extensionElements>
<flowable:field name="to">
<flowable:string><![CDATA[testemail#some.com]]></flowable:string>
</flowable:field>
<flowable:field name="from">
<flowable:string><![CDATA[testemail#some.com]]></flowable:string>
</flowable:field>
<flowable:field name="subject">
<flowable:string><![CDATA[Notify Partner]]></flowable:string>
</flowable:field>
<flowable:field name="text">
<flowable:string><![CDATA[Notify Partner about the assignment.]]></flowable:string>
</flowable:field>
</extensionElements>
</serviceTask>
With a application.properties as -
spring.flowable.mailServerHost=smtp.gmail.com
spring.flowable.mailServerPort=587
spring.flowable.mailServerUserName=testemail#some.com
spring.flowable.mailServerPassword=****
spring.flowable.mailServerUseTls=true
But all this results in no success. It gives the following error -
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
I even tried setting up all the properties in flowable.cfg.xml also
<property name="mailServerHost" value="smtp.gmail.com" />
<property name="mailServerPort" value="587" />
<property name="mailServerUseTls" value="true" />
<property name="mailServerUsername" value="testemail#some.com" />
<property name="mailServerPassword" value="****" />
Is there any other configuration missing?
Flowable version used is 6.2.1.
In case you are using the Spring Boot starter for Flowable 6.2.1 you will need to remove the leading spring. from your properties.
Your properties should look like:
flowable.mailServerHost=smtp.gmail.com
flowable.mailServerPort=587
flowable.mailServerUserName=testemail#some.com
flowable.mailServerPassword=****
flowable.mailServerUseTls=true
When using Spring Boot the flowable.cfg.xml is not being used to configure the engine.
NOTE: When migrating to Flowable 6.3.x the properties have slightly changed and they need to look like:
flowable.mail.server.host=smtp.gmail.com
flowable.mail.server.port=587
flowable.mail.server.username=testemail#some.com
flowable.mail.server.password=****
flowable.mail.server.use-tls=true

Apache Camel Spring DSL, referring to an environment variable HOSTNAME

I am using Apache Camel 2.16.0 with Spring DSL
I have a Spring context XML in which I have defined a Property PlaceHolder to read the properties from various files as follows-
<bean id="propertyPlaceholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
<property name="ignoreResourceNotFound" value="false"/>
<property name="locations">
<list>
<value>classpath:/properties/versioning.properties</value>
<value>classpath:/properties/#{inetAddress.hostName}.properties</value>
</list>
</property>
</bean>
<bean id="inetAddress" class="java.net.InetAddress" factory-method="getLocalHost"/>
The property values are used to construct other beans such as -
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.gjt.mm.mysql.Driver"/>
<property name="url" value="${${LIVE_}DATASOURCE_URL}"/>
<property name="username" value="${${LIVE_}DATASOURCE_USERNAME}"/>
<property name="password" value="${${LIVE_}DATASOURCE_PASSWORD}"/>
</bean>
This works fine, I can see the beans being created.
I also have another Spring Context XML in the same app which has a camel context and I want to use the some other properties defined in the same properties files. I know that camel supports Spring Property Placeholder, see below excerpts from the context -
<camelContext id="charge-process-context" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="properties"
location="classpath:/properties/versioning.properties,
properties/${env:HOSTNAME}.properties"
xmlns="http://camel.apache.org/schema/spring" />
.....
.....
</camelContext>
As part of this context, I have a route that uses https component that uses the values from the property file such as below -
<to uri="https4:{{LIVE_AUTH_RESPONSE_HOST}}:{{LIVE_AUTH_RESPONSE_PORT}}/{{LIVE_AUTH_RESPONSE_CONTEXT_PATH}}"/>
This route does not start and throws following exception -
Caused by: java.lang.IllegalArgumentException: Cannot find system environment with key: HOSTNAME
at org.apache.camel.util.FilePathResolver.resolvePath(FilePathResolver.java:54)
at org.apache.camel.component.properties.PropertiesComponent.parseLocations(PropertiesComponent.java:434)
at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:163)
at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:148)
at org.apache.camel.impl.DefaultCamelContext.resolvePropertyPlaceholders(DefaultCamelContext.java:2261)
at org.apache.camel.model.ProcessorDefinitionHelper.resolvePropertyPlaceholders(ProcessorDefinitionHelper.java:730)
at org.apache.camel.model.ProcessorDefinition.createOutputsProcessorImpl(ProcessorDefinition.java:427)
at org.apache.camel.model.ProcessorDefinition.createOutputsProcessor(ProcessorDefinition.java:413)
at org.apache.camel.model.ProcessorDefinition.createOutputsProcessor(ProcessorDefinition.java:165)
at org.apache.camel.model.ExpressionNode.createFilterProcessor(ExpressionNode.java:109)
at org.apache.camel.model.WhenDefinition.createProcessor(WhenDefinition.java:74)
at org.apache.camel.model.WhenDefinition.createProcessor(WhenDefinition.java:32)
at org.apache.camel.model.ProcessorDefinition.createProcessor(ProcessorDefinition.java:483)
at org.apache.camel.model.ChoiceDefinition.createProcessor(ChoiceDefinition.java:135)
at org.apache.camel.model.ProcessorDefinition.makeProcessorImpl(ProcessorDefinition.java:534)
at org.apache.camel.model.ProcessorDefinition.makeProcessor(ProcessorDefinition.java:495)
at org.apache.camel.model.ProcessorDefinition.addRoutes(ProcessorDefinition.java:219)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:1069)
Please Note: I am deploying my application as a war file on Tomcat 8 on an AWS instance.
I have a Dev Environment on Windows 10 and I have found this working on the Windows OS. I have also seen that the file FilePathResolver.java in Apache Camel 2.16 uses System.getenv(key) to obtain the value i.e. System.getenv("HOSTNAME") which returns a null on AWS instance and a correct value on Windows 10. I also tried using env:hostname (small case letters for unix) but still no luck ...
I found a solution at http://camel.apache.org/using-propertyplaceholder.html
at Bridging Spring and Camel Property Placeholders
It mentions following -
The Spring Framework does not allow 3rd party frameworks such as Apache Camel to seamless hook into the Spring property placeholder mechanism. However you can easily bridge Spring and Camel by declaring a Spring bean with the type org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer, which is a Spring org.springframework.beans.factory.config.PropertyPlaceholderConfigurer type.

EJB remote call from standalone client to JBoss server is not working in Multi threading

I am using spring container in a batch operation that realize many calls to an remote EJB. When the operation is single-thread everything works fine but when try to use multi-threading to gain performance it throws:
Exception in thread "taskExecutor-1" javax.ejb.EJBException: java.io.IOException: Channel Channel ID e9c80c0d (outbound) of Remoting connection 18f42160 to servername/ip:port has been closed
My stand-alone cliente dependecy is:
<dependency>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-ejb-client-bom</artifactId>
<type>pom</type>
<version>7.2.0.Final</version>
</dependency>
I'm also using spring's SimpleRemoteStatelessSessionProxyFactory to inject the lookup beans
<bean id="jndiProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<property name="location" value="file:/path/jboss-ejb-client.properties" />
</bean>
<bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment" ref="jndiProperties" />
</bean>
<bean id="operation" class="org.springframework.ejb.access.SimpleRemoteStatelessSessionProxyFactoryBean">
<property name="jndiName" value="ejb:remote/interface/location" />
<property name="jndiTemplate" ref="jndiTemplate" />
<property name="businessInterface" value="com.MyBussinesInterfae" />
</bean>
In my tests I verified that if a Thread-A creates the InitContext and does the JNDI EJB lookup and creates the instance of the EJB remote inferface (under the SRSSPFBean) it can invoke methods via the EJB remote interface, but if an Thread-B gets the reference of the EJB remote interface and tries to invoke a method the exception is thrown.
The only similiar problem found on my search is here and seems related to bugs on Jboss AS 7. I am using Jboss eap 6.2.
Thanks in advance for any help.

BoneCP sometimes cannot get config from spring property holder

I have been facing this issue for a while now. My config is as following
<!-- Load Properties Files -->
<context:property-placeholder location="classpath:*-${environment}.properties" ignore-unresolvable="true"/>
<bean id="dataSource" class="com.jolbox.bonecp.BoneCPDataSource" destroy-method="close">
<property name="driverClass" value="${jdbc.driverClassName}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="idleConnectionTestPeriodInMinutes" value="${boneCP.idleConnectionTestPeriodInMinutes}"/>
<property name="idleMaxAgeInMinutes" value="${boneCP.idleMaxAgeInMinutes}"/>
<property name="maxConnectionsPerPartition" value="${boneCP.maxConnectionsPerPartition}"/>
<property name="minConnectionsPerPartition" value="${boneCP.minConnectionsPerPartition}"/>
<property name="partitionCount" value="${boneCP.partitionCount}"/>
<property name="acquireIncrement" value="${boneCP.acquireIncrement}"/>
<property name="statementsCacheSize" value="${boneCP.statementsCacheSize}"/>
<property name="lazyInit" value="true"/>
<property name="maxConnectionAgeInSeconds" value="${boneCP.maxConnectionAgeInSeconds}"/>
</bean>
The project is running on Tomcat 7
On the local machine, the project deploy with no error as well as for the dev server. Unfortunately, the project cannot be deployed on dev server any more (server configuration remain same) while local machine is still fine. Every time I deploy the project on the dev server, Tomcat just hang at INFO: Deploying web application archive /etc/tomcat/webapps/project.war. But if I config BoneCP with real values, everything is fine.
Could any one tell me what's wrong with it?
It turned out to be the lazyInit problem. If I comment it out, the server can start normally. But now I'm facing the new issue though. Mybatis cannot access the db at all while the local machine is 100% fine. and yet dont throw any exception. But when I stop the server, I found the following exceptions
INFO: Illegal access: this web application instance has been stopped already. Could not load com.jolbox.bonecp.PoolUtil. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
at com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal(DefaultConnectionStrategy.java:94)
at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection(AbstractConnectionStrategy.java:90)
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:540)
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:131)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:80)
at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:66)
AND
INFO: Illegal access: this web application instance has been stopped already. Could not load org.apache.ibatis.reflection.ExceptionUtil. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:363)
at sun.proxy.$Proxy15.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:195)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:124)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:90)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:40)
at sun.proxy.$Proxy45.selectByExample(Unknown Source)
Well it could be many things but its most likely
properties are not being replaced with the values you think
the database number of connections has been exceeded or the wrong host
A combination of 1 + 2
For #1 I would a make a separate bean that needs com.jolbox.bonecp.BoneCPDataSource as a dependency and have it print out the getters of BoneCPDataSource.
For #2 I would turn on as much logging as possible (see log4j or logback or whatever your logging framework is).

Multiple MBean Servers found - Spring JMX with Websphere 7

I am using Spring 3 and exposed an MBean through Spring, everything works fine but I see below warning message in the Log file.
WARN org.springframework.jmx.support.JmxUtils - Found more than one MBeanServer instance. Returning first from list.
Here is my configuration:
<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
<property name="assembler" ref="assembler" />
<property name="namingStrategy" ref="namingStrategy" />
<property name="autodetect" value="true" />
<property name="registrationBehaviorName" value="REGISTRATION_REPLACE_EXISTING"/>
</bean>
<bean id="jmxAttributeSource"
class="org.springframework.jmx.export.annotation.AnnotationJmxAttributeSource" />
<bean id="assembler"
class="org.springframework.jmx.export.assembler.MetadataMBeanInfoAssembler">
<property name="attributeSource" ref="jmxAttributeSource" />
</bean>
<bean id="WASAdminService" class="com.ibm.websphere.management.AdminServiceFactory"
factory-method="getAdminService" />
<bean id="namingStrategy" class="com.xxxx.WebSphereNamingStrategy">
<constructor-arg ref="WASAdminService" />
</bean>
I put a debug point and found that it is finding 'com.ibm.ws.management.PlatformMBeanServer' and 'com.sun.jmx.mbeanserver.JmxMBeanServer' objects. Any idea why it is finding more than one? When I googled, I found that I can specify the 'agent id' to find the required MBean server, but again if we are deploying this code in multiple environments, agent id may not be same (cannot be static)...
Any inputs on this is appreciated...
Thanks, Kiran
My guess is that the two MBean servers correspond to:
the "Platform" MBean server - established by the JVM. This MBean server would have all of the builtin Mbeans, like: java.lang:type=Runtime
the WebSphere provided MBean server - which has MBeans like WebSphere:name=WebContainer,process=...
I'm new to this area, but am just about to investigate further. From my perspective, I'd like to specify the Platform Mbean server - so that it's easier to access from monitoring tools, like JConsole, etc

Resources