Active MQ clustering using http auto discovery with multi cast on Amazon EC2 - amazon-ec2

We are trying to set up the active MQ cluster on production environment on Amazon EC2 with Auto discover and multicast.
I was able to configure successfully auto discovery with multi-cast on my local active mq server but on Amazon EC2 it is not working.
From the link
I found that Amazon EC2 does not support multi-cast. Hence we have to use HTTP transport or VPN for multi-cast. I tried HTTP transport for multi-cast by downloading activemq-optional-5.6.jar (we are using Active-MQ 5.6 version). It requires httpcore and httpClient jars to servlet in it class path.
In broker configuration(activemq.xml)
`
&ltnetworkConnectors>
&ltnetworkConnector name="default" uri="http://localhost:8161/activemq/DiscoveryRegistryServlet"/>
&lt/networkConnectors>
&lttransportConnectors>
&lttransportConnector name="activemq" uri="tcp://localhost:61616" discoveryUri="http://localhost:8161/activemq/DiscoveryRegistryServlet"/>
&lt/transportConnectors>`
are added.
But broker is not identifying the DiscoveryRegistryServlet.
Any help is much appreciated.

Finally figured out how to setup active MQ auto discovery with HTTP
Active-MQ Broker configuration:
In $ACTIVEMQ_HOME/webapps folder create a new folder
|_activemq
|_WEB-INF
|_classes
|_web.xml
create a web.xml file with the following contents
&ltweb-app>
&ltdisplay-name>ActiveMQ Message Broker Web Application&lt/display-name>
&ltdescription>
Provides an embedded ActiveMQ Message Broker embedded inside a web application
&lt/description>
&lt!-- context config -->
&ltcontext-param>
&ltparam-name>org.apache.activemq.brokerURL&lt/param-name>
&ltparam-value>tcp://localhost:61617&lt/param-value>
&ltdescription>The URL that the embedded broker should listen on in addition to HTTP&lt/description>
&lt/context-param>
&lt!-- servlet mappings -->
&ltservlet>
&ltservlet-name>DiscoveryRegistryServlet&lt/servlet-name>
&ltservlet-class>org.apache.activemq.transport.discovery.http.DiscoveryRegistryServlet&lt/servlet-class>
&ltload-on-startup>1&lt/load-on-startup>
&lt/servlet>
&ltservlet-mapping>
&ltservlet-name>DiscoveryRegistryServlet&lt/servlet-name>
&lturl-pattern>/*&lt/url-pattern>
&lt/servlet-mapping>
&lt/web-app>
Place httpclient-4.0.3.jar, httpcore-4.3.jar, xstream-1.4.5.jar and activemq-optional-5.6.0.jar in $ACTIVEMQ_HOME/lib directory.
In $ACTIVEMQ_HOME/config directory, modify the jetty.xml file to expose activemq web app.
&ltbean id="securityHandler" class="org.eclipse.jetty.security.ConstraintSecurityHandler">
...
&ltproperty name="handler">
&ltbean id="sec" class="org.eclipse.jetty.server.handler.HandlerCollection">
&ltproperty name="handlers">
...
...
&ltbean class="org.eclipse.jetty.webapp.WebAppContext">
&ltproperty name="contextPath" value="/activemq" />
&ltproperty name="resourceBase" value="${activemq.home}/webapps/activemq" />
&ltproperty name="logUrlOnStart" value="true" />
&ltproperty name="parentLoaderPriority" value="true" />
...
...
&lt/list>
&lt/property>
&lt/bean>
&lt/property>
&lt/bean>
Modify activemq.xml file in $ACTIVEMQ_HOME/conf directory to use http protocol
&ltbroker name=”brokerName”>
...
&ltnetworkConnectors>
&ltnetworkConnector name="default" uri="http://&ltloadbalancer_IP>:&ltlocadbalancer_Port>/activemq/DiscoveryRegistryServlet?group=test"/>
&lt!--&ltnetworkConnector name="default-nc" uri="multicast://default"/>-->
&lt/networkConnectors>
&lttransportConnectors>
&lttransportConnector name="http" uri="tcp://0.0.0.0:61618" discoveryUri="http://&ltloadbalancer_IP>:&ltlocadbalancer_Port>/activemq/test"/>
&lt/transportConnectors>
...
&lt/broker>
make sure that the broker names are unique. “test” in url is the group name of brokers.
Client configuration:
1. Keep httpclient-4.0.3.jar, httpcore-4.3.jar, xstream-1.4.5.jar and activemq-optional-5.6.0.jar in classpath of client
2. URL to be use by client
discovery:(http://&ltloadbalancer_IP>:&ltlocadbalancer_Port>/activemq/test)connectionTimeout=10000
here “test” is the group name.

Related

IBM WebSphere Liberty working with IHS Server Problems with Certificate(s)

I have a liberty server and an IHS server (both on different hosts). This is on rhel7.
I've followed a number of guides that I found that walk thru the setup of keystores and sharing of certs but still run into issues. For reference, I tried the methods detailed https://jazz.net/wiki/bin/view/Deployment/CreateIHSPLUGINFORLIBERTYPROFILE and https://www.ibm.com/support/knowledgecenter/en/SSEQTJ_9.0.5/com.ibm.websphere.ihs.doc/ihs/tihs_install_config_liberty.html.
A quick summary of what I have done this past attempt... On the liberty server I added the following to my server.xml to add a plugin include (to keep things easier to read):
<include location="${server.config.dir}/plugin-join-include.xml" />
And the plugin-join-include.xml:
<featureManager>
<feature>ssl-1.0</feature>
</featureManager>
<keyStore id="libertyKey" password="xxxx"
location="${server.config.dir}/resources/security/libkey.jks"/>
<pluginConfiguration webserverPort="10500" webserverSecurePort="10447"
pluginInstallRoot="/opt/IBM/wasadmin/Plugins"
sslKeyRingLocation="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.kdb"
sslStashFileLocation="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.sth"
sslCertlabel="libertyKey"/>
On my IHS server the path for the root Plugin directory is as indicated above in the file at /opt/IBM/wasadmin/Plugins and the plugin-key.kdb and plugin-key.sth are located in the locations listed above.
I reboot the liberty server and it generated the plugin-cfg.xml in the logs/state directory:
<?xml version="1.0" encoding="UTF-8"?><!--HTTP server plugin config file for app generated on 2020.03.04 at 12:32:02 UTC-->
<Config ASDisableNagle="false" AcceptAllContent="false" AppServerPortPreference="HostHeader" ChunkedResponse="false" ConfigHash="1077723051" FIPSEnable="false" IISDisableNagle="false" IISPluginPriority="High" IgnoreDNSFailures="false" RefreshInterval="60" ResponseChunkSize="64" SSLConsolidate="false" TrustedProxyEnable="false" VHostMatchingCompat="false">
<Log LogLevel="Error" Name="/opt/IBM/wasadmin/Plugins/logs/webserver1/http_plugin.log"/>
<Property Name="ESIEnable" Value="true"/>
<Property Name="ESIMaxCacheSize" Value="1024"/>
<Property Name="ESIInvalidationMonitor" Value="false"/>
<Property Name="ESIEnableToPassCookies" Value="false"/>
<Property Name="PluginInstallRoot" Value="/opt/IBM/wasadmin/Plugins"/>
<!-- Configuration generated using httpEndpointRef=defaultHttpEndpoint-->
<!-- The default_host contained only aliases for endpoint defaultHttpEndpoint.
The generated VirtualHostGroup will contain only configured web server ports:
webserverPort=10500
webserverSecurePort=10447 -->
<VirtualHostGroup Name="default_host">
<VirtualHost Name="*:10500"/>
<VirtualHost Name="*:10447"/>
</VirtualHostGroup>
<ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="true" LoadBalance="Round Robin" Name="affms_default_node_Cluster" PostBufferSize="0" PostSizeLimit="-1" RemoveSpecialHeaders="true" RetryInterval="60" ServerIOTimeoutRetry="-1">
<Server CloneID="412b3187-16c4-41b0-86e8-1e327c1c6b1b" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="20" MaxConnections="-1" Name="default_node_affms" ServerIOTimeout="900" WaitForContinue="false">
<Transport Hostname="libertyhost" Port="10500" Protocol="http"/>
<Transport Hostname="libertyhost" Port="10447" Protocol="https">
<Property Name="keyring" Value="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.sth"/>
<Property Name="certLabel" Value="libertyKey"/>
</Transport>
</Server>
<PrimaryServers>
<Server Name="default_node_app"/>
</PrimaryServers>
</ServerCluster>
<UriGroup Name="default_host_app_default_node_Cluster_URIs">
<Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/app/ui/*"/>
<Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/adminCenter/*"/>
</UriGroup>
<Route ServerCluster="app_default_node_Cluster" UriGroup="default_host_app_default_node_Cluster_URIs" VirtualHostGroup="default_host"/>
I copy the plugin-cfg.xml over to the IHS server at /opt/IBM/wasadmin/Plugins/config/webserver1/plugin-cfg.xml
The plugin stuff works only the SSL portion between IHS and the app server give me issues.
On the IHS server I have created the plugin-key.kdb keystore with stash plugin-key.sth and imported the cert I extracted from the liberty server /opt/IBM/wlp/usr/servers/app/resources/security/key.p12 the default personal cert and copy it to the IHS server and add the default liberty cert to the plugin-key.kdb keystore.
At the bottom of the httpd.conf is:
LoadModule was_ap22_module /opt/IBM/wasadmin/Plugins/bin/64bits/mod_was_ap22_http.so
WebSpherePluginConfig /opt/IBM/wasadmin/Plugins/config/webserver1/plugin-cfg.xml
I bounce the apachectl and go to the url for the application on port 8443 https://hostname:8443/app/ui and get a "500 Internal Server Error".
The http_plugin.log plugin logs show:
[04/Mar/2020:18:23:48.31652] 00002f26 3cff9700 - ERROR: lib_stream: openStream: Failed in r_gsk_secure_soc_init: GSK_ERROR_BAD_CERT(gsk rc = 414) PARTNER CERTIFICATE DN=CN=hostname,OU=app,O=ibm,C=us, Serial=xxxxxxxxxxx
[04/Mar/2020:18:23:48.31655] 00002f26 3cff9700 - ERROR: Ensure correct certificate is marked as default certificate in plugin-key.kdb. Consult documentation regarding Administering application security and Securing communications for more information.
[04/Mar/2020:18:23:48.31657] 00002f26 3cff9700 - ERROR: Last validation error [575010]: GSKVAL_ERROR_NO_CHAIN_BUILT
[04/Mar/2020:18:23:48.31658] 00002f26 3cff9700 - ERROR: Subject [[Class=]GSKVALMethod::PKIX[Issuer=]OU=memberRoot,O=xxxx-xxx-xxx-xxx-xxxxxxxx,DC=com.ibm.ws.collective[#=]082d6a83e5ec[Subject=]CN=hostname,OU=app,O=ibm,C=us] failed certificate validation
[04/Mar/2020:18:23:48.31659] 00002f26 3cff9700 - ERROR: X509 Certificate validation log: [[Class=]GSKVALMethod::PKIX[Time=]2020:3:4:18:23:48.316[buildChain=][Error=]GSKVAL_ERR_NO_CHAIN_BUILT[Info=]OU=memberRoot,O=xxx-xxx-xxx-xxx-xxx,DC=com.ibm.ws.collective[Cert=][Issuer=]OU=memberRoot,O=xxx-xxx-xxx-xxx-xxxx,DC=com.ibm.ws.collective[#=]082d6a83e5ec[Subject=]CN=hostname,OU=app,O=ibm,C=us[=Cert][=buildChain]^M
]
[04/Mar/2020:18:23:48.31664] 00002f26 3cff9700 - ERROR: ws_common: websphereGetStream: Could not open stream
[04/Mar/2020:18:23:48.31667] 00002f26 3cff9700 - ERROR: ws_common: websphereExecute: Failed to create the stream
[04/Mar/2020:18:23:48.31668] 00002f26 3cff9700 - ERROR: ws_common: websphereHandleRequest: Failed to execute the transaction to 'default_node_app' on host 'hostname:10447'; will try another one
[04/Mar/2020:18:23:48.31669] 00002f26 3cff9700 - ERROR: ws_common: websphereWriteRequestReadResponse: Failed to find an app server to handle this request
[04/Mar/2020:18:23:48.31670] 00002f26 3cff9700 - ERROR: ESI: getResponse: failed to get response: rc = 2
[04/Mar/2020:18:23:48.31671] 00002f26 3cff9700 - ERROR: [xxx.xx.xx.17://app/ui/] ws_common: websphereHandleRequest: Failed to handle request rc=2
Thank you!
You said "imported" I assume this means an "add" operation in ikeyman/gskcmd/gskcapicmd? The two verbs in these tools have a different meaning.
I suspect you added the issued certificate (CN=hostname...) instead of the CA that issued the certificate (memberroot). The debug information in the WAS Plugin message implies that it couldn't chase the certificate up to a trusted root (as opposed to finding some X509 error w/ the root it had access to).
But, it could also be that you imported the "wrong" certificate with cn=memberroot. WebSphere does not always add the necessary Subject Key Identifier/Authority Key Identifier to be sure, so check carefully.
Ok, I got this working finally. It seems having a collective setup which used DefaultKeyStore id was somehow causing conflict with my Liberty plugin keystore.
I ended up re-creating everything (plugin-key.kdb keystore and the liberty plugin keystore which I named LibertyKeystore.jks). I dumped the collective stuff (not needed at this time anyways) and made an include as above called plugin-join-include.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<server description="IHS plugin join include file">
<featureManager>
<feature>ssl-1.0</feature>
</featureManager>
<keyStore id="defaultKeyStore" password="password"
location="${server.config.dir}/resources/security/LibertyKeystore.jks" />
<pluginConfiguration webserverPort="80" webserverSecurePort="8443"
pluginInstallRoot="/opt/IBM/wasadmin/Plugins"
sslKeyRingLocation="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.kdb"
sslStashFileLocation="/opt/IBM/wasadmin/Plugins/config/webserver1/plugin-key.sth"
sslCertlabel="LibertyKeystore"/>
</server>
I exchanged keys between the liberty server LibertyKeystore.jks and the IHS plugin-key.kdb keystores and pushed the newly generated plugin to the IHS server and bounced everything and it all seems to function without issue.
I do now notice in the /opt/IBM/wasadmin/HTTPServer/logs the error log streams:
[Wed Mar 04 21:32:28 2020] [error] [client xxx.xx.xx.17:65261] [7ff5a0000910] [19831] SSL0279E: SSL Handshake Failed due to fatal alert from client. Client sent fatal alert [level 2 (fatal), description 46 (certificate_unknown)] [xxx.xx.xx.17:65261 -> xxx.xx.xx.87:8443] [21:32:28.000934048] 0ms
The access logs show 200's. Maybe a non plugin "issue" at this point.

Wildfly 10 for jms messaging

I tried a simple spring jms example and deployed in tomee worked fine (din't start activemq server). Then I migrated to wildfly 10. But here i need to start activemq server. Is it needed wildfly is already integrated with activemq right? On starting the server (standalone.bat) should start the activemq isn't?
EDIT: Actually some configuration has been changed (I was not aware of the change). I unzipped wildfly 10 again and tried to deploy using "standalone.bat -c standalone-full.xml". Server started successfully but got the below given error.
10:29:26,172 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 10.0.0.Final (WildFly Core 2.0.10.Final) started in 13367ms - Started 401 of 691 services (397 services are lazy, passive or on-demand)
10:29:29,101 INFO [org.springframework.jms.listener.DefaultMessageListenerContainer] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) JMS message listener invoker needs to establish shared Connection
10:29:30,111 ERROR [org.springframework.jms.listener.DefaultMessageListenerContainer] (org.springframework.jms.listener.DefaultMessageListenerContainer#0-1) Could not refresh JMS Connection for destination 'myMessageQueue' - retrying in 5000 ms. Cause: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused: connect
I managed to fix this problem by adding an embedded broker to my springcontext.xml
<amq:broker id="broker" useJmx="false" persistent="false">
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:61617" />
</amq:transportConnectors>
</amq:broker>
Refered : http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html

hazelcast : unable to connect to any address in config

I have an hazelcasrCLient-xml and have configured the port to as i have limitation on using the 5701 port :
<hazelcast-client>
<group>
<name>dev</name>
<password>dev-pass</password>
</group>
<network>
<cluster-members>
<address>135.46.61.34:28019</address>
</cluster-members>
<smart-routing>true</smart-routing>
<redo-operation>true</redo-operation>
<connection-attempt-limit>10</connection-attempt-limit>
</network>
</hazelcast-client>
also for hte server side the configuration in hazelcast.xml is :
<hazelcast>
<group>
<name>dev</name>
<password>dev-pass</password>
</group>
<instance-name>hzpunInstance1</instance-name>
<network>
<port auto-increment="true">28019</port>
</network>
<partition-group enabled="false" />
<executor-service name="default">
<pool-size>16</pool-size>
<!--Queue capacity. 0 means Integer.MAX_VALUE. -->
<queue-capacity>0</queue-capacity>
</executor-service>
<hazelcast>
the server is running on cloud whereas the client in on another VM
so when the client tries to connect to the hazelcast server i get an error :
8/18/16 10:36:23:982 GMT] 00000022 ServletWrappe E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0014E: Uncaught service() exception root cause appServlet: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException: Unable to connect to any address in the config! The following addresses were tried:[/135.46.61.34:28019]
.........
Caused by: java.lang.IllegalStateException: Unable to connect to any address in the config! The following addresses were tried:[/135.46.61.34:28019]....
an so on
Can anyone suggest what could be the fix or where am i going wrong?
What I understand from your config is that hazelcast nodes (server-side) are configured to use port 28019 with auto-increment option activated. So potentially, used port is anywhere between 28019 and 28119 (default value of port-count is 100).
However you client is only configured to try port 28019. There is no auto-increment option for the client, it only attempts to connect to addresses specified in the client configuration (135.46.61.34:28019 in your case)... and fails.
If you are using auto-increment for your cluster, then you must explicitly add all possible addresses int the client conf. For example:
Serverver-side config
<port portcount="10" auto-increment="true">28019</port>
Client-side config
<cluster-members>
<address>135.46.61.34:28019</address>
<address>135.46.61.34:28020</address>
<address>135.46.61.34:28021</address>
<address>135.46.61.34:28022</address>
<address>135.46.61.34:28023</address>
<address>135.46.61.34:28024</address>
<address>135.46.61.34:28025</address>
<address>135.46.61.34:28026</address>
<address>135.46.61.34:28027</address>
<address>135.46.61.34:28028</address>
</cluster-members>

OpenAM with OpenDJ - NameNotFoundException: ldap/idp/userDN - when starting up JBoss

I'm using OpenAM, with its embedded OpenDJ as the LDAP service, to protect my web application running on JBoss 7.
When I start my JBoss I get this error:
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ldapUserDN'
...
Caused by: javax.naming.NameNotFoundException: ldap/idp/userDN -- service jboss.naming.context.java.ldap.idp.userDN
So apparently Spring is looking for the JNDI node ldap/idp/userDN. But the jboss configuration file that I got with the project has these entries:
<simple name="ldap/opendj/url" value="ldap://localhost:50389"/>
<simple name="ldap/opendj/userDN" value="cn=Directory Manager"/>
<simple name="ldap/opendj/password" value="mypassword"/>
<simple name="ldap/opendj/baseDN" value="dc=opensso,dc=java,dc=net"/>
And these properties are added to my JNDI tree on JBoss.
If I change these to "ldap/idp/userDN", for instance, then I get rid of the error, but I was wondering if there's anywhere, where "ldap/opendj/userDN" should be mapped to "ldap/idp/userDN", that I've missed.
If you're using Spring LDAP, the actual configuration of the ldap-context-source goes in the a spring config file, and might look like this:
<jee:jndi-lookup jndi-name="ldap/idp/url" id="ldapUrl"/>
<jee:jndi-lookup jndi-name="ldap/idp/userDN" id="ldapUserDN"/>
<jee:jndi-lookup jndi-name="ldap/idp/password" id="ldapPassword"/>
<jee:jndi-lookup jndi-name="ldap/idp/baseDN" id="ldapBaseDN"/>
<ldap:context-source url="#{ldapUrl}"
username="#{ldapUserDN}"
password="#{ldapPassword}"
base="#{ldapBaseDN}"
native-pooling="true"/>
So the jndi entries in your jboss config file should match the ones above.

JMX connection to tomcat on VirtualBox

I have enabled JMX on my tomcat server with
-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost and I have a spring app that exposes JMX beans with a JmxRemoteLifecycleListener bean:
<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" rmiRegistryPortPlatform="10000" rmiServerPortPlatform="10001" />
When I run this tomcat instance on virtualbox (using vagrant) I forward ports 10000 and 10001, but when I try to connect to the JMX service (tried with VisualVM and JRockit Mission Control), I am unable to connect. Is there special configuration that needs to be done to connect since it is running on VirtualBox?
You need to do the Port forwarding with IPTABLES. Just check whether port is enabled in Iptables.

Resources