Wildfly 15 configuration for HA on EC2 using AWS_PING - amazon-ec2

I need to run 2+ Wildfly servers on EC2 so I thought that AWS_PING could be a fine way to discover peers.
I run on EC2 the sample JGroups chat based on AWS_PING as described at https://github.com/meltmedia/jgroups-aws and it works fine.
So I put the following configuration in standalone-ha.xml:
<subsystem xmlns="urn:jboss:domain:jgroups:6.0">
<channels default="ee">
<channel name="ee" stack="tcp" cluster="ejb"/>
</channels>
<stacks>
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp"/>
<protocol type="com.meltmedia.jgroups.aws.AWS_PING" module="org.jgroups.aws:awsping">
<property name="tags">foobar</property>
<property name="port_number">7600</property>
</protocol>
<protocol type="MERGE3"/>
<protocol type="FD_SOCK"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
</stacks>
</subsystem>
and the socket-binding-group is described as:
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
...
<socket-binding name="jgroups-tcp" interface="private" port="7600"/>
...
</socket-binding-group>
and I configured the Wildfly's modules using the tar file generated with https://github.com/soulwing/wildfly-jgroups-aws-ping
When I restart Wildfly the JGroups subsystem seems correctly started:
$ grep jgroups log/server.log
2019-07-04 08:52:56,732 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 57) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.0.15
However Wildfly is not listening on port 7600:
$ sudo netstat -plnt | grep java
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 13123/java
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 13123/java
tcp 0 0 127.0.0.1:9990 0.0.0.0:* LISTEN 13123/java
tcp 0 0 0.0.0.0:8009 0.0.0.0:* LISTEN 13123/java
Did I miss some configuration? Why isn't Wildfly listening on port 7600?

Related

How do you configure JBOSS to allow port 8080 over HTTPS?

I have a JBOSS server (7.0) running an application that uses ServiceWorkers, which requires an HTTPS connection. I was able to update the standalone.xml and Eclipse launch configuration to bind my JBOSS server to my local IP (I'll worry about port forwarding later). Connecting to http://192.168.0.197:8080/[application] works just fine, except that ServiceWorkers won't start because it isn't an HTTPS connection. If I try https://192.168.0.197:8080/[application], the connection fails with the browser reporting "unable to connect".
I've researched several documentation sources and can't figure out what needs to be updated. Please forgive any terminology errors - my background is with application programming and networking tends to be the bane of my existence.
This is the pertinent standalone.xml configuration:
<subsystem xmlns="urn:jboss:domain:webservices:2.0">
<wsdl-host>${jboss.bind.address:192.168.0.97}</wsdl-host>
<endpoint-config name="Standard-Endpoint-Config"/>
<endpoint-config name="Recording-Endpoint-Config">
<pre-handler-chain name="recording-handlers" protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP ##SOAP12_HTTP_MTOM">
<handler name="RecordingHandler" class="org.jboss.ws.common.invocation.RecordingServerHandler"/>
</pre-handler-chain>
</endpoint-config>
<client-config name="Standard-Client-Config"/>
</subsystem>
<subsystem xmlns="urn:jboss:domain:weld:4.0"/>
</profile>
<interfaces>
<interface name="management">
<inet-address value="${jboss.bind.address.management:192.168.0.97}"/>
</interface>
<interface name="public">
<inet-address value="${jboss.bind.address:192.168.0.97}"/>
</interface>
</interfaces>
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
<socket-binding name="txn-recovery-environment" port="4712"/>
<socket-binding name="txn-status-manager" port="4713"/>
<outbound-socket-binding name="mail-smtp">
<remote-destination host="localhost" port="25"/>
</outbound-socket-binding>
</socket-binding-group>
And the Eclipse launch configuration:
-mp "C:\JBOSS-EAP70\modules" org.jboss.as.standalone --server-config=standalone.xml -Djboss.server.base.dir=C:\JBOSS-EAP70\standalone
"-Dprogram.name=JBossTools: Red Hat JBoss Enterprise Application Platform 7.0 at localhost" -server -Xms1303m -Xmx1303m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dorg.jboss.resolver.warning=true -Djava.net.preferIPv4Stack=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true "-Dorg.jboss.boot.log.file=C:\JBOSS-EAP70\standalone\log\boot.log" "-Dlogging.configuration=file:C:\JBOSS-EAP70\standalone\configuration\logging.properties" "-Djboss.home.dir=C:\JBOSS-EAP70" -Dorg.jboss.logmanager.nocolor=true
It's there in your configuration:
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
You'd need to change the http port and then the https port.

make wildfly listen on port 443 not 8443

so I have added a SSL certificate to my wildfly 9 and it's working, but I want to configure my standalone.xml to listen to https on port 443 not on port 8443 as the default configuration, so when I update the value ${jboss.https.port:8443} to ${jboss.https.port:443} it generate an error.
this what I have in my standalone.xml :
<server name="default-server">
<http-listener name="default" socket-binding="http" redirect-socket="https"/>
<https-listener name="httpsServer" socket-binding="https" security-realm="ApplicationRealm"/>
<host name="default-host" alias="localhost">
<location name="/" handler="welcome-content"/>
<location name="/images" handler="ImagesDirHandler"/>
<filter-ref name="server-header"/>
<filter-ref name="x-powered-by-header"/>
</host>
</server>
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
<socket-binding name="txn-recovery-environment" port="4712"/>
<socket-binding name="txn-status-manager" port="4713"/>
<outbound-socket-binding name="mail-smtp">
<remote-destination host="localhost" port="25"/>
</outbound-socket-binding>
</socket-binding-group>
Please, provide more accurate details about your environment and errors.
I had similar needs like you. The users access our system trough a network where the only requests availables are on port 80 or 443. Than, when a costumer calls the system on port 80, wildfly redirects to port 8443 and the user cannot connect to the system. The solution was to make wildfly redirect to port 443 instead 8443. Follow some instruction for all looking for help in this issue:
In case of a linux based operational system, ports up to 1024 are
available to bind only with root privilegies.
It isn't a great idea run wildfly or any other web/app server with root privilegies in a production oriented server.
In other hands, try to run wildfly with a 'regular' user directly bind to port 443 or 80 generates permission denied like errors.
The solution for the problem I described above was to bind wildfly to ports 8080/8443 (without root privilegies) and ask the operational system to redirect traffic from port 80 to port 8080 and port 443 to port 8443. After it, config wildfly to redirect http requests to https requests on port 443 instead 8443.
So, assuming wildfly is working with http on port 8080 and https on port 8443 in a Linux based OS as service:
1) Stop wildfly: sudo service wildfly stop
2) Add iptables commands in the startup /etc/init.d/wildfly script like:
if [ $launched -eq 0 ]; then
log_warning_msg "$DESC hasn't started within the timeout allowed"
log_warning_msg "please review file \"$JBOSS_CONSOLE_LOG\" to see the status of the service"
else
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8443
fi
PS: You add a rule on a table called "nat", from man pages of iptables:
nat:
This table is consulted when a packet that creates a new connection is encountered.
So, if you have requested https://localhost:443 before the rule creation, the connection wal already created, so the nat table is not applied. Try from a new device.
Where $launched is a bash variable to represent the state of wildfly
2) In the standalone.xml, create an additional socket-binding entry:
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
<socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
<socket-binding name="http" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
<socket-binding name="https-external" port="443"/>
<socket-binding name="txn-recovery-environment" port="4712"/>
<socket-binding name="txn-status-manager" port="4713"/>
...
</socket-binding-group>
Take attention to new tag entry <socket-binding name="https-external" port="443"/>
3) Change the http-listener to redirect to https-external instead https:
<http-listener name="default" socket-binding="http" redirect-socket="https-external" max-header-size=...
Where the change is redirect-socket="https-external"
4) Restart wildfly: sudo service wildfly start
After wildfly starts, verify the console.log file to see any errors report.
Thus, if your web.xml section assure confidential transport:
....
<security-constraint>
...
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
...
Wildfly will redirect the requests on port 80 or 8080 to directly to port 443 instead 8443.
Obs: It is a good idea to make backup copies of your /etc/init.d/wildfly script and standalone.xml file configuration before make any changes on them.

IHS and WAS configuration OK, but can't connect to application via WebServer

I've successfully setup IBH HTTP Server, generated the plugin, propagated it from WAS admin console to the remote WebServer, but still cannot connect to the application via the Web Server.
I've followed with success the steps for configuring the HTTP server as stated in IBM manuals here.
The WebServer is up on port 880 and I can see the welcome page of IHS from a remote machine when requesting: http://ihs-ip#:880
Requesting: http://ihs-ip#:880/app_name throws a 404 Not Found message, no errors in the log files.
Firewall and SELinux are disabled on the IHS machine.
I tried restarting the webserver, the DMGR and the two cluster app nodes with no success.
WebSphere AS version is: 7.0.0.31 ND
I can post my httpd.conf and plugin-cfg.xml content if needed.
Any clues?
Thanks.
UPDATE:
Here is the content of my plugin-cfg.xml file (I've also noticed that I don't have a *.880 alias in the default_host virtual host):
<?xml version="1.0" encoding="ISO-8859-1"?><!--HTTP server plugin config file for the webserver DmgrBrokerClusterCell.ws-005056878644.xxxxx.com-node.webServerBroker880 generated on 2015.02.06 at 01:00:02 AM CET-->
<Config ASDisableNagle="false" AcceptAllContent="false" AppServerPortPreference="HostHeader" ChunkedResponse="false" FIPSEnable="false" FailoverToNext="false" HTTPMaxHeaders="300" IISDisableFlushFlag="false" IISDisableNagle="false" IISPluginPriority="High" IgnoreDNSFailures="false" KillWebServerStartUpOnParseErr="false" MarkBusyDown="false" OS400ConvertQueryStringToJobCCSID="false" RefreshInterval="60" ResponseChunkSize="64" SSLConsolidate="true" TrustedProxyEnable="false" VHostMatchingCompat="false">
<Log LogLevel="Error" Name="/opt/IBM/HTTPServer/Plugins/logs/webServerBroker880/http_plugin.log"/>
<Property Name="ESIEnable" Value="true"/>
<Property Name="ESIMaxCacheSize" Value="1024"/>
<Property Name="ESIInvalidationMonitor" Value="false"/>
<Property Name="ESIEnableToPassCookies" Value="false"/>
<Property Name="ESICacheidFull" Value="false"/>
<Property Name="PluginInstallRoot" Value="/opt/IBM/HTTPServer/Plugins/"/>
<VirtualHostGroup Name="default_host">
<VirtualHost Name="*:9080"/>
<VirtualHost Name="*:80"/>
<VirtualHost Name="*:9443"/>
<VirtualHost Name="*:5060"/>
<VirtualHost Name="*:5061"/>
<VirtualHost Name="*:443"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:9080"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:80"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:9443"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:5060"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:5061"/>
<VirtualHost Name="ws-005056874449.xxxxx.com:443"/>
<VirtualHost Name="*:9083"/>
</VirtualHostGroup>
<ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="true" LoadBalance="Round Robin" Name="BrokerCluster" PostBufferSize="64" PostSizeLimit="-1" RemoveSpecialHeaders="true" RetryInterval="60" ServerIOTimeoutRetry="-1">
<Server CloneID="19da3jqec" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="BrokerClusterNode001_Broker01" ServerIOTimeout="60" WaitForContinue="false">
<Transport Hostname="10.1.140.206" Port="9083" Protocol="http"/>
<Transport Hostname="10.1.140.206" Port="9446" Protocol="https">
<Property Name="keyring" Value="/opt/IBM/HTTPServer/Plugins/config/webServerBroker880/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/IBM/HTTPServer/Plugins/config/webServerBroker880/plugin-key.sth"/>
</Transport>
</Server>
<Server CloneID="19da3jrkm" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="BrokerClusterNode002_Broker02" ServerIOTimeout="60" WaitForContinue="false">
<Transport Hostname="10.1.140.207" Port="9083" Protocol="http"/>
<Transport Hostname="10.1.140.207" Port="9446" Protocol="https">
<Property Name="keyring" Value="/opt/IBM/HTTPServer/Plugins/config/webServerBroker880/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/IBM/HTTPServer/Plugins/config/webServerBroker880/plugin-key.sth"/>
</Transport>
</Server>
<PrimaryServers>
<Server Name="BrokerClusterNode001_Broker01"/>
<Server Name="BrokerClusterNode002_Broker02"/>
</PrimaryServers>
</ServerCluster>
<UriGroup Name="default_host_BrokerCluster_URIs">
<Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/Broker/*"/>
</UriGroup>
<Route ServerCluster="BrokerCluster" UriGroup="default_host_BrokerCluster_URIs" VirtualHostGroup="default_host"/>
<RequestMetrics armEnabled="false" loggingEnabled="false" rmEnabled="false" traceLevel="HOPS">
<filters enable="false" type="URI">
<filterValues enable="false" value="/snoop"/>
<filterValues enable="false" value="/hitcount"/>
</filters>
<filters enable="false" type="SOURCE_IP">
<filterValues enable="false" value="255.255.255.255"/>
<filterValues enable="false" value="254.254.254.254"/>
</filters>
<filters enable="false" type="JMS">
<filterValues enable="false" value="destination=aaa"/>
</filters>
<filters enable="false" type="WEB_SERVICES">
<filterValues enable="false" value="wsdlPort=aaa:op=bbb:nameSpace=ccc"/>
</filters>
</RequestMetrics>
</Config>
You need to add the port your client addresses the webserver via (880) to the default_host host alias in WAS for it to be recognized as associated with the application. Then generate and propagate plugin-cfg.xml.
I think you should begin healcheck file plugin-cfg.xml in folder /opt/IBM/WebSphere/Plugins/config/http30WebSrv1
Because you Generate Plug-in maybe have error
InformationPLGC0063E: The propagation of the plug-in configuration file from /u01/IBM/WebSphere/AppServer/profiles/Dmgr01/config/cells/tpxCell01/nodes/tpx04IHSNode01/servers/trn03WebSrv011/plugin-cfg.xml to /opt/IBM/WebSphere/Plugins/config/trn03WebSrv011/plugin-cfg.xml on the Web server computer failed.
So you can add permission for folder:
chown -R ihsuser:ihsgroup /opt/IBM/WebSphere/Plugins/config/trn03WebSrv011/
and generate Plug-in and Propagate Plug-in
Thanks and regards
tandn

How do I set the number of "webcontainer worker-threads" in jboss as 7?

Most app. servers provide a way of tuning the number of WebContainer worker-threads when it goes down to tuning. Is it possible to do that in JBoss AS 7.x?
Thanks.
you can tune the HTTP Conector of the AS7 web subsystem. The available attributes you can tune for the HTTP Connector are described here The Http Connector. To define the max-connections for this connector you need change it in $JBOSS_HOME/standalone/configuration/standalone.xml or $JBOSS_HOME/domain/configuration/domain.xml
See this piece of configuration:
<subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">
<connector name="http"
protocol="HTTP/1.1"
scheme="http"
socket-binding="http"
max-connections="250"/>
...
</subsystem>
To define a thread pool specific for the HTTP Connector you need to use the AS7 threads subsystem like this one:
<subsystem xmlns="urn:jboss:domain:threads:1.0">
<bounded-queue-thread-pool name="http-executor" blocking="true">
<core-threads count="10" per-cpu="20" />
<queue-length count="10" per-cpu="20" />
<max-threads count="10" per-cpu="20" />
<keepalive-time time="10" unit="seconds" />
</bounded-queue-thread-pool>
</subsystem>
and then you need to reference it in the executor attribute of the HTTP Connector. See this piece of config:
<subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">
<connector name="http"
protocol="HTTP/1.1"
scheme="http"
socket-binding="http"
max-connections="250"
executor="http-executor"/>
...
</subsystem>
For more details about tuning the AS7 see this post JBoss AS 7 Performance tuning - Tuning Web server thread pool on the masterjboss.com.

Infinispan and JGroups discovery on EC2

I'm trying to use my application on AWS EC2 on some Linux boxes with Tomcat servers. Previously I used my application with Infinispan on LAN and I used UDP multicasting for JGroups member discovery. EC2 does not support UDP multicasting and this is the default node discovery approach used by Infinispan to detect nodes running in a cluster. I looked into using the S3_PING protocol, but I have not figured out why it doesn't work.
Does anyone have any ideas what the problem might be here?
Here is my configuration files:
1. applicationContext-cache.xml
<!-- Infinispan cache -->
<cache:annotation-driven/>
<import resource="classpath:/applicationContext-dao.xml"/>
<bean id="cacheManager" class="org.infinispan.spring.provider.SpringEmbeddedCacheManagerFactoryBean">
<property name="configurationFileLocation" value="classpath:/infinispan-replication.xml"/>
</bean>
<context:component-scan base-package="com.alex.cache"/>
2.infinispan-replication.xml
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
xmlns="urn:infinispan:config:5.1">
<global>
<transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport">
<properties>
<property name="configurationFile" value="/home/akasiyanik/dev/projects/myapp/myapp-configs/jgroups.xml"/>
</properties>
</transport>
</global>
<default>
<!-- Configure a synchronous replication cache -->
<clustering mode="replication">
<sync/>
<hash numOwners="2"/>
</clustering>
</default>
</infinispan>
3. jgroups.xml
<config>
<TCP bind_port="${jgroups.tcp.port:7800}"
loopback="true"
port_range="30"
recv_buf_size="20000000"
send_buf_size="640000"
discard_incompatible_packets="true"
max_bundle_size="64000"
max_bundle_timeout="30"
enable_bundling="true"
use_send_queues="true"
sock_conn_timeout="300"
enable_diagnostics="false"
thread_pool.enabled="true"
thread_pool.min_threads="2"
thread_pool.max_threads="30"
thread_pool.keep_alive_time="60000"
thread_pool.queue_enabled="false"
thread_pool.queue_max_size="100"
thread_pool.rejection_policy="Discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="2"
oob_thread_pool.max_threads="30"
oob_thread_pool.keep_alive_time="60000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.queue_max_size="100"
oob_thread_pool.rejection_policy="Discard"
/>
<S3_PING location="r********s" access_key="AK***************SIA"
secret_access_key="y*************************************BJ" timeout="2000" num_initial_members="2"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD timeout="3000" max_tries="3"/>
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER />
<pbcast.NAKACK use_mcast_xmit="false"
exponential_backoff="500"
discard_delivered_msgs="true"/>
<UNICAST />
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
max_bytes="4M"/>
<pbcast.GMS print_local_addr="true" join_timeout="3000"
view_bundling="true"/>
<UFC max_credits="2M"
min_threshold="0.4"/>
<MFC max_credits="2M"
min_threshold="0.4"/>
<FRAG2 frag_size="60K" />
<pbcast.STATE_TRANSFER/>
</config>
Use this: https://github.com/meltmedia/jgroups-aws
It is an implementation of JGroups discovery protocol for AWS using AWS API (multicast replacement)

Resources