I know it is a similar problem to:
Spring Integration. Unknown host and tcp-connection-factory
However, in my particular case the tcp-connection is established based on the si message flow. I am writing a program that "fakes" a user login to a particular site through tcp/ip connection. The host/port changes weekly so I prefer to dynamically set-up the connections. Hence, those host/ip for the destination server is not static.
I am still wondering whether there is any methods other than the [dynamic ftp sample] which set-up a whole new applicationContext for each tcp/ip connection and modifying the connection-factory which is non-trivial work.
My ideal case is:
<int-ip:tcp-outbound-gateway id="outGateway"
request-channel="input"
reply-channel="clientBytes2StringChannel"
connection-factory="client"
connection-host="#{headers.dest.host}"
connection-port="#{headers.dest.port}"
request-timeout="10000"
reply-timeout="10000"/>
<int-ip:tcp-connection-factory id="client"
type="client"
host="#{headers.dest.host}"
port="#{headers.dest.port}"
single-use="true"
so-timeout="10000"/>
Where the destination host and port is inside the message header.
I know my use-case is rare but it's extremely useful in my particular business logic. My whole webapp runs based on the messages from those raw tcp-ip connections.
It's not currently possible/easy - even if you customize or extend the class for tcp-connection-factory to be able to connect to changing hosts. There is an open new feature request in JIRA to provide that functionality. Simillar question here.
Related
In spring Documentation --> 32.6 TCP Adapters it is mentioned that we use clientMode = "true" then the inbound adapter is responsible for the connection with external server.
I have created a flow in which the TCP Adapter with client connection factory makes connection with external server the code for the flow is :
IntegrationFlow flow = IntegrationFlows.from(Tcp.inboundAdapter(Tcp.nioClient(hostConnection.getIpAddress(),Integer.parseInt(hostConnection.getPort()))
.serializer(customSerializer)
.deserializer(customSerializer)
.id(hostConnection.getConnectionNumber())).clientMode(true).retryInterval(1000).errorChannel("testChannel").id(hostConnection.getConnectionNumber()+"adapter"))
.enrichHeaders(f->f.header("CustomerCode",hostConnection.getConnectionNumber()))
.channel(directChannel())
.handle(Jms.outboundAdapter(ConnectionFactory())
.destination(hostConnection.getConnectionNumber()))
.get();
theFlow = this.flowContext.registration(flow).id(hostConnection.getConnectionNumber()+"outflow").register();
I have created multiple flow by iterating over the list of connections and
iterate the above code in for loop and register them in flowcontext with unique ID.
My clients are created successfully with no issue and then establish there connection as supported by topology.
Issue :
I have counted the number of client connection created successfully so I have counted that 7 client connection (7 Integration flow) made successfully and they initiate connection from themselves.
when I create 8th client connection (8th flow created and registered successfully) but the .clientMode(true) is not working means the client don't initiate connection itself after first failure means it try for the first time to make connection if connected successfully then no issue but in case of failure it don't retry again.
Also my other created clients i.e 7 clients connection which are created successfully they also stopped initiating connection from itself when they got disconnected.
Note: There is no issue with flow only the TCP Adapters they stop initiating the connection
The flow is created and registered successfully as there is no issue it is because when I run a control bus command #adapter_id.retryConnection() it got connected with the server.
I don't understand that what is the issue with my flow that i couldn't initiate a connection after a particular count i.e seven or is there limitation in creating number of clients.
One possibility is the taskScheduler's thread pool is exhausted - that shouldn't happen with the above configuration, but it depends on what else is in the application. Take a thread dump (e.g. jstack) to see what the taskScheduler threads are doing.
See the documentation for information about how to configure the threads in the scheduler. However, if it solves it, you should really figure out what task(s) are using scheduler threads for long tasks.
Also turn on DEBUG logging to see if it provides any clues.
When I open the haproxy statistics report page of my http proxy server, I saw something like this:
Cum. connections: 280073
Cum. sessions : 3802
Cum. HTTP requests: 24245
I'm not using 'appsession' and any other cookie related command in the configuration. So what's 'session' means here?
I guess haproxy identify http session by this order:
Use cookie or query string if it's exists in the configuration.
Use SSL/TLS session.
Use ip address and TCP connection status.
Am I Right?
I was asking myself the very same question this morning.
Searching through http://www.haproxy.org/download/1.5/doc/configuration.txt I came accross this very short definition (hidden in a parameter description) :
A session is a connection that was accepted by the layer 4 rules.
In your case, you're obviously using Haproxy as a layer7/HTTP loadbalancer. If a session is a TCP connection, due to client-side / frontend Keep-Alive, it's normal to have more HTTP reqs than sessions.
Then I guess the high connection number shows that a lot of incoming connections were rejected even before being considered by the HTTP layer. For instance via IP-based ACLs.
As a far as I understand, the 'session' word was introduced to make sure two different concepts were not mixed :
a (TCP) connection : it's a discrete event
a (TCP) session : it's a state which tracks various metadata and has some duration; most importantly Haproxy workload (CPU and memory) should be mostly related to the number of sessions (both arrival rate and concurrent number)
In fact sessions were not introduced after but before connections. An end-to-end connection was called a "session". With the introduction of SSL, proxy protocol and layer4 ACLs, it was needed to cut the end-to-end sessions in smaller parts, hence the introduction of "connections". Zerodeux has perfectly explained what you're observing.
I have a network of brokers with the following configuration
<transportConnectors>
<transportConnector name="tomer-amq-test2" uri="tcp://0.0.0.0:61616" updateClusterClients="true" rebalanceClusterClients="true" updateClusterClientsOnRemove="true"/>
</transportConnectors>
I expect that when I connect using the URL
failover:\(tcp://tomer-amq-test2:61616\)?backup=true
the broker shall update th client with the full list of brokers it is conencted too, and the client shall connect to one randomly, and to a second as backup
however this is not happening and the client connects only to the specified broker in the url
help anyone?
Tx Tomer
figured out the problem (at least on my env)
when a broker updates another broker that it is up, it identifies itself by the server name.
once the server name of all brokers was added to /etc/hosts on the client side, all was well
:)
I guess this is bad practice, and the broker should identify itself by ip and not by hostname
I was running activeMQ 5.5.1 on ubuntu 10.4
Your client will only be updated with the full broker-list if some or the following properties are true: updateClusterClients rebalanceClusterClients and updateClusterClientsOnRemove.
you have to set them manually on your client as they are false by default.
see: http://activemq.apache.org/failover-transport-reference.html
Most proxy servers perform the job of forwarding data to an appropriate "real" server. However, I am in the process of designing a distributed system in which when the "proxy" receives a TCP/IP socket connection, the remote system actually connects with a real server which the proxy nominates. All subsequent data flows from remote to the real server.
So is it possible to "forward" the socket connection request so that the remote system connects with the real server?
(I am assuming for the moment that nothing further can be done with the remote system. Ie the proxy can't respond to the connection by sending the IP address of the actual server and the remote connections with that. )
This will be under vanilla Windows (not Server), so can't use cunning stuff like TCPCP.
I assume your "remote system" is the one that initiates connection attempts, i.e. client of the proxy.
If I get this right: when the "remote system" wants to connect somewhere, you want the "proxy server" to decide where the connection will really go ("real server"). When the decision is made, you don't want to involve the proxy server any further - the data of the connection should not pass the proxy, but go directly between the "remote system" and the "real server".
Problem is, if you want the connection to be truly direct, the "remote system" must know the IP address of of the "real server", and vice versa.
(I am assuming for the moment that nothing further can be done with
the remote system. Ie the proxy can't respond to the connection by
sending the IP address of the actual server and the remote connections
with that. )
Like I said, not possible. Why is it a problem to have the "proxy" send back the actual IP address?
Is it security - you want to make sure the connection really goes where the proxy wanted? If that's the case, you don't have an option - you have to compromise. Either the proxy forwards all the data, and it knows where the data is going, or let the client connect itself, but you don't have control where it connects.
Most networking problems can be solved as long as you have complete control over the entire network. Here, for instance, you could involve routers on the path between the "remote system" and the "real client", to make sure the connection is direct and that it goes where the proxy wanted. But this is complex, and probably not an option in practice (since you may not have control over those routers).
A compromise may be to have several "relay servers" distributed around the network that will forward the connections instead of having the actual proxy server forward them. When a proxy makes a decision, it finds the best (closest) relay server, tells it about the connection, then orders the client to connect to the relay server, which makes sure the connection goes where the proxy intended it to go.
There might be a way of doing this but you need to use a Windows driver to achieve it. I've not tried this when the connection comes from an IP other than localhost, but it might work.
Take a look at NetFilter SDK. There's a trial version which is fully functional up to 100000 TCP and UDP connections. The other possibility is to write a Windows driver yourself, but this is non-trivial.
http://www.netfiltersdk.com
Basically it works as follows:
1) You create a class which inherits from NF_EventHandler. In there you can provide your own implementation of methods like tcpConnectRequest to allow you to redirect TCP connections somewhere else.
2) You initialize the library with a call to nf_init. This provides the link between the driver and your proxy, as you provide an instance of your NF_EventHandler implementation to it.
There are also some example programs for you to see the redirection happening. For example, to redirect a connection on port 80 from process id 214 to 127.0.0.0:8081, you can run:
TcpRedirector.exe -p 80 -pid 214 -r 127.0.0.1:8081
For your proxy, this would be used as follows:
1) Connect from your client application to the proxy.
2) The connection request is intercepted by NetFilterSDK (tcpConnectRequest) and the connection endpoint is modified to connect to the server the proxy chooses. This is the crucial bit because your connection is coming from outside and this is the part that may not work.
Sounds like routing problem, one layer lower than TCP/IP;
You're actually looking for ARP like proxy:
I'd say you need to manage ARP packets, chekcing the ARP requests:
CLIENT -> WHOIS PROXY.MAC
PROXY -> PROXY.IP is SERVER.IP
Then normal socket connection via TCP/IP from client to server.
I am doing a JMS connection using Java. The command I am using to establish connection is
QueueConnectionFactory factory =
new com.tibco.tibjms.TibjmsQueueConnectionFactory(JMSserverUrl);
Where JMSServerUrl is the varible which stores my JMS URL.
Now the problem is that I need to add the fault tolerance URL i.e two different URL's. So can any one tell me how can I specify two URLs together in the above code sample such that if first URL is not accessible it should try connecting to the other URL.
Put all URLs in a single string with a comma between them.
new TibjmsQueueConnectionFactory("ssl://host01:20302,ssl://host02:20302");
Caution, I am a Tibco EMS newbie, but this seems to work, as evidenced by the error I can get ...
javax.jms.JMSSecurityException: Failed to connect to any server at:
ssl://host01:20302,ssl://host02:20302
[Error: Can not initialize SSL client: no trusted certificates are set:
url that returned this exception = SSL://host01:20302 ]
The .NET documentation for tibco(I know your using java) suggests that you can provide a comma delimited list of server URL's for messaging connections. Bear in mind that I don't have any real tibco experience, but this is a common way to handle initial connection fault tolerance(i.e. prior to establishing a connection and receiving information about the cluster, after which failover is typically handled by the connection). It may be worth a try. Another solution that I have seen to this problem is creating a virtual IP and handling fault tolerance at the Network Level.