When trying to receive a (large, approx. 100MB) file using an FTP adapter in BizTalk 2006, we run into the following problem, which causes the file to be processed over and over again.
Retrieving the file succeeds; it is placed into the MessageBox and processed properly
When the FTP adapter issues the DELE statement, it never reaches the FTP server the file is on (we have verified this by taking a look at the FTP server's logs)
there are no signs of timeouts on the FTP server; the FTP server log does not mention a timeout occurring
After the interval time set on the adapter expires, the FTP server will still find the large file that we have already processed in the previous run, because the DELE statement failed
The event log in BizTalk states that ‘The connection to the FTP server was broken prematurely’. That is why we think there is a timeout issue.
We have seen that retrieval of the file takes around 35 minutes. The FTP server timeout is set to 1 hour. no problems there I guess.
Then we found the following article: http://www.ncftp.com/ncftpd/doc/misc/ftp_and_firewalls.html#FirewallTimeouts. It states that a firewall / routing device might be responsible for the timeouts. The team managing our firewalls and routers told us that there were no timeouts set here.
Which leaves us in the dark on the cause of our problem. Does anyone of you have any suggestions? Or even better, the solution!!
Have you tried the solutions in this article?
I avoid using the FTP adapter. Instead I use a third party utility to retrieve files and move the transferred file to a file adapter receive location. Third party utilities allow you to configure rules, recovery actions etc, freeing BizTalk from having to manage the transfer.
Related
I want to write a CLLE Program to transfer Files from the IFS to a Webserver.
The Webserver is registered in the Hosttable from the Iseries.
(commandline: CFGTCP -> Option 10 -> Scroll to the Webserver IP)
Current CLLE Program: (only the ftp-access)
PGM
OVRDBF FILE(INPUT) TOFILE(CMDDTAR7_2/US9FIP) OVRSCOPE(*JOB)
OVRDBF FILE(OUTPUT) TOFILE(CMDDTAR7_2/US9FOP) +
OVRSCOPE(*JOB)
FTP RMTSYS('10.1.2.99')
ENDPGM
After i call my Program i get some results in a PF-File.
The Results are in German but i translated shortly for you.
The Connection is unavalable to 10.1.2.99 on Port 21. Try it again later.
[...]
My Question at the end.
Do you have some solutions for me to transfer xml files to my Webserver from the iseries?
You're writing about a web server and you want to transfer files by ftp. Perhaps you're mixing up protocols here. I'll concentrate to resolve your apparent ftp connection problem.
Seems that the destination host 10.1.2.99 doesn't accept connections on Port 21.
Please make sure, there's no firewall or other stuff blocking your request to the destination host. Moreover, on the host 10.1.2.99 try telnet 10.1.2.99 ftp or ftp 10.1.2.99 in a command window, depending on OS and installed components.
Before automating, just test manually if you can successfully connect and log in via standard command line: ftp 10.1.2.99.
Transferring data by FTP is just fine in a local LAN. Beware that any data including username and password will be transferred in clear text and thus can be possibly read by others as the intended hosts.
Following up to your comment: Actually there are many possibilities to copy files. If you're running a recent version of IBM i, I'd opt for using scp. It's encrypted and thus safe for running outside of local LANs. Great to automate by utilizing Key-Authentication instead of passwords. And many web hosting companies offer scp/sftp access.
i have a big problem. I need tranfer a lot of files by a server to another server, but the second server isnt a local server. If i tranfer by a local server i cant 100mbs but if i send for another server out the speed is 2mbs. my network is 1gbs. I use a command line 7z.
If your servers are (as you wrote) on the same network and connected through the same line you are most likely to have a network connection problem.
I've often seen that the duplex settings of network cards are not set up correctly which leads to a lot of collisions.
Check your network card settings and try to force for example 100mbps full duplex.
I work for a company where this happens daily when trying to connect IBM network cards with Cisco switches. Have a look here how to set up duplex settings: https://superuser.com/questions/86581/how-do-you-check-the-current-duplex-value-of-a-network-card-set-to-auto-negotiat.
If this doesn´t help you might be better off asking at superuser.com
I recently faced a problem with FTP adapter in BizTalk 2013.
I configured send adapter to send files to ftp server, files are picked up from folder (file transfer) and "passthroutransmit" to send port ftp adapter.
My problem is that file never end in send location (folder on ftp server) however I do not get any error in Biztalk.
I try to unenlist send port and then i get error no subscriber.
I ran out of options, I do not know where to look for error.
Have you guys any idea what it could be?
(as per comments/responses above)
If there are no errors reported and nothing is showing as suspended, it might well be that the files made it to their destination...and were perhaps moved/deleted by something on the target machine (e.g. an automated task or virus checking software).
I am working on a Windows (Microsoft Visual C++ 2005) application that uses several processes
running on different hosts in an intranet.
Processes communicate with each other using TCP/IP. Different processes can be on the
same host or on different hosts (i.e. the communication can be both within the same
host or between different hosts).
We have currently a bug that appears irregularly. The communication seems to work
for a while, then it stops working. Then it works again for some time.
When the communication does not work, we get an error (apparently while a process
was trying to send data). The call looks like this:
send(socket, (char *) data, (int) data_size, 0);
By inspecting the error code we get from
WSAGetLastError()
we see that it is an error 10054. Here is what I found in the Microsoft documentation
(see here):
WSAECONNRESET
10054
Connection reset by peer.
An existing connection was forcibly closed by the remote host. This normally
results if the peer application on the remote host is suddenly stopped, the
host is rebooted, the host or remote network interface is disabled, or the
remote host uses a hard close (see setsockopt for more information on the
SO_LINGER option on the remote socket). This error may also result if a
connection was broken due to keep-alive activity detecting a failure while
one or more operations are in progress. Operations that were in progress
fail with WSAENETRESET. Subsequent operations fail with WSAECONNRESET.
So, as far as I understand, the connection was interrupted by the receiving process.
In some cases this error is (AFAIK) correct: one process has terminated and
is therefore not reachable. In other cases both the sender and receiver are running
and logging activity, but they cannot communicate due to the above error (the error
is reported in the logs).
My questions.
What does the SO_LINGER option mean?
What is a keep-alive activity and how can it break a connection?
How is it possible to avoid this problem or recover from it?
Regarding the last question. The first solution we tried (actually, it is rather a
workaround) was resending the message when the error occurs. Unfortunately, the
same error occurs over and over again for a while (a few minutes). So this is not
a solution.
At the moment we do not understand if we have a software problem or a configuration
issue: maybe we should check something in the windows registry?
One hypothesis was that the OS runs out of ephemeral ports (in case connections are
closed but ports are not released because of TcpTimedWaitDelay), but by analyzing
this issue we think that there should be plenty of them: the problem occurs even
if messages are not sent too frequently between processes. However, we still are not
100% sure that we can exclude this: can ephemeral ports get lost in some way (???)
Another detail that might help is that sending and receiving occurs in each process
concurrently in separate threads: are there any shared data structures in the
TCP/IP libraries that might get corrupted?
What is also very strange is that the problem occurs irregularly: communication works
OK for a few minutes, then it does not work for a few minutes, then it works again.
Thank you for any ideas and suggestions.
EDIT
Thanks for the hints confirming that the only possible explanation was a connection closed error. By further analysis of the problem, we found out that the server-side process of the connection had crashed / had been terminated and had been restarted. So there was a new server process running and listening on the correct port, but the client had not detected this and was still trying to use the old connection. We now have a mechanism to detect such situations and reset the connection on the client side.
That error means that the connection was closed by the
remote site. So you cannot do anything on your programm except to accept that the connection is broken.
I was facing this problem for some days recently and found out that Adobe Acrobat Reader update was the culprit. As soon as you completely uninstall Adobe from the system everything returns back to normal.
I spent a long time debugging a 10054/10053 error in s3 pre-signed uploads
Turns out that the s3 server will reject pre-signed s3 uploads for the first 15 minutes of it's life.
So - If you're debugging s3 check it's not a new bucket.
If you're debugging something else - this is most likely a problem on the server side not client side.
stupid problem. I get those from a client connecting to a server. Sadly, the setup is complicated making debugging complex - and we run out of options.
The environment:
*Client/Server system, both running on the same machine. The client is actually a service doing some database manipulation at specific times.
* The cnonection comes from C# going through OleDb to an EasySoft JDBC driver to a custom written JDBC server that then hosts logic in C++. Yeah, compelx - but the third party supplier decided to expose the extension mechanisms for their server through a JDBC interface. Not a lot can be done here ;)
The Symptom:
At (ir)regular intervals we get a "Address already in use: connect" told from the JDBC driver. They seem to come from one particular service we run.
Now, I did read all the stuff about port exhaustion. This is why we have a little tool running now that counts ports and their states every minute. Last time this happened, we had an astonishing 370 ports in use, with the count rising to about 900 AFTER the error. We aleady patched the registry (it is a windows machine) to allow more than the 5000 client ports standard, but even then, we are far far from that limit to start with.
Which is why I am asking here. Ayneone an ide what ELSE could cause this?
It is a Windows 2003 Server machine, 64 bit. The only other thing I can see that may cause it (but this functionality is supposedly disabled) is Symantec Endpoint Protection that is installed on the server - and being capable of actinc as a firewall, it could possibly intercept network traffic. I dont want to open a can of worms by pointing to Symantec prematurely (if pointing to Symantec can ever be seen as such). So, anyone an idea what else may be the cause?
Thanks
"Address already in use", aka WSAEADDRINUSE (10048), means that when the client socket prepared to connect to the server socket, it first tried to bind itself to a specific local IP/Port pair that was already in use by another socket, either an active one or one that has been closed but is still in the FD_WAIT state. This has nothing to do with the number of ports that are available.
I'm having the same issue on a Windows 2000 Server with a .Net application connecting to a SQL Server 7.0. There's like 10 servers with the same configuration and only one is showing this error several times a day. With a small test program I'm able to reproduce the error by just establishing a TCP connection on the SQL Server listening port. Running CurrPorts (http://www.nirsoft.net/utils/cports.html) shows there's still plenty of available ports in range 1024-5000.
I'm out of ideas and would like to know if you've found a solution since you've posted your question.
Edit : I finally found the solution : a worm was present on the server (WORM_DOWNAD.A) and exhausted local ports without being noticed.