Compress and copy same time to another server - slow - ftp

i have a big problem. I need tranfer a lot of files by a server to another server, but the second server isnt a local server. If i tranfer by a local server i cant 100mbs but if i send for another server out the speed is 2mbs. my network is 1gbs. I use a command line 7z.

If your servers are (as you wrote) on the same network and connected through the same line you are most likely to have a network connection problem.
I've often seen that the duplex settings of network cards are not set up correctly which leads to a lot of collisions.
Check your network card settings and try to force for example 100mbps full duplex.
I work for a company where this happens daily when trying to connect IBM network cards with Cisco switches. Have a look here how to set up duplex settings: https://superuser.com/questions/86581/how-do-you-check-the-current-duplex-value-of-a-network-card-set-to-auto-negotiat.
If this doesn´t help you might be better off asking at superuser.com

Related

Using FTP Programs with Automatic Proxy Configuration URL

I've literally searched the internet for the last 5 hours and I have tried every suggestion out there and I'm starting to wonder if what I want to do is simply not possible....
Most webservers only allow X simultaneous connections for uploading/downloading. I simply want to upload my many files faster, by connecting/uploading through various proxies. However, no program I can find has anything for automatic proxy configuration, and only for a specific proxy IP. I have an account with a proxy service that gives you a different IP address for every request/connection made through it. I can connect to this fine from any FTP program but it appears that the servers are confused when they see different IP's connecting, and there's no way to manually whitelist/authenticate them on the server side, so it simply closes all connections. I even have a list of IP addresses with port/user/pass that I am willing to use, but I can't figure out how to do anything other than use a specific proxy to upload/download from servers.... Is this even possible????
ANY HELP/INPUT IS GREATLY APPRECIATED!!

TCP: Address already in use exception - possible causes for client port? NO PORT EXHAUSTION

stupid problem. I get those from a client connecting to a server. Sadly, the setup is complicated making debugging complex - and we run out of options.
The environment:
*Client/Server system, both running on the same machine. The client is actually a service doing some database manipulation at specific times.
* The cnonection comes from C# going through OleDb to an EasySoft JDBC driver to a custom written JDBC server that then hosts logic in C++. Yeah, compelx - but the third party supplier decided to expose the extension mechanisms for their server through a JDBC interface. Not a lot can be done here ;)
The Symptom:
At (ir)regular intervals we get a "Address already in use: connect" told from the JDBC driver. They seem to come from one particular service we run.
Now, I did read all the stuff about port exhaustion. This is why we have a little tool running now that counts ports and their states every minute. Last time this happened, we had an astonishing 370 ports in use, with the count rising to about 900 AFTER the error. We aleady patched the registry (it is a windows machine) to allow more than the 5000 client ports standard, but even then, we are far far from that limit to start with.
Which is why I am asking here. Ayneone an ide what ELSE could cause this?
It is a Windows 2003 Server machine, 64 bit. The only other thing I can see that may cause it (but this functionality is supposedly disabled) is Symantec Endpoint Protection that is installed on the server - and being capable of actinc as a firewall, it could possibly intercept network traffic. I dont want to open a can of worms by pointing to Symantec prematurely (if pointing to Symantec can ever be seen as such). So, anyone an idea what else may be the cause?
Thanks
"Address already in use", aka WSAEADDRINUSE (10048), means that when the client socket prepared to connect to the server socket, it first tried to bind itself to a specific local IP/Port pair that was already in use by another socket, either an active one or one that has been closed but is still in the FD_WAIT state. This has nothing to do with the number of ports that are available.
I'm having the same issue on a Windows 2000 Server with a .Net application connecting to a SQL Server 7.0. There's like 10 servers with the same configuration and only one is showing this error several times a day. With a small test program I'm able to reproduce the error by just establishing a TCP connection on the SQL Server listening port. Running CurrPorts (http://www.nirsoft.net/utils/cports.html) shows there's still plenty of available ports in range 1024-5000.
I'm out of ideas and would like to know if you've found a solution since you've posted your question.
Edit : I finally found the solution : a worm was present on the server (WORM_DOWNAD.A) and exhausted local ports without being noticed.

Redirect Traffic from NIC to Another NIC On Separate Networks While Using Remoting

The project I'm working on is to handle data capture from scan guns (Pocket PC 2003) and process this data on a host (Win XP) then into our inventory database on a separate server (Win 2000). This is all driven by the Remoting framework provided by MS and As Good As It Gets (http://gotcf.net). The application is complete enough for a general proof of concept with both the client and server working properly while in the emulator.
All is well until I began to test using actual scan guns. Due to security concerns, the scanners are on a separate network (for clarification the 10 network) than the server (the 15 network). My development machine has dual NIC connected to both networks and can communicate with both independently. However, I am having issues with my application receiving information from the 10 network using .Net Remoting, and then sending out information to the server on the 15 network via a third party app (Combination of ODBC, Btrieve, and OLE).
Is there anyway to process information from one network then update the server on another?
Any suggestions will be greatly appreciated!
Note: I'm not very familiar with networking, thus I may be calling it the wrong name but the gun IP's start with 10...* and the server IP's start with 15...*
So long as the computer's routing table is properly configured, you shouldn't have to worry about this from your application. So long as you're using the proper IP addresses, the networking stack should take care of delivering things to the right place.
You might want to check the output of "route print" (at least I think that was available on WinXp -- if not, someone else will likely post the correct command for XP soon). In any way, you should see what network destinations are configured for which interfaces. You'll need to make sure that the server's IP on the 15 network will properly route via the interface you want (ie. the lowest-cost matching destination/netmask lists your 15 interface).
The issue seems to stem from both the NIC cards not set up properly and a so far unresolved issue with the frameworks I've chosen.
To solve the NIC problem, the easiest solution I'd found had me clear the default gateway on the 10 network.
The other issue deals with recreating the remoting objects after they've been destroyed. I currently have to warm boot the scanner in order to re-connect to the host. In order to correct this issue I'm going to contact As Good As It Gets to see what their input is. Damn firewall

How do I detect hosts on my LAN?

To help users, I would like my code to discover Oracle databases on the LAN. I thought to do this by first detecting all hosts, then checking each host to see if it is listening on Oracle's default port.
Any ideas how to go about this? Preferably in Java, but any language or algorithm would do.
Are you using DHCP? If so, your DHCP server has a list of the leases it has passed out. That should do you for a list of hosts on the LAN. Then try opening a connection to the Oracle port on each of those hosts and see if it accepts the connection.
It should be pretty simple to implement as a shell script with half a dozen lines or so. Java seems like overkill for something like this. Loop through the leases file, grab the IP from each lease, and telnet to the Oracle port; if it connects, disconnect and print the IP to standard out.
If you want to stay platform-independant, and unless you have access to some kind of database that lists the hosts, the only way to get a list is to try each IP address in the local network - might as well try to connect to the Oracle port on each of them.
There are lots of problems with this approach:
Will only search through the local network, which may only be a small part of the LAN (in case of large companies with lots of subnets)
Can take a long time (you definitely want to reduce the timeout for the connection attempts, but if someone has configured his LAN as a class A network, it will still take forever)
Can trigger all kinds of alerts, such as desktop users' personal firewalls, and intrusion detection systems - because you're doing exactly the same thing someone trying to exploit a security hole in Oracle servers would do
As brazzy points out, scanning for hosts is likely to cause problems, especially if there is a bug in your scanner.
A better approach may be to get the owners of the databases to register them somewhere, for example in a local DNS service (or does Oracle have zeroconf support?), or simply on some intranet webpage or wiki.
You better register the SID names/addresses to some server with a fixed address(maybe with a simple web service), and then query the list from there. Another approach is the bruteforce one (explained by #brazzy) by scanning one or more subnets, but this isn't really a good thing to do.
In case you are looking for a tool Loo#Lan can do this for you. Unfortunately there's no source available...
All of these smart answers are the reasons why many companies do not use the default port. Using a different port for each database is entirely possible, you know.

How do you diagnose network issues on Windows?

I often run into problems where I can't get something to connect to something else. I usually forget to check something obvious. Can you help with:
A tip/technique for diagnosing a connection issue
The name of a tool or application that can help (and the situation in which it's useful)
I know the question is a little non-specific, but hopefully the answers can form a useful starting point for anybody who's stuck trying to get computers/programs talking to each other.
Please can you give one answer per answer so the best ones can be voted up.
Simple checks to run when debugging network problems:
Has each machine got an IP address, Go to command prompt and run ipconfig. Key things to check here are the interfaces and ensuring the appropriate ones have IP addresses.
Check both machines IP addresses are in the same range and subnet if you are running it on an internal or Virtual network.
Try pinging each machine from the other to see if they can communicate with each other. Note that some firewalls will block ping requests.
If Pinging fails then check to see if firewalls are active. If the communication is within a 'safe' internal network then try disabling the firewalls and re-pinging.
If the connections are over a wireless network then check signal strength.
If pinging fails and you are connecting through several networks then try running a tracert to see at which will may show you where on the network the connection is failing.
If you are able to ping but not connect then check firewall settings and network connection settings. Windows 2000+ has the capability of setting port an ip access on a connection properties.
Try drawing a network diagram of the connections to help in visualising the problem.
If you are connecting through routers, firewalls and loadbalancers then check that all devices are not tied to any specific ip addresses and that the IP address redirection (if in place) is correct. Also check any NAT logs to see if connections are being received and properly re-directed.
Wireshark
Latest versions of ProcMon
netstat
Wireshark www.wireshark.org
Wireshark is a network protocol analyzer for Unix and Windows.
Features:
Deep inspection of hundreds of protocols, with more being added all the time
Live capture and offline analysis
Standard three-pane packet browser
Multi-platform: Runs on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many others
Captured network data can be browsed via a GUI, or via the TTY-mode TShark utility
The most powerful display filters in the industry
Rich VoIP analysis
Read/write many different capture file formats: tcpdump (libpcap), Catapult DCT2000, Cisco Secure IDS iplog, Microsoft Network Monitor, Network General Sniffer® (compressed and uncompressed), Sniffer® Pro, and NetXray®, Network Instruments Observer, Novell LANalyzer, RADCOM WAN/LAN Analyzer, Shomiti/Finisar Surveyor, Tektronix K12xx, Visual Networks Visual UpTime, WildPackets EtherPeek/TokenPeek/AiroPeek, and many others
Capture files compressed with gzip can be decompressed on the fly
Live data can be read from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and others (depending on your platfrom)
Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2
Coloring rules can be applied to the packet list for quick, intuitive analysis
Output can be exported to XML, PostScript®, CSV, or plain text.
work the OSI model from the bottom up
Physical (Do you have a network adapter/connection)
Link layer (arp, ethernet port blocked by network team (I've seen this where locked down environments see two MAC addresses coming from one workstation port and shut down the port)
Network layer (ipconfig, tracert, ping,)
Do you have a network address (DHCP, fixed)
Are you on a proper subnet/have routing between subnets
Is something in the middle blocking you
firewalls, routing tables
When in doubt, check to see if the windows firewall is messing with your communications. 8 times out of 10, it's at fault.
Using tracert is a good start to see how far along the chain you are getting.
For virtual machines it's usally a good idea to make sure you have the loopback adapter set correctly in the Host os.
Most frequently used tool is the ping. It can be used both to test your connection and the availability of a target
Second tool is the tracert if you want to see where the packets get lost.
For more advanced debugging I use the following tools: nmap, wireshark, etc.
Windows has a netstat utility which is pretty similar to the Unix netstat and can do a number of different things that might help you solve network issues.
Random example:
netstat -r displays routing information
netstat /? for usage information
Since you said you're using 2 virtual machines I would hazard a guess that both machines are setup in a NAT configuration (rather than a unique network device) -- In the NAT configuration, neither machine would (typically) be able to ping the other.
If you're familiar with the command line, you can try the "netstat" command.
You can also try "arp -a" to list all the IP/MAC addresses known to your PC.
The "tracert [ip address]" command will show you how many gateways/routers your packets jump through on their way to their destination. (This is probably not helpful if both machines are on the same network, though.)
And don't forget to check your Windows firewall settings.
Otherwise, if you want to get down and dirty, you can try the packet sniffer known as Wireshark: http://www.wireshark.org/ (aka. Ethereal)
Pull the network cable out
If you can get some communications to a device (eg a ping), but can't get your program to talk to a service on the computer. Then, try pulling the network cable out and see if the ping stops. This will verify you're communicating with the computer you really think you are.
On windows i user PortQueryUI : http://www.microsoft.com/en-us/download/details.aspx?id=24009
DNS activity: Portable DNS Cache and Firewall;
General network activity: Wireshark, Network Monitor;
Windows utilities: ping, netstat, nslookup.
You need to be use the process of elimination, for example if you can ping the ip address but not the hostname then there's DNS issues. If you can ping the system but not connect to a share etc.
DNS out of sync
If you're using a virtual machine and you perform a roll-back on it, then it could become out of sync with the DNS (Domain name server). Try to remove and re-add the machine to the domain, or if you've got access to the DNS machine, then get it to flush its cache.

Resources