I'm having a very strange problem. We're running a Java application, using Hibernate on top of JDBC, on a Windows Server 2012, on a VM.
When we try to read a lot of data from an Oracle 12c database, it's systematically super slow.
But once we run Wireshark once... It's instantly 100x faster! And it stays like this until we reboot the machine, even if we close Wireshark afterwards.
Any explanations? It really sounds like issue with Windows network cards drivers..
Edit 1 : We ruled out Hibernate : the problem happens as well with only JDBC
Edit 2 : We ruled out WinPcap, Wireshark without it still fixes the problem.
I had a similar issue, but using a .Net application accessing a Oracle 12c database, and uninstalling WinPcap/Wireshark didn't solved the slowness problem.
As the environment in question runs under vmware, we have looked into issues related to the vmxnet3 driver (under Windows 2012 R2) and found, as a contouring solution, how to disable Receive Side Coalescing (RSC) feature on it. Without reboot, the problem is gone.
In Powershell:
Disable-NetAdapterRsc *
Source: After upgrading a virtual machine to hardware version 11, network dependent workloads experience performance degradation (2129176)
Related
I'm having a very strange problem with an application in windows 10. It consists of several .exe in the same computer communicating between them with sockets using system.net.sockets library.
The problem I have is that after installing Windows 10 in a new computer, install all windows updates and then installing that application, connection to sockets doesn't work correctly and the application fails. The strangest thing is that if you leave the computer alone for 1-2 days the applications starts working just fine. The same has happened after installing version 1803 update, it stops working and then works one or two days later.
Any idea of what can it be? Has anyone seen something similar?
It really seems to be related to the 1803 update you mentioned.
Symptoms:
Running an application from a network share will fail when creating a socket;
Copying the very same application to a local drive/path will work just fine, without any further modification.
We are also struggling with this while connecting to an Oracle database (both ODBC and ODP.NET) and it seems the issue has recently been acknowledged:
https://support.oracle.com/knowledge/Oracle%20Database%20Products/2399465_1.html
It also seems this is a recurrent Windows bug:
Win Socket Creation fails with Error code 10022 if non super user
https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/3076a9cd-57a0-418d-8de1-07adc3b486bb/socket-fails-with-error-10022-when-application-is-run-from-certain-network-shares-on-vista-and?forum=wsk
Sorry, no effective solution at the time (other than copying the app binaries to a local folder). I'll update this answer once we get a better solution.
OK, looking a little further I found here in SO that this might be related to a SMBv1 network share, which describes the environment we had here (the network share was disabled because of another bug we faced - thanks MSFT).
Re-enabling SMBv2 / SMBv3 on the server solved the issue.
Related post:
After Windows 10 update 1803 my program can't open a socket when running from network share
For historical reasons,we have some legacy vb6 server side code running on two identical Windows 2008 server boxes that uses dao 3.6 to get to the back end MS access databases. This has worked fine for years, and we are currently in the process of migrating all the code. However, one of the servers the code is running on is getting flakey, so we need to move the code to a new server as the migrated solution will not be ready for a while.
The server that works ok is running Windwos Server Web 2008 Sp1 64 bit. The new server is Windows Server Standard 2008 R2 Standard Sp1.
In running tests on this new server, we started getting random freezing in the application. There is a simple loop that runs a select query on every table in the database.After adding a bunch of logging, it was freezing on a call to the OpenRecordset method of the DAO.Database object. Sometimes it would freeze for a few seconds, other times up to 10 - 11 minutes and then carry on. The queries being run are of this format, and should all return 0 records :
SELECT * FROM BlahBlah WHERE WriteTime > #25 Oct 2012 10:09:43#
When run directly on the database (i.e. in Access), they return fine. It's not the same query that locks up each time either. I thought that it might be something specific to our software that I hadn't managed to track down until I found a very similar post here (unfortunately with no reply except mine!)
http://www.vbforums.com/showthread.php?653166-Using-Dao-3.6-on-Windows-server-2008&highlight=dao+3.6+server+2008
The only thing I have found so far is that the version of DAO360.dll on the server that it runs on is 3.60.9704, and the version on the server with problems is 3.60.9756 (i.e. a newer version of Dao). I did not install or register anything manually, but installed Access 97 and Access 2003 on the new server, the same as the old server. I should also point out that If i move the back end code as-is to the old server, then the tests work fine, so I'm pretty sure it is not the code.
Now I know that DAO is outdated and all that, but I'm stuck with this codebase until the migrated solution is available and thoroughly tested, so any thoughts of where to look would be very welcome indeed. In the meantime, I'm going to try and manually register the 3.60.9704 version on the new server and run more tests to see if that is the issue, but it would be kind of weird if it is...
Aha ! Although we are only using DAO, not full blown access, this seems to be the answer
http://social.technet.microsoft.com/Forums/en-US/officesetupdeployprevious/thread/2a34fc07-0a1e-4248-b866-2b1c60aabba2
When I looked a bit more into it, the server that the code is running on fine is Windows 2008 webserver, which, although it is 64 bit, is a completely different beast to the OS on the new machine which is Windows Server 2008 R2. Solution seems to be to go back to an older OS on the server, rather than re-writing the whole app, which is an easy thing to do for us.
Luckily, this only has to run for another few months or so, as the whole thing is being re-written.
Tomcat is running a webapp under Windows. After a few days (under very low load), the exception mentioned in the title starts to appear in the logs, no new connections can be established from that point on, the only fix is then to reboot the server.
Environment:
Latest Tomcat 6
Windows Server 2008 R2
JDK 6 update 30
SQL Server 2008
Kerberos authentication
Evidence collected so far:
netstat shows no excessive amount of connections
ProcessExplorer shows no excessive amount of open file handles
system main memory usage is average
JVM heap usage is average
restarting Tomcat does not solve the problem
Open questions:
if we were leaking connections, shouldn't they show up in netstat?
shouldn't a restart of the appserver resolve the problem, because the OS should free all process resources?
is there a way to trace the problem to its origin? E.g. installing monitoring software, maybe something similar to lsof etc.?
I'm out of ideas, any hints appreciated!
The reason we got this error is a bug in Windows Server 2008 R2 / Windows 7. The kernel leaks loopback sockets due to a race condition on machines with more than one core, this patch fixes the issue:
http://support.microsoft.com/kb/2577795
I was running Alfresco Community 4.0d on Windows 7 64 bit and had the same symptoms and errors.
The problem was fixed with Microsoft's patch: "Kernel sockets leak on a multiprocessor computer that is running Windows Server 2008 R2 or Windows 7" (http://support.microsoft.com/kb/2577795) (ie. Buddy Casino's answer (see below)).
Another observation I'd like to add is that Windows connections (Internet Explorer, Remote Desktop etc) would work again about 5-10 mins after the Alfresco services were shutdown.
Alfresco is an excellent product and I was afraid I would have to scrap it. Fortunately stackoverflow came to the rescue !
Thanks again to Buddy Casino's answer.
Boo to the person who down-voted the Question.
We are seeing the same thing on a similar setup, W2008R2, Tomcat 6.0.29, Java 1.6.0.25. Restarting tomcat does not help, but restarting the server itself does, at least for a while. After the last time we started shutting down individual services and believe we have it narrowed down to either an instance of Alfresco that is also running on the server or the Backup Exec Agent services. After those services (four in total) were stopped, the applications in Tomcat started working again, although we were still seeing the buffer/connections error in the stdout log which was strange. Will need to wait for the problem to return before confirming which are the culprit, which could be anywhere from a few days to a week or more.
Any chance you are running either Alfresco or BE on your server?
I'm having a problem with vfp9 on Windows 7 64-bit. I've found that create sql view is taking 5-6 seconds. These happen instantaneous in XP. When my app starts up, I'm doing a few of these, so in Win 7, my app is taking 30+ seconds longer to start up than in XP. My views look like this:
create sql view MyView remote connection MyConn as select * from MyTable
I've also found that calling dbsetprop is adding another 1-2 seconds in Win 7. Again its instantaneous in XP.
dbsetprop('MyView.MyPk', 'Field', 'KeyField', .T.)
dbsetprop('MyView.MyPk', 'Field', 'Updatable', .T.)
Once created, the views work as they should. No slowness on with platform.
Does anyone have any ideas about what I could try or any info on what is/could be causing this?
Thank you in advance.
I don't know why as I haven't worked with Windows 7 yet with VFP... However, what I would check within VFP and try changing some settings to see if it helps.
From the VFP/IDE menu, go to Tools, then Options. On the multi-tab form, click on the "Remote Data" tab.
I don't know if/what its trying to do, but maybe for testing, make sure the "Records to fetch at a time" is NOT set to "All" (checkbox).
I would also look into SQLSETPROP() function to see if any of those settings might help.
I can't reproduce this on Windows 7 64 bit, either with VFP9 RTM or VFP9 SP2. I don't have a database of any size to work with but on the sample database Northwind the commands you list seem to work instantaneously.
A couple of questions:
Is this reproducible on any machine running Windows 7?
Where is your database? Is it on the local machine, a local network, or the internet?
There seems to be more scope for speed problems with Windows 7 and Visual FoxPro (and similar) applications, and I think this is down to the different network stack in Windows 7, immature network card drivers, an increased susceptibility to cabling and network switch problems, or any combination of these.
Ensure that all your Windows 7 boxes are on SP1 (and any Server 2008 boxes with shared DBF files also), as this fixes a file corruption issue that affected Visual FoxPro indexes.
Ensure that your network card drivers are 100% up to date. This can make a big difference.
One thing that I have seen which can give a massive improvement to the speed of networked Visual FoxPro applications is the network card driver Interrupt Moderation setting. This is present on Intel, Broadcom and many other NICs, although with possibly slightly different names.
I have personally seen situations where disabling this has changed a networked VFP application from taking 30 seconds to start to about 6 seconds.
Found the solution.
Write caching was being disabled on the drive by the raid controller software included with the machine.
Write caching was enabled under Device Manager > Disk Drive > Properties > Polices. However the software was overriding this setting.
It can be reproduced without the raid software by unchecking it in Windows 7 Polices.
The LAN which has about a half dozen windows xp professional pcs and one windows 7 professional pc.
A jet/access '97 database file is acting as the database.
The method of acccess is via dao (DAO350.dll) and the front end app is written in vb6.
When an instance is created it immediately opens a global database object which it keeps open for the duration of its lifetime.
The windows 7 machine was acting as the fileserver for the last few months without any glitches.
Within the last week what's happened is that instances of the app will work for a while (say 30 mins) on the xp machines and then will fail on database operations, reporting connection errors (eg disk or network error or unable to find such and such a table.
Instances on the windows 7 machine work normally.
Moving the database file to one of the xp machines has the effect that the app works fine on ALL the xp machines but the error occurs on the windows 7 machine instead.
Just before the problem became apparent a newer version of the app was installed.
Uninstalling and installing the previous version did not solve the problem.
No other network changes that I know of were made although I am not entirely sure about this as the hardware guy did apparently visit about the same time the problems arose, perhaps even to do something concerning online backing up of data. (There is data storage on more than one computer) Apparently he did not go near the win 7 machine.
Finally I know not very much about networks so please forgive me if the information I provide here is superfluous or deficient.
I have tried turning off antivirus on the win 7 machine, restarting etc but nothing seems to work.
It is planned to move our database from jet to sql server express in the future.
I need some suggestions as to the possible causes of this so that I can investigate it further. Any suggestions would be gretly appreciated
UPDATE 08/02/2011
The issue has been resolved by the hardware guy who visited the client today. The problem was that on this particular LAN the IP addresses were allocated dynamically except for the Win 7 machine which had a static IP address.
The static address happened to lie within the range from which the dynamic addresses were being selected. This wasn't a problem until last week when a dynamic address was generated that matched the static one and gave rise to the problems I described above.
Thanks to everyone for their input and thanks for not closing the question.
Having smart knowledgeable people to call on is a great help when you're under pressure from an unhappy customer and the gaps in your own knowledge mean that you can't confidently state that your software is definitely not to blame.
I'd try:
Validate that same DAO and ODBC-drivers is used on both xp- and vista machines.
Is LAN single broadcast domain? If not, rewire. (If routers required make
sure WINS is working)
Upgrade to ms-sql. It could be just a day of well worth work, ;-)
regards,
//t