I've been working on a VB.NET/VS2008/AJAX/SQL Server project for over 2 years now without any real issues coming up. However, we're in the last week of our project doing some heavy stress testing and the project starts failing once I get about 150 simultaneous users. I've even gone so far as to create a stripped down version of the site which only logs in a user, pulls up their profile and then logs off. That still fails under stress. When I says "fails" I mean the CPU's are spiked and the App Pool eventually crashes. This is running on a Windows 2008 R2 duo quad server w/ 16 gig of memory. The memory never spikes but the CPU tops out.
I ran YSlow on the site and it pointed out that I needed to compress the .axd files, etc... I did that by implementing Gzip compression on everything but that's what got me to the 150 users. I run YSlow now and it says everything is "A".
I'm really not sure where to go from here. I'd be more than willing to share the stripped down version of the site for anyone to review. I'm not sure if it's the server, my code or the web.config.
I know it is a bit late but have you considered increasing the number of worker processes in the application pool of your site to form a web garden? You can do this on the IIS Manager.
Related
We have multiple applications developed in Visual Foxpro 8.0 running in a data center on Windows 2008 R2 on VMware. We also have a Citrix farm on the same network where users run yet another VFP 8.0 application in Citrix sessions. All applications share the same set of data tables located on a file server (also Windows 2008 R2 VM). Virtual hosts are connected by 10Gb LAN (managed switch).
Since mid-July we started seeing random 1104 "Error reading file..." errors on multiple different applications on multiple servers. All of them reference different files on the file server.
The problem started mid-July and it frequency gradually increased. Earlier it was most frequent in the afternoons by 3 pm, now it happens from early morning till late afternoon. It affects EDI servers (these run batch jobs in unattended mode) and Citrix servers and a variety of applications. It occurs when a VFP application (any of them) tries to open a database container file or individual tables most often with USE command but some times executing a SQL Select statement, or when loading a VFP form that opens tables in DataEnvironment
We caught a moment when the same exact error happened on two different servers running different applications at the same exact moment (up to a second). We also saw two different applications running on the same computer erroring out at the same moment.
We replaced the file server with a new virtual machine with no relief (we since changed it back to the old file server ).
We disabled the antivirus.
We updated VMware on all hosts to the latest version.
Sysinternals Process Monitor displays "INVALID_NETWORK_RESPONSE" event when the error occurs.
We captured traffic on both the server side and client side when the error occurred and had it analyzed by a network analysis specialist. He observed a peculiar pattern, where client OS starts retrieving the file in question from the file server AFTER VFP application had thrown an error. It seems that VFP application requests a file from OS, then it either gets an abnormal response or just times out and only after that the OS sends packets requesting the file. Again, this happens sporadically.
OpLocks and SMB2 have been disabled on all computers both on the server and client side of the equation for many years and everything was running smoothly until now...
Any advice would be greatly appreciated.
My first piece of advice would be to re-enable OpLocks and SMB2. There is no reason to mess with either of those items as things stand today and you are losing a huge amount of performance running at SMB1 level.
In my experience these issues have almost always been caused by one of the following.
Antivirus/antimalware software.
Replication or online backup software like MozyPro.
The Windows Search indexing service.
You should consider installing the Windows 7 / Server 2008 R2 Enterprise Hotfix Rollup if you haven't already.
That problem mostly related by SMB2!
Some Antivirus Software!
Windows updates! If you use VFP apps by DBF/DBC file. Do not update your system/OS. That is my personal suggestion. Windows Server 2012+ or Windows 10+ prorbably would big problems at near future.
And the point high probably is:
What is your I/O request per secs? if your IO request bigger than 1000~2000 per secs for a dbf file that is a bottle neck; and your storage device is HDD -> you need to switch/update your HDD to SSD. I suggest m.2 pro series SSD.
I have TFS 2015 installed on one of the company's servers. I try to access TFS using web access and it is extremely slow, it takes more than 5 minutes for a page to load and sometimes even longer. If I restart the server, TFS becomes a little bit faster (a page would need only a minute or so to load), but soon it becomes slower.
The server itself is okay. The CPU and memory are not even fully utilized (~20% - ~40% is utilized).
Other applications that are installed on the server are working fine, so it's just TFS.
Any suggestions?
Log in the application tier machine to try to access the web access to see whether you can see the same behavior.
Check the network connection between the application tier machine and data tier machine if you set up TFS in a multiple server configuration. You may try to turn off the firewall and anti-virus software on the machines.
Clean the cache folder on the application tier, usually the folder locates in: C:\TfsData\ApplicationTier\_fileCache
Check the Requirements and compatibility, to see whether your TFS set up on a appropriate environment.
If the items above is not helpful. You may need to consider move your TFS to another hardware.
On Jenkins v1.528, after a few hundred builds, web page access and build job completion slows to many times slower than normal, all the way to unusable. Restart fixes the issue. The host machine, a Mac circa 2012, has gigabytes of available memory and CPU usage is normal.
But I notice a persistent spike in disk usage when accessing a Jenkins web page (such as one for a Jenkins job/task/build). It seems that Jenkins has possibly run out of heap space. Yet I can't think of anything that has changed on this Jenkins instance or on the host machine to cause the issue (it didn't used to be a problem).
Also, I've seen some fixes for major slowness recently, but they went into prior versions.
We have a similar issue after restart. This is what I figured our still today.
Your issue seems to be related to this issues:
jenkins is slow resopnding to main page request
Jenkins gets very slow after saving some Jobs
Jenkins' homepage loading very slowly
Jenkins is very slow on first visit
First off, I'm very new to Azure.
I've successfully deployed an ASP.NET MVC 3 web application to Azure, using a web role. The app uses Entity Framework and SQL Azure.
Recently I've done some changes (some including adding appsettings), and tried to upgrade the application. When upgrading, it took quite a long time, before Aborting. I've always deployed through the management portal Silverlight application at http://windows.azure.com.
When trying again to no avail, I setup remote desktop and deployed again. The remote desktop session was extremely slow, and it turned out to be because WaIISHost was putting the CPU to 100%.
The IIS Manager shows that the application is deployed and 'started', however I cannot navigate to the site in the VM, and the deployment constantly seems to be trying to update without success and eventually aborting and retrying, (as I write this, it's currently Busy and Waiting for role to start...).
Does anyone have any ideas as to what the problem could be?
I believe all the right dependencies are set to copy local, which is a possible problem. It is extremely hard to debug this issue, as the remote desktop session hangs so often due to the 100% CPU utilization, and the recycling/restarting/reupdating of the web role from time to time.
Thanks,
James
P.S. Hope some of that made at least some sense...
I doubt that there's something doing in your WebRole.OnStart and/or Run, which caused the WaIISHost uses 100% CPU. Can you remove all codes from the WebRole.OnStart and/or Run and try again.
And it might be helpful to turn on the IntelliTrace when deploying, so that you can download the trace and find out any exceptions occurred when your application started, even before the website started.
Setup TFS 2010 on a pretty oldish server (actually an oldish desktop machine running server 2003 - single core, pre Core2 P4 so outdated...)
I'm finding a first adding and first getting of a website with about 700 files is quite slow (over 20 mins already over a VPN line).
Once you do that, the checkin / checkout operations are reasonably ok.
One thing I haven't done yet is get one of the guys at work to make a change and for me at home to do a get latest. We were running VSS up to this and that operation used to be a killer!
Anyway, few questions:
1)We set it up as a basic installation on server 2008 express. Would there be any performance gain with full sql server 2008?
2) We have the option of moving the drive to a better core 2 machine that should be a lot faster - will that make any difference?
Or are we simply running into a typical slowness of TFS over a LAN (bearing in mind we as a team work mainly in the office but sometimes from home over VPN when the speed issue seems to get worse).
TFS in it self isn't slow. We run a TFS on a dedicated VM and with the other VM's on the actual server also taking up ticks, or TFS is decently fast and reliable. Even when checking in and out code, running reports etc... So maybe the 2 core machine would help, but your P4 shouldn't be that bad in running it. 700 Files should be fairly quick within a minute or so. I think its your VPN that makes it slow. Everyone knows how slow VPN's can be.
Just an update. We moved to a faster machine (since we were getting one anyway). The speed of the machine makes no difference.
The good news is, its only slow the first time you add a project to TFS or take it down from TFS.
Daily checkin / checkout is fine. Also, doing a get latest from home is much improved over VSS - it no longer does that horrible freeze while it spends 10 mins figuring out if files have changed.
It was worth the upgrade for this alone.