I have a 9Mb PBIX containing small tables and one table with 250k rows. Data imported from various xlsx & JSON sources. Machine is Windows 10 Pro, 2.6GHz, 64 bit, 16GB RAM.
On the Power BI service online the performance is ok, but on desktop it's practically unworkable. With task manager I can see that it is using 7Mb of memory, but almost 100% CPU, half an hour after opening - while on a blank tab with no visualisations.
I don't understand what it is doing in the background and how I can improve the situation.
There is the 'Allow data preview to download in the background' setting, but I think this is only relevant to the query editor? Would clearing the cache or changing cache settings help?
I am aware of performance analyzer and the query diagnostics tools, but neither seem relevant since the queries are not refreshing and there are no visualisations loading.
Am at a bit of a loss - any help greatly appreciated.
Thanks
Update: Having disabled parallel load and background refresh in Data load settings I noticed that finally the issue seemed to go away (though not immediately). Eventually, when reopening the pbix, mashup containers did not appear and CPU and memory was not being killed. Then at some point Power BI got stuck and had to close and the problem reappeared even though the data load settings were still disabled. Restarting the machine seemed to clear the problem once again.
It seems then, that some zombie processes can persist through application close and re-open. Has anyone else noticed this, can confirm or refute it, suggest what is going on or any steps on how to avoid/prevent? It's very annoying!
Thanks
I have also noticed the same issue, for opening 5 mb pbix file, power bi eating 12 GB of memory, and 90%+ CPU utilization, Power BI Desktop is poorly managed product by Microsoft.
Related
I've recently had my laptop replaced and I've had to install Visual Studio 2015 and SQL Server 2014 Express with Management Studio.
My previous environment was Visual Studio 2015 with SQL Server 2008 R2 Express with Management Studio.
I restored the 2008 R2 databases into SQL Server 2014 Express, same database names, logins etc.
Now when I run any of my ASP.NET MVC 5 applications (using Entity Framework 6) on my laptop using Visual Studio, I'm getting sporadic timeout errors. Please see below.
Occasionally the application database calls will perform as expected, but mostly they are either very slow or timeout.
I'm finding it difficult to understand why this is as on my previous laptop using SQL Server 2008 R2 Express I never had any of these issues.
Also, these applications are on a live web server and being used by 1000s of users each day without any of these problems. This makes me think there is something possibly wrong with the installation of SQL Server 2014 Express on my laptop.
I have seen others comment on extending the Command Timeout on my DbContext
public class MyDatabase : DbContext
{
public MyDatabase ()
: base(ContextHelper.CreateConnection("Connection string"), true)
{
((IObjectContextAdapter)this).ObjectContext.CommandTimeout = 180;
}
}
But I don't see this as a solution, as I didn't need this with my previous laptop/ environment and the current live applications also don't need it.
I'm stumped here and would really appreciate any help or guidance.
Thanks.
Update
Thanks to the suggestions from Steve Py I decided to check the memory performance from my new laptop when running Visual Studio 2015 and SQL Server Express 2104 concurrently. I've included a screen shot below which shows that 90% of the available memory is used (3.5G out of 3.9G). I'm far from an expert in tuning up a device for software development, however, this seems it may be a reason as to why when I run my applications locally that they are timing out.
Is there anyone on Stackoverflow who ca inform me if this looks like the possible problem?
Thanks.
Firstly I'd look at hooking up a profiler to capture the queries coming from EF. For SQL Server you can use ExpressProfiler. This will give you the actual SQL EF is trying to run, the # of row reads, writes, and execution time. Copy the SQL queries and paste them into a new query window on the DB and execute them, plus have a look at the execution plan. Does the execution time correlate with EF? (change parameters and re-run in SQL to ensure you aren't getting cached results)
Other factors are the hardware on the two laptops. You'd hope that the new laptop would have more grunt than the old one, more cores, better cores, more RAM, but how do they compare? How much memory is free when nothing is running? What kind of HDD was in the two machines? For instance dropping from an i7 with an SSD down to an i3 with 5400rpm HDD, and half the RAM will be extremely noticeable, even if the clock speed is higher.
When it comes to databases there are a number of factors that can impact performance, even when backing up and restoring. For instance the Isolation Level and recovery model settings for the database can play a part, especially for larger databases. I'd also look at server settings such as how much RAM the database server is allocated to be able to use. Feel free to paste some results from the profiler for slow queries.
Edit: Based on the screenshot of the resource use, my guess is your new laptop is potentially underpowered. 4GB of RAM is bare-bones with Windows 10 especially to be running Visual Studio and SQL Server, even for just a development use database. The history graph for CPU and disk also show heavy activity. If that's all you've got to work with then the next step would be to look at what is using the memory. SQL Server by default will attempt to use whatever is available, and it can be quite greedy, but it's generally a good idea to set boundaries on the server. From SQL Management you can bring up the properties of the server and select "Memory" to set a minimum and maximum memory size. For 4GB I'd set the minimum to 500MB and the max to 2000MB. For processors you can use "Boost SQL Server Priority"
Next, on the database side of things look into the file and recovery options. What is the size of the database MDF file, and transaction log? (LDF file) From the database properties window under "General" you should see the "Size" which is the MDF size. For the LDF you will probably need to check on the file system. A large LDF can be bad for performance and indicate your database should be backed up and the log compressed/truncated. Lof files default to grow by percent so they can grow fast and churn the disk. In the "Options" tab you cna check the "Recovery Model" and set that to "Simple" for a dev database to significantly cut on log file churn/growth. Production databases will use "Full".
For development purposes it helps to have a bit more grunt from a laptop. While things like "ultra books" look like good options and are nice and lightweight, they rarely have the grunt and resources needed for a dev environment. (plus generally poor keyboards and displays to boot! :) ) There is also a significant gap in price between ultra books and workstation replacement laptops. What I've found fit in that gap and serve as exceptional development PC replacements are gaming laptops. They are tuned for performance and usually come with 8GB minimum with expansion available. They also happen to come with exceptional keyboards and displays. They're typically a fair bit cheaper than workstation replacements that seem to price in a premium. I use an MSI GE65 series which came with 16GB, SSD+HDD, a great keyboard, and was over $1000 cheaper than the closest "workstation" laptop. It does draw a couple stares coming into a client site with a gaming laptop with it's LED keyboard and lid badge, not a single game on it though! :)
I have VS 2013 and Microsoft Windows 8.1
The issue appeared at the ending of last week. Without any updating or important changing, when I do somethings in VS, disk usage reaches 100%. For example, when I click on "Check In" button in the "Team Explorer" window, disk usage raises up to 100%. Sometimes by a simple right-click in text editor this problems happens.
I googled about 100% disk usage problem but there are some things about this problem on windows 8.1 but on my computer, all applications are running without any problem, just VS2013 has a "full disk usage" problem.
Some information about my system:
OS Name: Microsoft Windows 8.1 Pro
OS Version: 6.3.9600 N/A Build 9600
System Type: x64-based PC
Processor(s): 1 Processor(s) Installed.
Intel64 Family 6 Model 60 Stepping 3GenuineIntel ~3500 Mhz
Total Physical Memory: 8,131 MB
Available Physical Memory: 3,836 MB
Virtual Memory: Max Size: 10,947 MB
Virtual Memory: Available: 5,275 MB
Virtual Memory: In Use: 5,672 MB
Page File Location(s): C:\pagefile.sys
(Comment for others landing here as #Marta explains that the problem no longer persists on their machine.)
In general, any performance issue in Visual Studio should be reported to Microsoft. It's easy to do this directly from VS using the Report a Problem tool. That feature will automatically attach logs/traces which are shared privately with Microsoft. Internally, tooling will analyse those attachments to assign a ticket to the relevant team. With such attachments, there is a high likelihood that the problem can be diagnosed and fixed in a future release of Visual Studio.
Instructions on the Report a Problem tool:
https://learn.microsoft.com/en-us/visualstudio/ide/how-to-report-a-problem-with-visual-studio?view=vs-2019
If you prefer to diagnose high disk IO yourself, FileMon can be a useful tool:
https://learn.microsoft.com/en-us/sysinternals/downloads/filemon
I am using Visual Code 1.71.2 as well as Visual Studio 2022 (Community Edition) on Windows 10. I am also facing the same issue.
After lot of checking, found disabling superfetch mitigates this issue. But again, Windows, applications startup take lot of time.
As a workaround, I found that by clearing %temp% folder after using visual studio or code eliminates this issue and disk activity is normal.
But every time, I may not remember this cleanup and hate it for forgetting :(
Hope this helps someone in similar situation.
It could be related to Visual Studio updates - which would show under C:\ProgramData\Package Cache.
A disk space management tool like TreeSize Pro will help figure it out though ... it will show which directory is using the most space. You can then target what aspect of Visual Studio is eating up your drive space.
There is a free trial at https://www.jam-software.com/treesize/
You can also use this tool to export and post a screenshot / export of the usage here and it may help identify what is going on.
I had a similar issue that turned out to be the built-in git provider having issues with large codebases containing a moderate-to-large amount of changes before a commit.
Changing to a third-party one fixed the issue.
The operating system manages the resources (CPU cores, disk drives, GPU) to deliver what you have asked of it.
Ideally (what the OS designers are hoping for), when you perform an action, all the resources spin up into action and due to a well balanced system, they all go to 100% utilization, for a brief length of time, then go back to idle.
This form of utilization is, in practice impossible to achieve, as the PC builders would have to know what your system is going to be used for.
When the task manager describes the CPU as 100% utilized, it means that all the cores on the box, are busy running code, and are the bottleneck.
When the task manager describes the disk as 100% utilized, it (as far as I can tell), means that there is always a queue of items to be read, or written to/from the disk. Even with 100% utilization, it may be that the metric is the only reason you are concerned, and the system is otherwise responsive.
In either of these cases, it shows that for a given workload, the CPU or the disk drive has become the rate determining step.
In practice, it should not matter, unless the length of time the system is at 100% is longer than a few minutes, or that your machine feels otherwise sluggish.
Further diagnosis can be performed by using the tool Sys internals : procmon, or the Microsoft : ADK
I would look using the procmon, at what files are being accessed during the 100%disk usage period, and decide whether
The behavior is sensible (if not raise a bug with Microsoft)
The machine is working usably (if not consider a hybrid or ssd disk)
I've had some exasperating problems with disk usage and source control explorer.
What fixed the issue for me was making sure that I never opened Source Control Explorer in more than one project at a time, keep it closed when I could and limit the amount of VS instances you have open.
An SSD may can solve this... Are you sure this is caused by visual studio? when I was using Windows 8.1, the Windows Defender get to 100% disk usage from time to time. If you're sure it occurs when you use Visual Studio, you can try to repair it using the installer. Hope these would help you.
Try moving the source code to SSD drive.
HDDs have much slower disk I/O performance compared to SSD drives.
Generally in windows C drive comes as SSD drive.
So the times you see is an example of typical development. You fire up your server and mysql database, then login to the backend and try to add a simple thing like a menu item.
The times shown are only for the server to start responding, not for the page to actually finish loading. So this is time, passing on the server in the code, executing queries etc. All the JS files and CSS is not part of this measurement.
I can keep going. Clicking on "New Menu Item" and Hitting "Save" will take just as long.
So for a simple thing like adding a menu item the user spends roughly a minute looking at a blank screen (assuming the user knows joomla by heart and makes no wrong clicks and thus never has to go back).
Caching
So I read about caching. If you enable Page Caching I cannot keep developing because it seems my changes are not getting refreshed, and you really want this feature when you develop.
The View Caching actually speeds up the backend and the frontend a lot. But you still have to access the page once slowly before it gets cashed, and you have to access it again in the timeframe of the existence of the cash to profit from it. So for me, this means the backend is basically always slow. Unless I try to do something like adding 10 menu items within 15 minutes.
I btw run on a brand new notebook which really should not be the problem.
Is there something I am missing out on?
Is this actually normal?
EDIT
I could improve my times to around 2 seconds. The profile shows a lot of red colors though, someone has an idea? The picture is for the view menu manager, main menu menu items.
My times are all below 2 seconds, usually approx 1 both on my development server (a VM running CentOS 6 in virtualbox hosted by a Win7, i7 / 6Gb RAM / SSD disk) and my production server (Xeon dual 2GHz / 4Gb / 10000 rpm SATA disks).
Enable the debug for your site and see in the bottom of the page the times each module / component / event takes to run, this will make it possible to determine if it's a single extension / piece of Joomla eating up all the time, or it's just your machine.
I don't have a particularly good local machine (just a cheap W8 and using EasyPHP) and my times are all much faster than wither yours or those other people are reporting. One of the things you can do is turn on debug and look at the profiling data. When I load the admin login page even with debug on I can see it's onAfterDispatch which is the slowest part of the process.
A lot of times upgrading MySQL will give massive speed improvements.
When working on a localhost, the loading time usually depend on the PC performance. I work a hellish amount of time using wampserver (localhost) at work and on my computer at home.
When installing a fresh copy of Joomla 3.2 on Wamp at home, the step to create the database and insert the default content takes around 7-9 seconds, where as at work, it literally takes under 2 seconds. The reason? Because my work computer performance is much better than my personal computer.
It's the same concept for loading pages in the backend.
Hope this bit of info helped you.
I have an MVC3 C#.Net web app. It is running on IIS 7 on Windows Server 2008 R2. We are noticing significant performance issues when initially loading a page. We are using nHibernate and have found that performance to be slow in some instances. But all pages, even simple ones, behavinging similarly. I'm not really an IIS stud so.....
Am I missing something in IIS....a setting or action that I can tweak to improve performnce?
I had a similar issue running a site on a shared host that only allocated 100MB RAM to an application pool. When you exceeded that IIS was set to recycle it. The app generally ran at about 120MB so was constantly recycling. Each page was loading painfully slow as the whole thing started up again. Increasing the RAM available to the app pool fixed it.
Another thing that I'd try would be to set up SQL profiler and watch the queries being sent to the db. You can configure it to report the duration column in a smaller increment than the default (microseconds perhaps?) which makes the painful ones stand out. You can then pick up ones that are suspect, run them through query analyser with "display execution plan" switched on and examine the subtree costs. Perhaps NHibernate is generating nasty queries or perhaps too many?
Not sure what has happened on my dev machine but I can barely use visual studio 2010 these days. I have a copy of professional edition installed on a win7 pro x64 build running on top of a i5 M430 and 6 gigs of ram.
With only VS2010 open i've seen the process leak away to 600,000k+.
The editor is extremely slow. Every character I type sends the gui into "Not Responding" for 5 seconds and starting/stopping the debugger is a ~30 second operation.
I've done a repair install. No change.
I've removed productivity power tools and installed the perfwatson extension.
When I installed perfwatson the GUI sped up a bit while opening/loading a project. But the text editor still has an awful delay.
What else can I do? Harware rendering is off in my environment options.
an example of the slowness (literally): typing Height="1024" takes about 30 seconds to display in the text editor and do its update flash to go out of not responding. The word "Height=" takes 5 seconds to show. The intellesense and blank " " takes another 5 seconds. Each digit pops in every five seconds after that.
Needless to say even trying to edit existing work is a frustrating experience.
edit: rolled back one version on video driver. No noticeable changes after reboot.
edit: did some winforms projects today. No slow issues with this project type. Must be something with just wpf/sl projects.
edit 8/18/11: Took troublesome project to the production server. VS2010 editor works great there. Very snappy and responsive. Not at all slow. So it's not something inside my project. It's something in my machine. But a full out OS rebuild is something I can't just do now. Probably will start a bounty soon.
Delete the .sdf and .sou that have the same base file name as your solution file.
If your solution file is
c:\MyProject\project.sln
You should delete
c:\MyProject\project.sdf
c:\MyProject\project.sou
This solves 98% of the problems of slow VS.
These files contain intermediate information that is not important for the functionality of the solution and as time goes by they swell up and become bloated and fragmented. VS relies on these files for and if dealing with them is slow, everything is slow.
I know this is an old post but I have just had an issue where my visual studio project has been working fine for about 2 1/2 years and suddenly every time I clicked run I had to wait about 3 minutes and the same when clicking stop. I tried the old windows reboot but to no avail.
I found a post about deleting the projects .suo file (it was only 4MB).
I deleted the .suo file and everything is completely back to normal. I guess the file had corrupted or something.
Having a large number of breakpoints or a large number of files open can cause serious performance problems, but it sounds like your problems are worse than that...
A bios update and a intel chipset update on my machine solved nearly all my performance issues.
The slowness started to creep out into the OS and I was pegging the cpu at idle. I've got 4 cores and 8gb ram. It shouldn't do that. Now its happy at 8% load at idle.
Thanks to those that tried to help.
had the same problem. not sure what causes this ridiculous performance nightmare, but eventually i had to re-install windows. This same issue was posted on Microsoft forums but the best answer was to re-install VS or windows.