I have a report that uses a shared dataset. It also has several different slicers for viewing the data. The dataset is very large, so I created a cache for it so it doesn't take an eternity to load every time the user clicks on a slicer. The cache is set to expire every morning at 3:30am and refresh at 4am. The report is going to be used by 15 different clients and my company has a separate database set up for each client. So there are 15 versions of the report, each with a different data source.
The problem I'm having is that the cache is not working consistently. One day, all the reports run off the morning cache, the next day only a few reports use the morning cache and the others pull the live data (which means it takes several minutes to load). I've gone in and cleared the cache for each client, and the next day everything works fine, but a couple days go by and it's back to inconsistent.
One thought I had was there may be multiple copies of the same cache being stored and the report doesn't know which to use, so it doesn't use any. This shouldn't happen because the cache is cleared a half hour before it is refreshed, but is this possible? I would think if there were multiple copies of a cache, the report would use the most recent.
Another idea I had was that because there are 15 reports caching with the same parameters at the same time, maybe this is confusing the report. I would think it would use the cache associated with its data source, but could this be happening? Should I add a parameter to the dataset that has the client name, so there is no confusion?
Any other thoughts on what could be causing this would be helpful, thanks.
I figured out what was happening. Since all the reports were trying to cache at the same time, not all the caches were being successfully saved to the report server. I staggered the cache times, and that fixed my problem
Related
We have Magento EE 1.14. Admin was working fine till last two days its speed dropped dramatically. Frontend is not affected. Also no changes in code or server configuration. here is my attempt to fix the problem but nothing worked:
Log cleaning is properly configured
removed two unused extensions. but no improvement
tried to disable non-critical extensions to see if speed will improve but also not luck.
I can NOT use REDIS cache at this time. but configured new server which is using REDIS cache and move to it next month.
sometimes backend will gain speed for few minutes
I enabled profilers the source of the delay is mage ( screenshot attached ).
here are my question:
Is there anyway to know the exact reason for Mage delay ?
do I have other test i can use to identify the cause of delay ?
Thanks in advance,
It could be delay on external resources connection. Do you have new relic or similar software? Check there for slow connections. If You don't have NR, profile admin by blackfire.io. Magento profiler is really unhelpful :)
Follow below steps:
Delete unused extensions
It is best to remove unused extensions rather than just disabling them. If you disable an extension, it will still exist in the database. It would not only increase the size of your database (DB) but it also adds to the reading time for DB. So, keep your approach clear: If you don’t need it, DELETE it.
Keep your store clean by deleting unused and outdated products
One should keep in mind that a clean store is a fast store. We can operationalize the front-end faster by caching and displaying only a limited set of products even if we have more than 10,000 items in the back-end, but we cannot escape their wrath. If the number of products keeps on increasing at the backend, it may get slower, so it is best to remove unused products. Repeat this activity in every few months to keep the store fast.
Reindexing
One of the basic reasons why website administrators experience slow performance while saving a product is because of reindexing. Whenever you save a product, the Magento backend starts to reindex, and since you have a lot of products, it will take some time to complete. This causes unnecessary delays.
Clear the Cache
Cache is very important as far as any web application is concerned so that a web server does not have to process the same request again and again.
Even though Chrome is caching static files (JS, images, etc.,) in the Network tab, these files are taking sometime as shown in the below picture.
Where as many of the cached files are loading in just 0ms. Can someone please tell me even though the files are loading from cache, why are they loading in >0ms?
At first glance, it does look quite strange to see Chrome spending time downloading resources even though they are coming from the cache. It's not the time spent downloading from a web server you're seeing. Rather, I believe it is the time spent downloading from a local database cache.
The retrieval of any data has some amount of cost involved. The resources are essentially stored in a database in Chrome, and to retrieve data requires a lookup, which is not instant. As well as looking up the data in a table, there is likely some processing involved to push the correct data into memory, since the data is not stored exactly how it is going to be used. It is likely to be compressed, and decompressing data can be a slow process.
You can see in the Network tab that, although it appears to take 0 ms to retrieve some resources, when you look at the Timings tab, you will see that it is actually rounded down. For example, I see both 0.08 ms stalled and 0.02 ms download in the request below, despite it showing 0 ms in the grid.
Update:
I looked further into this and found that Chrome Extensions seem to have an effect on the retrieval times from both the cache and the web, particularly ones that inject content into the page. Adblock seems to be the cause of some delay for me - explanation above still very much applies for the rest.
Oddly, timings in Chrome are a bit... quirky... the time is not purely network time. If the JS engine gets blocked somehow, it is included in that total time...
If you hit this issue go to "timeline" tab and record a full timeline.
I have a default cache that is fairly small and static. It contains just string keys and a string objects.
Since I won't be using anywhere near the allowed amount of memory, I'd like to just preload all of the objects into the cache on startup and have them never expire. I added a log message on start indicating that the cache was loaded.
Right now the project is still in development so the cache isn't being hit often (other than by web spiders/crawlers/scripts). The problem I'm seeing is that every hour to few hours, I'm seeing the log message that my cache was loaded. I'd expect it to load once and then not reload until I force it to.
Is there any way to keep the cache "alive" so that it doesn't have to frequently reload? Is it like an IIS worker process that dies out after some amount of inactivity?
FYI I have the cache configured for Expiry Policy: Never, Time: 0min, Eviction: Disabled. Also the way I check if the cache is still alive is that on load I add a special object to the cache. Then I check to see if that object exists and if it doesn't I assume the cache needs to be reloaded.
For anyone else who stumbles across this I ended up creating a scheduled task that hit the cache every 5 minutes. Since then, I haven't seemed to have any issues with it reloading. Not sure if this is the best answer, but it worked for me.
While our website is not yet complete graphically and design wise, most of the backend operations are near completion.
However, after optimising the mysql database we are still receiving a significant initial receiving period when tested on pingdom.com:
http://tools.pingdom.com/fpt/#!/IuoBna86v/http://foscam-uk.com
According to Pingdom:
The yellow part is the time it takes to resolve the hostname and similar (before the connection is initiated to the web server), the green part is connecting to the web server, and the blue part is the time it takes to retrieve the content from the webserver.
Upon asking our managed VPS support team we got the response : 'Have you tried optimizing your script? I believe that the high wait time on there indicates actual website loading time (meaning for your script to load); not actual connection to the website/server.'
Now, pingdom shows the js/css loading relatively quickly, the mysql database side of things doesn't seem to be slowing anything down either - does anyone have any suggestions of what this could be or might be causing it?
Thank you very much for your time and help.
89 requests are too many.
Reduce number of image request by creating sprites.This is pretty important from what is shown in pingdom.
Keep Alive should be set to On and Keep alive time should be a bit higher(15 seconds or so).
Use of compiler plus merge and minify js/css is recommended.
Change the hosting provider. 8 second loading is very very slow. It means that it actually is around 15-17 seconds for a user that doesn't have cached parts of your site (first time visitor). My site www.bebepunk.ro loads according to pingdom in 2.5 seconds and users still complain about the slowness of the site. Check also with http://www.webpagetest.org for both values.
We're running 3 different Drupal (Pressflow to be specific) sites on the same server. The 2nd and 3rd sites were cloned from the first one and load just fine. The first one, though, is taking a few seconds to connect and start sending data back from the server. Same box, same config (as far as we know), same modules, and generally the same theme. Here's what Pingdom shows...
Fast site:
http://i.stack.imgur.com/YZilC.png
Slow site:
http://i.stack.imgur.com/6Um1M.png
Edit: Those are from Pingdom, the yellow indicating "The web browser is waiting for data from the server"
The configs are the same, performance options same, server configs, as far as we can tell are the same. The delay occurs before any page elements are visible so it's not an on-page object problem or a page speed problem.
Could this be a config issue with the server? Where should we be investigating?
Thanks in advance!
I would try increasing the size of your MySQL cache. It's possible the fast sites have their particular queries cached and the slow site has a query that doesn't quite fit the cache, so the MySQL query results need to be regenerated each time.
Just a guess!