I have developed application using ZF.The app is little big with a lots of features.
I use Zend_Application(already using autoloader in constructor),Zend_Layout,Zend_view,Zend_form,etc. My current issue is, the page loading is very slow and that too in localhost with XAMP.
I have enabled xdebug, to investigate the issue, got a cachegrind file in "tmp" folder and tried to view it with WinCachegrind software. There i can a see a lot of processes and functions being run for each and every request or page load.
Also, i have installed YSlow add-on for firefox and observed the speed of page loads in seconds...I have compare the speed with ZF and non ZF applications. And from the comparison, the pages for non zf app takes less than 1 sec to load and for the ZF app, it takes atleast 6-7 seconds. What a huge difference.
Main Things happen in the app are :
1) Database connection happens for each request.
2) Im not adding the view to layout explicitly,ZF just appends it automatically, to layout.phtml, based on the action name.
3) Some windows have forms with few drop down boxes which fetches data from the database.
4) Have menus with ACL implimented, before it was loading the privilges from DB for each and every request, but now i have optimized it, so that it will work only duiring the login and rest of the time it will take from the Zend_Registry.
I would like to attach the cachegrind file so that some one can see whats happening in the background, but i cant see an option here for attaching.
Someone please help me to find a solution for this. Any kind of help is really appreciated. Thanks a lot
Let's try to give some hints.
First database connection should happen only once (except if you use several privileges access on the database or several databases). So check that you use Singleton patterns with you Zend_Db_Tables object
Then you do not use Zend_Cache. You should really start to use Zend_Cache and build several cache objects. Let's say for example a File cach, with long term storage, and a memcache or Apc Cache, storing objects. Then use these cache in several layers:
gives the FileCache to Zend_Db_Table (defaultMetaDataCache), this way you will avoid a loot of metadata queries, queries that ask for description of each columns of the tables you use.
Store one or more Acl object (depends on how you use Acl, if you have one big Acl with all rules or several with subsets). And store them in mid-duration caches when they are built.
Think of other usages, detect heavy loops, semi-static contents (like you select lists, how many time should they be considered static?)
Finally, get a whole mental image of how your application engine works, and how your data will grow and be used.You will need that step to use application levels caches in the very best way (for example should some elements be cached for groups of users?, should Acl objects be build for groups, for each user, for everybody, is ther some blocks in the layout that should be rendered the same for everybody?).
Related
Given: Each call to a BE module takes several seconds even with a SSD drive. (A well configured setup runs below 1 second for general BE tasks.)
What are likely bottlenecks?
How to check for them?
What options to speed up?
On purpose I don't give a special configuration, but ask for a general checklist, so that the answer is suitable for many people as first entry point.
General tips on performance tuning for TYPO3 can be found here: https://wiki.typo3.org/Performance_tuning
However, in my experience most general performance problems are due to one of a few reasons:
Bad/no caching. Usually this is a problem with one or more extensions (partly) disabling cache. Try disabling all third party extensions and enabling them one by one to see which causes the site to slow down the most. $GLOBALS['TSFE']->set_no_cache() will disable all cache, so you could search for that. USER_INT and COA_INT in TypoScript also disable cache for anything that's configured inside there.
A lot of data. Check the database for any tables containing a lot of data. How many constitutes "a lot", depends on a lot of factors, but generally anything below a million records shouldn't be too much of a problem unless for example you do queries with things like LIKE '%...%' on fields containing a lot of data.
Not enough resources on the server. To fix this, add more memory and/or CPU cores to the server. Or if it's a shared server, reduce the number of sites running on it.
Heavy traffic. No matter how many resources a server has, it will always have a limit to the number of requests it can process in a given time. If this is your problem you will have to look into load balancing and caching servers. If you don't (normally) have a lot of visitors, high traffic can still be caused by robots crawling your site too quickly. These are usually easy to block on IP address in your firewall or webserver configuration.
A slow backend on a server without any other traffic (you're the only one who can access it) rules out 1 (can only cause a slow backend if users are accessing the frontend and causing a high server load) and 4 (no other traffic).
one further aspect you could inspect: in the user record a lot of things are stored, for example the settings you used in the log module.
one setting which could consume a lot of memory (and time to serialize and deserialize) is the state of the pagetree (which pages are expanded/ which are not).
Cleaning the user settings could make the backend faster for this user.
If you have a large page tree and the user has to navigate through many pages the effect will stall. another draw back: you loose all settings as there still is no selective cleaning.
Cannot comment here but need to say: The TSFE-Object does absolutely nothing in the TYPO3 Backend. The Backend is always uncached. The TYPO3-Backend is a standalone module to edit and maintenance the frontend output. There are tons of Google search results that will ignore this fact.
Possible performance bottlenecks are poor written extensions that do rendering or data processing. Hooks to core functions are usually no big deal but rendering of many elements for edit forms (especially in TYPO3s Fluid Template Engine) can cause performance problems.
The Extbase-DBAL-Layer can also cause massive performance problems. The reason is the database model does not know indexes. It' simple but stupid. A SQL-Join on a big table of 2000 records+ will delay the output perceptibly, depending on the data model.
Also TYPO3 Backend does not really depend on the Typoscript-Configuration but in effect to control some output or loaded by extensions, the full parsing of the *.ts files is needed. And this parser is very slow.
If you want to speed things up you need to know what goes wrong. The only way to debug this behaviour is to inspect the runtime with a PHP profiling tool like xdebug because the TYPO3 Framework is very complex. It's using some kind of Doctrine Framework and will load tons of files, by every request. Thus a good configured OpCache is a must.
Most reason the whole thing is slow is because it is poor written. You can confirm that fact by inspecting the runtime.
In addition to what already has been said, put the runtime environment onto your checklist:
Memory:
If heavy IDE and other tools are open at the same time, available memory can become an issue. To check the memory profile, you may start a tool that monitors the memory usage of the machine.
If virtualization is used, check the memory assigned to the box. Try if assigning more memory improves behaviour.
If required and possible spend more memory to your machine. This should not be a bugfix to poorly written code. Bad code can blow up any size of memory.
File access:
TYPO3 reads and writes thousands of files. If you work with a contemporary SSD, this is surprisingly fast. I did measure this. Loading all class files of TYPO3 takes just a fraction of a second.
However this may look different if you do not work with a standard setup. Many factors may slow you down:
USB-Sticks as storage.
Memory cards as storage.
All kind of external storage may be limited due to slow drivers.
Virtualization can become an issue. Again it's a question of drivers.
In doubt test and store your files and DB on a different drive to compere the behaviour.
Routing
The database itself may be fast. A bad routing of your request may still slow you down. Think of firewalls, proxies etc. even on your local machine and specially if virtualisation is used.
Database connection:
I fast database connection is crucial. If the database access is slow TYPO3 can't be fast.
Especially due to Extbase TYPO3 often queries much more data than really required and more often than really required, because a lot of relations are resolved in the PHP layer instead of the DB layer itself. Loading data structures like the root line may cause a lot of ping-pong between the PHP and the DB layer.
I can't give advice, how to measure your DB-connection. You have to as your admin for that. What you always can do is to test and compare with another DB from a completely different environment.
The speed of the database may depend on the type of the database itself. Typically you use MySQL/Maria-DB which should be fast. It also depends on the factors mentioned above, memory, file access and routing.
Strategy:
Even without being and admin and knowing all performance tools, you can always exchange parts of your system and check if matters improve. By this approach you can localise the culprit without being an expert. Once having spotted the culprit, Google may help you to get more information.
When it comes to a clean and performant setup of routing or virtualisation it's still the best idea to ask an experienced admin.
Summary
This is all in addition to what others have already pointed to.
What would be really helpful would be a BE-Plugin, that analyses and measures the environment. May there are some out there I don't know.
I have an application build in ColdFusion MVC framework "Mach-II" and hosted on blue dragon ColdFusion server.
It causes caching issue. When i added a new page with some contents and load the page than it's working fine. But when i made some changes in the same file and hit it again its not update my changes. Its always showing me the content that i have made in the very first time. Its seems like that the server is caching my page and did not consider further changes. I have tried many solutions but failed to solve the problem.
Please let me know if you have any solution for that.
This is a bit too long for a comment - but it's not much of an answer.
First off, your question is quite broad for StackOverflow. If you aren't looking at the code yourself, and have nothing to show us, there is no guarantee we can help you at all.
It sounds like maybe this service is using query caching - which looks something like this.
<cfquery datasource="CRM" name="testQuery" cachedwithin="#CreateTimeSpan(0,0,30,0)#">
-SQL logic-
</cfquery>
Basically it stores a query's result in memory on the server. It can really help reduce strain on the database. It's possible that they've set a time limit on this caching feature that's longer than you'd like.
If you don't have access to the code, THIS is the issue you want to ask about first.
Edit: It may be entirely different.
https://docs.oracle.com/cd/E13176_01/bluedragon/621/BlueDragon_621_WL_User_Guide.html#_Toc121303111
From source:
Where ColdFusion (5 and MX) defines a ‘template cache” as a place to
holds templates in memory once rendered from source code, BlueDragon
has the same notion but refers to this as the “file cache”. In both
engines, a template once rendered from source will remain in the cache
until the server (or J2EE or .NET web app) is restarted.
The cache size, specified in the Admin Console, indicates how many of
these cached templates to keep. It defaults to 60 but that number may
need to change for your application, depending on how many CFML
templates your application uses. One entry is used for each template
(CFM or CFC file) requested.
It’s very important to understand that this is not caching the OUTPUT
of the page but rather the rendering of the template from source into
its internal objects. One cached instance of the template is shared
among all users in the application.
As in ColdFusion, once the file cache is full (for instance, you set
it to 60 and 60 templates have been requested), then the next request
for a template not yet cached will force the engine to flush the
oldest (least recently used) entry in the cache to make room.
Naturally, if you set this file cache size too low, thrashing in the
cache could occur as room is made for files only to soon have the
flushed file requested again.
It sounds like you might have to either restart the ColdFusion application or clear the Template Cache in the CFAdmin.
Okay, so I have an old ASP Classic website. I've determined I can reduce a huge number of DB calls by caching the data daily. Our site data is read only, and changes very slowly. I think based on our site usage, I would be able to cache pages by query string for every visit each day, without a hit to our server.
My first thought was to use Output Caching, but the problem I discovered right away was that it wasn't until the third page request was generated that I gained any performance. I verified this using SQL profiler, but I'm not sure why.
My second thought was to add this ObjPageCache include file from https://web.archive.org/web/20211020131054/https://www.4guysfromrolla.com/webtech/032002-1.shtml After some research I discovered that this could cause more issues than it may solve http://support.microsoft.com/kb/316451
I'm hoping someone on here will tell me that since 2002 the issue with Sending ServerXMLHTTP or WinHTTP Requests to the Same Server has been resolved with Microsoft.
Depending on how your data is maintained you could choose from a number of ways to cache it.
If your data is changed and saved in one single place you could choose to generate an html-file which you save to the serverdisk and refer to in your linking. This will require write access for the process running your site though (e.g. NETWORK SERVICE). This will produce fast pages as the server serves these pages without any scriptingengine getting involved.
Another option is reading the data into an DomDocument which you store in the Application object and refer to on the page that needs it (hence saving the roundtrip to the database). You could keep two timestamps together with the cached data (one for the cachingtime and one for the time of change of data in the database). Timestamps will allow for fast check for staleness of the cached data: cached timestamp <> database timestamp => refresh data; otherwise use cached data. One thing to note about this approach is that Application does not accept objects other than multithreaded object so you will have to use the MSXML2.FreeThreadedDomDocument.6.0
Personally I prefer the last one as it allows for a more dynamic usage and I don't have to worry about write access permissions for the process running my site (which would probably pose security risks anyways).
I have an application that was built using Fusebox 4 with ColdFusion. Can anyone recommend a good caching solution, that is a plugin, which works directly with this older version of the framework?
Another idea I've been tinkering around with is to take the most commonly used queries in the system and applying cachedWithin. The value would be a variable stored in the application scope. Basically anytime we update any of the most commonly accessed tables in the db, we update the application.cachedwithin variable as well. So whenever these tables are updated the data is refreshed. Anything else that isn't used frequently will simply query the DB to get the content.
Also to add to this very simple caching methodology would be to simply store strings, or other frequently used content, directly within the application scope.
This bulk of this application is around 30 pages, comprised of approximately 200 products. So its quite a small website.
Can anyone recommend a good Fusebox 4 cache plugin or confirm if this simple caching methodology is a good idea? If not, could you recommend a simple alternative?
thanks in advance
I would suggest you to use cfcache to store all pages output into statistics HTML files.
Then on any update, you can clear the cache of the updated pages or all the cache:
<cfcache action="flush" />
<cfobjectcache action="clear" />
make sure to disable the urlSessionFormat() in URL.
I'm not sure that you even need to be caching given the size of the site, unless you are getting a huge amount of traffic. If you are currently having performance problems, the first thing to do is make sure that Fusebox is in production mode, so that it isn't recreating the parsed files on each request.
Caching the queries should certainly aid performance - how long are the queries currently taking to execute? With Fusebox 4, it can be problematic to have "Report execution times" turned on in CF when debugging, as it can significantly affect the time the request takes to execute.
In the beginning i built my site - bemcapaz.net - on Wordpress. But after having to hack the core and build lots of stuff through direct programming I decided to move on to Drupal.
Drupal besides being a CMS focused more on community websites is great for doing anything you can imagine in a really simple way, even a blog which was what I created.
My question now is, which one offers the best performance? I think Drupal looks to be really heavier than Wordpress but since I'm not an advanced programmer I have no idea how to evaluate which one offers the fastest MySQL requests and loading times of the web pages.
Thanks.
Drupal is definitely heavier in the sense that it runs more queries per page once you've customized it. Using modules like Views, you can also build your own dynamic queries to drive widgets and pages. Those can be as speedy or as slow as the underlying combination of joins allows.
On the other hand, Drupal does have much more robust caching controls. Full-page output caching for anon users, granular caching of widget output, and granular caching of any data retrieved by a Views query can all combine to help quite a bit. There are also plugin modules like "Boost" or "Memcached" that let you augment that underlying cache system with materialized HTML files in the filesystem (bypassing Drupal directly in favor of apache), or a memcached server that stores all the cached information in memory rather than the database.
If you're looking to discover hot spots in a Drupal site you should also install the Devel module; it allows you to get query counts and detailed query times for each page on the site, and track them down to the module that's running them.
Don't know about Drupal, but in WP you can estimate query time with following code:
Just add it to your footer, after any queries.
<?php echo get_num_queries(); ?> queries. <?php timer_stop(1); ?> seconds.
I suppose, performance for both CMS depends on numbers and complexity of queries and caching mechanism. If you are using them both wisely, your performance gonna be OK. I mean - ask your database only for info you really need ;)
If someone (like me) wants a raw and simple time-comparasion, I wrote the exact same App 3 times (with 3 frameworks), which's result will share below.
Note that I didn't do any heavy queries.
Or anything which would affect the results
(in favor of one framework over the other).
In my local with Core-i7 CPU, and SSD storage:
Drupal toke 5 seconds to show a simple page:
And that because of caching,
If I clear cache manually, takes 25 seconds (and recreates cache, so the next run takes 5 seconds again),
But don't worry a server's hosting-machine is far stronger than my local-host (>ლ)
While WordPress toke 17 seconds for same page (and that always).
Again, don't worry a server's host is far stronger than my local.
Rewriting with Laravel, same App takes 850 mili-seconds ;-)
So, if you don't have the time, money or knowledge to create a basic CMS with Laravel, then Drupal is the obvious winner (but harder to learn compared to WordPress).