I have a Play Framework 2 application with intertionalization.
I use the CacheApi embedded in Play (ehCache) with standard configuration.
I made an example with index page:
#Inject #NamedCache("session-cache") CacheApi cache;
then
#Cached(key = "index")
public Result index() {
//Controller.changeLang("fr");
return ok(index.render(this.userProvider,"c-layout-header-fullscreen",NEWS_FORM));
}
When i launch first page and refresh it seems to work but i have a method to change the lang (or detect automatically the browser lang/country)
public Result changeLanguage(String code) {
Controller.changeLang(Language.findByCode(code).code);
return redirect(controllers.routes.Application.index().toString());
}
when i change the language the same page as before cached in first language page ordered is displayed.
How i can specify the cache for a language (if i have 3 languages, 3 differents index page cached). Same problem for DB translation i want to chache result query but following the language it will be different. I want to cache the translation query because it will use DB lot of.
what is difference between session-cache, db-cache, user-cache ?
Related
How to show not cached fe_user data in TYPO3? According to Prevent to Cache Login Information.
The ViewHelper sets $GLOBALS['TSFE']->no_cache = 1 if the user logged in. Is there a better way? Because not the whole page should not be cached, only some parts of it.
Unfortunately this is not possible.
The best way is, you render the not cached fe_user data with a AJAX called eID or TypeNum and the whole page is completly cached.
like this one:
http://www.typo3-tutorials.org/cms/typo3-und-ajax-wie-geht-das.html
your example code disabled cache for the complete page. but you only need to disable cache for the part where you display the user specific data. As you can except any part from caching you need to select whether to cache only
one content element (especially for plugins this is standard behaviour: just declare your plugin as uncachable in your ext_localconf.php)
one column (make sure you use a COA_INT (or other uncached object) in your typoscript)
one viewhelper (make your viewhelper uncachable [1] or use the v:render.uncache() VH from EXT:vhs)
[1]
as a viewhelper is derived from AbstractConditionViewHelper, which uses the Compilable Interface, which caches the result, the compile() method from AbstractConditionViewHelper must be rewritten and return the constant
\TYPO3\CMS\Fluid\Core\Compiler\TemplateCompiler::SHOULD_GENERATE_VIEWHELPER_INVOCATION
like this:
public function compile(
$argumentsVariableName,
$renderChildrenClosureVariableName,
&$initializationPhpCode,
\TYPO3\CMS\Fluid\Core\Parser\SyntaxTree\AbstractNode $syntaxTreeNode,
\TYPO3\CMS\Fluid\Core\Compiler\TemplateCompiler $templateCompiler
) {
parent::compile(
$argumentsVariableName,
$renderChildrenClosureVariableName,
$initializationPhpCode,
$syntaxTreeNode,
$templateCompiler
);
return \TYPO3\CMS\Fluid\Core\Compiler\TemplateCompiler::SHOULD_GENERATE_VIEWHELPER_INVOCATION;
}
I am using Spring Data with EclipseLink JPA to do server side pagination on a database result set. I have everything working and I get the expected paged results, but I noticed performance suffering on large data sets (several million rows). It is taking about 5 minutes to return a page of 20 results. Perhaps this is to be expected, but what concerned me was the query output.
My log output:
SELECT COUNT(filename) FROM document
SELECT filename, datecaptured, din, docdate, docid, doctype, drawer, foldernumber, format, pagenumber, tempfilename, userid FROM document ORDER BY din ASC
I would understand that in order to page, Spring would need to know the max row count, so the first query makes sense.
The second query is pulling the entire database when I specifically only asked for 20 results with a 0 offset (page).
Does Spring/EclipseLink/JPA in fact grab the entire data set and then only return the subset paged request?
If that is the case, how should I modify my repository class to be more efficient?
My test case:
#Test
public void getPagedDocumentsTest() throws IOException {
Page<Document> requestedPage = documentRepository.findAll(new PageRequest(0, 20, Sort.Direction.ASC, "din"));
Assert.assertNotNull("Page is null", requestedPage);
Assert.assertNotNull("Page is empty", requestedPage.getContent());
List<Document> documents = requestedPage.getContent();
LOG.info("{}", documents);
LOG.info("{}", documents.size());
}
My repository class:
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;
import com.example.data.model.Document;
#Repository
public interface DocumentRepository extends PagingAndSortingRepository<Document, String> {
}
Edit - per #Chris's suggestion
Tried adding the platform to my properties, but it didn't make a difference:
eclipselink.weaving=static
eclipselink.allow-zero-id=true
eclipselink.target-database=SQLServer
eclipselink.logging.level=FINE
Also tried adding it to my configuration (I'm using Java Config):
#Bean
public LocalContainerEntityManagerFactoryBean entityManager() {
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setPersistenceUnitName("ExampleUnit");
factory.setPackagesToScan("com.example.data.model");
EclipseLinkJpaVendorAdapter eclipseLinkVendorAdapter = new EclipseLinkJpaVendorAdapter();
eclipseLinkVendorAdapter.setDatabase(Database.SQL_SERVER);
eclipseLinkVendorAdapter.setDatabasePlatform("SQLServer");
factory.setJpaVendorAdapter(eclipseLinkVendorAdapter);
factory.setDataSource(dataSource());
factory.setJpaProperties(jpaProperties());
factory.setLoadTimeWeaver(new InstrumentationLoadTimeWeaver());
return factory;
}
Looks like the platform is set correctly.
[EL Config]: connection: 2015-08-06 12:04:05.691--ServerSession(686533955)--Connection(1896042043)--Thread(Thread[main,5,main])--connecting(DatabaseLogin(
platform=>SQLServerPlatform
user name=> ""
connector=>JNDIConnector datasource name=>null
))
But neither helped. The SQL query output remained the same as well.
Edit
Found a related question with a similar answer from #Chris:
EclipseLink generated SQL doesn't include pagination
EclipseLink 2.5 source that I checked I believe has support for database level filtering built into the following database platform classes:
DB2Platform
DerbyPlatform
FirebirdPlatform
H2Platform
HANAPlatform
HSQLPlatform
MySQLPlatform
OraclePlatform
PostgreSQLPlatform
SymfowarePlatform
Each of these override the printSQLSelectStatement method to take advantage of their respective database features to allow filtering in the SQL itself. Other platforms will need to use JDBC filtering, which depend on the driver to restrict rows - they may be able to optimize queries, but it is driver specific and I believe it is why your query takes longer then you desire.
I don't know SQLServer well enough to say what equivalent functionality it has that can be used within the SQL, but if you find it, you would need to create a SQLServerPlatform subclass, override the printSQLSelectStatement method as is done in the above classes, and then specify that platform class be used instead. Please also file a bug/feature to have it included in EclipseLink.
Other options are described here:
http://wiki.eclipse.org/EclipseLink/Examples/JPA/Pagination
One thing you should consider is whether you actually need to know the number of pages / total number of elements. If you are returning a page from a result set that has milions of elements, chances are your users will not be interested in looking through all those pages either way :). Maybe your front end shows the data in a an infinite scroll that just needs to know, if there are any more pages, instead of number of pages.
If any of those cases apply to you, you should consider returning a Slice instead of a Page, as in:
public Slice<MyClass> findByMyField(..);
This way, instead of doing the expensive Count, Spring Data will just ask for one more element than you originally wanted. If that element is present, the Slice will return true from the hasNext method.
Where I work we recently used Slices for several large data sets and with the right indexes (and after clearing the database cache :) we have seen some really significant gains.
I am trying to setup nhibernate second level caching and i see in this article, and i am trying to understand the difference between query caching and entity caching. It says you need to add
Cache.ReadOnly(); or Cache.ReadWrite();
on every single entity mapping like this:
public class CountryMap : ClassMap<country>
{
public CountryMap()
{
Table("dropdowns");
Id(x => x.Id, "pkey");
Map(x => x.Name, "ddlong");
Map(x => x.Code, "dddesc");
Where("ddtype = 'COUNTRY'");
//Informing NHibernate that the Country entity itself is cache-able.
Cache.ReadOnly();
}
}
But when using nhibernate profiler, i see things hitting the second level cache and I don't have this Cache.ReadOnly() value set.
Is that really required? Should I be doing this for every single entity (no matter how often that entity changes?).
If the answer is yes, that i should be doing this for all entities, I saw a page that mentioned there is a risk of setting an entity with this line as it might lead to Select n + 1 query problem if you are trying to join that entity with other entities in a query. I am using nhibernate profiler and it looks like someething are hitting the second level cache just from the code below. In my session setup, i have the following code:
return configuration
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<ApplicationMap>().Conventions.Add(typeof(Conventions)))
.ExposeConfiguration(
c => {
c.SetProperty("cache.provider_class", "NHibernate.Caches.SysCache.SysCacheProvider, NHibernate.Caches.SysCache");
c.SetProperty("cache.use_second_level_cache", "true");
c.SetProperty("cache.use_query_cache", "true");
c.SetProperty("expiration", "86400");
})
.BuildSessionFactory();
and i have a generic "Query" method that does this:
ICriteria c = Session.CreateCriteria(typeof(T));
c.SetCacheable(true);
return c.Future<T>().AsQueryable();
so basically I am trying to confirm if i setup caching correctly as I see some second level cache hits when I using nhibernate profiler but I have not set the Cache in the entity mapping code. I am trying to determine if there are other things i need to do to get caching working (or working better)
When I use nhibernate profiler (without having the Cache.ReadWrite() set at an entity level), it still seems like it does hit the second level cache. (see screenshot below)
Query cache only stores the identifiers of entities returned as result of a query. Actual entities are stored in entity cache region. Therefore entity must be configured cacheable to be used with query cache. If query cache is used without setting entities cacheable, still only identifiers of query results will be stored in query cache. As stated in a blog
Query cache does not cache the state of the actual entities in the
result set; it caches only identifier values and results of value
type. So the query cache should always be used in conjunction with the
second-level cache.
When re-executing the same query, what NHibernate does is, it gets list of identifiers in query result from query cache, and fetch each entity from entity cache and if not found in cache, queries that element from database (ending up in multiple queries; one for each entity).
Therefore it is always recommended to use 2nd level cache for entities along query cache i.e. you need to specify Cache.ReadOnly(); or Cache.ReadWrite(); in entity mapping else query caching will even further reduce your application performance by making multiple database queries against one cached query result.
I would like to provide some summary about the (2nd level) Caching options we do have in NHibernate. First of all, there are 4 kinds of settings. These in fact, do represent the real power of a very granular caching settings.
<property name="cache.use_second_level_cache"> - Global switch
<property name="cache.use_query_cache"> - Global switch
<cache usage="read-write" region="xxx"/> - Class/Instance level
.SetCacheable(true).SetCacheMode(CacheMode.Normal).SetCacheRegion("yyy") - per Query
The first two are global enablers/disablers. They must be turned on before we can use the next/last two settings.
But in fact, there is already the answer. The global means - support caching, local means - decide 1) how whill be 2) which class/query handled - if will or not at all.
for example, this is a snippet of the SessionFactory.cs:
...
bool useSecondLevelCache = PropertiesHelper
.GetBoolean(Environment.UseSecondLevelCache, properties, true);
bool useQueryCache = PropertiesHelper
.GetBoolean(Environment.UseQueryCache, properties);
if (useSecondLevelCache || useQueryCache)
{
// The cache provider is needed when we either have second-level cache enabled
// or query cache enabled. Note that useSecondLevelCache is enabled by default
settings.CacheProvider = CreateCacheProvider(properties);
}
else
...
Let me explicitly point out the comment:
The cache provider is needed when we either have second-level cache enabled
or query cache enabled. Note that useSecondLevelCache is enabled by default
(NOTE: also see that the first PropertiesHelper.GetBoolean() call passes the last default value true)
That would mean, that if the 3. setting (<cache usage="read-write" region="xxx"/>) would not be important... all the mapped instances would be cached ... unmanaged, default way...
Luckily this is not true. The 3. setting, class level, is important. It is a must. Without explicit setting like this:
// xml
<class name="Country" ...>
<cache usage="read-write" region="ShortTerm" include="non-lazy/all"/>
// fluent
public CountryMap()
{
Cache.IncludeAll() // or .IncludeNonLazy
.Region("regionName")
.NonStrictReadWrite();
The 2nd level cache (class/instance level) won't be used. And this is great - because it is in our hands how to set it.
Fluent NHibernate - apply it as a convention
There is a Q & A discussing how to apply (in fact not apply) these settings globally - via special Fluent NHibernate feature - Convention (In my case, this Q & A was suggested aside of this question):
NHibernate second level cache caching entities with no caching configuration
Small code snippet cite:
public class ClassConvention : IClassConvention
{
public void Apply(IClassInstance instance)
{
instance.Table(instance.EntityType.Name);
instance.LazyLoad();
instance.Cache.NonStrictReadWrite();
}
}
Finally, we should mention here that the 4. option (query level .SetCacheable(true)) should come together with the 2nd level cache:
19.4. The Query Cache
Query result sets may also be cached. This is only useful for queries that are run frequently with the same parameters. To use the query cache you must first enable it:
<add key="hibernate.cache.use_query_cache" value="true" />
... Note that the query cache does not cache the state of any entities in the result set; it caches only identifier values and results of value type. So the query cache should always be used in conjunction with the second-level cache...
Summary: Caching with NHibernate is really powerful. The reason is 1) because it is plugable (see how many providers we can use out of the box e.g. here) and 2) it is configurable. The fact, that we can use different Concurrency strategies, regions, lazy loading handling ... or even NOT to use them... is essential. Some entities are "almost-read-only" (e.g. Country code list), while some are highly changing...
The best we can do is to play with. To experiment. At the end we can have well oiled machine with a good performance.
Yes you have to do this for all entities. However this can be done via xml configuration rather than the mapping:
Configuring NHibernate second level caching in an MVC app
It's a simple question with no answer in search(google/bing/stackoverflow). The answer of course could be complicated.
I've read a couple articles on FPC within Magento, and have yet to really nail down where I need to add or create code so that when certain URL parameters are sent it serves up cached version of the page and not try and re-cache with the URL parameters.
http://www.kingletas.com/2012/09/how-does-magento-full-page-cache-works.html
So for example, when you go to http://www.example.com/shoes it loads the correct cached version. however, with google analytics and any other type of 3rd party reporting, especially with unique identifiers, it will reload the page as if it wasn't cached. So http://www.example.com/shoes?utm_key=A1537BD94EF07 would create a new cached version of that page and so on.
I would like to be able to exclude certain URL parameters and not all. Mainly any parameter I am using for tracking of customers.
As far as code, I have not come up with anything, due to the fact of the complexity of FPC and not having a dev site currently setup to test on.
Any leads as to where I can add this exception would be helpful, thanks!
EDIT: I would like to add that I am working with the Enterprise Edition. And using the Redis for cache.
I developed my own extension on the fix.
In short the get parameters are used in the cache ID. So in order to bypass this, I created an extension that changed the following:
/app/code/core/Enterprise/PageCache/Model/Processor/Category.php
Two functions where changed
protected function _getQueryParams()
AND
public function getPageIdWithoutApp(Enterprise_PageCache_Model_Processor $processor)
/app/code/core/Enterprise/PageCache/Model/Processor/Default.php
One function was changed
public function getPageIdWithoutApp(Enterprise_PageCache_Model_Processor $processor)
Once changed, it no longer created the cache ID with my specified tracking parameters.
example:
public function getPageIdWithoutApp(Enterprise_PageCache_Model_Processor $processor)
{
$queryParams = $_GET;
ksort($queryParams);
/**
* unset known tracking codes
*/
unset($queryParams["trk_msg"]);
unset($queryParams["trk_contact"]);
unset($queryParams["utm_source"]);
unset($queryParams["utm_medium"]);
unset($queryParams["utm_term"]);
unset($queryParams["utm_campaign"]);
unset($queryParams["utm_content"]);
/** End Edit */
$queryParamsHash = md5(serialize($queryParams));
return $processor->getRequestId() . '_' . $queryParamsHash;
}
I have an application where each user can choose a custom layout. The layouts can be different and it's not just css styles but html as well.
I know that mvc would cache the layout, but having so many layouts I doubt it would fit in cache. So what would it be better to save templates in DB or on the disk?
FYI: DB that I'm using is MongoDB.
I would save the layouts on disk because at the moment I don't see any advantage in a database (unless you do). But one thing that is worth mentioning is that you can create a class derived from OutputCacheAttribute and have your saved result depend on the layout you're using.
Does the layout depend on user? You could use the VaryByCustom property to have it vary by user.
EDIT
Are your users allowed to change layouts dinamically? If yes, you should also have a guid associated to your users change it each time the layouts change so you return on your VaryByCustom method:
return string.Format("User-{0}-{1}", user.Id, user.LayoutUpdateGuid);
See the meaning of this? This way, when a user changes the layouts, they will see their pages updated immediately.
How to apply the VaryByCustom attribute in your situation
In your action method, you may use:
[OutputCache(Duration = 3600, VaryByCustom = "UserLayouts")]
public ActionResult Details(string param)
{
// Returning the view
}
Then, in your VaryByCustom method in your Global.asax.cs file:
protected override string VaryByCustom(string custom)
{
switch (custom)
{
case "UserLayouts":
//// Here you fetch your user details so you can return a unique
//// string for each user and "publishing cycle"
//// Also, I strongly suggest you cache this user object and expire it
//// whenever the user is changed (e.g. when the LayoutUpdateGuid is
//// changed) so you achieve maximum speed and not defeat the purpose
//// of using output cache.
return string.Format("User-{0}-{1}", user.Id, user.LayoutUpdateGuid);
break;
}
}
The missing piece
The missing piece here is that you need to store a value that I called LayoutUpdateGuid (I'm sure you'll find a better name) and change that value whenever a user changes his layouts => this will lead to a different string being returned by the VaryByCustom(string) method in the Global.asasx.cs which in turn will force your action method to run again and return the result with the updated layout.
Makes sense to you?
Note: I can't test the specific code I wrote here, but I am sure (apart from typos) it is correct.