How would you handle a situation when you want to cache some sublayout that relies on external data(search index, etc.)?
Here's the example:
After updating content you click
"Publish" in Sitecore.
HTML cache is automatically cleared after
publishing.
Sublayout is rendered and put into the cache on
first request, but the search index
is not yet updated.
Search index is updated.
Until the next cache cleanup, sublayout will display the cached (obsolete) date.
Are there any simple ways to fix this issue?
I've got only one idea so far - trigger cache cleanup when the index update is over, but it might be complicated for many reasons.
You can cache the sublayout and vary by params, where you define the custom params. Those params can be a unique string from Lucene, e.g the last time it was rebuilt.
E.g.
<sc:sublayout ID="slNews" Path="NewsList.ascx" Cacheable="true" VaryByParm="true" runat="server" />
Note: the Sitecore code has a typo and it's "VaryByParm" not "VaryByParam"
In C#:
string lastIndexRebuild = GetLastRebuildTimeOfIndex().ToString();
slNews.Parameters = "lastIndexRebuild=" + lastIndexRebuild;
If you can somehow define a method to determine when the index was last rebuilt, you can use that as a parameter to define custom cache instances based on when the index freshness or staleness. You can even tack on additional parameters, like a datasource, etc.
John West has recently posted related blog post -
Schedule Expiration for Output Cache Entries with the Sitecore ASP.NET CMS
You can set cache expiration on sublayouts, in most cases it seems to be the easiest solution when your control relies on external datasource.
Related
I'm using Google Data Studio to visualize results from various queries (from different tables within the same BigQuery-database).
For this reason, I created and use multiple data-sources-connectors. Each one of them has a SQL query included and makes use of an defined input parameter (which can be changed by report editors) - called "userid". It is the same id for all queries and resulting charts.
However, when I click "Manage URL parameters", I'm not allowed to use the same URL parameter for more than one data source (instead they are called ds0, ds1, ds2 etc - although they all end up being used as "userid").
If I add a data source under File - Report Settings, a new field "userid" appears, which I can alter - this will update ALL charts in the report with the very same userid (as expected). This works, but I do want to make use of an url which delivers an report with all updated queries depending on ONE userid.
Therefore, I guess I'm overseeing something - it should be possible to just use one query parameter to update the same "userid" for all queries in all data connectors? Or have I overlooked the possibility to fire multiple queries within one data source connector? Or is it expected to create a looong url full of redundant query parameters in this case?
I'm curious for your input!
Best regards :)
There does not seem to be any good solution for this.
For now the best workaround seems to be to just repeat the parameter multiple times -- it's ugly but it works. For example, use the URL parameter mapping screen to call the parameter u1, u2, etc., and then just pass all of them:
?params={"u1":"foo","u2":"foo"}
(URL encoded of course)
The ugliness is mostly for us developers: it violates our sense of DRY and clean code, and makes the URL much longer than it needs to be. However, most people don't care or know about the URL parameters so its irrelevant to them.
The bigger downside is that when the URL is distributed to clients (bookmarks, mobile apps), every time new data sources are added that require the same URL parameters, a new URL has to be distributed to clients for no good reason. A workaround for this is to build the URL dynamically via a simple redirector function.
This issue https://issuetracker.google.com/issues/180705297 is a feature request to implement this capability.
If you group the elements that you want to control with the same parameter (select and then shift G) then it will give you options to select the data source and the params box to apply to the group.
Is there any method on elasticsearch for fully (not partially) updating documents and not create new ones in case it doesn’t already exists?
Until now, I found that the _update method, while passing a doc attribute inside the json request body to partially updating documents, however, I would like to replace the entire document in this case, not only partially.
I have also found that, the index method, where sending a PUT request works fine, although creating a new document in case the id not yet indexed.
Setting the op_type parameter to create will enforce document creation instead update.
I was wondering if there is any way to always enforce update and never create a new one?
Or perhaps is there another method that would allow me to achieve such task?
If I understand correctly, you want to index a doc, but only if it already exists? Like an op_type option of update?
You can mostly do it with the update API, given that your mapping remains consistent. With an _update, if the document doesn't exist, you'll get back a 404. If it does exist, ES will merge the contents of doc with whatever document exists there. If you make sure you're sending over a new doc with all the fields in the mapping, then you're effectively replacing it outright.
Note, however, that you can do it without the document merge rather efficiently in two requests; the first one checking for doc existence with a HEAD request. If HEAD /idx/type/id is successful, then do a PUT. This is essentially what's happening internally anyway with the update API, with a little extra overhead. But HEAD is really cheap because it's not shuffling any payload around. It simply returns an HTTP 200/404.
I have an ASP.NET MVC3 application which has multilingual support. Almost every word has multilingual support and at each page request I get all the words in the currently selected language from the database into a List and I use it for each Word: I hold MeaningID for each element and print out the matched one from the List. Costly approach, but better than reaching to database for every Word.
Still, I wonder if there's a data structure I can use globally throughout the project, which is only loaded from the database when the user changes the selected language. Is there a session like list structure can I use for such a purpose?
EDIT: To make things clearer I'm posting my database tables.
--Word-- --WordBase-- --Language--
ID ID ID
Text Text Name
BaseID
LanguageID
As it's seen, WordBases are meanings that Words depend on by a Language. Example data is:
--Word-- --Base-- --Language--
1;Hallo;1;1 1;Hello 1-Deutsch
2;Hello;1;2 2;Good 2-English
3;Gut; 2;1
4;Good; 2;2
Your web app is like a dictionary? I mean... your "words" are the data of your application... or are you talking about internationalization?
If it is internationalization, I think there are better ways to do it... using the tools built in. Check this: http://afana.me/post/aspnet-mvc-internationalization.aspx
If the translatable data is too large... may be you could have an hybrid approach... keeping tokens in database... and translation in resource files. Then, caching would be useful, specially if your data doesn't change very often (you can set caching for 30min... and for that time you avoid SQL queries to retrieve words in every request).
You should cache this using Cache. Then, you can manage the Cache to hold the information during the user session or by time expiration.
Take a look here: Walkthrough: Caching Application Data in ASP.NET.
I've been searching for a day or 2 for an answer to this question, but I haven't found one yet. I've got an external application which is modifying a TSV file (adding data) periodically. I'm using the Basic Line Chart example to display the data and it looks really nice:
Now I want the data to update when the TSV file is updated. I want to be able to set an auto-refresh on the data where it pulls from the tsv file and repopulates the graph without refreshing the entire page.
I tried just wrapping up the current code in a function and calling setInterval on that function, but the data remains the same each time (maybe because it's cached?).
Ideally the solution to this would be a function which can be called to Update whenever I'd like (based on a user event, timer, whatever).
Any ideas, links, or suggestions for alternate ways to accomplish the same goal would be much appreciated!
As a bonus question: I understand D3 may not be the right choice for this sort of Psudo-Real-Time data display. Are there other packages which lend themselves to this sort of thing more? The app generating the data is a C# application (in case that ends up mattering).
Edit: As a supplementary explanation, imagine this example but with the data being read from a file: http://mbostock.github.com/d3/tutorial/bar-2.html
If you are executing an Ajax call to fetch the data from the server and you think caching is a problem, you can try busting the cache by setting the cache parameter in jquery's ajaxSetup to false anywhere in your code:
$.ajaxSetup({cache: false});
From the docs:
If set to false, it will force requested pages not to be cached by the
browser. Note: Setting cache to false will only work correctly with HEAD and
GET requests. It works by appending "_={timestamp}" to the GET parameters. The
parameter is not needed for other types of requests, except in IE8 when a
POST is made to a URL that has already been requested by a GET.
I've been having a look at several MVC frameworks (like rails, merb, cakephp, codeignitier, and similars...)
All the samples I've seen are basically plain and simple CRUD pages, carrying all the infr needed in the querystring and the posted field values.
I've got a couple of apps made with some sort of framework built with classic asp.
This framework handles some CRUD stuff a little more complex than the examples I found.
Something like master-detail, filtering by example, paging, sorting and similars.
I have a controller class that it's just a finite state machine, that goes thru diferent states (like new, browse, filter, show, etc.), then performs the appropiate action depending on the event raised and finally retrieves the neede info to the calling page.
To achieve this I have several hidden inputs to keep the state of the web page (like current id, filter criterias, order criterias, previous state, previous event, well, you get the idea)
What do you think would be the finnest approach to achieve this kind of funcionality?
hidden inputs built in the view and used from the controller??? (I guess that would be the equivalent of what I'm doing right now in classi asp)
--
(added in response to tvanfosson)
basically, my question refers to the third category, the context-dependent setting (in respect to the other two categories I agree with you) the info I was storing in hidden fields to store them on the querystring, I guess that when you click on the "next page" you include everything you need to save in the querystring, right? so that piece of query string gets appended in each and every link that performns some kind of action...
I'm not sure, what are the advantages and disadvantages of using the querystring instead of hidden inputs???
I use different strategies depending on the character of the actual data. Things that are preferences, like default page size, I keep in a Preferences object (table) that is associated with the current logged in user and retrieve from there when needed.
Persistent settings associated with the current logon, like filter settings for a page, are stored in the user's session. Generally these are things that if a user sets them in the current session they should remain sticky. I think filter settings and visibility are like this. If I filter a list, navigate away from it to drill down into a particular item, then come back to the list, I want my filter settings to be reapplied -- so I make it part of the session.
Context-dependent settings -- like the current sort column or page number, are controlled using query parameters. Paging and sort controls (links) are built with the appropriate query parameters to "do the right thing" when clicked and pass any necessary query parameters to maintain or update the current context of the control. Using the query parameters allows you to use an HTTP GET, which is bookmarkable, rather than a POST. Using hidden form parameters makes it much harder for the user to save or enter a URL that takes them directly where they want to go. This is probably more useful for sorting than it is for paging, but the principle applies equally.