How Multi-Language Magento store work with Varnish.
Is there any configuration available in varnish,so we can create cache base on cookies?
If you don't mind the languages being at different urls, Turpentine can handle this for you: https://github.com/nexcess/magento-turpentine/issues/36
If you want them to behave as they do out of the box, lets keep going.
You have to modify how varnish generates the has in your VCL
Reference: https://www.varnish-cache.org/trac/wiki/VCLExampleCachingLoggedInUsers
We would modify this to also take into account the store cookie that Magento sets based on the language selector. (Following the behavior here: http://demo.magentocommerce.com) Unfortunately this gets tricky as Varnish tends to either not pass cookies back to the server or not cache things when it sees cookies flying around
This would have Varnish cache based on the value of the cookie as well as the default url and host:
sub vcl_hash {
hash_data(req.url);
hash_data(req.http.host);
if (req.http.Cookie ~ "(?:^|;\s*)(?:store=(.*?))(?:;|$)"){
hash_data(regsub(req.http.Cookie, "(?:^|;\s*)(?:store=(.*?))(?:;|$)"));
}
return (hash);
}
But, with this method you might have to tweak the rest of your VCL to cache the page properly AND send the cookies back to the server
Another option is to use the cookie to vary caching on an arbitrary header, lets call it X-Mage-Lang:
sub vcl_fetch {
#can do this better with regex
if (req.http.Cookie ~ "(?:^|;\s*)(?:store=(.*?))(?:;|$)"){
if (!beresp.http.Vary) { # no Vary at all
set beresp.http.Vary = "X-Mage-Lang";
} elseif (beresp.http.Vary !~ "X-Mage-Lang") { # add to existing Vary
set beresp.http.Vary = beresp.http.Vary + ", X-Mage-Lang";
}
}
# comment this out if you don't want the client to know your classification
set beresp.http.X-Mage-Lang = regsub(req.http.Cookie, "(?:^|;\s*)(?:store=(.*?))(?:;|$)");
}
This pattern is also used for device detection with varnish: https://github.com/varnish/varnish-devicedetect/blob/master/INSTALL.rst
Then, you would have to extend Mage_Core_Model_App to use this header instead of the 'store' cookie. In Magento CE 1.7 its _checkCookieStore :
protected function _checkCookieStore($type)
{
if (!$this->getCookie()->get()) {
return $this;
}
$store = $this->getCookie()->get(Mage_Core_Model_Store::COOKIE_NAME);
if ($store && isset($this->_stores[$store])
&& $this->_stores[$store]->getId()
&& $this->_stores[$store]->getIsActive()) {
if ($type == 'website'
&& $this->_stores[$store]->getWebsiteId() == $this->_stores[$this->_currentStore]->getWebsiteId()) {
$this->_currentStore = $store;
}
if ($type == 'group'
&& $this->_stores[$store]->getGroupId() == $this->_stores[$this->_currentStore]->getGroupId()) {
$this->_currentStore = $store;
}
if ($type == 'store') {
$this->_currentStore = $store;
}
}
return $this;
}
You would set the current store on $_SERVER['X-Mage-Lang'] instead of the cookie
Add Following lines in Varnish Config,
if(beresp.http.Set-Cookie) {
return (hit_for_pass);
}
Related
Let's say we want to provide a unique CSRF token for each response cached by Varnish.
Is there a way hook Varnish output so that we can change a specific part of the content programmatically?
Varnish Cache solution with ESI & vmod_digest
In the open source version of Varnish, you can use Edge Side Includes for this.
You can put the following place holder in your output:
<esi:include src="/csrf-token-esi" />
This ESI tag will be parsed by Varnish and the /csrf-token-esi endpoint can be intercepted by Varnish and changed.
Be sure to return a Surrogate-Control: Varnish=ESI/1.0 response header on the page that contains the ESI tag. This header will force Varnish to parse the ESI tag.
Via synthetic responses, we can change the response body of /csrf-token-esi.
Here's an example where we use vmod_digest to create an HMAC signature as the CSRF token:
vcl 4.1;
import digest;
backend default {
.host = "localhost";
.port = "8080";
}
sub vcl_recv {
set req.http.Surrogate-Capability = "Varnish=ESI/1.0";
if(req.url == "/csrf-token-esi") {
return(synth(777,digest.hmac_sha256("secret-key", now)));
}
}
sub vcl_backend_response {
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
}
sub vcl_synth {
if(resp.status == 777) {
set resp.body = resp.reason;
set resp.http.csrf-token = resp.reason;
set resp.status = 200;
return(deliver);
}
}
FYI: you can download the vmod_digest source from https://github.com/varnish/libvmod-digest. You need to compile this from source or use another mechanism to create a unique value.
The value of the CSRF token is generated by the following line of code:
return(synth(777,digest.hmac_sha256("secret-key", now)));
The secret-key is the HMAC's signing key and now returns the current timestamp in Varnish which is the value that will be signed.
There are plenty of other ways to generate a CSRF token. vmod_digest also has other ways to generate unique output. Just remember that the 2nd argument of the synth() function is the response body and effectively the value of the CSRF token.
Varnish Enterprise solution with vmod_edgestash & vmod_crypto
In Varnish Enterprise all the necessary VMODs are packaged, so no need to compile them from source. The vmod_edgestash and vmod_crypto VMODs are enterprise VMODs.
The solution consists of adding a {{csrf}} placeholder in your response. The format of this placeholder is Mustache syntax and is parsed by vmod_edgestash.
Unlike the Varnish Cache solution, no extra roundtrips and request interception are required. Varnish Enterprise will cache the page that includes the placeholder and will customize the CSRF token upon delivery.
Here's the VCL code:
vcl 4.1;
import crypto;
import edgestash;
backend default {
.host = "localhost";
.port = "8080";
}
sub vcl_backend_response{
if (beresp.http.Content-Type ~ "html") {
edgestash.parse_response();
}
}
sub vcl_deliver {
if (edgestash.is_edgestash()) {
set req.http.csrf = crypto.hex_encode(crypto.urandom(16));
edgestash.add_json({"
{"csrf":""} + req.http.csrf + {""
}
"});
edgestash.execute();
}
}
In this case we're using crypto.hex_encode(crypto.urandom(16)) to generate a unique value. But again, you can use any function that vmod_crypto provides to generate this unique value.
See https://docs.varnish-software.com/varnish-cache-plus/vmods/edgestash/ for more information about vmod_edgestash and https://docs.varnish-software.com/varnish-cache-plus/vmods/total-encryption/ for more information about vmod_crypto.
if caching is turned off then everything works correctly. if you enable caching, the plugin at the beginning works correctly, but after 5 minutes it stops processing the line. normal or progressive caching — it doesn’t matter. when you delete the cache — processing is turned on again, and again after 5 minutes it disappears.
here is the complete plugin code. what could be the reason?
code inserted into the material for example such {robokassa 5}
class plgContentRobokassa extends JPlugin
{
public $cont='';
public function onContentPrepare($context, &$row, &$params, $page = 0)
{
$doc = JFactory::getDocument();
$doc->addStyleSheet(JURI::root(true).'/plugins/content/robokassa/css/robokassa.css');
$this->cont=$context;
}
public function onAfterRender()
{
$is_test='0';
$mrh_pass1='*****';
$mrh_login='******';
$app = JFactory::getApplication();
if ($app->getName() != 'site') {
return true;
}
// Получаем кодовое слово из параметров
$varname = 'robokassa';
//Получаем тело сайта
$html = $app->getBody();
// Если тегов нет
if (strpos($html, $varname) === false)
{
return true;
}
$bodyPos = stripos($html, '<body');
$preContent = '';
if ($bodyPos > -1)
{
$preContent = substr($html, 0, $bodyPos);
$html = substr($html, $bodyPos);
}
//Задаем шаблон поиска
$pattern = '#\{' . $varname . ' ([0-9]+)\}#i';
//Закидываем все найденные шаблоны в массив
if (preg_match_all($pattern, $html, $matches))
{
$db = JFactory::getDbo();
$query = $db->getQuery(true);
foreach ($matches[0] as $i => $match)
{
*replace code here*
}
$html=$preContent.$html.$script_alert;
//Запихиваем всё обратно в тело
$app->setBody($html);
}
}
}
The plugin events are invoked in the order they have in the Plugin administration.
Cache is part of the System plugins so when their cache is valid the response is retrieved from Cache, and the plugins which come before it are not executed.
You could be able to solve this by moving your plugin after Cache (or last)
If the plugin is of a different kind, i.e. Content, there is no way to achieve this.
You can change it into a System plugin (all events from any plugin category is available also in the System plugins).
Another alternative, you could put your code in a module, prevent caching in the module; but this will not work with page cache.
Finally, and I don't know why I didn't write it first, make an Ajax call to a page which you will NOT cache (by excluding it in the page cache plugin configuration: that way the page will be cached and each time it will retrieve the current data.
Using varnish 4 to cache different content of same request from multiple servers. It looks like it caches the first request from one server and keeps giving the same content for every subsequent request.
doing curl gives response with two caches and different age.
Are there any factors like load or anything else for stickiness behaviour?
Used Jmeter and apache benchmark with load but still got the same behaviour.
Is my vcl_hash is good? Want to save the object with hash combination of url and ip of backend server.
Atleast in my case, looks like after the ttl of the cache object, varnish is caching from second server and returns the same until ttl is done. But this is not what we expect it to behave?
am I missing anything?
using round robin and hash_data. below is my config.vcl
backend s1{
.host = "190.120.90.1";
}
backend s2{
.host = "190.120.90.2";
}
sub vcl_init {
new vms = directors.round_robin();
vms.add_backend(s1);
vms.add_backend(s2);
}
sub vcl_recv {
set req.backend_hint = vms.backend();
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return(lookup);
}
First thing to consider is that you are going to have the backend ip only after the object has been fetched from it. So you are not able to use that ip on your hash method because vcl_hash occurs before fetch.
Second is about the round robin. It takes place only when Varnish is fetching the objects, hence it will not take place when an object has already been cached.
To answer your question precisely it is needed to know why your application delivers different content for the same request. How do you indicate which backend is being requested if the request is always the same? There must be something like a cookie, a header or the origin IP of the request that should dictate which one must respond to that request.
Knowing that it is possible to set the specific backend and use it in your vcl_hash. For example purposes, let's assume that you want to set your backends based on the presence of a header named backend_choice:
sub vcl_recv {
if (req.http.backends_choice == "s1") {
set req.backend_hint = s1;
# If the header is not "s1" or does not exist
} else {
set req.backend_hint = s2;
}
...
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# We use the selected backend to hash the object
hash_data(req.backend_hint);
return(lookup);
}
Hope this answer address your needs. If there was something that I missed, feel free to comment or add to your question and I'll be glad to add some info to my answer.
We have a CI installation that has the following setting in our config...
$config['enable_query_strings'] = TRUE;
We need this in order for another area of our application to run correctly with a third party API. What's happening, however, is that pagination is defaulting to a query string method of doing pagination, which doesn't play well with caching.
Right now, they look like this...
http://localhost/something/?&page=6
It's not playing well with caching, mainly because every page URL is the same page to CI. My goal is to get switched over to the below example without messing with global settings for the rest of my application.
I've been trying for hours to find a way to disable the above setting only within this single part of the application, so that we can properly have separate URLs for the pagination, like this...
http://localhost/something/1
http://localhost/something/2
http://localhost/something/3
So far, I have been unable to overide that setting for this controller, and honestly, I'm not sure there's even a way to actually do it. Any help is appreciated. There's got to be some method of disabling a feature for a single controller somehow.
Could you use routing?
$route['something/page/(:num)'] = "something?&page=$1";
edit: to turn off pagination query strings with $config['enable_query_strings'] = TRUE;
system/libraries/Pagination.php
~line 134
change
if ($CI->config->item('enable_query_strings') === TRUE OR $this->page_query_string === TRUE)
{
if ($CI->input->get($this->query_string_segment) != 0)
{
$this->cur_page = $CI->input->get($this->query_string_segment);
// Prep the current page - no funny business!
$this->cur_page = (int) $this->cur_page;
}
}
else
{
if ($CI->uri->segment($this->uri_segment) != 0)
{
$this->cur_page = $CI->uri->segment($this->uri_segment);
// Prep the current page - no funny business!
$this->cur_page = (int) $this->cur_page;
}
}
to
if ($CI->uri->segment($this->uri_segment) != 0)
{
$this->cur_page = $CI->uri->segment($this->uri_segment);
// Prep the current page - no funny business!
$this->cur_page = (int) $this->cur_page;
}
~line 196
if ($CI->config->item('enable_query_strings') === TRUE OR $this->page_query_string === TRUE)
{
$this->base_url = rtrim($this->base_url).'&'.$this->query_string_segment.'=';
}
else
{
$this->base_url = rtrim($this->base_url, '/') .'/';
}
to
$this->base_url = rtrim($this->base_url, '/') .'/';
that might do it. Or maybe better form would be to hook into the page...
Simple solution...
$this->config->set_item('enable_query_strings',FALSE);
Just put this before you call your pagination logic in the controller. Thanks go to Taftse in the #codeigniter IRC channel for this simple override.
right now I'm using an antiflood function in all my websites :
function flood($name,$time)
{
$name = 'tmptmptmp'.$name;
if(!isset($_SESSION[$name]))
{
$_SESSION[$name] = time();
return true;
}
else
{
if(time()-$time > $_SESSION[$name])
{
$_SESSION[$name] = time();
return true;
}
else
{
return false;
}
}
}
I use it this way :
if(flood('post',60)) do something;
else 'you're posting too fast';
Is this way safe ? Or do I need to replace it/complete it with a db table stocking ips and checking if they did a request earlier ?
It depends. How likely are your users going to clear their cookies to get past your anti-flood protection? I'll say that if they have to login again, 99% of the users won't even bother.
But sure, if you really want better method, store the ips in the DB. But even that can be defeated by getting a new IP.