Symfony 1.4 functional test not displaying new data - caching

I'm running a functional test against my Symfony 1.4 project and it keeps failing because it doesn't fetch the latest data.
The test makes up a new site entry, then a survey at the site, then adds data to the survey. Each of these are on separate pages and each work. The data is definitely present in the database. After saving the last form, the survey_data, it returns to the survey page where it should get the survey details and a list of all the data items added to it. This works in dev and prod environments but in my functional tests the survey_data list is empty. Looking through the logs it doesn't try to fetch the data from the database (Doctrine & Postgres). But if I manually load the page in a browser just seconds later the list is there, and if I run a test that goes directly to the page (without clearing the database) the list is there, so the test login has the rights to see that page and the contents, but won't show it at first, as if it has cached the page survey page before the survey_data was added.
So my question is, how can I ensure my functional tests get the latest data from the DB or how can I refresh the local data object cache after saving a new item to the database?
Additional:
My functional tests extend sfPHPUnitBaseFunctionalTestCase and I use the sfBrowser class to check the contents of the pages and navigate through them

The issue appears to be with Doctrine caching any objects it updates for the duration of the execution. This gets cleared out after each web request in a production environment - one request per execution - but in the test environment it appears to persist between requests as they're all happening in the same execution.
It becomes troublesome when testing the process of, say, adding a new Item to a List - the List is in memory (test browser has been to its page before), the new Item gets created, saved but the link between them isn't formed in-memory.
Targeted refreshing of related objects
If you know what object you're looking to forcibly refresh you can:
$SomeDoctrineRecordObject->refreshRelated(); // for all relationships
$SomeDoctrineRecordObject->refreshRelated($relation); // for a specific one
( for the above List and Item example, you'd $list->refreshRelated('item') )
This is only useful if you're using the Doctrine object already, else you have to go pull it out of the route or database each time you need to refresh it. A more general approach is to extend the sfTestFunctional class and override the methods which might result in a relationship change and thus need to trigger a refresh.
General refreshing of all objects
The code to clear out Doctrine's object cache:
$manager = Doctrine_Manager::getInstance();
$connection = $manager->getCurrentConnection();
$tables = $connection->getTables();
foreach ( $tables as $table ) {
$table->clear();
}
And an example of how to hook it into a custom functional test object:
class myTestFunctional extends sfTestFunctional
{
public function get( $uri, $parameters = array(), $changeStack = true )
{
$this->clearEntityCache();
return parent::get( $uri, $parameters, $changeStack );
}
public function click( $name, $arguments = array(), $options = array() )
{
$this->clearEntityCache();
return parent::click( $name, $arguments, $options );
}
protected function clearEntityCache()
{
$manager = Doctrine_Manager::getInstance();
$connection = $manager->getCurrentConnection();
$tables = $connection->getTables();
foreach ( $tables as $table ) {
$table->clear();
}
}
}
Thus every time myTestFunctional gets a url or clicks to change page it clears out any Doctrine objects stored in memory. Not subtle, but effective and doesn't make the tests themselves more labourious to write!

Related

Invalidate Shopware 6 page cache on entity update via API

We created a custom CMS element which displays entries which are managed via API.
Now when entries are updated and Shopware 6 runs ins production mode, the changes are not reflected on the page. I believe this is due to the page cache. (APP_ENV=prod)
What do we have to do to invalidate the cache automatically?
I checked the docs, but did not find the necessary information.
For the product box it works: When I place a product CMS element on the main page and change the product, the page is updated when I reload in the browser.
I was expecting to find some hint in \Shopware\Core\Content\Product\Cms\ProductBoxCmsElementResolver but there are no cache tags or something like this added there.
EDIT: Actually I was a bit inaccurate. The page I have lists the entities, and it is a category page.
So I believe I have too hook into CategoryRouteCacheTagsEvent.
For testing I hard-coded into:
\Shopware\Core\Content\Category\SalesChannel\CachedCategoryRoute::extractProductIds
$slots = $page->getElementsOfType('jobs');
/** #var CmsSlotEntity $slot */
foreach ($slots as $slot) {
$box = $slot->getData();
$ids = array_merge($ids, $box['jobs']->getIds());
}
But this does not yet work.
PS: Also I noticed some code duplication in the core in \Shopware\Core\Content\Category\SalesChannel\CachedCategoryRoute::extractProductIds and \Shopware\Core\Content\LandingPage\SalesChannel\CachedLandingPageRoute::extractIds
The Shopware\Core\Framework\Adapter\Cache\CacheInvalidationSubscriber listens to a lot of events, including indexer and entity-written events. This in turn uses the CacheInvalidator to invalidate cached data based on tags/cache keys.
You should be able to add invalidation based on your own entity in a similar fashion.
For this to work with a custom entity, you will probably have to tag any cache entries with something you can generate on invalidation. For CMS pages, I would probably start with the CachedLandingPageRoute as a reference.
I suggest you should have a look at the CacheInvalidationSubscriber and its service definition. You can see that there are already a bunch of events that are dispatched when write operations to certain entities occur. When you then look at the respective handler you can see how it invalidates the cache for whatever kind of routes it should affect.
When you speak of entries I assume you implemented your own custom entities for use in your CMS element? If that is the case just replicate the listener for your own entities. Otherwise you'll have to look for another event that is dispatched once you save your changes and then invalidate the cache likewise.
Based on the answers of dneustadt and Uwe, as for the job listings I solved it like with this two subscribes. I do not need any single ID here, because the full listing page has to be invalidated in case a job is deleted or added. This is why I went with the any-jobs tag:
use Shopware\Core\Content\Category\Event\CategoryRouteCacheTagsEvent;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
class CacheKeyEventSubscriber implements EventSubscriberInterface
{
public static function getSubscribedEvents(): array
{
return [
CategoryRouteCacheTagsEvent::class => 'generateTags'
];
}
public function generateTags(CategoryRouteCacheTagsEvent $event): void
{
$page = $event->getResponse()->getCategory()->getCmsPage();
$slots = $page->getElementsOfType('jobs');
if (!empty($slots)) {
$event->setTags(
array_merge($event->getTags(), ['any-jobs'])
);
}
}
}
and
class CacheInvalidationSubscriber implements EventSubscriberInterface
{
private CacheInvalidator $cacheInvalidator;
public static function getSubscribedEvents(): array
{
return [
EntityWrittenContainerEvent::class => 'invalidateJobs'
];
}
public function __construct(CacheInvalidator $cacheInvalidator)
{
$this->cacheInvalidator = $cacheInvalidator;
}
public function invalidateJobs(EntityWrittenContainerEvent $event): void
{
if (!empty($event->getPrimaryKeys(\ApplicationManagement\Core\Content\JobAd\JobAdDefinition::ENTITY_NAME))) {
$this->cacheInvalidator->invalidate(
['any-jobs']
);
}
}
}

Laravel Auth::User() performance

I have a question that whenever we call Auth::User() then its execute the query to fetch record or it have a saved instance?
Example 1
echo Auth::User()->name;
echo Auth::User()->email;
echo Auth::User()->phone;
Example 2
$userInfo=Auth::User();
echo $userInfo->name;
echo $userInfo->email;
echo $userInfo->phone;
Which one should be used performance wise?
Answer and example
Call to the database will be made only the first time you call Auth::user(), after that Laravel will store the user data and each call after that will get the stored instance rather then query the database again.
You can take a look at the vendor\laravel\framework\src\Illuminate\Auth\SessionGuard.php file under user() method. This is the code I copied from my current project which uses Laravel 7.x and this is the function called by Auth::user().
/**
* Get the currently authenticated user.
*
* #return \Illuminate\Contracts\Auth\Authenticatable|null
*/
public function user()
{
if ($this->loggedOut) {
return;
}
// If we've already retrieved the user for the current request we can just
// return it back immediately. We do not want to fetch the user data on
// every call to this method because that would be tremendously slow.
if (! is_null($this->user)) {
return $this->user;
}
$id = $this->session->get($this->getName());
// First we will try to load the user using the identifier in the session if
// one exists. Otherwise we will check for a "remember me" cookie in this
// request, and if one exists, attempt to retrieve the user using that.
if (! is_null($id) && $this->user = $this->provider->retrieveById($id)) {
$this->fireAuthenticatedEvent($this->user);
}
// If the user is null, but we decrypt a "recaller" cookie we can attempt to
// pull the user data on that cookie which serves as a remember cookie on
// the application. Once we have a user we can return it to the caller.
if (is_null($this->user) && ! is_null($recaller = $this->recaller())) {
$this->user = $this->userFromRecaller($recaller);
if ($this->user) {
$this->updateSession($this->user->getAuthIdentifier());
$this->fireLoginEvent($this->user, true);
}
}
return $this->user;
}
Debugbar options
Also as the comment before me pointed out, it is good to download Debugbar for Laravel https://github.com/barryvdh/laravel-debugbar. It will enable you to take a look into queries being executed, views being rendered, requests being sent, and much more.
Other option is Laravel's native solution https://laravel.com/docs/8.x/telescope. I have never personally used it and IMO the first one is simpler to use.
Notes on good practice
Although both examples will essentially do the same thing, I think it is much better to use the second example. Not because of the performance, but rather to make your code readable in future. If you define the user only one time and assign Auth::user() result to it, in future it will be more obvious what it is, and plus, you can easily change what are you assigning to the $userInfo variable (maybe you want to get user from another guard in future, etc.) without having to change it on N places throughout the code.

beforeFind() in Yii ActiveRecord and cache

In some model classes I want to implement cache. And I wanted to do that like:
UsersModel::model()->findByAttributes([...])
In that class I wanted to override method beforeFind() to send request first to cache server, but it seems that method does not take any additional parameters, nor does have object with attributes.
Putting additional conditions/checks in top level code something like :
$response = Yii::app()->cache->get('userUserLogin');
if(empty($response) == true) {
//fetch data from db and set to cache
$userModel = UsersModel::model->findByAttributes([...])
Yii::app()->cache->set('user' . $userModel->username, $userModel->getAttributes());
}
is not nice and trivial, leading to many branches.
You should not use beforeFind() for that. Besides technical problems in implementation, you may get many side effects and hard to debug bugs because of that. That is because cache may be out of date and many internal Yii logic may rely on assumption, that findByAttributes() (and other methods) always fetches fresh data from database. You will also not be able to ignore cache and get model directly from database.
In general you have 2 options:
1. Use CActiveRecord::cache()
$model = UsersModel::model()->cache(60)->findByAttributes([...])
This will query cache results for 60 seconds.
2. Custom helpers
You may add custom methods, which will simplify using cached active records:
public static function findByAttributesFromCache($attributes = []) {
$result = Yii::app()->cache->get(json_encode($attributes));
if ($result === false) {
//fetch data from db and set to cache
$result = static::model()->findByAttributes($attributes);
Yii::app()->cache->set(json_encode($attributes), $result, 60);
}
return $result;
}
You can add such method to trait and reuse it in multiple models. Then all you need is:
$userModel = UsersModel::findByAttributesFromCache([...]);

Laravel DatabaseTransactions for PHPunit have no effect

Writing tests for an existing API, there are many cases where the database has been modified. What I have been doing is something as follows:
public function testPut()
{
//setup
/*Copy an existing record and take its data as an array.
* the function being tested will take an array of data
* and create a new record. Using existing data guarantees
* the data is valid.
*/
$customerObj = Customer::getInstance(); //regular instantiation disabled in this api
$cust = ArCust::first()->toArray();
$oldNum = $cust['CUST_NO'];
unset($cust['CUST_NO']);
$custNo = rand(1, 9999999999999);
//test
/*put() creates a new customer record in the database
and returns the object.
*/
$this->assertInternalType('object', $customerObj->put($custNo, $cust));
//cleanup
/*manually remove the newly created record.
*/
ArCust::whereNam($cust['NAM'])->whereNotIn('CUST_NO', [$oldNum])->delete();
}
I am now running into instances where the API creates or updates many tables based on foreign keys. It would take far too much time to go through and manually reset each table.
The DatabaseTransaction trait provided by Laravel is supposed to take care of resetting everything for you. However, when I use it, I still find the test-created records in the database.
Here is how I have used it:
class CustomerTest extends TestCase
{
use DatabaseTransactions;
public function testPut()
{
//setup
$customerObj = Customer::getInstance();
$cust = ArCust::first()->toArray();
$oldNum = $cust['CUST_NO'];
unset($cust['CUST_NO']);
$custNo = rand(1, 9999999999999);
//test
$this->assertInternalType('object', $customerObj->put($custNo, $cust));
}
}
Am I using this incorrectly? Getting DatabaseTransactions to work correctly will save an incredible amount of time, as well as make the testes more readable to other people.
The issue was that we had multiple database connections defined in config > database.
In the database.php conf file, I changed the default connection to the correct database using its name as defined in the setup:
$connection = 'counterpoint';
and DatabaseTransactions now works.
This next step to this solution is to direct the connection of each test to the appropriate database rather than change the global connection.

Hole punching Mage_Catalog_Block_Product_Price in Magento EE FPC

I'm having a heck of a time figuring out the code/parameters to hole-punch the Full Page Cache in magento for the Mage_Catalog_Block_Product_Price block. I can get the price to show the first time the page is loaded, but when the cache id is unique, it's not rendering the price properly (it does cache it correctly when it's supposed to be cached) . I know I need to send it parameters, such as product_id etc, but not clear about what (eg 'xx') needs to be sent from getCacheKeyInfo into the cache container for use in $this->_placeholder->getAttribute('xx'). And what needs to be prepared and sent from _renderView() to the price layout/view.
So far I have done the following successfully (they each output testing data)
Created the cache.xml in my module /etc folder
Created the cache container model and verified works (just need settings)
Rewrote/extended the Mage_Catalog_Block_Product_Price into my own model to add the getCacheKeyInfo()
So the problem is that I have tried many variations within the container model's _getCacheId() and _renderBlock() in combination with the getCacheKeyInfo(), like described above. But I am hitting a stumbling block. If anyone can lead me in the right direction, it would be greatly appreciated.
I've been struggling with the Full Page Caching as well.
These are my findings and have been very helpful to me.
Please take a look at: app/code/core/Enterprise/PageCache/Model/Processor/Default.php Line 47
/**
* Check if request can be cached
*
* #param Zend_Controller_Request_Http $request
* #return bool
*/
public function allowCache(Zend_Controller_Request_Http $request)
{
foreach ($this->_noCacheGetParams as $param) {
if (!is_null($request->getParam($param, null))) {
return false;
}
}
if (Mage::getSingleton('core/session')->getNoCacheFlag()) {
return false;
}
return true;
}
Looking at this function there seem to be two ways of avoiding (disabling) the Full Page Cache:
GET Parameter:
You can use the parameters 'store' or 'from_store' prefixed with three underscores to avoid the cache.
Example:
http://magentourl.com/catelog/category/view/id/123?___store
Mage::getUrl('catalog/category/view', array('id' => 123, 'query' => array('___store' => '')))
Session variable:
You could also avoid the Full Page caching by setting a (temporary) session variable:
Mage::getSingleton('core/session')->setNoCacheFlag(true)

Resources