I would like to be able to set all indexes to "Update on schedule" and then have them all update automatically like Magento says they should in the background. The problem is, this doesn't happen. There is no cron job that automatically reindexes (See this related question).
So, if I have to create my own cron job, how do I do this exactly in an efficient way? I don't want to run "php shell/indexer.php reindexall". That does full rebuilds of index tables. Sure, I could do that nightly, but that means that no changes will be reflected on the frontend until the next day. That's not an acceptable solution. If I run full reindexes throughout the day, I end up with the same problem that I have right now - table locks and slowness due to reindexing while people are working in the admin.
Magento's new "partial reindexing" should fix this right?
This is my understanding of how it works:
I edit an entity that has a related index (e.g. A product).
A database trigger adds a record to related change log tables.
Some process later reads the change log tables and reindexes these specific entities
Concrete example
I update a value in "catalog_product_entity_varchar".
The database trigger "trg_catalog_product_entity_varchar_after_update" flags this product as changed by inserting a new version into "catalog_product_flat_cl" and "catalogsearch_fulltext_cl".
A partial reindex process reads these change log tables and reindexes only the products mention to "catalog_product_flat" and "catalogsearch_fulltext" respectively.
If this were the case, the reindexing process would be minimal and could be run often. Even every minute to where indexing becomes almost unnoticeable to admin users. (I say every minute, because Magento tells us this is possible)
In this release, however, the flat catalog is updated for you — either every minute, or according to your Magento cron job.
Where is this mystical partial reindex? How do I call it instead of reindexing everything?
Is there a reindexPartial()?
The enterprise_refresh_index cron job appears to run this. It runs every time the Magento cron runs. See Enterprise_Index_Model_Observer::refreshIndex().
This is not intended to run manually because of the need to establish a lock file. It is easiest just to run the cron.php file if you need a manual reindex.
I believe I just have a project specific issue with this not running.
The partial reindexing is executed through the cron job operator built into Magento. You do not need to run the actual indexer.php file. Instead, you must setup Magento's built in Cron scheduler based on the documentation.
Documentation: http://www.magentocommerce.com/wiki/groups/227/setting_up_magento_in_cron
You simply execute the cron.php file, which will in turn call the partial reindexing process.
php5-cli -f /home/USERNAME/public_html/cron.php
How it works:
A change to the an entity is made and is flagged to be reindex.
A cronjob executes the cron.php file
Magento checks to see which cron tasks it will run, and runs the partial reindexing process
The indexing process will see the changed entity and update the index tables with the new values.
Related
We have one Magento 2 project where we configured cron every min. Ideally the site has no change but still all cron jobs running too frequently.
Questions :
How Magento consider to re-run a index even there is no change.
Magento cron running synchronize or parallels way.
How to prevent run indexing if there is no change any lock or anything Magento manage ?
If your indexer is update by schedule.
then only changed data will be reindexed not all.
There is a schedule set for the crons. If you find that the frequency is too short, you can change the frequency of crons in a crontab.xml file.
You need to know the cron's name and its instance.
You can refer following link for configure cron time.
https://amasty.com/blog/configure-magento-cron-job/
Problem: Set up a workflow in CRM Dynamics 365 Sales that starts when the value of a specific field changes. But it turned out that the process does not start if changes are made to old CRM records (which were created before starting the process itself).
Question: Is there any method how can I make CRM start the process even for old records? I am sorry that everything is in Russian. I work in this version.
The process works correctly when creating a record and when editing a field in a new record. And when editing a field in an existing record, the process does not start
To make that Workflow to trigger on all records, make the scope as “Organization” instead of “User” - it should work as intended. Read more
It’s not about when it is created, probably those records are owned by somebody else. That’s why user scoped WF is not triggering at all.
Normally workflows trigger any future relevant changes. Otherwise, we should create a scenario where we cause that trigger. A couple of options,
As you have already set the workflow to run on a specific field change, you can make an update to that field and save the record which should trigger the workflow. If it's a very less number of records it's possible otherwise it's not a good idea to manually update all these records. If you don't want to do this manually you can update the records using any other option like a console app which makes updates to all the records (this would be faster assuming this is a one-time activity you have to do.)
Make this workflow on demand and trigger the same manually for all the records you want to run the workflow. Again this is a manual process but cleaner than the first one.
You do not need to do any manual update. The workflow you creates should be enough to kick in.
make sure your workflow has trigger on change of field. Screenshot for reference. It does not matter when the Workflow is created. As long as it satisfies condition it will kick in.
How to make sure that every 24 hours run script automatically, which compares the current time with the time that is stored in the database, and if the current time is greater then I need to make changes to the database. I use Codeigniter.
CRON Jobs is the solution, but it is not script, it's server based application that runs every given time. You can setup it from hosting panel, but not all of the hosting providers support it :(
Scenario: The system needs to check the Product table in the database DAILY to check every product's expiration date. The system needs to get a list of products with expiration dates matching the current date. Then, removes these products in the database.
Things to consider:
- Runs a single query to retrieve products that has matching expiration date.
- Remove these products in the database.
- We are talking about thousands of products here.
QUESTION: Is there a need for me to create a Spring Batch Job supported with Scheduler for this or just a Scheduled Job to do this efficiently?
Because in Scheduled Job, I can just schedule the checking and removing daily. And its done. At the same time, I can also do it using Spring Batch with Scheduler. But, what do you think is the more efficient way?
i think spring batch would be a wise decision, if you need to restart your job with more steps inside. otherwise, if it's really just one job, you could solve that restart functionality anyway manually...
all those spring-batch configuration aren't that heavy in your applicationcontext, but you need to create tables for the according repository. (perhaps you gotta stage those tables as well..)
Spring Batch:
if you need restart-functionality
Common Scheduling: Easy and fast, ain't need that much knowledge about the framework
How can I stop the indexing process? I had imported 15,000 products and started the indexing, but now it's stuck on processing for "Category search index". It has been stuck on processing for two days now, how can I stop the indexing process?
Any help would be appreciated, thank you in advance.
When Magento starts indexing of each item in the 'Index Management' list it sets 'status' field of correlative entire of 'index_process' table to 'working'. If error happens during the indexing process script halts and status remains as 'working'. As a result 'processing' image hangs at the 'Index Management' page. Of course there's no way to make sure that indexing process is still not running but checking your server's processes list (or brutal restarting or your apache).
The only intelligent way I know to solve this issue is to find out what causing this error and fix it. You can do it by running indexing from your shell and looking for the output. Here is the command:
php shell/indexer.php reindexall
If you don't have an SSH access you can also try locating this error from your server's error logs.
If you just want to remove 'processing' image you can just run the following MYSQL query:
UPDATE `index_process` SET `status` = 'pending' WHERE `status` = 'working'
However this will not solve the problem, but only hide the output.
Probably the easiest way is to restart Apache. It will gracefully close any open visitor connections, and should terminate the PHP process.
--------- EDIT ----------
There are a set of tables in the database that look to be related to the indexing, index_process and index_process_event. You could try reverting the state of the relevant record there to pending and try again...
Keep in mind, if you are running magento under a different user than the one you are currently using (www-data vs mySshLoginUser), you might open yourself up to permissions problems on the lock files. If you have manually run the shell indexer, those lock files will belong to your ssh user.
Worst case scenario, delete those lock files in var/locks/* and try it.