Filter product collection by multiple categories in Magento

During the past week I’ve dealed with a very weird and stressing thing with my colleagues: filtering a product collection by multiple categories in Magento.

  • Filter by only 1 category

Magento offers a type of filter that can be used directly from the collection:

$_category = Mage::getModel('catalog/category')->load($category_id);

$collection->addCategoryFilter($_category);
  • Filter by 2 or more categories

This is a completely different story. It’s very important to proceed correctly in order to avoid errors of the type:
[blockquote]Item (Mage_Catalog_Model_Product) with the same id already exist[/blockquote]

Proceed in the following way:

1. Call your collection and filter by the attributes you want to filter (all but the category one). For example:

$collection= Mage::getResourceModel('catalog/product_collection')
           ->addFieldToFilter('is_saleable',1)
           ->addFieldToFilter('type_id', array('eq' => 'configurable'));

2. After doing this, you can apply your multiple category filter as follows:

$categories_to_filter = array('3', '4', '5');
$ctf = array();

foreach ($categories_to_filter as $k => $cat) {
     $ctf[]['finset'] = $cat;
}

$collection->joinField('category_id', 'catalog/category_product', 'category_id', 'product_id = entity_id', null, 'left')
            ->addAttributeToFilter('category_id',array($ctf));

Important: do it in this order if you don’t want to face weird Magento issues.

Best way of disabling the Region select for addresses in Magento

Region select is not perfect in Magento. For example, for UK, there are missing counties, probably due how official they are or maybe just because Magento didn’t take the correct database.

One way or another, region field is not considered one of the required fields for most sellers when integrating orders on their ERP.

What is the best and cleanest way for disabling this select box? From my point of view, this is the best way:

Launch this mysql query on your database:

TRUNCATE TABLE directory_country_region;
  • You don’t need code changes
  • You can easily recover this table
  • This will affect throught all the web. No more select menus for region.

If you’re not convinced, you might check other options.

Magento EE 1.13 & EE 1.8

Magento EE 1.13 is finally out (last version was released in December. The focus of this version has been performance, especially regarding indexing. These are good news, as our experience with indexing problems and performance during the last months has been really bad.

Reindex Section in Magento

Reindex Section in Magento

Magento CE 1.8 will follow soon, something completely necessary, as the last version was released in July 2012 (that means nearly 8 months without any updates, not even security updates)

The most important changes are the following ones. You can also see the complete list on the Magento Website:

  • Reindexing (Enterprise Edition only):
    • Most indexing processes now run only to update products, categories, URL redirects, and so on that have changed—eliminating the need for manual full reindexing
    • Your Magento web store is not locked at any point during reindexing
    • Reindexing is now a background process.
  • Caching (Enterprise Edition only):
    • Full page caching now invalidates only pages that are affected by product or category changes
    • Improved cache adapter for single-host systems
    • Additional option of using Redis NoSQL for cache and session storage in multi-host deployments (recommended for new deployments)
  • EE customers also get detailed performance and tuning guidelines geared toward enterprise installations
  • Sales, Value Added, and Fixed Product Tax calculations (Magento CE and EE):
  • Checkout performance (Magento CE and EE) achieved by:
    • Eliminating unnecessary calls to gift wrapping when loading the Shipping Method checkout step
    • Eliminating unnecessary RSS cache cleanups when RSS functionality is disabled
    • Eliminating unnecessary calls to the translation module when sending new order e-mails if the current locale is the same as the locale set in Magento
    • Improving the overall checkout process performance by loading the information for the current checkout step only
    • Improving the overall checkout process performance by loading the progress information for the current checkout step only

[Extension] Easily generate custom grid in Magento Backoffice

A colleague of mine has recommended to me this great extension that lets the backoffice user customize all the grids on the Magento Backoffice. You can add new columns to your order, products or other grids with just few clicks.

Custom grids on Magento Backoffice

Custom grids on Magento Backoffice

Programmatically set product position in a category in Magento

Today I had to reorder products in a category. Basically, our client wanted to put the products with special price at the end. After trying to do this with the category model in Magento, I finally run into the Category Api. It worked like a charm, even though it works kind of slow. Here you have the solution (with an example):

$categories = Mage::getModel('catalog/category')->getCollection();
$new_order = 300;
foreach($categories as $category){
    
    $cat_api = new Mage_Catalog_Model_Category_Api;
    $products = $category->getProductCollection();

    foreach ($products as $product){
       $cat_api->assignProduct($category->getId(), $product->getId(), $new_order);
    }
    
}

Passing arguments to a Magento Shell script

Using Magento Api to create shell scripts seems to be easy, but there are too many things to take into account to get the most of it. In this case, I want to explain how to pass an argument from the command line when executing a magento shell script.

You have to call the script passing the argument title and argument value:

php shellscript.php --argument_title argument_value

Then, on the script code, you can retrieve this value using from your class using:

$this->getArg('argument_title')

Solving reindex issues with Product Flat Data in Magento

For the past 2 years we’ve run into a reindex issue when reindexing product flat data in Magento more than once. Basically, we wet something like:

Product Flat Data index process unknown error:
exception 'PDOException' with message 'SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`DATABASE_NAME`.<result 2 when explaining filename '#sql-1803_1aa01'>, CONSTRAINT `FK_CAT_PRD_FLAT_2_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` FOREIGN KEY (`entity_id`) REFERENCES `catalog_product_entity` ()' in /app/magento_oneline/lib/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /app/magento_oneline/lib/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /app/magento_oneline/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#2 /app/magento_oneline/lib/Zend/Db/Statement.php(300): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#3 /app/magento_oneline/lib/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)

The key is here:

FK_CAT_PRD_FLAT_2_ENTT_ID_CAT_PRD_ENTT_ENTT_ID

That is telling you which table is failing. As it is explained in this StackOverflow answer, this exception tells that the referential integrity between: catalog_product_flat_2.entity_id and catalog_product_entity.entity_id is corrupted. This usually means some record(s) contain(s) null in the field catalog_product_entity.entity_id, where they are expected to contain existing entity_ids.

Fixing this

In order to fix this issue, you need to find what the corrupted entries are. This is easy. In this case, you’ll need to create a query to get the empty registers:

SELECT a.entity_id FROM catalog_product_flat_2 AS a LEFT JOIN catalog_product_entity AS b ON a.entity_id = b.entity_id WHERE ISNULL(b.entity_id);

This will display the corrupted entities. You only need to delete them and that’s all.

+-----------+
| entity_id |
+-----------+
|     35427 |
|     35428 |
+-----------+
2 rows in set (0.04 sec)

As example:

DELETE FROM catalog_product_flat_2 where entity_id = '35427';

 

The most useful [active] Magento blogs

Ever since I started programming in Magento/PHP, I had the need to find Magento blogs with useful tips and up-to-date information (Magento changes very quickly along with how things have to be done with it as well). After almost two years I’ve only found a few blogs that are up to date and useful. I want to share them with you.

Inchoo Best Magento blog ever. These people know what they do and explain it in a way that all of us can understand
Magento Quickies This blog was created by Alan Storm, you’ve heard of him, I’m sure. He’s the teachaaa
Mukesh Chapagain’s Blog Looking for Magento tutorials in Google you will have ended up in this blog more than once…
Magento Tutorials Simple, nice and useful Magento blog
ASchroder.com This guy is the author of one of the best Magento Performance tests ever!
Magento Speed Test Test your Magento performance! (this is not a blog… but I had to mention it)
Bloggento.fr This French Magento blog is super-complete. Not only programming but also news about Magento
MagentoFox This is a nice Magento blog with useful tips
MXWest@westwideweb.com Another useful Magento Blog

Do you follow other Magento Blogs that are ‘active’? Let us know!!

Executing cron tasks manually in Magento

Usually I’ve been asked about how to run a cron task manually. A cron task defined in Magento is not anything else than a function that belongs to a model that is loaded and executed.

For example, we have this cron task on a config.xml file of a specific module:

<catgento_sap_sync_nonimages>
   <schedule>
       <cron_expr>*/15 * * * *</cron_expr>
   </schedule>
   <run>
       <model>sap/cron_sync_nonimages::run</model>
   </run>
</catgento_sap_sync_nonimages>

It’s easy to identify the model and function that is being loaded and executed as it is inside the <model> tags. The model that is being loaded is sap/cron_sync_nonimage, and the function inside that model that is being executed is run.

Knowing this, we can create a new script that loads and executes that cron task like this (create a script.php file and put something like this inside):

<?php

//Load Magento API
require_once 'app/Mage.php';
Mage::app();

//First we load the model
$model = Mage::getModel('sap/cron_sync_nonimage');

//Then execute the task
$model->run();

The last step would be running this script manually (php script.php).

How to use Solr as a search engine for Magento

We recently upgraded a Magento Community Edition 1.4 to a Magento Enterprise Edition 1.11. Once the upgraded was done and everything was working fine, our client started complaining about the fact that the search results where not good in comparison to the previous version they had (with Magento CE 1.4).

After doing some research we found no specific reason for this: we changed the order criteria: tested it using relevance or name, but the results were not good. At this point we decided to use Solr.

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, rich document (e.g., Word, PDF) handling, and geospatial search. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of many of the world’s largest internet sites.

Please note that Solr/Magento integration in Magento EE is already out-of-the-box. But with Magento CE you will need to install a module (I haven’t tested it).

Installing Solr

Note: You need to install java in your system.

If you want to use Solr with your Magento Enterprise, first of all you have to download the last version from the Official Website:

Download and uncompress the file in your home directory. Note that you’ll find a lot of files inside this zip file. The Solr files needed for using the engine are inside the «Example» folder along with other example files that can be deleted:

Once the zip has been uncompressed in your home directory, copy the files inside the folder file in a new folder called «solr» in /opt:

mkdir /opt/solr
cp -R ~/apache-solr-3.6.1/example/* /opt/solr

Now copy all the files that are in your_magento/lib/Apache/Solr/conf/ to /opt/solr/solr/conf. I used something like this:

cp * /path_to_magento/lib/Apache/Solr/conf/* /opt/solr/solr/conf/

And paste them in /opt/solr/solr/conf. This is the default configuration that Magento provides for the Solr server. In order to check that Solr works fine in the system, start the solr server:

java -jar /opt/solr/start.jar

This will start the server (a lot of information will appear and the last lines should be something like this):

Solr is started. As you can see, by default the Solr port is 8983. You can finally check that everything is OK going to your magento url and using that port. In my case: http://magentosolr.lxc:8983/solr. You will get something like this:

Now we need to set up Magento in order to use the Solr server instead of the default Magento search. In the backoffice, go to:

System » Configuration » Catalog » Catalog Search

On the Catalog Search tab, select the Solr search engine. Click on the ‘Test connection’ button in order to check that the connection is OK. You should get a ‘Successful, test again’?

Reindex And Start Using Your New Solr Search Engine

Everything is set up. There is only one missing point: reindexing. This is completely necessary so that Solr can index all the information about products and other data. This is also a final test. When reindexing, the terminal where we started Solr should start ‘moving’. The same will happen when performing a search from now on.

I’ve recorded a video so that you can see what will happen when reindexing and searching:

Conclusion And Further Steps

It’s actually very easy to install Solr and use it along with Magento. By default Solr works pretty fine. However, Solr can work much better by tweaking many different configuration files. We will work on that in following posts.