Magento’s Mini Error Framework

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

If you’re a long time reader, you’re probably aware of my article on Magento’s many 404 pages. It’s been one of the more useful articles I’ve written here — I own the Google results for the phrase “Which 404 page” in Magento land.

One thing I didn’t cover was how Magento displays the store exception 404 page. That’s because the how is an article series unto itself, and I’m starting that series today. Magento’s store exception 404 page, as well as its 503 page and exception/error reporting page are all controlled by a mini framework that lives in the errors/ folder.

Over the next few weeks we’ll explore this mini framework — both from a “how can I customize these misunderstood Magento error pages” point of view, as well as a framework design point of view. Today we’re going to walk through the code path of Magento handling a 404 store exception.

Magento Application Main Exception Block

The best place to start is here

#File: app/Mage.php
public static function run($code = '', $type = 'store', $options = array())
{
    try {
        Varien_Profiler::start('mage');
        self::setRoot();
        if (isset($options['edition'])) {
            self::$_currentEdition = $options['edition'];
        }
        self::$_app    = new Mage_Core_Model_App();
        if (isset($options['request'])) {
            self::$_app->setRequest($options['request']);
        }
        if (isset($options['response'])) {
            self::$_app->setResponse($options['response']);
        }
        self::$_events = new Varien_Event_Collection();
        self::_setIsInstalled($options);
        self::_setConfigModel($options);
        // throw new Mage_Core_Model_Store_Exception('wtf?'); 
        // throw new Exception("Wtf");
        self::$_app->run(array(
            'scope_code' => $code,
            'scope_type' => $type,
            'options'    => $options,
        ));

        Varien_Profiler::stop('mage');
    } catch (Mage_Core_Model_Session_Exception $e) {
        header('Location: ' . self::getBaseUrl());
        die();
    } catch (Mage_Core_Model_Store_Exception $e) {
        require_once(self::getBaseDir() . DS . 'errors' . DS . '404.php');
        die();
    } catch (Exception $e) {
        if (self::isInstalled() || self::$_isDownloader) {
            self::printException($e);
            exit();
        }
        try {
            self::dispatchEvent('mage_run_exception', array('exception' => $e));
            if (!headers_sent() && self::isInstalled()) {
                header('Location:' . self::getUrl('install'));
            } else {
                self::printException($e);
            }
        } catch (Exception $ne) {
            self::printException($ne, $e->getMessage());
        }
    }
}

This is the main Magento run method. This is the method that Magento’s main index.php calls to start Magento’s page/URL processing. There’s a lot of important-but-extraneous-to-us code in there, so let’s consider this simplified version

#File: app/Mage.php
public static function run($code = '', $type = 'store', $options = array())
{
    try {
        //...
        self::$_app    = new Mage_Core_Model_App();
        //...
        self::$_app->run(array(
            'scope_code' => $code,
            'scope_type' => $type,
            'options'    => $options,
        ));
    } catch (Mage_Core_Model_Session_Exception $e) {
        //...
        die();
    } catch (Mage_Core_Model_Store_Exception $e) {
        //...
        die();
    } catch (Exception $e) {
        //...
        die();
    }
}

The Mage::run method is, at its heart, a simple try/catch block. Magento instantiates a Mage_Core_Model_App object, and then calls its run method. If no exceptions occur during processing, everything continues as normal. However, if an exception occurs, three different error/catch branches are considered. One catches a Mage_Core_Model_Session_Exception exception, another looks for a Mage_Core_Model_Store_Exception exception, and the final one looks for a plain old PHP Exception exception (which will also include any exception type not caught above, including the Mage_Core_Exception exception). Magento handles each exception type differently.

The exception type we’ll look at today is the Mage_Core_Model_Store_Exception exception.

Store Exception

All Magento requests have a global “store” object. This object is responsible for keeping track of the currently set store id. Anytime the core code needs to know the store id (to lookup a store specific configuration, price, behavior, etc.) it uses the Magento core/store singleton (a Mage_Core_Model_Store object)

If Magento can’t instantiate this store object, or find a store_id/code/scope-type for the store object, this means there’s something seriously wrong. Most of the application functionality will break under these condition. The Mage_Core_Model_Store_Exception exists to signal that this is the case, and that Magento should shut down the request.

If we take a look at the store exception catch block, we see the following

#File: app/Mage.php
catch (Mage_Core_Model_Store_Exception $e) {
    require_once(self::getBaseDir() . DS . 'errors' . DS . '404.php');
    die();
catch (Exception $e) {

This means Magento will require in the errors/404.php file. This brings us to today’s real topic — Magento’s mini error framework.

Error Processor

If you take a look inside 404.php, you’ll see the following

#File: errors/404.php
require_once 'processor.php';

$processor = new Error_Processor();
$processor->process404();

When considering the case of the Magento store exception above, this looks like a file that requires another file, instantiates an object, and then calls a method. However, consider that this 404.php file is also accessible directly via the browser

http://store.magento.com/errors/404.php

In this context, the Error_Processor object starts to look like a simple controller, and the process404 method starts to look like a corresponding controller action. This idea is strengthened if you look at the 503.php page.

#File: errors/503.php
require_once 'processor.php';

$processor = new Error_Processor();
$processor->process503();

Here we have the same pattern. Magento instantiates the same Error_Processor object (a controller), and calls the process503 method (a controller action).

The rest of this article will walk though the execution of the process404 method, and we’ll start to see other ways these error handling pages/scripts resemble a simple, rudimentary PHP framework.

Processing an Error

If we take a look at the process404 method, we see a pretty simple definition

#File: errors/processor.php    
public function process404()
{
    $this->pageTitle = 'Error 404: Not Found';
    $this->_sendHeaders(404);
    $this->_renderPage('404.phtml');
}

Magento sets a page title property on the object, calls the _sendHeaders method, and then calls the _renderPage method. We’ll return to the pageTitle property and renderPage method later, but first let’s look at the _sendHeaders method.

protected function _sendHeaders($statusCode)
{
    $serverProtocol = !empty($_SERVER['SERVER_PROTOCOL']) ? $_SERVER['SERVER_PROTOCOL'] : 'HTTP/1.0';
    switch ($statusCode) {
        case 404:
            $description = 'Not Found';
            break;
        case 503:
            $description = 'Service Unavailable';
            break;
        default:
            $description = '';
            break;
    }

    header(sprintf('%s %s %s', $serverProtocol, $statusCode, $description), true, $statusCode);
    header(sprintf('Status: %s %s', $statusCode, $description), true, $statusCode);
}

Here we see a simple method that takes a numeric $statusCode, translates it into a description, and then issues two HTTP headers. The first header call

header(sprintf('%s %s %s', $serverProtocol, $statusCode, $description), true, $statusCode);

also uses the $serverProtocol variable. Consider the first line of this method

$serverProtocol = !empty($_SERVER['SERVER_PROTOCOL']) ? $_SERVER['SERVER_PROTOCOL'] : 'HTTP/1.0';

The above code will populate $serverProtocol with HTTP/1.0 or whatever is set in PHP’s $_SERVER['SERVER_PROTOCOL'] variable (mostly likely HTTP/1.1). If we expand the sprintf call, this would look something like

header('HTTP/1.1 404 Not Found', true, 404);

The second header call

header(sprintf('Status: %s %s', $statusCode, $description), true, $statusCode);

is similar, in that it outputs a status header that looks something like this

header('Status: 404 Not Found');

If we use curl to look at the headers of a 404.php request, we’ll see both these headers

$ curl -I 'http://store.example.com/errors/404.php'
HTTP/1.1 404 Not Found
//...
Status: 404 Not Found
//...    
Content-Type: text/html

There’s a few interesting things here worth talking about. First, the initial header call uses the header function’s seldom seen second and third parameters. If we take a look at the function prototype from the docs

void header ( string $string [, bool $replace = true [, int $http_response_code ]] )

We see the second $replace parameter

indicates whether the header should replace a previous similar header, or add a second header of the same type. By default it will replace, but if you pass in FALSE as the second argument you can force multiple headers of the same type.

and the third $http_response_code

Forces the HTTP response code to the specified value. Note that this parameter only has an effect if the string is not empty.

The use of this syntax is puzzling. The $replace variable defaults to true, and the $http_status_code parameter is meant for use with the Location: header and setting 301/302/etc. status codes. Whether these explicit parameters point to the original core team’s inexperience with PHP/HTTP, or their encounters with strange PHP systems in the wild that behaved erratically (or both!) is hard to say.

The second interesting thing is the protocol header being explicitly set to match PHP’s. This seems like a needless bit of extra programming busy work since both 404 and 503 are HTTP 1.0 status codes, and near all web browsers understand them. However, if you scroll down to the comments on the PHP docs, you’ll see this

I had big troubles with an Apache/2.0.59 (Unix) answering in HTTP/1.0 while I (accidentally) added a “HTTP/1.1 200 Ok” - Header. Most of the pages were displayed correct, but on some of them apache added weird content to it: …

Here we have a case where, to an outsider who works on a single projects, Magento’s code looks extra explicit, busy and verbose, but to the Magento core team it looks like code that will help their users deal with an obscure bit of weird apache behavior.

Finally, I don’t know why Magento adds a second, explicit Status header, but given the range of systems Magento needed to work with, I wouldn’t be surprised if it’s another obscure “good enough” edge case bug in some web server or browser.

All these problems are a nice illustration of why, as an application developer, you want your framework of choice handling these sorts of low level details so that all you need to say is “Send a not found header”, and be back to concentrating on your application instead of 15 years of HTTP specifications and edge-casey browser/server behavior.

Rendering the Page

If we jump back to our main “action”

#File: errors/processor.php    
public function process404()
{
    $this->pageTitle = 'Error 404: Not Found';
    $this->_sendHeaders(404);
    $this->_renderPage('404.phtml');
}

After our _sendHeaders call, we have a _renderPage method call. If we’re thinking about this in a framework context, this is our request to render a view layer. If we pop down to the renderPage method

#File: errors/processor.php
protected function _renderPage($template)
{
    $baseTemplate = $this->_getTemplatePath('page.phtml');
    $contentTemplate = $this->_getTemplatePath($template);

    if ($baseTemplate && $contentTemplate) {
        require_once $baseTemplate;
    }
}

we can see rendering a page means

  1. Fetching the path to a base template

  2. Fetching the path to a content template

  3. If we can fetch both paths, then require in the base template, effectively rendering the page.

We’re going to save the actual template path fetching for another time, and tell you the paths fetched are

$baseTemplate    = '/path/to/magento/errors/default/page.phtml
$contentTemplate = '/path/to/magento/errors/default/404.phtml';

If you look at page.phtml (the $baseTemplate), you’ll see a mostly static HTML file. There’s a few lines of PHP that are worth calling out

#File: errors/default/page.phtml
//...
1. <title><?php echo $this->pageTitle?></title>
//...
2. <base href="<?php echo $this->getSkinUrl()?>" />
//...
3. <a href="<?php echo $this->getBaseUrl()?>" 
//
4. <?php require_once $contentTemplate; ?>

The first line prints our page title. You’ll remember back up in process404 we set this page title

#File: errors/processor.php    
public function process404()
{
    $this->pageTitle = 'Error 404: Not Found';
    $this->_sendHeaders(404);
    $this->_renderPage('404.phtml');
}

The second line outputs a URL for the <base/> tag. This lets the browser know where it can find image and css files for the 404 page. The third line outputs a base URL inside of a link. This links end-user-customers back to the main Magento site from the 404 page. Although getSkinUrl and getBaseUrl share names with methods in the main Magento framework, they’re different methods, defined on the Error_Processor class in errors/processor.php. We’ll return to these methods later.

Finally, we reach the most important line

#File: errors/default/page.phtml
<?php require_once $contentTemplate; ?>

The page.phtml file is a base template for the HTML page. The $contentTemplate path is the actual page we’re rendering. You’ll recall we set $contentTemplate up in the _renderPage method. When you include/require a file in PHP inside a method/function, the included file has the same variable context as the method. Also, this is what allows our template to reference the special PHP variable $this, which will point back to the Error_Processor object.

As we mentioned earlier, the $contentTemplate variable points to

$contentTemplate = '/path/to/magento/errors/default/404.phtml';

If we pop this file open, we’ll see a simple HTML fragment that contains our 404 text

<!-- File: errors/default/page.phtml -->
<div id="main" class="col-main">
<!-- [start] content -->
    <div class="page-title">
        <h1>404 error: Page not found.</h1>
    </div>
<!-- [end] content -->
</div>

With this final phtml file included, that completes our page rendering. When invoked via a URL

http://store.magento.com/errors/404.php

the require is the last bit of PHP code invoked. When invoked via the store exception catch block, Magento adds an explicit die to make sure the code exits.

catch (Mage_Core_Model_Store_Exception $e) {
    require_once(self::getBaseDir() . DS . 'errors' . DS . '404.php');
    die();
}

With that, our page rendering is complete.

Wrap Up

Although I’ve described this as a framework, I’m sure some of your are skeptical. Compared to robust systems like Symfony, Laravel, Zend, and Magento proper itself, the Error_Processing class hardly seems like a framework.

With the information presented so far, that’s a fair assessment. However, in our next few articles we’ll discuss some of the framework like features of this class. How it handles different actions, how it has its own global configuration parsing similar to the main Magento framework, and how it’s used for a variety of different error cases.

Beyond that though, it’s sometimes useful to think of all programming projects as having a framework. Even when developers are banging out code in an unfamiliar system under an unrealistic deadline, unwritten rules develop about what code goes where, and how to handle certain common tasks. A framework always develops out of projects longer than a weekend hackathon. Being able to put yourself in the mind of the programmers who worked on a particular sub-system is the first step to working effectively with that sub-system.

Originally published July 24, 2014

Validating a Magento Connect Extension

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

If you’ve followed my work here, on Stack Overflow, or on Twitter, you know I have a perpetual bee in my bonnet over Magento Connect’s file permissions issues. There’s a few common cases where Magento Connect will tell you an extension is installed, but in actuality Connect couldn’t install the extension because of insufficient file permissions. Permissions are always a thorn in a developer’s side — but the Connect application’s willingness to lie to its users about an extension being installed crosses some invisible line of unacceptable behavior.

The Magento 2 team at eBay is aware of the problem, and working on it, but that leaves working Magento developers with the problem of never knowing what’s actually installed when clients or customers use Magento Connect. That’s why I’ve created a new command for n98-magerun called extension:validate. Today we’ll cover how to use the command, as well as discuss some implementation details so you understand what the command is doing.

Using the Command

The extension:validate command requires no arguments. If you run it you’ll see something like this

$ n98-magerun extension:validate

Customchecoutstep
--------------------------------------------------


Pulsestorm_Commercebug
--------------------------------------------------


Auctionmaid_Matrxrate
--------------------------------------------------

Problem: /path/to/magento/app/code/community/Webshopapps/Matrixrate/etc/config.xml
    Hash: MISMATCH
Problem: /path/to/magento/app/etc/modules/Webshopapps_Matrixrate.xml
    Path: FILE NOT FOUND

When invoked with no options, extension:validate will look at all the third party Magento Connect extensions a user has installed on their system, and validate their contents against the package.xml manifest.

In the above example, our system has the Magento Connect extensions Customchecoutstep, Pulsestorm_Commercebug, and Auctionmaid_Matrxrate installed. Customchecoutstep and Pulsestorm_Commercebug were fine, but the Auctionmaid_Matrxrate was missing a file (Webshopapps_Matrixrate.xml), and one of the files didn’t match the stored hash (config.xml). Not matching the stored hash means the file’s been modified/edited.

Troubleshooting Connect Details

While this tool is useful, it’s not a cure all. Once you’ve detected problems with an extension you’ll need to figure out why they happened. In other words, are they real problems or intended changes from more reckless developers. There’s a good chance the original package for an extension will be in the

downloader/.cache/community/

folder. This where Magento Connect downloads the packages prior to installing them. Copying a package from here and extracting the files will let you replace missing files or run a diff to see what’s changed.

The extension:validate command reads the package information from the

downloader/cache.cfg

file. Despite being named cache.cfg, this really isn’t a cache file — it’s the file Magento Connect reads from when it lists out the installed packages on the system, and effectively acts as the source of truth for what is and isn’t installed.

If I speculate, the reason this file is named cache is it’s a cached list of what’s installed on the system. In other words the entire Magento system itself is the actual thing, and this file is just a cache. However, this falls apart when you consider there’s no way to query the Magento system itself for a list of installed packages, and the Magento Connect downloader always reads from this file. Another example of “The best laid plans …” problem.

Command Options

The extension:validate command will also let you check a single extension. Just pass in the extension’s name as the first argument

$ n98-magerun extension:validate Auctionmaid_Matrxrate

Auctionmaid_Matrxrate
--------------------------------------------------

Problem: /path/to/magento/app/code/community/Webshopapps/Matrixrate/etc/config.xml
    Hash: MISMATCH
Problem: /path/to/magento/app/etc/modules/Webshopapps_Matrixrate.xml
    Path: FILE NOT FOUND

This is useful to run right after you’ve installed an extension to make sure it’s installed correctly. Keep in mind you need to use the Magento Connect Name for an extension, and not the Namespace_Modulename module name. (Auctionmaid_Matrxrate vs Webshopapps_Matrixrate above)

If you only want to do a specific type of check — that is only check if a file is missing, or if the hash matches — there’s options to skip each test type.

$ n98-magerun help extension:validate
...
 --skip-file           If set, command will skip reporting the existence of package files
 --skip-hash           If set, command will skip validating the package file hashes

You use them like this.

n98-magerun extension:validate --skip-file 
n98-magerun extension:validate --skip-hash

n98-magerun extension:validate Auctionmaid_Matrxrate --skip-file 
n98-magerun extension:validate Auctionmaid_Matrxrate --skip-hash

There’s also the --full-report command which will list all the files for an extension — not just the problematic ones.

$ n98-magerun extension:validate Auctionmaid_Matrxrate --full-report

Auctionmaid_Matrxrate
--------------------------------------------------

Checking: /path/to/magento/app/code/community/Webshopapps/Matrixrate/Block/Adminhtml/Shipping/Carrier/Matrixrate/Grid.php
    Path: OK
    Hash: OK
Checking: /path/to/magento/app/code/community/Webshopapps/Matrixrate/Block/Adminhtml/System/Config/Form/Field/Exportmatrix.php
    Path: OK
    Hash: OK
//...
Checking: /path/to/magento/app/etc/modules/Webshopapps_Matrixrate.xml
Problem: /path/to/magento/app/etc/modules/Webshopapps_Matrixrate.xml
    Path: FILE NOT FOUND

Useful if you’re curious what, exactly, a Magento Connect package has dropped on your server.

Magento Connect for Upgrades

In addition to installing third party extensions, Magento Connect’s other big feature is enabling package based updates of the Magento system itself. It does this by having the following 36 packages installed by default when you initialize a Magento Connect system.

'Cm_RedisSession','Interface_Adminhtml_Default','
Interface_Frontend_Base_Default','Interface_Frontend_Default','
Interface_Frontend_Rwd_Default','Interface_Install_Default','Lib_Cm','
Lib_Credis','Lib_Google_Checkout','Lib_Js_Calendar','Lib_Js_Ext','
Lib_Js_Mage','Lib_Js_Prototype','Lib_Js_TinyMCE','Lib_LinLibertineFont',
'Lib_Mage','Lib_Magento','Lib_Phpseclib','Lib_Varien','Lib_ZF','
Lib_ZF_Locale','Mage_All_Latest','Mage_Centinel','Mage_Compiler','
Mage_Core_Adminhtml','Mage_Core_Modules','Mage_Downloader','
Mage_Locale_de_DE','Mage_Locale_en_US','Mage_Locale_es_ES','
Mage_Locale_fr_FR','Mage_Locale_nl_NL','Mage_Locale_pt_BR','
Mage_Locale_zh_CN','Magento_Mobile','Phoenix_Moneybookers'

This has always struck me as a little bit crazy. Because Magento Connect installations aren’t transactional, this creates a situation where an install that fails due to network connectivity issues or un-handled corner cases can leave a Magento installation in a “half-upgraded” state. If you’ve spent any time on the Magento forums or the various Stack Exchange sites, you know this is more than a theoretically concern.

Regardless, if you or your client are managing upgrades via Magento Connect, you may include these packages with the --include-default flag.

$ n98-magerun extension:validate --include-default

Be warned though, since it’s effectively running through every Connect managed file in your Magento application it’ll take a bit longer to finish. This is also a potentially useful way to keep your application “core hack” free. Finally, these extension names are hard-coded into the extension:validate command. If Magento adds any packages in the future, extension:validate will see them as 3rd party extensions until the n98-magerun command is updated with a new extension list. If anyone know a way to query something for a list of official packages, pull requests are welcome.

Wrap Up

The extension:validate command is available in the develop branch of the n98-magerun source. It looks like the application is getting monthly-ish releases, so this should be available in the official release soon. If you run into problems or have feature suggestions, please let me know.

Originally published June 9, 2014

Commerce Bug 2.3.1 with Theme Inheritance

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

In case you missed it in the waterfall of post Magento Imagine press releases, I’ve got a new version of Commerce Bug 2 available. Free for registered users — this update brings support for the theme inheritance features of Magento CE 1.9 and Magento EE 1.14. You can read more on the Pulse Storm press release blog. Actually, there’s not much more there — but if you want notices when I update an extension or other product, you should defiantly subscribe.

If you’re new to Magento or new to my work, Commerce Bug is my commercial Magento debugging extension. It provides programmers, developers, system administrators, and designers with the sort of diagnostic information they needed to work with unfamiliar Magento systems. It’s useful for newbies and old-hands alike. The developing Magento with Commerce Bug article series demonstrate how a developer can use Commerce Bug in their job day to day. If you’re being paid to do Magento work Commerce Bug will pay for itself it the first day of use.

I released Commerce Bug four years ago at the first volcano interrupted Magento Imagine. Since then I’ve been slowly but steadily updating it — adding new features and keeping up to date with Magento’s various version changes. Charging for the software lets me provide my users with top notch support. It also allows me the sort of career where I can do the deep research need to write the articles you see here, answer questions on Stack Exchange sites, and contribute other projects to the open source community. Without that income I couldn’t do what I do.

I try not to say it too often, but if you’ve gotten any value out of my Magento work over the past half-decade, I’d really appreciate it if you took a look at the products I sell and give one a try. Your support, as always, is appreciated.

Originally published June 4, 2014