Magento 2: Understanding Access Control List Rules

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

The Magento backend application, (sometimes called “The Admin” or adminhtml area), is where a system owner manages their Magento store. This is where users interact with web forms to add new products, change configurations, etc. Magento is a multiuser application — i.e. a business owner may have a backend account for herself, but also give each individual member of her staff an account to access the Magento backend. Furthermore, that business owner can turn off features for different accounts.

For example, the customer support staff may only have access to the customer and orders sections, while the sales staff may have access to both these sections and the marketing section. In the Magento backend, a system owner can accomplish this via the System -> Users and Systems -> Roles sections. These two sections implement an authentication and authorization system.

For those of you too busy to read the wikipedia article, authentication is the act of ensuring a user is who they say they are. In simplified terms, this is the user entering an account name and password in a login screen. Systems with a higher level of general security or special PCI compliance requirements are often required to implement two factor authentication. A common two factor authentication process is a password, combined with an SMS/Text message sent to their phone.

Once a user proves who they are, the next step is an authorization system. Authorization systems implement rules that say what a user is allowed to do in a system. Magento’s authorization system allows a system owner to

  1. Create an unlimited number of logical Roles. Some example of roles might include Sales Staff, Support Staff, IT Staff, Contract Developers, Executive Team, etc.

  2. Assign a set of Access Control List (ACL) rules to each individual role

Each access control rule defines a specific permission granted to the user in the system. You can see a list of these rules by navigating to

System -> User Roles -> Add/Edit Role -> Role Resources

and selecting Custom from the drop down menu

Each individual rule controls access to a system feature. Tailoring a set of rules into a set of Roles that an individual business can use to run their online store is one of the many things a Magento system integrator or store owner will need to do.

The special Resource Access: all role is a super user role. These users are granted access to every resource in the system.

ACL Rules for Developers

As a module developer, ACL rules present a few interesting challenges. First, there are several places that you, as a module developer, are expected to add ACL rule checks to your module. A few examples

  1. Every URL endpoint/controller in the admin application must implement an _isAllowed method that determines if a user can access the URL endpoint.

  2. Every Menu Item in the left hand navigation also has a specific ACL rule that controls whether or not the menu displays for the logged in user. This is often the same rule from _isAllowed)

  3. Every configuration field in System -> Configuration has a specific ACL rule that controls whether or not the menu displays

Despite being required fields, there are no hard and fast rules as to how a module developer should setup and structure their own rules. Also, a module developer will likely want additional rules that are specific to their module. This article can’t answer these hard questions for you, but we will show you how to check the current user against a specific ACL rule, look up ID values for existing rules, and how to create your own tree of ACL rules.

Looking up a Rule ID

In Magento 2, each specific ACL rule is assigned an arbitrary string ID. You can find these IDs in a module’s etc/acl.xml file. For example, Magento defines the Stores -> Settings -> Google API ACL rule here

<!-- File: vendor/magento/module-google-analytics/etc/acl.xml -->
<?xml version="1.0"?>
<!--
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Acl/etc/acl.xsd">
    <acl>
        <resources>
            <resource id="Magento_Backend::admin">
                <resource id="Magento_Backend::stores">
                    <resource id="Magento_Backend::stores_settings">
                        <resource id="Magento_Config::config">
                            <resource id="Magento_GoogleAnalytics::google" title="Google API" />
                        </resource>
                    </resource>
                </resource>
            </resource>
        </resources>
    </acl>
</config>

You can find the specific rule by its title attribute. If you’re having trouble, it’s this one

<!-- File: vendor/magento/module-google-analytics/etc/acl.xml -->    
<resource id="Magento_GoogleAnalytics::google" title="Google API" />    

The ACL rule with the title “Google API” has an ACL ID of Magento_GoogleAnalytics::google. This is the ID you’ll use to check if the logged in user has access to this resource.

ACL IDs are mostly arbitrary strings. By convention, a Magento 2 ACL rule ID name is

  1. The name of the module
  2. Followed by two colon characters ::
  3. Followed by a lower case string describing the rule’s purpose

Also, Magento’s new XSD/XML validation system

<!-- File: vendor/magento/framework/Acl/etc/acl.xsd -->
<xs:simpleType name="typeId">
    <xs:annotation>
        <xs:documentation>
            Item id attribute can has only [a-z0-9/_]. Minimal length 3 symbol. Case insensitive.
        </xs:documentation>
    </xs:annotation>

    <xs:restriction base="xs:string">
        <xs:pattern value="([A-Z]+[a-zA-Z0-9]{1,}){1,}_[A-Z]+[A-Z0-9a-z]{1,}::[A-Za-z_0-9]{1,}" />
    </xs:restriction>
</xs:simpleType>

forces the rule to conform to the following regular expression

([A-Z]+[a-zA-Z0-9]{1,}){1,}_[A-Z]+[A-Z0-9a-z]{1,}::[A-Za-z_0-9]{1,}

All in all, while it’s possible to deviate slightly from Magento’s convention, it’s best to stick to it.

If you’re curious about the placement of the Magento_GoogleAnalytics::google rule inside the node structure of acl.xml files (i.e. what are its parent nodes), an acl.xml file is an infinitely deep XML tree of <resource/> nodes, with each node defining a new level in the ACL hierarchy. If that didn’t make sense, don’t worry, we’ll be running through a few examples later that should clear things up.

Before we move on to creating our own ACL rules, Magento 1 developers will want to take note. Magento 1 ACL rules did not have an explicit ID. Instead, the system derived an ID from the names of the XML nodes that defined the rules (i.e. foo/baz/bar). This means, in Magento 2, you can no longer look at an ID and be sure where it lives in the hierarchy, you can only be sure of the module that created it.

When I’m hunting through a Magento 2 installation for a specific ACL rule, I use a number of different unix commands in a terminal to search through the acl.xml files. For example, to find the Google API rule above, I used the following

$ find vendor/magento/ -name 'acl.xml' -exec grep -i 'Google API' '{}' +
vendor/magento//module-google-analytics/etc/acl.xml:
<resource id="Magento_GoogleAnalytics::google" title="Google API" />

This searches every acl.xml in vendor/magento for the string Google API.

Creating Your Own ACL Rules

When you create Magento admin features, you’ll need to add ACL rules to your module. We’re going to show you how to do this using pestle, and then explain what each file pestle created is for. If you’re not familiar with it, pestle is a free and open source PHP command line system that features a number of useful Magento 2 code generation tools.

First, we’ll want to create a blank module named Pulsestorm_AclExample. You can create the base module files using pestle’s generate_module command.

$ pestle.phar generate_module Pulsestorm AclExample 0.0.1

and then enable the module in Magento by running the following two commands

$ php bin/magento module:enable Pulsestorm_AclExample
$ php bin/magento setup:upgrade    

If you’re interested in creating a module by hand, or curious what the above pestle command is actually doing, take a look at our Introduction to Magento 2 — No More MVC article.

Next, we’re going to use pestle’s generate_acl command.

# with interactive input

$ pestle.phar generate_acl
Which Module? (Pulsestorm_HelloWorld)] Pulsestorm_AclExample
Rule IDs? (Pulsestorm_AclExample::top,Pulsestorm_AclExample::config,)] 
Created /path/to/magento/app/code/Pulsestorm/AclExample/etc/acl.xml

#without interactive input
$ pestle.phar generate_acl Pulsestorm_AclExample Pulsestorm_AclExample::top,Pulsestorm_AclExample::config

The first argument (Which Module?) lets pestle know which module’s acl.xml file it should use or create. The second argument (Rule IDs), is a comma separated list of Rule IDs — each comma represents a <resource/> sub-node in acl.xml.

If that didn’t make sense, open up the generated acl.xml

<?xml version="1.0"?>
<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Acl/etc/acl.xsd">
    <acl>
        <resources>
            <resource id="Magento_Backend::admin">
                <resource id="Pulsestorm_AclExample::top" title="TITLE HERE FOR">
                    <resource id="Pulsestorm_AclExample::config" title="TITLE HERE FOR"/>
                </resource>
            </resource>
        </resources>
    </acl>
</config>

Here you can see pestle created a Pulsestorm_AclExample::top rule that’s a parent of the Pulsestorm_AclExample::config rule (i.e. the first two rules in our comma separated list). You’ll also notice pestle created both these rules under the Magento_Backend::admin resource — all Magento ACL rules go under this node — it’s the top level node.

Let’s edit this file to give our rules some less generic titles

<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    

<resource id="Pulsestorm_AclExample::top" title="Pulse Storm ACL Example Module">
    <resource id="Pulsestorm_AclExample::config" title="The First Rule"/>
</resource>

And, while we’re at it, lets add a second rule!

<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    

<resource id="Pulsestorm_AclExample::top" title="Pulse Storm ACL Example Module">
    <resource id="Pulsestorm_AclExample::config" title="The First Rule"/>
    <resource id="Pulsestorm_AclExample::more_rules" title="The Second Rule"/>        
</resource>

With the above in place, clear your cache, and take a look at the Roles menu at System -> User Roles -> Add/Edit Role -> Role Resources. You should see your rules added to the system

That’s all there is to it! Your rules can now be assigned to admin roles in the system, and those admin roles can be assigned to users.

Programmatically Checking ACL Rules

The following assumes some base familiarity with Magento’s object and automatic constructor dependency injection systems, although if you’re not 100% comfortable with these concepts, just power through for the code snippets you’ll need!

Magento provides an abstract type, Magento\Framework\AuthorizationInterface, which a client programmer (you!) can use to validate the currently logged in user against a specific access control rule. i.e., if you were playing fast and loose with Magento’s Don’t use the Object Manager guidelines, the following

$auth = $object_manger->get('Magento\Framework\AuthorizationInterface');
if($auth->isAllowed('Pulsestorm_AclExample::config'))
{
    //user is logged in here
}
else
{
    //user is not logged in here
}

would check if the currently logged in user was assigned our Pulsestorm_AclExample::config rule. If you’re not playing fast and loose with Magento’s Don’t use the Object Manager guidelines, you can inject the auth checking object with something like this

public function __construct(Magento\Framework\AuthorizationInterface $auth)
{
    $this->authorization = $auth;
}

If you’re in a controller that extends the \Magento\Backend\App\Action controller, you automatically have access to the authorization checking object via the _authorization property.

namespace Pulsestorm\HelloAdmin\Controller\Adminhtml\Index;
class Index extends \Magento\Backend\App\Action
{
    protected function someControllerMethod()
    {
        return $this->_authorization->isAllowed('Pulsestorm_HelloAdmin::pulsestorm_helloadmin_index_index');
    }            

}

Regarding the controller method above — if you’re injecting additional arguments via the __construct method, don’t forget to include the admin context object (Magento\Backend\App\Action\Context). This context object is where the auth checking object is, itself, instantiated and injected.

class Index extends \Magento\Backend\App\Action
{
    protected $resultPageFactory;
    public function __construct(
        \Magento\Backend\App\Action\Context $context,
        \Magento\Framework\View\Result\PageFactory $resultPageFactory)
    {
        $this->resultPageFactory = $resultPageFactory;        
        return parent::__construct($context);
    }
    //...
}

Finally, for the curious, in a stock Magento install (circa spring 2016), the Magento\Framework\AuthorizationInterface object type resolves to a Magento\Framework\Authorization object. The class for this object is found here

#File: vendor/magento/framework/Authorization.php

If you’re having trouble with ACL rule debugging, this is where you’ll want to start.

Wrap Up

Access Control Rules are an important, but often overlooked part, of Magento extensions. While an extension’s functionality and ability to solve a business or technical problem is paramount, giving your extension users the ability to turn certain admin features on and off for certain users can often be the differences between a manager or store owner choosing your extension over a competitor’s.

However, even if you don’t want to slice your extension’s functionality into narrow bands, there are places in the Magento Admin where you’ll need to add ACL rules. Next time we’ll explore one of these sections, when we cover how to create Magento Admin MCV/MVVM controller endpoints.

Originally published May 2, 2016

Magento 2: Composer and Components

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

One question I keep getting from new Magento 2 developers, (and we’re all new Magento 2 developers) is, “How should I organize my project files?”. Since Magento has heavily restructured code organization around Composer, it’s not always clear how files should be organized during development, organized for distribution, and how (if at all) to move between these two modes.

While this article won’t definitively answer those questions, we will dive into some of the formalization behind Magento 2 components, as well as how Magento 2’s Composer integration works. With this information in hand, you should be able to come up with a project structure that works for you and your team.

Magento 2 Components

Magento 1 had an informal, inconsistent idea of components. The best way to think about Magento components is

A group of source files, in various formats, with a single purpose in the system

If that’s a little vague, see our previous comments about informal and inconsistent. Speaking more concretely, the four component types in Magento are

  • Modules
  • Themes
  • Language Packs
  • Code Libraries

In Magento 1, modules were pretty well defined and self contained

A folder of files with an etc/config.xml, with developers making Magento aware of the module via an app/etc/modules/Package_Namespace.xml file

Themes were a little less defined and a little less self contained

A collection of files under app/design/[area]/[package]/[name], with developers making Magento aware of the theme via a setting in core_config_data. Unless it’s the admin theme in which case you need a module. Also, modules are responsible for adding the layout XML files to themes. Also, while we’re here, modules aren’t all that self contained either because if they want to use phtml templates then the templates need to be in a theme folder

Things start to get really vague with language packs

A collection of key/value csv files located in app/locale/[language]/Packagename_Modulename.csv. Also we’re just going to drop email templates in here because reasons

And Magento 1 barely had the concept of a generic code library.

Um, yeah, maybe just drop them in lib and add that path to PHP’s autoloader? And look in the code pool folders too? And maybe just add a top level js folder for javascript libraries? Unless they go in skin?

Oh right! Skins! Magento 1 also had a (now dropped) concept of skins. Skins were best defined as

Any CSS or javascript file that doesn’t belong in a theme.

Where CSS or javascript file that doesn’t belong in a theme was defined as

Any CSS or javascript file that doesn’t belong in a skin

While, in practice, norms developed over time and development wasn’t as chaotic as I’m describing, Magento 1’s lack of formalization around components did make the system harder to work with, particularly if you were trying to redistribute Magento 1 code for reuse.

Magento 2 formalizes the idea of components, and this formalization means the core Magento system, and other external systems (i.e. Composer) can deal with these components in a sane and reasonable way.

Magento 2 Components

In Magento 2, a component is

A group of files, under a top level directory (with sub-folders allowed), with a registration.php file defining the type of component.

That’s it. There’s nothing about how the components work, interact with the system, or interact with other components. Those things aren’t the concern of the component system.

As of this writing, there are four component types in Magento 2

  • Modules
  • Themes
  • Libraries
  • Language Packs

Let’s take a look at this in action. Consider the Magento_AdminNotification module. This is a collection of files, under a top level directory (AdminNotification), with a registration.php file. If we take a look at registration.php

#File: app/code/Magento/AdminNotification/registration.php
<?php
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_AdminNotification',
    __DIR__
);

We can see this file registers a component via the static \Magento\Framework\Component\ComponentRegistrar::register method. This component is a module (\Magento\Framework\Component\ComponentRegistrar::MODULE), its identifier is Magento_AdminNotification, and you can find its files in the __DIR__ folder, (i.e. the same directory registration.php is in via PHP’s magic __DIR__ constant).

Next, consider the Luma theme. Again, a collection of files, under a top level directory (luma), with a registration.php file.

#File: app/design/frontend/Magento/luma/registration.php
<?php
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::THEME,
    'frontend/Magento/luma',
    __DIR__
);

Here the component type is a theme (\Magento\Framework\Component\ComponentRegistrar::THEME), and its name is frontend/Magento/luma.

Even though the Magento GitHub project has these files in familiar locations, (app/code, app/design, etc.), thanks to Magento 2’s new component system, these directories can be located anywhere, so long as as the module, theme, library, or language pack correctly defines its registration.php file.

How Magento Loads Components

At this point, the systems minded among you are probably wondering how Magento 2 loads and identifies components. It’s one thing to say so long as the module, theme, library, or language pack correctly defines its registration.php file, but the system still needs to load these files, and that means there are rules.

In order to get Magento to recognize your module, theme, code library, or language pack (i.e. your component), you need Magento to read your component’s registration.php file. There are two ways to get Magento to read your registration.php file

  1. Place your component in one of several predefined folders
  2. Distribute your module via Composer, and use Composer’s autoloader features

Of the two methods, the second is the preferred and recommended way of distributing Magento 2 modules. For development, the first offers a convenient way to get started on a component, or checkout/clone a version control repository to a specific location. The first also offers a non-composer way for extension developers to distribute their components.

Predefined Folders

At the time of this writing, Magento 2 will scan the following folders/files for components (the patterns below are for the glob function).

app/code/*/*/cli_commands.php
app/code/*/*/registration.php
app/design/*/*/*/registration.php
app/i18n/*/*/registration.php
lib/internal/*/*/registration.php
lib/internal/*/*/*/registration.php

This is what allows you to place modules in app/code/Packagename/Modulename, or themes in app/design/[area]/[package]/[name], etc. Magento will explicitly look for registration.php files to load at these locations. Also of interest are the app/code/*/*/cli_commands.php files — this appears to be a way for a module to register command line classes without using di.xml.

It’s not clear if these folders were added as a stop-gap measure while Magento 2 gets everyone moved over to Composer distribution, or if they’ll stick around for the long term. If you’re curious, Magento does this registration check in the following file

#File: app/etc/NonComposerComponentRegistration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

$pathList[] = dirname(__DIR__) . '/code/*/*/cli_commands.php';
$pathList[] = dirname(__DIR__) . '/code/*/*/registration.php';
$pathList[] = dirname(__DIR__) . '/design/*/*/*/registration.php';
$pathList[] = dirname(__DIR__) . '/i18n/*/*/registration.php';
$pathList[] = dirname(dirname(__DIR__)) . '/lib/internal/*/*/registration.php';
$pathList[] = dirname(dirname(__DIR__)) . '/lib/internal/*/*/*/registration.php';
foreach ($pathList as $path) {
    // Sorting is disabled intentionally for performance improvement
    $files = glob($path, GLOB_NOSORT);
    if ($files === false) {
        throw new \RuntimeException('glob() returned error while searching in \'' . $path . '\'');
    }
    foreach ($files as $file) {
        include $file;
    }
}    

If you’re researching how a future version of Magento 2 handles scanning for components, this would be a good place to start.

Composer Distribution

The other way to have Magento notice your component is to distribute your component via Composer. We’re going to assume you have a basic familiarity with Composer, but for the purposes of this article, all you really need to know is

Composer allows you to ask for a package of PHP files, and have that package downloaded to the vendor/ folder

If you’re interested in learning more about Composer, the previous articles in this series, my Laravel, Composer, and the State of Autoloading, and the Composer manual are a good place to start.

So, assuming you have your Magento component in GitHub (or a different source repository Composer can point at), and your component has a registration.php file, the only question left is How do we get Magento to look at our registration.php file.

Rather than have Magento scan all of vendor/ for registration.php files, (an approach that could quickly get “O^N out of hand” as the number packages grows), Magento uses Composer’s file autoloader feature to load each individual component’s registration.php file.

If that didn’t make sense, an example should clear things up. Assuming you’ve installed Magento via the Composer meta-package, or installed it via the archive available via magento.com (which is based on the meta-package), take a look at the catalog module’s composer.json file.

#File: vendor/magento/module-catalog/composer.json
{
    "name": "magento/module-catalog",
    //...
    "autoload": {
        "files": [
            "registration.php"
        ],
        "psr-4": {
            "Magento\\Catalog\\": ""
        }
    }
    //...
}

The autoload section is where you configure the PHP class autoloader for a Composer package. This is covered in great detail in my Laravel, Composer, and the State of Autoloading series. The section we’re interested in today is here

#File: vendor/magento/module-catalog/composer.json
"files": [
    "registration.php"
],

The files autoloader section was originally intended as a stop gap measure for older PHP packages that had not moved to a PSR-0 (and later, PSR-4) autoloader system. Composer’s autoloader (not during install or update, but when your application is running) will automatically include any files listed in files (with the specific package as the base directory), and package developers can do whatever they need to do to setup their pre-PSR autoloaders.

Over the years, many frameworks have taken the simplicity and flexibility of the files autoloader and turned it to different purposes. Magento 2 is no exception. The above autoload configuration ensures Composer will always load the file at

#File: vendor/magento/module-catalog/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_Catalog',
    __DIR__
);    

This, as we’ve already learned, will register the component. The same holds true for third party components (i.e. yours!) — make sure you’ve created a registration.php file with the correct registration code for your component type, and then include an identical files autoloader.

Here’s an example of each component type from Magento’s core.

Module

#File: vendor/magento/module-weee/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_Weee',
    __DIR__
);

Theme

#File: vendor/magento/theme-frontend-luma/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::THEME,
    'frontend/Magento/luma',
    __DIR__
);

Library

#File: vendor/magento/framework/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::LIBRARY,
    'magento/framework',
    __DIR__
);

Language Pack

#File: vendor/magento/language-de_de/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::LANGUAGE,
    'magento_de_de',
    __DIR__
);

Invalid Assumptions

There’s one last important thing to take away from this, even if you’re not responsible for packaging your company’s Magento work. It’s no longer safe to make assumptions about where a folder is located in located in the Magento hierarchy. If you’re trying to find a specific file in Magento, it’s more important than ever to learn your way around the Magento\Framework\Module\Dir helper class.

Daily Work

So, now that we have a better understanding of what a component is, and how Magento loads components into the system, that still leaves us with our original question. Where should the code for our in progress Magento projects go? How should we store our projects in source control?

Unfortunately — there’s no clear answer, and a lot will depend on the sort of project you’re working on. Are you an extension developer? A theme developer? A system integrator/store builder or someone integrating with a Magento system? Do you want your working source repository to be the same repository Composer reads from? What tooling is your team is familiar with? While there certainly are approaches that are “better” for each scenario, from a programmer’s point of view Magento 2’s still too new to know for sure.

For what its worth, I’ve been creating symlinks to my source repositories so that NonComposerComponentRegistration.php finds my components, using a build process to create the final Composer accessible repository, and temporarily patching over any issues Magento has with symlinks.

Part of being a Magento 2 developer will be figuring this out for your own team, even if you’re just a team of one.

Originally published April 22, 2016

Magento 2: Composer Plugins

Like this article? Frustrated by Magento? Then you’ll love Commerce Bug, the must have debugging extension for anyone using Magento. Whether you’re just starting out or you’re a seasoned pro, Commerce Bug will save you and your team hours everyday. Grab a copy and start working with Magento instead of against it.

In our last article, we talked a bit about Magento 2’s use of Composer, and touched on the Composer meta-package installation method. One high level take away was, when you use Composer’s create-project method to start a new Magento project, you are

  1. Fetching the latest version of a specific Composer package (magento/project-community-edition) hosted at repo.magento.com

  2. Running composer install for that project’s composer.json file

The curious among you may have noticed something strange about this. If we use the --no-install option to skip step #2 above (i.e. only fetch magento/project-community-edition),

$ composer create-project --no-install --repository-url=https://repo.magento.com/ magento/project-community-edition

we’ll see a pretty sparse project folder

$ ls project-community-edition
README.md   composer.json   update    

The README.md file is a basic Welcome to Magento affair, and the composer.json file contains the actual Composer packages you’ll need to install Magento. That’s why magento/project-community-edition is called a “meta” package — it’s not he actual package, and most software engineers aren’t english or philosophy majors and enjoy stretching the definition of “meta”.

The update folder contains a snapshot of the Magento Component Manager application (i.e Market Place updater), which is a separate project from the Magento core. Why this feature is a separate application, and why Magento distributes it like this is a story for another time.

All in all, relatively straight forward. However, after running composer install (or running create-project without the --no-install flag), we end up with the following

$ ls project-community-edition/
CHANGELOG.md                        dev
CONTRIBUTING.md                     index.php
CONTRIBUTOR_LICENSE_AGREEMENT.html  lib
COPYING.txt                         nginx.conf.sample
Gruntfile.js                        package.json
LICENSE.txt                         php.ini.sample
LICENSE_AFL.txt                     phpserver
README.md                           pub
app                                 setup
bin                                 update
composer.json                       var
composer.lock                       vendor

That’s a huge difference. A plethora of files and folders. Many of these files are necessary for Magento to run, others are informational, and still others are sample configurations. However, if you’re new to the Composer eco-system, you may be wondering

Where the heck did all these extra files and folders come from? I thought Composer would only update files in /vendor.

Today we’re going to explain how these files get here, and why you’ll need to be hyper aware of this as a Magento 2 developer. To start, we’ll need to dive into some less known features of Composer

Composer: Plugins and Scripts

Composer bills itself as a Dependency Manager for PHP. While this is true, and dependency management is an important part of a PHP project, Composer is really a foundational framework for PHP development, and serves the same role that linkers do in the C/C++ world.

Yes yes, I know, from a computer science point of view linkers and Composer couldn’t be further apart. However, the end result of a linker is, the C programmer stops needing to worry about how they incorporate code from other libraries into their program. In a similar way, Composer does the same thing for PHP — if a project conforms to what Composer expects in terms of directory structure and autoloading, and a PHP developer conforms to what Composer expects from a PHP program (i.e., includes the Composer autoloader), the developer stops needing to worry about how they should include other people’s code in their own systems.

When considered from this point of view — that Composer is, itself, just another programmatic framework that your code sits on top of — it makes more sense that Composer would have a plugin system for changing, altering, and extending its behavior. There are two main systems programmers have for altering the behavior of Composer. These systems are scripts, and plugins.

Scripts and plugins share a base set of concepts, but have a few key distinctions. Scripts provide a way, in the project composer.json file, to take additional programmatic action when composer triggers certain events. These events are listed in the Composer manual, and include things like pre-and-post composer install running.

Plugins, on the other hand, provide the same mechanism for individual packages that are part of a larger project. In addition to listening for Composer events, plugins also have the ability to modify composer’s installation behavior.

Put another way, you configure scripts in your main composer.json file, you (or third parties) configure plugins in composer.json files that live in the vendor/ folder.

While understanding both systems is important for a well rounded Composer developer, today we’re going to focus on the plugin system.

Plugin Example

Rather than try to describe things from scratch, we’ve created a simple Composer plugin that should demonstrate the plugin lifecycle, and help you understand what Magento 2 is doing with Composer plugins.

If you take a look at the plugin class

#File: src/Plugin.php
//...
public static function getSubscribedEvents()
{
    return array(
        'post-install-cmd' => 'installOrUpdate',
        'post-update-cmd' => 'installOrUpdate',            
    );
}    
//...    
public function installOrUpdate($event)
{
    file_put_contents('/tmp/composer.log', __METHOD__ . "\n",FILE_APPEND);
    file_put_contents('/tmp/composer.log', get_class($event) . "\n",FILE_APPEND);            
}

you can see that this plugin listens for the post-install-cmd and post-update-cmd events. You tell a plugin which events it should listen to by defining a getSubscribedEvent method that returns an array in the above format. Keys are the event, and values are the method, (on the plugin class), that Composer calls as an observer.

In our case, both the post install and post update events call the installOrUpdate method, and this method logs some simple information to the /tmp/composer.log file in our temp directory.

The plugin class also has an activate method.

#File: src/Plugin.php
public function activate(Composer $composer, IOInterface $io)
{
    file_put_contents('/tmp/composer.log', __METHOD__ . "\n",FILE_APPEND);
}

Composer calls the activate method when it detects the plugin every time Composer runs. The activate method is where you instantiate any other objects your plugin will need. In our case, we’ve added a line to log when the method is called.

All in all, a mostly useless plugin, but one that’s useful to diagnose how plugins work.

Adding a Plugin to Your Project

Adding a plugin to your project is the same as adding any other Composer package to your project. Create a new folder

$ mkdir test-plugin
$ cd test-plugin

and then create the following composer.json file in that folder.

//File: composer.json
{
    "repositories":[
        {
            "type":"vcs",
            "url":"git@github.com:astorm/composer-plugin-example.git"
        }
    ],
    "require":{
        "pulsestorm/composer-plugin-example":"0.0.1"
    }
}

In the require section we’ve added our plugin (pulsestorm/composer-plugin-example) and the desired version (0.0.1). The plugin’s name comes from the plugin’s composer.json file file. The version, 0.0.1, comes from the tagged releases.

Since I didn’t create a packagist.org listing for this package, we need the repositories section. This tells Composer to use the git (vcs/version control system) repository at the provided URL as a repository.

This Composer file in place, there’s one last thing we’ll want to do before we run composer install. In a separate terminal window, run the following.

$ touch /tmp/composer.log
$ tail -f /tmp/composer.log

This creates our composer.log file, and then tails it. Tailing a file means showing the last few lines of output. When we run tail with the -f option, we’re telling tail to show us the last line of the file whenever the file is changed. This is a decades old technique for monitoring log files in the *nix world.

Composer Plugin Lifecycle

OK! We’re ready to install our simple project. Run

$ composer install

and composer will install the plugin.

$ composer install
Loading composer repositories with package information
Updating dependencies (including require-dev)                       
  - Installing pulsestorm/composer-plugin-example (0.0.1)
    Loading from cache

Writing lock file
Generating autoload files

More interesting to us though is the output in our /tmp/composer.log file.

Pulsestorm\Composer\Example\Plugin::activate
Pulsestorm\Composer\Example\Plugin::installOrUpdate
Composer\Script\Event

Here, we see Composer called the activate method, and then (per our events) called the installOrUpdate method. If we were to run update

$ composer update

We’d see the same lines

Pulsestorm\Composer\Example\Plugin::activate
Pulsestorm\Composer\Example\Plugin::installOrUpdate
Composer\Script\Event    

because we’re also listening for the update event.

A Composer plugin developer can, (via the Composer\Script\Event object Composer passed to our handler or the Composer\Composer object Composer passes to the active method), examine and change Composer’s state at run time, implementing all sorts of extra functionality whenever a Composer project updates.

Covering that functionality in full is beyond the scope of this article, but with Composer being open source, there’s nothing stopping you from diving right in.

What Makes a Package a Plugin?

As we mentioned earlier, a Composer plugin is just a standard Composer package. However, it’s a standard Composer package with a special composer.json file. Let’s take a look at our plugin’s composer.json file.

//File: composer.json
{
    //...
    "type": "composer-plugin",
    //...        
    "require": {
        "composer-plugin-api": "^1.0"
    },
    "autoload":{
        "psr-4":{
            "Pulsestorm\\Composer\\Example\\":"src/"
        }
    },    
    "extra":{     
        "class":"Pulsestorm\\Composer\\Example\\Plugin"
    }
    //...
}

The first configuration a plugin package needs is the following

//File: composer.json

"type": "composer-plugin"

This tells Composer that this is a plugin package.

The second configuration a plugin package needs is

//File: composer.json

"require": {
    "composer-plugin-api": "^1.0"
},    

This looks like a standard Composer require — but it’s not. When Composer encounters a package named composer-plugin-api, this indicated which Plugin API version your plugin targets.

Finally, in the extra section, the following configuration

//File: composer.json

"extra":{     
    "class":"Pulsestorm\\Composer\\Example\\Plugin"
}

points to our plugin class (Pulsestorm\Composer\Example\Plugin). Since Composer will need to instantiate this class, that means you’ll need something in your autoload section that ensures PHP will load the class definition file. In our case, we used a standard PSR-4 autoloader

//File: composer.json

"autoload":{
    "psr-4":{
        "Pulsestorm\\Composer\\Example\\":"src/"
    }
}

That’s all you’ll need for a Composer plugin!

Finding Magento 2’s Composer Plugins

Now that we have a better understanding of Composer plugins, we can come back to our Magento problem. As a reminder, we’re trying to figure out how the stock create-project files

$ ls project-community-edition
README.md   composer.json   update 

become a full fledged, many files outside of vendor, Magento installation

$ ls project-community-edition/
CHANGELOG.md                        dev
CONTRIBUTING.md                     index.php
CONTRIBUTOR_LICENSE_AGREEMENT.html  lib
COPYING.txt                         nginx.conf.sample
Gruntfile.js                        package.json
LICENSE.txt                         php.ini.sample
LICENSE_AFL.txt                     phpserver
README.md                           pub
app                                 setup
bin                                 update
composer.json                       var
composer.lock                       vendor

If it’s not obvious by now, these additional files are placed here by a plugin in one of the Composer packages that make up Magento 2.

Unfortunately, Composer doesn’t provide an easy way to check your project for any installed plugins. You’ll need to use some good old fashioned unix command line searching to figure out which Magento packages have plugins.

In plain english, we’ll want to

  1. Create a list of all our project’s composer.json files
  2. Search those files for the all important "type": "composer-plugin", text

In unix english, that’s

$ find vendor/ -name composer.json | xargs grep 'composer-plugin'
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v1/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v1/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v2/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v2/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v3/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v3/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v4/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v4/composer.json:        "composer-plugin-api": "1.0.0"
vendor//magento/magento-composer-installer/composer.json:    "type":"composer-plugin",
vendor//magento/magento-composer-installer/composer.json:        "composer-plugin-api": "^1.0"

We can safely ignore the results in composer/composer/tests — these are tests in the main Composer package. The result we are interested in is

vendor//magento/magento-composer-installer/composer.json

It looks like the magento/magento-composer-installer package is actually a Composer plugin. If we take a look at the contents of this composer.json file

#File: vendor//magento/magento-composer-installer/composer.json 
{
    //...

    "type":"composer-plugin",

    //...

    "extra":{
        //...
        "class":"MagentoHackathon\\Composer\\Magento\\Plugin"
    }
}

We see the composer-plugin type-tag our command line searching found, as well as the required extra configuration that configures the MagentoHackathon\Composer\Magento\Plugin class as a plugin.

Without getting into the specific technical details, this is the plugin that installs those extra files at the root level, above the vendor folder. In short, the MagentoHackathon\Composer\Magento\Plugin will

  1. Listen for composer install and composer update events

  2. Look at the extra->map section(s) for any composer.json file in the just installed or updated composer vendor packages

  3. Use that information to copy file from the installed package, to the root level project folder

If that didn’t make sense, let’s walk through it. First, let’s find any composer.json files with a "map" section.

$ find vendor/ -name composer.json | xargs ack '"map"'
vendor/magento/magento2-base/composer.json
75:        "map": [

vendor/magento/magento2-base/dev/tests/api-functional/_files/Magento/TestModuleIntegrationFromConfig/composer.json
12:    "map": [

vendor/magento/magento2-base/dev/tests/api-functional/_files/Magento/TestModuleJoinDirectives/composer.json
12:    "map": [

Again, we can safely ignore the files in the tests folder — this leaves us (at the time of this writing) with a single result in the magento/magento2-base package. If we look at a snippet of this file

//File: vendor/magento/magento2-base/composer.json
{
    "name": "magento/magento2-base",
    //...
    "extra": {
        //...
        "map": [
            [
                "lib/internal/Cm",
                "lib/internal/Cm"
            ],
            [
                "lib/internal/LinLibertineFont",
                "lib/internal/LinLibertineFont"
            ],
            [
                "lib/internal/Credis",
                "lib/internal/Credis"
            ],
            //...
            [
                "LICENSE_AFL.txt",
                "LICENSE_AFL.txt"
            ],
            [
                "vendor/.htaccess",
                "vendor/.htaccess"
            ]
        ]
    }
}

When the MagentoHackathon\Composer\Magento\Plugin finds the above map section, it will start running PHP code that’s roughly equivalent to

cp -r vendor/magento/magento2-base/lib/internal/Cm lib/internal/Cm
cp -r vendor/magento/magento2-base/lib/internal/LinLibertineFont lib/internal/LinLibertineFont

cp -r vendor/magento/magento2-base/lib/internal/Credis lib/internal/Credis

//...
cp vendor/magento/magento2-base/LICENSE_AFL.txt LICENSE_AFL.txt

cp vendor/magento/magento2-base/vendor/.htaccess vendor/.htaccess"

This is how the non-vendor files Magento needs to operate get from Magento core vendor packages into the root folder of your project.

History of magento/magento-composer-installer

Before we wrap up, it’s worth noting that the magento/magento-composer-installer Composer plugin is a fork of the original Magento 1 Composer installer plugin built at a Magento hackathon, promoted by Firegento, and maintained by Daniel Fahlke. The original goals of this plugin were to build a system that allowed developers to use Composer to fetch Magento 1 plugins into vendor, and then install them into a Magento 1 system via a number of different strategies. This was necessary since Magento 1 never officially adopted Composer.

The Magento core team has repurposed the project as an automatic installer which, on one hand, shows the power and usefulness of open source. On the other hand, if you’re not familiar with the project history and you start exploring the plugin’s implementation in

vendor/magento/magento-composer-installer//src/MagentoHackathon/Composer/Magento/Plugin.php

you may be left scratching your head.

However, if you keep the project’s original goals in mind, the source should make a little more sense.

Consequences

Between this article and last week’s, you should have a pretty good understanding of where all Magento’s files come from in a Composer “meta-package” installation. Understanding this is critical for Magento 2 developers and consultants looking to use and develop day-to-day on Magento’s systems. For example, it may be temping to drop some functions into

app/functions.php

as a shortcut way to get your code into a Magento system, but once you understand that any future composer update will wipe these changes away, the true cost of such short cuts become apparent. Don’t edit the core is as true as it ever was, but with Magento 2 it’s not always clear what is, and what isn’t, a core Magento file.

Next time we’ll be diving into Magento 2’s component system — the system that makes Composer distribution possible.

Originally published April 20, 2016