Magento 2: Adding Frontend Files to your Module

Last time we discussed how Magento serves and generates front end (javascript, css) files to end users from its own modules. Like most of Magento’s feature, if it’s done in a core module, third party developers can do it in their own modules. This time we’ll be creating a module of our own, and using it to add front end javascript/css files to the system.

The specifics in this article refer to the official Magento 2.0 released in the fall of 2015. While the specifics may change in future versions, the concepts should apply to all versions of Magento 2.

A Quick Note on File Permissions

If you take a close look at how Magento handles unix file permissions in its code generation systems, one thing is clear. The core team are not fans of Apache’s mod_php module, and probably run their systems using some sort of PHP-FPM/FastCGI implementation.

As a result, if you’re running PHP with the Apache mod_php module (the most common out of the box way of running PHP) you may end up running into problems with Magento created files. Specifically, files created via command line mode that PHP can’t read/write when running in web server mode, or vice versa.

“The right” thing to do here would be to ensure the apache user and your own shell user are in the unix same group. That’s a non-trivial thing to do though. If you’re not up for it, and you understand the security implications, another way of dealing with it is just chmod 777ing your development files in pub/static and var/ folders. Here’s two quick find commands that will do this for you.

$ find /path/to/magento2/pub/static -exec chmod 777 '{}' +
$ find /path/to/magento2/var/ -exec chmod 777 '{}' + 

This isn’t my favorite approach, but until Magento 2 gets its permission situation in order it’s the only simple way of dealing with your development environment.

Creating the Module

Concerns about file permissions aside — step one is creating a new module. We’re going to run through the steps, cookbook style, below. If you’re curious on a more in depth look at creating modules in Magento, try our Introduction to Magento 2 — No More MVC article.

We’re going to name our module Pulsestorm_FrontendTutorial1. To create this module, we’ll need to create two files. First, create the module.xml file.

#File: app/code/Pulsestorm/FrontendTutorial1/etc/module.xml
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../lib/internal/Magento/Framework/Module/etc/module.xsd">
    <module name="Pulsestorm_FrontendTutorial1" setup_version="0.0.1" />
</config>

Then, create the register.php file

#File: app/code/Pulsestorm/FrontendTutorial1/registration.php
<?php
    \Magento\Framework\Component\ComponentRegistrar::register(
        \Magento\Framework\Component\ComponentRegistrar::MODULE,
        'Pulsestorm_FrontendTutorial1',
        __DIR__
    );?>

With these two files in place, use Magento’s CLI program to enable your module.

$ php bin/magento module:enable Pulsestorm_FrontendTutorial1
The following modules have been enabled:
- Pulsestorm_FrontendTutorial1

To make sure that the enabled modules are properly registered, run 'setup:upgrade'.
Cache cleared successfully.
Generated classes cleared successfully. Please re-run Magento compile command
Info: Some modules might require static view files to be cleared. Use the optional --clear-static-content option to     clear them.

And then run the CLI’s setup:upgrade command.

$ php bin/magento setup:upgrade
Cache cleared successfully
File system cleanup:
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Composer
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Magento
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Symfony
Updating modules:
Schema creation/updates:
Module 'Magento_Store':

//...

Module 'Pulsestorm_FrontendTutorial1':

Please re-run Magento compile command

After doing the above, your (functionless) module will be installed into the Magento system.

System Assumptions

If you finished the article from last week, you know that Magento’s front end file serving behaves differently depending on

1. The root folder you’ve chosen for your website
2. The “mode” Magento is running in

We’re going to start this article assuming

1. That you’ve setup your web site’s root folder as the pub/ folder
2. That you’re running in developer mode

If you’re unsure of how to do this, or curious why we need to have this disclaimer, be sure to read Magento 2: Serving Front End Files

Adding Front End Files

Our goal for this article is to have URLs similar to the following

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

return javascript files from our module. Let’s start with hello.js. First, add the following file to our Magento module.

#File: app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js
alert("Hello World");

With the above in place, load the following URL in your browser or via a command line program like curl

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

You should see the contents of your file! Congratulations, you’ve just added your first front end file to a Magento 2 system

What Just Happened — Module Files

Magento 2 allows a module developer (some might say forces them) to include javascript and CSS files under their main module folder. The old Magento top level skin folder is gone, and app/design is reserved exclusively for Magento theme files.

The top level module view folder (app/code/Pulsestorm/FrontendTutorial1/view) above, is where a module developer places all files related to Magento 2’s user interface. This includes the front end files we’re interested in today, as well as Magento’s layout handle XML files and phtml template files.

The next folder, base, is the area folder. Areas area a way to split individual Magento 2 applications into different areas of functionality, based on the URL. i.e. The cart application is Magento’s frontend area, the backend admin console is the adminhtml area.

So what’s the base area? This is one of those places where Magento 2 has improved on Magento 1 — the base area is a special folder that will allow you to serve your files from either the frontend or adminhtml areas. We’ll talk more about this below.

Next up is the web folder. Files in web are ones that will be served via http or https. While they’re beyond the scope of this article, other folders at this level are email, layout, page_layout, templates, and ui_component.

Finally, we have our file, hello.js. Notice, unlike Magento 1, there’s no need for us to create a sub-folder for our assets. Because these files already live in a Magento module folder, Magento’s smart enough to know where they should go. That said, some Magento core modules still separate out these files with an additional folder/namespace. While you’re free to do this, there’s no need to do this.

What Just Happened — Asset URLs

Module files explained, next up let’s take a look at that URL.

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

We’re going to examine each URL segment and describe where it comes from. For the most part, you won’t need to remember all this — Magento will do most of the URL generating for you. However, understanding the URL asset path will be useful if you’re debugging a system that’s returning 404s, 500s, or some other error when browsers request front end asset files.

The first part of a Magento 2 front end asset URL is static. This points to the actual

pub/static

folder in your Magento install. If you’re using the root level index.php file instead of pub/index.php, this should be

http://magento.example.com/pub/static/...

The next URL segment is the Magento area. In our example URL above, this is frontend. However, because we created a file in the base area folder, we can also fetch this file via the following URL

http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

i.e., using adminhtml instead of frontend. We’ll talk more about this in the area section below — for now all we need to know is this URL segment is the area.

The next URL segment is Magento — this is the vendor prefix for the theme name. This is similar to the design package in Magento 1. This is Magento because the theme we’re using is in the app/design/frontend/Magento folder.

After the vendor prefix is the theme’s actual name. In our case, we’re using the blank theme in the URL. However, we could also use the luma theme that ships with Magento 2.0, or any theme installed in the system.

The penultimate folder, en_US, is the locale folder. Magento allows different front end asset files per locale — think of hard coded language strings in javascript, or locale specific images in CSS files. If we wanted a version of our file for french, we’re use fr_FR in the URL

http://magento.example.com/static/adminhtml/Magento/blank/fr_FR/Pulsestorm_FrontendTutorial1/hello.js

and create our french version of hello.js in

app/code/Pulsestorm/FrontendTutorial1/view/base/web/i18n/fr_FR/hello.js

Notice the i18n folder — this stands for internationalization, and is topic well beyond the scope of this, or any single, article.

Our final url segment is the name of our module — Pulsestorm_FrontendTutorial1. This is why Magento 2 doesn’t need you to self-organize your files in the view/[area] folder. By generating URLs with the module name in them, Magento 2 has enough information to find the file in your module folder. Change this segment of the URL, and Magento won’t find your file.

http://magento.example.com/static/science/Magento/blank/en_US/Some_OtherModuleWithAHelloJs/hello.js

With developer mode enabled, Magento will use each of these URL segments to find the correct file, read it into memory via PHP, and echo it back out to the end user. While this is appropriate for an individual developer’s machine, this sort of dynamic lookup would be unacceptable for a medium to high traffic production system.

That’s where static asset generation comes into play.

Generating Static Assets for Production

If you’ve read the previous article in this series, you know the first time we accessed each of the above URLs, Magento automatically generated a file for us in pub/static. In the first non-beta Magento 2.0 release, these files generated during development mode are symlinks to the actual file. You can see this for yourself with ls -lh

$ ls -lh pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js 
lrwxrwxrwx  1 _www  staff   112B Dec 27 11:19 pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/ hello.js -> /path/to/magento2/app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js

This is a common approach for many PHP frameworks. However, another common approach is to just copy the file over the first time its requested. There’s no clear right way here, and I wouldn’t be surprised to see the Magento core team flip-flop its approach in a future release.

Regardless of whether they’re symlinks or actual files, generating these on the fly is inappropriate for a production environment. In addition to putting additional burden on the server’s file system, it also introduces a potential attack vector for black hat hackers. That’s why if we remove the generated file(s)

$ rm pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
$ rm pub/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

and then switch our system into production mode

#If you're having trouble, did you check pub/.htaccess for mode setting?
#File: .htaccess
SetEnv MAGE_MODE production

Magento will return a blank screened, 404 for the file.

$ curl -i 'http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js'
HTTP/1.1 404 Not Found
Date: Sun, 27 Dec 2015 20:30:30 GMT
Server: Apache/2.4.10 (Unix) PHP/5.6.16
X-Powered-By: PHP/5.6.16
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=UTF-8

When you deploy your Magento application, you need to generate these files on your own. You can do this by running the following command

$ php bin/magento setup:static-content:deploy
Requested languages: en_US
=== frontend -> Magento/blank -> en_US ===
.....................................

...

---

New version of deployed files: 1451248980

After the above finishes running, you’ll have a statically generated file for every web file in every Magento module. We can see this if we use the command line find program to look for generated hello.js file.

$ !find
find pub/ -name hello.js
pub//static/adminhtml/Magento/backend/en_US/Pulsestorm_FrontendTutorial1/hello.js
pub//static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
pub//static/frontend/Magento/luma/en_US/Pulsestorm_FrontendTutorial1/hello.js

Here we can see that Magento has created three hello.js files. One is for the backend theme in the Magento adminhtml area. The other two are for the frontend area — one for the blank theme, another for the luma theme.

Before we move on to our promised discussion of areas — there’s a few things to take note of. First (at the time of this writing), the files generated for production are not symlinks, and are actual copies of the file

$ ls -lh pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js 
-rw-rw-rw-  1 alanstorm  staff    21B Dec 27 13:10 pub/static/frontend/Magento/blank/en_US/ Pulsestorm_FrontendTutorial1/hello.js

This means if you change something directly in a module folder on your production system, you won’t see the change reflected on the live site. Also, if you do directly edit the files in the pub/static sub-folders, your changes will be deleted the next time someone deploys to the production server.

Another thing to watch out for here: If the symlinks from development mode are still present in the pub/static sub-folders when you run setup:static-content:deploy, Magento will not remove them. On one hand — this shows whomever implemented setup:static-content:deploy cared enough to make sure their command wasn’t destructive. On the other hand — if your deployment procedure isn’t super tight, this means you may end up with symlinks on your production website.

Understanding the Area Hierarchy

The first thing we had you do in this tutorial was create the following file

#File: app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js
alert("Hello World");

This added the hello.js file to both the adminhtml and frontend areas.

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

It’s also possible, and far more common, to have a javascript or css file that’s only needed in one of the two areas. For example, try creating the following file

#File: app/code/Pulsestorm/FrontendTutorial1/view/frontend/web/hello-2.js
alert("Hello Frontend World");

Notice its location is almost exactly the same as our original file — the only difference is we’ve replaced the base folder with a folder named frontend. With the above in place, and developer mode reenabled, try loading the file’s corresponding URL

$ curl -i 'http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello-2.js'
HTTP/1.1 200 OK
Date: Mon, 28 Dec 2015 02:06:30 GMT
Server: Apache/2.4.10 (Unix) PHP/5.6.16
Last-Modified: Mon, 28 Dec 2015 02:04:32 GMT
ETag: "1e-527ebb9d38c00"
Accept-Ranges: bytes
Content-Length: 30
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Cache-Control: public
Content-Type: application/javascript

alert("Hello Frontend World");

You should see your file returned without a hitch. However, if you attempt to load the same file using an adminhtml area, you should see a 404 Not Found error.

$ curl -I 'http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello-2.js'
HTTP/1.1 404 Not Found

If there’s a file in both a specific area folder and the base area folder

app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello-2.js
app/code/Pulsestorm/FrontendTutorial1/view/frontend/web/hello-2.js

the file in the specific area folder (frontend above) will win out.

Generally speaking, you should try to keep your front end asset files in the appropriate area folder. If you need a javascript file that adds features for the backend admin console, it’s best to keep that in the adminhtml folder. However, if there’s a library file your module needs in both locations, then using the base folder is appropriate.

Developer Mode Quirks

Finally, when you’re working with your system in developer mode, there are a few quirks to be aware of. Consider the URL for our original helloworld.js file

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

If you change the URL to use a fake area like the one below

http://magento.example.com/static/fake-area-that-is-not-there/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

You’ll be surprised to find out Magento still returns the file found in the base folder

$ curl -I http://magento.example.com/static/fake-area-that-is-not-there/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
HTTP/1.1 200 OK

The same is true for the theme vendor, theme name, and locale portions of the URL (replaced with bar, foo, and baz below)

http://magento.example.com/static/fake-area-that-is-not-there/bar/foo/baz/Pulsestorm_FrontendTutorial1/hello.js

This behavior can go from curious anomaly to crazy making if you’re manually creating URLs (say, for a tutorial). Consider the following typos

http://magento.example.com/static/frotnend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

http://magento.example.com/static/front-end/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/fronternd/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

Fortunately though, as hinted at throughout this article, you shouldn’t need to manually create paths to front-end static assets. Magento 2 contains a number of different systems for pulling in javascript and CSS files, and a number of PHP objets and methods for generating these paths programmatically.

Now that we have a better understanding of how Magento serves front end static assets, and also understand how we can add individual front end assets to our own modules, our next steps in exploring the various Magento systems that will let you use these assets. That’s where we’ll start next time.