Magento’s SOAP V2 Adapater

If you’ve been following along, it may seem like we’ve reached the end of our API series. We’ve covered routing and controller dispatch, the API server, configuration, adapters, handlers, and the use of the response object for direct API output. I imagine there were members of the core team who thought they were done at this point too. Unfortunately, the best laid plans often fall afoul of the real world. Today we’ll be covering additions made to the API in response to complaints/requests from the SOAP community.

This article relies heavily on concepts covered in the entire API series, particularly themes from the last article. You can proceed without having read them, but you won’t be getting the full story.

For reasons we’ll cover at the end of this article, the code examples we’re using here come from the older Magento 1.5.0x branch of community edition, but (as always) the concepts apply across all versions of Magento.

An Abstraction Too Far

Before we can understand the first problem with Magento’s V1 API, we need a little CS 101. Most people using Magento come from a background in PHP development, (maybe they know a little ruby or python too). Declaring a variable in these language is as simple as using the variable.

#PHP
$foo = 'bar';

#Ruby
foo  = "bar"

There’s another class of languages, .NET and Java most prominently, where declaring variables is a little more complicated. In addition to specifying the variable’s name, you also need to declare its type

#Java
String foo = "bar";

#.NET/C#
string foo = "bar;

As you can see above, the word string is used to indicate the variable foo will contain a string. In addition to this bit of boilerplate you (typically) can’t change the type of the value in a variable

#Invalid Java code
String foo = "bar"
foo = 1

Also, when you define a method/function in these language, you need to explicitly state a return type for these methods. For example, the following method must return a string.

string public function helloWorld()
{
    String theMessage = "Hello World"
    return theMessage;
}

If we wrote code that failed to return a string, the compiler would refuse to compile it

//will not compile
string public function helloWorld()
{        
    return 42;
}

You could fill a masters degree program with the myriad reasons for this, but in simple terms these languages are explicitly compiled by the programmer/developer. That is, you write the code (say, in Java), and then tell another program to compile that Java code into something called bytecode. This bytecode is (again, grossly over-simplifying things) a program that will run super efficiently. So with .NET and Java, the compiler needs to know what each variable contains so it can set aside the exact amount of memory needed for that variable. This is part of why compiled programs have a reputation for running faster and more efficiently than non-compiled programs.

Of course, in 2012 things are much more complicated. The so-called “non-compiled” (or worse, “dynamic”) language like PHP, Ruby, Python, Perl and Javascript all have some sort of behind the scenes process that looks an awful lot like compilation. For example, with PHP when you run a program/script it’s compiled into something called opcode, and then this opcode is is what’s actually run. When you’re using a PHP accelerator you’re telling PHP to compile your program/script into opcode once, and then use the saved optcode for future requests. This means compilation is skipped on every request, and program performance improves across the board.

Instead of separating languages into compiled and non-compiled, it’s probably fairer to say there are languages where compilation is an explicit step and must be accounted for in code, and languages where compilation is hidden from user-programmers. That, however, is a discussion that can quickly get mired in organization politics with no easy answers. For our purposes, we just need to be aware that Java and .NET developers need to deal more explicitly with types.

Why do we need to be aware of this? Consider a theoretical Magento API call in one of these languages.

result = client.call('product.info',...);

Remember, the Magento API uses a keyhole method named call. With that in mind, what should result‘s type be? More importantly, what should the call method’s return type be? Some methods will return a string, some a boolean. Some (most?) will return a PHP array, and if you think variables are complicated in these languages, you’d be shocked at the sort of boilerplate code you need to get into for collections and dictionary type objects.

The larger point here is this: SOAP was an API system designed for use with the explicitly compiled languages, and people who use SOAP in these languages have an expectation about how their API methods will work. All of the above problems can be worked around, but not without tremendous effort by the developers involved. More effort than an API client programmer should need to go through. That’s why a second version of Magento’s SOAP APi was needed.

The purposed of Magento’s V2 SOAP API is give .NET and Java developers an API that fits their version of the world. This means the keyhole method must be replaced with actual client methods, such as

#$client->call('resourcename.methodname',...);
$client->resoucenameMethodname(...);

In addition to ensuring our handler class can respond to these methods, we’ll also need to ensure our WSDL file has a list of these methods as well. SOAP client libraries in these languages use the WSDL to help prepare variable types.

Changes for the V2 API

At first, this may seem like an epic task. The naive approach would be to add every API method to a new API handler class. The problem with this approach is, you’d loose the ability to add methods to the Magento API. You’d also create a gargantuan, possibly unmanageable handler class. Ironically (or appropriately, depending on your take), although its abstractions that got us into this mess, it’s abstractions that will get us out.

To start, let’s take a look at the WSDL for the V2 API

http://store.example.com/api/v2_soap?wsdl

If you load the above URL in your Magento system you’ll see a much larger XML file than the V1 WSDL. That’s because it contains a definition for each and every API method. If you remove the wsdl from the URL

http://store.example.com/api/v2_soap

You’ll see the familiar Invalid XML response

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
    <SOAP-ENV:Body>
        <SOAP-ENV:Fault>
            <faultcode>Sender</faultcode>
            <faultstring>Invalid XML</faultstring>
        </SOAP-ENV:Fault>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

So far we’re in familiar territory. The V2 SOAP API is following the same patterns as the original V1 API. If we take a look at the controller action responsible for dispatching the API server request

#from Magento 1.5.0x
#File: app/code/core/Mage/Api/controllers/V2/SoapController.php
class Mage_Api_V2_SoapController extends Mage_Api_Controller_Action
{
    public function indexAction()
    {
        $server = Mage::getSingleton('api/server');

        /* @var $server Mage_Api_Model_Server */
        $this->_getServer()->init($this, 'soap_v2', 'soap_v2')
            ->run();
    }
}

we see a familiar pattern. However, this time we’re passing in the string value soap_v2 for both adapter and handler parameters. This means our handler and adapter class aliases will be puilled from the following API configuration nodes

<!-- File: app/code/core/Api/etc/api.xml -->
<adapters>
    <!-- ... -->
    <soap_v2>
        <model>api/server_v2_adapter_soap</model>
        <handler>soap_v2</handler>
        <active>1</active>
        <required>
            <extensions>
                <soap />
            </extensions>
        </required>
    </soap_v2>
    <!-- ... -->        
</adapters>

<!-- ... -->    

<handlers>
    <!-- ... -->
    <soap_v2>
        <model>api/server_v2_handler</model>
    </soap_v2>
</handlers>

This makes our adapter a api/server_v2_adapter_soap model object, and our handler a api/server_v2_handler model object. This, by the way, is what object oriented folks mean when they say “real” object oriented programming. You’ll recall from past articles that our server object will instantiate an adapter object, and call that adapter object’s run method.

By changing the configuration, we’re dropping an entirely new object type into the system without needing to change system code. For the V2 API, the run method will be called on the api/server_v2_adapter_soap object. The server object doesn’t need to care what it’s calling run on, it just needs to do it. That, to many people, is what object oriented means.

Let’s take a look at the new adapter’s class definition

#File: app/code/core/Mage/Api/Model/Server/V2/Adapter/Soap.php
class Mage_Api_Model_Server_V2_Adapter_Soap extends Mage_Api_Model_Server_Adapter_Soap
{
    /**
     * Run webservice
     *
     * @param Mage_Api_Controller_Action $controller
     * @return Mage_Api_Model_Server_Adapter_Soap
     */
    public function run()
    {
        $apiConfigCharset = Mage::getStoreConfig("api/config/charset");

        if ($this->getController()->getRequest()->getParam('wsdl') !== null) {
            $wsdlConfig = Mage::getModel('api/wsdl_config');
            $wsdlConfig->setHandler($this->getHandler())
                ->init();
            $this->getController()->getResponse()
                ->clearHeaders()
                ->setHeader('Content-Type','text/xml; charset='.$apiConfigCharset)
                ->setBody(
                      preg_replace(
                        '/<\?xml version="([^\"]+)"([^\>]+)>/i',
                        '<?xml version="$1" encoding="'.$apiConfigCharset.'"?>',
                        $wsdlConfig->getWsdlContent()
                    )
                );
        } else {
            try {
                $this->_instantiateServer();

                $this->getController()->getResponse()
                    ->clearHeaders()
                    ->setHeader('Content-Type','text/xml; charset='.$apiConfigCharset)
                    ->setBody(
                            preg_replace(
                                '/<\?xml version="([^\"]+)"([^\>]+)>/i',
                                '<?xml version="$1" encoding="'.$apiConfigCharset.'"?>',
                                $this->_soap->handle()
                            )
                    );
            } catch( Zend_Soap_Server_Exception $e ) {
                $this->fault( $e->getCode(), $e->getMessage() );
            } catch( Exception $e ) {
                $this->fault( $e->getCode(), $e->getMessage() );
            }
        }

        return $this;
    }
}

The first thing to notice about the Mage_Api_Model_Server_V2_Adapter_Soap class (which is what api/server_v2_adapter_soap resolves to) is that it extends the V1 adapter class (Mage_Api_Model_Server_Adapter_Soap). Again, this is good OOP in practice. To start, our V2 adapter has the exact same functionality, and we’ll only change the methods we need to. Or, more accurately, method. Singular.

The only method the V2 adapter changes is the run method, and it contains the same if/else clause for a WSDL generating code path and an API call handling code path the V1 adapter did. The API handling code path uses the same pattern to instantiate a SOAP server and then handle the SOAP call.

#File: app/code/core/Mage/Api/Model/Server/V2/Adapter/Soap.php
$this->_instantiateServer();
...
$this->_soap->handle()

Remember, the _instantiateServer method is the same as our V1 adapter, which means the same Zend soap handling class will be used.

#File: app/code/core/Mage/Api/Model/Server/V2/Adapter/Soap.php
$this->_soap = new Zend_Soap_Server($this->getWsdlUrl(array("wsdl" => 1)), array('encoding' => $apiConfigCharset));
...
$this->_soap
    ->setReturnResponse(true)
    ->setClass($this->getHandler());    

The only difference, (at the risk of getting ahead of ourselves), is the $this->getHandler() call will return our V2 handler class, which was set during the Magento API server’s init method.

In fact, the only reason we need a new adapter for the V2 API is the WSDL generation code. You’ll remember from our previous article the V1 WSDL generation was a weird combination of a Varien_Io_File object and an email template object. For the V2 API this has been cleaned up a bit

#File: app/code/core/Mage/Api/Model/Server/V2/Adapter/Soap.php
$wsdlConfig = Mage::getModel('api/wsdl_config');
$wsdlConfig->setHandler($this->getHandler())
            ->init();
...
$wsdlConfig->getWsdlContent()

As you can see, with for the V2 API Magento has created a api/wsdl_config model class which contains the PHP code we’ll be using to generate our WSDL file for the V2 API. This is another example of the various things the word “model” means in Magento’s landscape. The api/wsdl_config model contains the “business logic” for solving the problem of generating a WSDL file that reflects all the API methods (core or otherwise) available to the system. It has nothing to do with Magento’s CRUD/database models.

Generating the WSDL

Generating the WSDL using the api/wsdl_config object is a four step process.

1. The model is instantiated

2. We set the handler class string on the object

3. We init the object

4. We call getWsdlContent to generate the WSDL

Before we dive into the WSDL generation, let’s think about the problem. We know a V2 WSDL will need an entry for every available API method. We also know the Magento API philosophy is

Allow each module developer to to expose whatever API methods they wish via API resources

The fact that any module can define an API method via its api.xml file means we’re probably going to see some module traversing code in the api/wsdl_config class. Let’s take a look

The api/wsdl_config class alias resolves to the class Mage_Api_Model_Wsdl_Config.

#File: app/code/core/Mage/Api/Model/Wsdl/Config.php
class Mage_Api_Model_Wsdl_Config extends Mage_Api_Model_Wsdl_Config_Base
{
    //...
}

This class extends the Mage_Api_Model_Wsdl_Config_Base class,

app/code/core/Mage/Api/Model/Wsdl/Config/Base.php
class Mage_Api_Model_Wsdl_Config_Base extends Varien_Simplexml_Config
{
}

which in turn extends the Varien_Simplexml_Config class. Long time readers will recognize this pattern as extremely similar to the configuration loading pattern in The Magento Config. For those not familiar, this class hierarchy is used in Magento load and merge a number of XML files from various sources in the file system. The Magento Config isn’t required reading here, but we will be breezing over a few concepts that are worth investigating further.

The core of the work in generating the new WSDL files is done by the init method. The setHandler method is a simple chained setter

#File: app/code/core/Mage/Api/Model/Wsdl/Config/Base.php
public function setHandler($handler)
{
    $this->_handler = $handler;
    return $this;
}

and the getWsdlContent method calls asXml on a SimpleXML node stored in the _xml object property.

#File: app/code/core/Mage/Api/Model/Wsdl/Config.php
public function getWsdlContent()
{
    return $this->_xml->asXML();
}

It’s the init method that will instantiate and populate _xml. Rather than go through init line-by-line (and re-explain Magento’s configuration merging), we’re going to go with a plain english explain of this method.

Every Magento module that exposes API methods is responsible for providing an XML file with the correct WSDL definitions for those methods in a wsdl.xml file. For example, the Mage_Catalog module exposes several API methods, and you can find it’s WSDL definitions in

app/code/core/Mage/Catalog/etc/wsdl.xml

The init method will search through all defined Magento modules and look for the these wsdl.xml files. These files will be combined into a single XML tree, and this tree will be set on the _xml property.

This allows each module developer to come up with a their own WSDL definition, but it also places the burden of correctly generating that WSDL file on the module developer. This includes more than just definitions for the defined methods. It gets into assigning complex types to PHP’s generic array elements. It’s beyond the scope of this article to get into things like complex types, but consider this bit of the Mage_Catalog‘s WSDL.

<!-- app/code/core/Mage/Catalog/etc/wsdl.xml -->
<complexType name="catalogProductEntity">
    <all>
        <element name="product_id" type="xsd:string" />
        <element name="sku" type="xsd:string" />
        <element name="name" type="xsd:string" />
        <element name="set" type="xsd:string" />
        <element name="type" type="xsd:string" />
        <element name="category_ids" type="typens:ArrayOfString" />
        <element name="website_ids" type="typens:ArrayOfString" />
    </all>
</complexType>

There’s also, as you’d expect, a few gotchas to be aware of. First, there’s these two lines

#File: app/code/core/Mage/Api/Model/Wsdl/Config.php
$mergeWsdl->addLoadedFile(Mage::getConfig()->getModuleDir('etc', "Mage_Api").DS.'wsdl.xml');
$baseWsdlFile = Mage::getConfig()->getModuleDir('etc', "Mage_Api").DS.'wsdl2.xml';

You may remember that the V1 API used the a file named wsdl.xml for it’s 8 (?) method WDSL definition. That creates a problem for the V2 API, as the module scanning code would normally pickup this file and attempt to merge in this different WSDL. The two lines above take care of this situation, by marking

app/code/core/Mage/Api/etc/wsdl.xml

as already loaded, and then adding

app/code/core/Mage/Api/etc/wsdl2.xml

as the base WSDL file that the other will be merged into. It’s not the most intuitive pattern — and it’s quick-hack nature points to a bug being discovered late in the development cycle.

Another gotcha is the the Magento style {{template}} variable tags

<definitions xmlns:typens="urn:{{var wsdl.name}}" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
             xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
             xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
             xmlns="http://schemas.xmlsoap.org/wsdl/"
             name="{{var wsdl.name}}" targetNamespace="urn:{{var wsdl.name}}">

In the V1 API, an email template object was instantiated to perform the variable replacement for these tags in the adapter object. This time, that replacement happens in the base configuration class. As we said, this isn’t an article covering that system, but if you’re looking for the method where this replacement happens it’s processFileData

#File: app/code/core/Mage/Api/Model/Wsdl/Config/Base.php
public function processFileData($text)
{
    /** @var $template Mage_Core_Model_Email_Template_Filter */
    $template = Mage::getModel('core/email_template_filter');

    $this->_wsdlVariables->setHandler($this->getHandler());

    $template->setVariables(array('wsdl'=>$this->_wsdlVariables));

    return $template->filter($text);
}

The final gotchas is that, like most multiple module configuration tress, the V2 WSDL is cached. That means if you’re working on your WSDL XML files you’ll need to clear Magento cache to see any changes. This is extra tricky, because the cache tag used (wsdl_config_global)

$this->setCacheId('wsdl_config_global');

Is not one of the 7 Magento cache types manageable from the Admin, which means you’ll need to blow away the entire cache to see your changes.

Handling API calls

With WSDL generation out of the way, we’re free to concentrate on how the V2 API actually handles an API call. For reference, the following V1 client call

$result         = $client->call($session_id,
                   'customer.info',
                   1);

would look something like this in the V2 API

$result         = $client->customerCustomerInfo($session_id,1);   

Instead of using the keyhole call method, we directly call a customerCustomerInfo method.

Knowing this, we might expect to see a customerCustomerInfo method defined on our api/server_v2_handler (Mage_Api_Model_Server_V2_Handler). Opening up that file

#File: app/code/core/Mage/Api/Model/Server/V2/Handler.php
class Mage_Api_Model_Server_V2_Handler extends Mage_Api_Model_Server_Handler_Abstract
{
    protected $_resourceSuffix = '_v2';    
    public function __call( $function, $args )
    {
        //...
        return $this->call($sessionId, $apiKey, $args);
    }
}

will either confuse you more or set off a light bulb. Rather than add every API method to the handler class, the core team is using PHP’s magic __call method. The Zend SOAP library will attempt to call customerCustomerInfo method, and when PHP finds it undefined the call will be passed to __call.

This means all the core needed to do was write a routine in __call that turns customerCustomerInfo into a a V1 API resource.method string, and then “call” the keyhole call method. This is, again, object oriented programming at it’s finest. The V2 API, despite exposing a whole new set of API methods, will be starting with the exact same method implementations as the V1 API.

Unpacking the //.. above, we start by shifting the session ID off the argument array and initializing a variable to hold our resource.method string.

#File: app/code/core/Mage/Api/Model/Server/V2/Handler.php
$sessionId = array_shift( $args );
$apiKey = '';

In addition to getting us the session ID, this leaves $args in a state to be passed directly to the call method as is. Next, we load a configuration node from the combined api.xml tree

$nodes = Mage::getSingleton('api/config')
->getNode('v2/resources_function_prefix')
->children();

To understand the resources_function_prefix node, we need to understand how Magento decided to turn it’s resource/method string into an API method call. When a V2 API call is made,

$client->customerCustomerInfo(...);

The method name is actually two strings combined. One represents the API resource. The second represents the API method. So that means the above call is split like this

Resource: customerCustomer
Method:   info

The string customerCustomer needs to be translated into the resource name customer. That’s where the resources_function_prefix comes in.

<config>
    <api>
        <v2>
            <resources_function_prefix>
                <!-- ... -->
                <customer>customerCustomer</customer>
                <customer_group>customerGroup</customer_group>
                <customer_address>customerAddress</customer_address>
                <!-- ... -->
            </resources_function_prefix>
        </v2>
    </api>
</config>

This node contains the mappings we’re looking for. The value of the node is the method prefix, while the name of the node is the actual resource node. Next up Magento starts looping over these nodes, trying to match the value in the node with the start of the current value in $function (in our case, that’s customerCustomerInfo

#File: app/code/core/Mage/Api/Model/Server/V2/Handler.php
foreach ($nodes as $resource => $prefix) {
    $prefix = $prefix->asArray();
    if (false !== strpos($function, $prefix)) {
        //...
    }
}

You’ll notice we’re not doing a == or === compare, we’re using strpos to match the start of the method name. If this method matches, the code inside the conditional runs

#File: app/code/core/Mage/Api/Model/Server/V2/Handler.php
$method = substr($function, strlen($prefix));
$apiKey = $resource . '.' . strtolower($method[0]).substr($method, 1);

At this point Magento’s already has a resource name (from the name of the node that matched). All it needs to do is come up with a value to use for the method. It does this by string replacing the prefix

$method = substr('customerCustomerInfo', strlen('customerCustomer'));
//$method = 'Info';

and then lower casing the camelCased method.

strtolower('I').substr('Info', 1);

This method name is combined with the resource node’s name, resulting in a resource.method pair of

customer.info

which matches our V1 API call. Finally, the keyhole call method is called, and its return value is passed back to the SOAP library code, which will serialize things back to the user.

return $this->call($sessionId, $apiKey, $args);

In this way the V2 API call will be implemented by the V1 method call. At this point, our API call is complete.

V2 API Resource Classes

There’s one last thing to cover before leaving for the day. You may have noticed an additional property on the V2 handler class

protected $_resourceSuffix = '_v2';

This seemingly innocuous property is vitally important when you’re debugging calls to the V2 API. To see how, we’ll need to revisit a single line in the definition of call in the base abstract handler class

#File: app/code/core/Mage/Api/Model/Server/Handler/Abstract.php
public function call($sessionId, $apiPath, $args = array())
{
    //...
    $modelName = $this->_prepareResourceModelName((string) $resources->$resourceName->model);
    //...
}

In case you need a refresher, at this point in call, a Magento class alias is pulled out of api.xml. Something along the lines of

$modelName = 'customer/customer_api';

This is used to instantiate a resource model object, which contains the definition of our API methods. However, last time we ignored the definition of _prepareResourceModelName. Let’s take a look at that now.

#File: app/code/core/Mage/Api/Model/Server/Handler/Abstract.php
protected function _prepareResourceModelName($resource)
{
    if (null !== $this->_resourceSuffix) {
        return $resource . $this->_resourceSuffix;
    }
    return $resource;
}

Ah Ha! Here we can see Magento is looking fro a defined _resourceSuffix property. If present, it will alter the resource name to include it’s value as a suffix. That means customer/customer_api will be transformed into customer/customer_api_v2. The larger implication of this is all V2 API resources have their own Magento resource class. If you’re creating a custom API method and want it supported by the V2 API, then you’ll need to provide your own V2 resource class

In our case, customer/api_resource_v2 resolves to

#File: app/code/core/Mage/Customer/Model/Customer/Api/V2.php
class Mage_Customer_Model_Customer_Api_V2 extends Mage_Customer_Model_Customer_Api
{
    ///...
}

As you can see, this class simply extends the original V1 resource class (Mage_Customer_Model_Customer_Api). This means, by default, the implementations for the methods will be the same, but a V2 API method might have a different implementation. Sticking with our example above, we can see the V2 API has a redefined items methods.

#File: app/code/core/Mage/Customer/Model/Customer/Api/V2.php
class Mage_Customer_Model_Customer_Api_V2 extends Mage_Customer_Model_Customer_Api
{
    //...
    public function items($filters)
    {
    }
    //...
}

From what I’ve seen, this is mostly done to fix places where the original method assumed something about the way a parameter would be serialized in a .NET or Java environment, but I haven’t done complete survey of all the V2 resource methods to know for sure, so your milage may vary.

Again, we’re seeing the Magento core team leverage OOP to reduce the amount of new code they need to write or generate. By default, an empty V2 resource class will suffice, and then method may be selectively defined as needed. Forward progress resumes without impacting existing code.

Wrap Up

So that’s the reasons for, and the implementation of, the V2 Magento API. You may be wondering why we used code examples from the 1.5.0 branch instead of something more recent. The answer lies in in the API dispatch from the 1.6.1x branch

#File: app/code/core/Mage/Api/controllers/V2/SoapController.php
if(Mage::helper('api/data')->isComplianceWSI()){
    $handler_name = 'soap_wsi';
} else {
    $handler_name = 'soap_v2';
}

/* @var $server Mage_Api_Model_Server */
$this->_getServer()->init($this, $handler_name, $handler_name)
    ->run();

New version of Magento introduced a third API adapter and handler which implements something called WSI compliance mode. This, along with some thought on architecture drift, and will be the topic of our next article in the Magento API series.