- Compatible XF Versions
- 2.1
- Additional Requirements
- You must have Composer installed
Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.
XenForo 2 uses Composer behind the scenes to include certain packages it uses - however, there is no way to automatically include your own packages for addons using this core Composer support. There is no
However, the framework provided by XenForo 2 has extension points we can hook in to which allow us to include packages in our addons and have them autoloaded by the XenForo autoloader. This tutorial describes how to do so and provides some code you may use in your own addons to help achieve this.
Note that there are some caveats here - dependency management can be complex, and it is entirely possible for you to introduce unexpected bugs in your addon, other addons, or even in the XenForo core by following this tutorial and including composer packages. Care must be taken and you would be well advised to avoid this approach if you are not already familiar with Composer dependency management.
Assumptions made in this tutorial:
This tutorial runs through a basic example of adding a Composer package to an addon. The addon itself is installable so you can see all of the code in operation and examine how it works. We will install the Carbon API extension for DateTime objects.
All code in this tutorial is licensed under the MIT license, which essentially allows you to use (or modify!) the code for any purpose (including commercial purposes) free of charge, with the only condition being that you include the relevant copyright notice and permission notice.
See the
Just FYI, Composer itself and the Carbon package we will install also use the MIT license.
Getting Started
I have created a basic addon called ComposerTutorial. My
XenForo is installed on my dev server at
The first step to adding a Composer package is to create a
We now have a basic Composer file created at
While there are dozens of fields available in the composer.json Schema - you actually only need the
If you created your
The other thing we now have is a
The
The
The
The
Finally, we have a
While you are developing your addon, you may run the
However, in a production environment, it is important to only ever run the
Either way, we'll cover installation in more detail later. For now, just recognise that the
So now that we have our package and dependencies installed, we need to get XenForo to autoload everything.
XenForo Autoloader
As mentioned, XenForo uses composer behind the scenes - although they hide the
So with XenForo already running an autoloader, we cannot simply call the
I've created a class to help achieve this. I copy this class to each of my addons which use Composer packages.
You can find this file in the addon root in the published addon:
This class provides four functions to look through the four different files which are auto-generated by Composer, identifies the classes and files and then adds them to the XenForo autoload queue.
I usually put this in the addon root - but it can go anywhere below your addon root, provided that you adjust the namespaces in this file and the initialisation functions (discussed later) appropriately.
Order of Class Loading
Note that the optional
When resolving a class during execution, the Composer autoloader simply runs through the autoload queue to find the first match for the class being called. If there are duplicate class definitions (for example if multiple addons use the same dependency, or if an addon uses the same depenency as the core XenForo packages), then only the first match is used.
By default, classes are appended to the autoload queue, meaning that the first classes added to the queue (the XenForo core dependencies!) will be given priority. This is generally a good thing, as it prevents you from clobbering the core dependencies - but be sure you check the versions being used by the core to ensure your package is not receiving a substantially different version of a class from the core rather than from your addon dependency packages.
If another addon is initialised before yours, its dependencies will be added to the queue before yours. If both addons contains the same classes - it is their versions which will be loaded, not yours. If this causes problems with other addons, you should be able to adjust the class loading queue order by adjusting the order of execution for the listener we'll set up in the next step. Assuming there are no significant bugs and the dependency versions are not significantly different, this should not cause issues - but this is where unexpected bugs can be introduced, so care is required.
In certain (rare) circumstances, you may need to replace one of the dependency classes provided in the XenForo core. You can set the
For my Guzzle 6 addon, I needed to do exactly this because I was replacing core functionality - XenForo v2.0 ships with Guzzle v5.3 and so if I appended my classes to the autoloader queue, the core Guzzle 5.3 versions of the classes would always be found first. By appending my dependencies to the front of the queue, I was able to have Guzzle 6 classes loaded instead. This is a rare occurrence and not generally recommended as it does impact on core functionality and could break things unexpectedly.
Initialising the Composer Autoloader
We can use a code event listener to initialise our addon and add our dependencies to the XF autoloader queue. I create the following
This function simply calls each of the four autoload functions (order matters!) from our Composer class to load the dependencies into the autoload queue.
It is here that we can use that
Now, we simply add the code event listener to execute the appSetup function we defined above:
Let's test this by adding a test tool to the admin area - see source code for details.
To run the test after installing the addon - go to the Admin > Tools > Test Composer Tutorial page and you'll see some output generated by Carbon.
Packaging the Addon
Once the addon is ready for release, we can simply package it up and publish it.
However, there are several different approaches here.
The "purist" approach would be to not check your vendor folder in to source control, but instead rely on the person installing the addon to run
The "pragmatic" approach is to check the entire vendor folder into source control, so that when the addon is deployed, it is complete and ready to use. This is the approach I take.
However, there are some things we should do before packaging our addon.
By default, Composer installs development dependencies (eg unit testing tools) specified in your
Similarly, there are some optimisations we can (and should!) do to our code for running in production environments. These optimisations can be handled for us by composer - we just need to execute the commands before packaging our addon.
Fortunately, the XenForo devs have been kind enough to provide us with a lovely little tool to run custom commands when building our addon.
Simply create a
This tells the XenForo build process to execute a
You can see the output of the composer commands along with the build output.
I have attached the built addon to this tutorial so you can install it, look at the code and experiment with other packages.
Alternatively, clone the git repository for the tutorial code - XenForo Composer Tutorial
XenForo 2 uses Composer behind the scenes to include certain packages it uses - however, there is no way to automatically include your own packages for addons using this core Composer support. There is no
composer.json file for XenForo 2 we can edit to add our own packages.However, the framework provided by XenForo 2 has extension points we can hook in to which allow us to include packages in our addons and have them autoloaded by the XenForo autoloader. This tutorial describes how to do so and provides some code you may use in your own addons to help achieve this.
Note that there are some caveats here - dependency management can be complex, and it is entirely possible for you to introduce unexpected bugs in your addon, other addons, or even in the XenForo core by following this tutorial and including composer packages. Care must be taken and you would be well advised to avoid this approach if you are not already familiar with Composer dependency management.
Assumptions made in this tutorial:
- you have Composer installed on your development server and you are familiar with how to use it
- you understand how Composer packages work
- all examples assume a Linux environment running bash, you will need to translate these for use on Windows or other platforms yourself
This tutorial runs through a basic example of adding a Composer package to an addon. The addon itself is installable so you can see all of the code in operation and examine how it works. We will install the Carbon API extension for DateTime objects.
All code in this tutorial is licensed under the MIT license, which essentially allows you to use (or modify!) the code for any purpose (including commercial purposes) free of charge, with the only condition being that you include the relevant copyright notice and permission notice.
See the
LICENSE file in the addon root for full license and copyright information. Alternatively, you can view the license file in the Git repository for the tutorial code: LICENSE.Just FYI, Composer itself and the Carbon package we will install also use the MIT license.
Getting Started
I have created a basic addon called ComposerTutorial. My
addon.json file contains nothing specific to using Composer, so don't worry about the contents of that file.XenForo is installed on my dev server at
/srv/www/xenforo2 - I will refer to this as the "XenForo root". My addon is installed at /srv/www/xenforo2/src/addons/ComposerTutorial - I will refer to this as my "addon root".The first step to adding a Composer package is to create a
composer.json file in the root of your addon. You can either do this by hand, or use the composer require command to do it for you.
Bash:
$ cd /srv/www/xenforo2/src/addons/ComposerTutorial
$ composer require nesbot/carbon
Using version ^1.32 for nesbot/carbon
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 3 installs, 0 updates, 0 removals
- Installing symfony/polyfill-mbstring (v1.8.0): Loading from cache
- Installing symfony/translation (v4.1.3): Downloading (100%)
- Installing nesbot/carbon (1.32.0): Downloading (100%)
symfony/translation suggests installing symfony/config ()
symfony/translation suggests installing symfony/yaml ()
symfony/translation suggests installing psr/log-implementation (To use logging capability in translator)
Writing lock file
Generating autoload files/srv/www/xenforo2/src/addons/ComposerTutorial/composer.json:
JSON:
{
"require": {
"nesbot/carbon": "^1.32"
}
}require field to make Composer work.If you created your
composer.json file by hand, you would instead run the composer update command to install the package and its dependencies.
Bash:
$ cd /srv/www/xenforo2/src/addons/ComposerTutorial
$ composer updatevendor folder containing the installed packages and all dependencies.
Bash:
$ ls -1p /srv/www/xenforo2/src/addons/ComposerTutorial/vendor/
autoload.php
composer/
nesbot/
symfony/autoload.php file is auto-generated by the composer commands and is for running the autoloader in a stand-alone project. This won't be suitable for using in XenForo, so we will ignore this file.The
composer directory is where the generated autoloader files are - it's worth familiarising yourself with the contents of this directory to understand how Composer finds and loads classes. We will be using some of the files in this directory to identify the classes and files we need to pass to the XenForo autoloader.The
nesbot directory is where the Carbon package is installed.The
symfony directory is where some Carbon dependency packages are installed (symfony/polyfill-mbstring and symfony-translation)Finally, we have a
composer.lock file that was created in our addon root directory. This file is auto-generated by the Composer commands and contains a list of the dependencies (and their specific versions!) that were resolved from composer.json the first time the packages were installed. You should consider the composer.lock file a "version-specific point-in-time snapshot" of the dependencies.While you are developing your addon, you may run the
composer update command from your addon root to update the dependencies, which will cause the composer.lock file to also be updated if new versions have been released. It is important to test that new versions have not included any breaking changes - although if using semantic versioning, this should (in theory) not occur.However, in a production environment, it is important to only ever run the
composer install command if you need to install packages and dependencies (which you shouldn't need to do if we have packaged the vendor folder with the addon - more discussion on this later). The install command does not resolve dependencies anew from the composer.json file, instead it relies on the composer.lock file to install exactly the versions specified. This means that we can be sure that the package versions installed in production are exactly those we have tested and found to work in development/test environments.Either way, we'll cover installation in more detail later. For now, just recognise that the
composer.lock file is important and should be checked into your source code control alongside your composer.json file.So now that we have our package and dependencies installed, we need to get XenForo to autoload everything.
XenForo Autoloader
As mentioned, XenForo uses composer behind the scenes - although they hide the
composer.json file from us, because we don't need to install or update any core packages and dependencies ourselves - their upgrade process takes care of that for us. See the src/vendordirectory under the XenForo root to see the packages used by XenForo.So with XenForo already running an autoloader, we cannot simply call the
autoload.php file from our addon's vendor directory - we need to hook into the XenForo autoloader and add our own dependencies to theirs.I've created a class to help achieve this. I copy this class to each of my addons which use Composer packages.
You can find this file in the addon root in the published addon:
Composer.php - alternatively, view it in our Git repository: Composer.php
PHP:
<?php namespace ComposerTutorial;
/**
* Copyright (c) Simon Hampel
* Based on code used by Composer, which is Copyright (c) Nils Adermann, Jordi Boggiano
*/
// TODO: the namespace will need to be changed when copying this file to a new addon
class Composer
{
public static function autoloadNamespaces(\XF\App $app, $prepend = false)
{
$namespaces = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_namespaces.php';
if (!file_exists($namespaces))
{
$app->error()->logError('Missing vendor autoload files at %s', $namespaces);
}
else
{
$map = require $namespaces;
foreach ($map as $namespace => $path) {
\XF::$autoLoader->add($namespace, $path, $prepend);
}
}
}
public static function autoloadPsr4(\XF\App $app, $prepend = false)
{
$psr4 = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_psr4.php';
if (!file_exists($psr4))
{
$app->error()->logError('Missing vendor autoload files at %s', $psr4);
}
else
{
$map = require $psr4;
foreach ($map as $namespace => $path) {
\XF::$autoLoader->addPsr4($namespace, $path, $prepend);
}
}
}
public static function autoloadClassmap(\XF\App $app)
{
$classmap = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_classmap.php';
if (!file_exists($classmap))
{
$app->error()->logError('Missing vendor autoload files at %s', $classmap);
}
else
{
$map = require $classmap;
if ($map)
{
\XF::$autoLoader->addClassMap($map);
}
}
}
public static function autoloadFiles(\XF\App $app)
{
$files = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_files.php';
// note that autoload_files.php is only generated if there is actually a 'files' directive somewhere in the dependency chain
if (file_exists($files))
{
$includeFiles = require $files;
foreach ($includeFiles as $fileIdentifier => $file)
{
if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
require $file;
$GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
}
}
}
}
}I usually put this in the addon root - but it can go anywhere below your addon root, provided that you adjust the namespaces in this file and the initialisation functions (discussed later) appropriately.
Order of Class Loading
Note that the optional
$prepend parameter can be used to change that behaviour of the first two functions.When resolving a class during execution, the Composer autoloader simply runs through the autoload queue to find the first match for the class being called. If there are duplicate class definitions (for example if multiple addons use the same dependency, or if an addon uses the same depenency as the core XenForo packages), then only the first match is used.
By default, classes are appended to the autoload queue, meaning that the first classes added to the queue (the XenForo core dependencies!) will be given priority. This is generally a good thing, as it prevents you from clobbering the core dependencies - but be sure you check the versions being used by the core to ensure your package is not receiving a substantially different version of a class from the core rather than from your addon dependency packages.
If another addon is initialised before yours, its dependencies will be added to the queue before yours. If both addons contains the same classes - it is their versions which will be loaded, not yours. If this causes problems with other addons, you should be able to adjust the class loading queue order by adjusting the order of execution for the listener we'll set up in the next step. Assuming there are no significant bugs and the dependency versions are not significantly different, this should not cause issues - but this is where unexpected bugs can be introduced, so care is required.
In certain (rare) circumstances, you may need to replace one of the dependency classes provided in the XenForo core. You can set the
$prepend parameter to true when calling the function (see next section), to have the autoloader prepend the dependencies to the autoload queue. This means that your addon packages will be loaded before the XenForo core packages.For my Guzzle 6 addon, I needed to do exactly this because I was replacing core functionality - XenForo v2.0 ships with Guzzle v5.3 and so if I appended my classes to the autoloader queue, the core Guzzle 5.3 versions of the classes would always be found first. By appending my dependencies to the front of the queue, I was able to have Guzzle 6 classes loaded instead. This is a rare occurrence and not generally recommended as it does impact on core functionality and could break things unexpectedly.
Initialising the Composer Autoloader
We can use a code event listener to initialise our addon and add our dependencies to the XF autoloader queue. I create the following
Listener.php file in the root of my addon:
PHP:
<?php namespace ComposerTutorial;
use XF\App;
class Listener
{
public static function appSetup(App $app)
{
Composer::autoloadNamespaces($app);
Composer::autoloadPsr4($app);
Composer::autoloadClassmap($app);
Composer::autoloadFiles($app);
}
}It is here that we can use that
$prepend parameter we mentioned earlier. To over-ride the core dependencies with your own, simply call Composer::autoloadNamespaces($app, true); and Composer::autoloadPsr4($app, true);. Again, this is not recommended other than in exceptional circumstances.Now, we simply add the code event listener to execute the appSetup function we defined above:
- Listen to event:
app_setup - Event hint:
- Execute callback:
ComposerTutorial\Listener::appSetup (namespace would be adjusted to suit your own listener namespace) - Callback execution order:
10(note - this is where we adjust the execution order to get our dependencies to load before another addon's - use with caution! I recommend leaving this set to 10 unless absolutely necessary) - Enable callback execution:
checked - Description:
Autoload Composer packages - Add-on:
Composer Tutorial(obviously, select your own addon from the list)
Let's test this by adding a test tool to the admin area - see source code for details.
To run the test after installing the addon - go to the Admin > Tools > Test Composer Tutorial page and you'll see some output generated by Carbon.
Packaging the Addon
Once the addon is ready for release, we can simply package it up and publish it.
However, there are several different approaches here.
The "purist" approach would be to not check your vendor folder in to source control, but instead rely on the person installing the addon to run
composer install from the addon root on the production server to install the dependencies specified in the composer.lock file. However, while this may work well in an automated deployment environment, it is a pain to have to do this manually and forgetting to do so will break your forum. Similarly, if there are issues connecting to the source of the packages such that the install command fails, your forum could be offline until those issues are resolved. The other downside to this approach is the requirement that Composer be installed on your production server, which may not be desirable.The "pragmatic" approach is to check the entire vendor folder into source control, so that when the addon is deployed, it is complete and ready to use. This is the approach I take.
However, there are some things we should do before packaging our addon.
By default, Composer installs development dependencies (eg unit testing tools) specified in your
require-dev block in composer.json, which are unnecessary in a production environment. Of course, if you don't use require-dev, this isn't an issue.Similarly, there are some optimisations we can (and should!) do to our code for running in production environments. These optimisations can be handled for us by composer - we just need to execute the commands before packaging our addon.
Fortunately, the XenForo devs have been kind enough to provide us with a lovely little tool to run custom commands when building our addon.
Simply create a
build.json file in the root of your addon with the following instructions:
JSON:
{
"exec": [
"composer install --working-dir=_build/upload/src/addons/{addon_id}/ --no-dev --optimize-autoloader"
]
}composer install (not an update!) to ensure we have all dependencies installed as per the current (tested!) version of composer.lock, and includes three composer install options:- --working-dir=_build/upload/src/addons/{addon_id}/: this tells composer that we actually want to optimise the build files, not our development files. These files only exist during the build process and are removed once the release zip file has been generated.
- --no-dev: Skip installing packages listed in
require-dev. The autoloader generation skips theautoload-devrules - --optimize-autoloader: Convert PSR-0/4 autoloading to classmap to get a faster autoloader. This is recommended especially for production, but can take a bit of time to run so it is currently not done by default
Bash:
$ cd /srv/www/xenforo2
$ php cmd.php xf-addon:build-release ComposerTutorial
Performing add-on export.
Exporting data for Composer Tutorial to /srv/www/xenforo2/src/addons/ComposerTutorial/_data.
25/25 [============================] 100%
Written successfully.
Attempting to validate addon.json file...
JSON file validates successfully!
Building release ZIP.
Loading composer repositories with package information
Installing dependencies from lock file
Nothing to install or update
Generating optimized autoload files
Writing release ZIP to /srv/www/xenforo2/src/addons/ComposerTutorial/_releases.
Release written successfully.I have attached the built addon to this tutorial so you can install it, look at the code and experiment with other packages.
Alternatively, clone the git repository for the tutorial code - XenForo Composer Tutorial
