Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cookbook/bundles/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ Bundles
extension
configuration
prepend_extension
tests
311 changes: 311 additions & 0 deletions cookbook/bundles/tests.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,311 @@
.. index::
single: Bundle; Testing third party bundles

Writing Tests for Third Party Bundles
=====================================

The Standard Edition's front controller looks like this::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually have a little introduction paragraph at the start of the article, instead of starting with the text directly. What about adding such a paragraph here as well? For instance:

When creating a shared bundle, you need to verify if the bundle works as it should. The best practice is to write automatic functional tests. This article will guide you through setting up and running such tests.


// web/app.php
use Symfony\Component\HttpFoundation\Request;

require_once __DIR__.'/../vendor/autoload.php';
require_once __DIR__.'/../app/AppKernel.php';

// Create the request from PHP variables super globals such as $_GET, $_POST, $_SERVER, etc
$request = Request::createFromGlobals();

// The application handles the Request
$kernel = new AppKernel('prod', false);
$response = $kernel->handle($request);

// Dump the Response's headers and print the Response's content
$response->send();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd remove this block - it's a bit of a maintenance problem. Instead, just describe how you could create a kernel and a "fake" request in order to test your bundle.


In this example, Symfony is used as an application that mimicks the HTTP protocol,
which means that it is possible to write transversal tests (System Tests) by building
"manually" a ``Request``, giving it to our application and then check the returned ``Response``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"[...] by manually building a Request [...], passing it to the application and checking the returned Response." (I used bold text to emphasize the changes)


This article provides examples on how to write such tests with third party bundles.

Creating a Minimal AppKernel
----------------------------

Third party bundles cannot be ran standalone, they need an application.
It is possible to create a minimalistic ``AppKernel`` inside the bundle for testing
and showcase purposes, instead of:

# Creating a separate application;
# Install the bundle in it;
# Then check how things turn out.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to use #. (not the dot) for numerated lists

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, to make everything readable, I propose to remove this list and integrate it in the sentence: "To test third party bundles, you can install the bundle in a Symfony application and manually verifying if everything works as expected. However, it is usually much quicker and easier if you create automated tests for this. You then need to create a minimalistic ``AppKernel inside the bundle to use for testing."


If the third party bundle is named ``AcmeMyBundle``, the ``AppKernel`` could look like this::

// Tests/app/AppKernel.php
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally find it weird to have your root app in Test. I would at least put it another folder like SymfonyApp or something

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's there because the application only makes sense for the bundle's tests, but I'm open to discussions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm fine with that part, what I'm saying is that rather having:

Tests/ app/ TestBundle/ Command/ ACommandTestCase.php ... ... (other tests directory as any `Tests` folder 

I would have:

Tests/ SymfonyApp/ app/ TestBundle/ Command/ ACommandTestCase.php ... ... (other tests directory as any `Tests` folder 
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's a matter of taste: some will argue that the tree grows unnecessarily deeper, others will agree that it keeps a "test application" separate from the tests.
To be fair, I used to put it in Resources/app, my point here is that there's many possibilities out there and each can be justified. This is a documentation article so nothing is forced on the developer, whatever we choose here won't prevent them to move it where they want. Maybe we should keep it this way and add a note?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnugat yeah it's a matter of taste and every developer who has a good understanding of how it works and is confortable with it will choose whatever he judge better (ex for my part I wouldn't even have a app folder). My point is more for the newcomer, at which this doc is directed. Based on some FOSBundles like FOSRestBundle, the symfony app is always in a subfolder not at the root of Tests.

But yeah, a note on this would be welcomed :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to use the convention of the Symfony core bundles here: Create a Tests/Functional directory and put the AppKernel in Tests/Functional/app. This convention is also used by FOS, KnpLabs and Sonata.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 :)

namespace Acme\MyBundle\Tests;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This results in an issue with KernelTestCase if I remember right. Might have to override it:

namespace Acme\MyBundle\Tests; use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase as SymfonyKernelTestCase; class KernelTestCase extends SymfonyKernelTestCase { protected static $class = 'Acme\MyBundle\Tests\SymfonyApp\AppKernel'; }
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually AppKernel doesn't need a namespace, I'll fix that part

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't but it makes sense no? I mean it looks weird to me to having everything under a namespace, bundles declared only for this app for testing purpose too, except the Kernel

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well in the scope of this article there's no TestBundles so it might not look that shocking, especially since we stick to the same skeleton as the standard edition.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 to remove the namespace


use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
public function registerBundles()
{
return array(
new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
new Acme\MyBundle\AcmeMyBundle(),
);
}

public function registerContainerConfiguration(LoaderInterface $loader)
{
$loader->load(__DIR__.'/config.yml');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use the Closure loader here and then not have an external configuration file.

}
}

.. note::

``AppKernel`` adds a direct dependency on Symfony's FrameworkBundle, which can be
installed by running the following command: ``composer require --dev symfony/framework-bundle``.

In its minimum state ``AppKernel`` requires only one configuration parameter, ``secret``:

.. configuration-block::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to have a configuration block, as we only show one format


.. code-block:: yaml

# Tests/app/config.yml
framework:
secret: Th1s1sS3cr3t!

Symfony applications generate cache and logs directories, which can be ignored by
adding those lines to ``.gitignore``:

.. code-block::

# .gitignore
/Tests/app/cache
/Tests/app/logs
/vendor
/composer.lock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the last 2 aren't related to testing, so I propose to remove them and replace them with // ...


Browsing Pages
--------------

If the third party bundle provides a controller, it can be helpful to create a
front controller similar to the one that can be found in Symfony's Standard Edition::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the purpose of this? Why would a third-party bundle need a front controller?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the point is simply to make it convenient to "try" your controllers, then I'd much rather a third-party bundle author install their bundle into a real project to try it. If we're doing this to functionally test the bundle, we should just instantiate the kernel directly as you've shown below.


// Tests/app/web.php
use Symfony\Component\HttpFoundation\Request;

require_once __DIR__.'/autoload.php';

$kernel = new AppKernel('prod', false);
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();

With this it becomes possible to browse the page. A way to do it without having to configure
a web server is to run the following command:

.. code-block:: bash

$ php Tests/app/console.php server:run -d Tests/app
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this relies on the next section to create the console, so it look weird

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to improve this part: should I remove it, or move it around? I could use some advice here :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to convert this to a simple sidebar and adding it after the Running Commands section. Like:

.. sidebar:: Visiting Pages in the Browser As you can run commands now, it is possible to run the internal webserver to visit the pages in a browser. In order to do this, you have to add a minimal front controller (just like the one that can be found in the Symfony Standard Edition::  // Tests/Functional/app/web.php  use Symfony\Component\HttpFoundation\Request;  require_once __DIR__.'/autoload.php';  $kernel = new AppKernel('prod', false);  $request = Request::createFromGlobals();  $response = $kernel->handle($request);  $response->send(); Now, you can execute the ``server:run`` comand: .. code-block:: bash   $ php Tests/Functional/app/console.php server:run -d Tests/app
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I propose to rename web.php to app_test.php and then use php Tests/Functional/app/console.php server:run --env=test -d Tests/Functional/app. This way, everything works as expected


.. note::

If the third party bundle uses the Twig templating engine to render HTML pages
or if it uses the Form Component or anything else, then more dependencies and
configuration parameters should be added.

Running Commands
----------------

If the third party bundle provides a command, it can be helpful to create a ``console``
similar to the one that can be found in Symfony's Standard Edition::

// Tests/app/console.php
set_time_limit(0);

require_once __DIR__.'/autoload.php';

use Symfony\Bundle\FrameworkBundle\Console\Application;

$kernel = new AppKernel('dev', true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$kernel = new Acme\MyBundle\Tests\AppKernel('dev', true);
$application = new Application($kernel);
$application->run();

With this it becomes possible to run manually the console:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"[...] to manually run the console"


.. code-block:: bash

$ php Tests/app/console.php

Automated Tests
---------------

Manual tests are great to get a quick idea of what the bundle does. But writing
automated tests is even better!

The first step is to install a test framework like PHPUnit:

.. code-block:: bash

$ composer require --dev phpunit/phpunit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Symfony relies on PHPUnit installed globally. So insted of showing the composer installation here, you should link to the PHPUnit installation doc and let users choose what they prefer

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do


.. note::

The steps should be similar with other tests frameworks.

Then the second one is to configure it to use Composer's autoloading:

.. configuration-block::

.. code-block:: xml

<?xml version="1.0" encoding="UTF-8"?>
<!-- phpunit.xml.dist -->

<!-- http://phpunit.de/manual/4.3/en/appendixes.configuration.html -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.3/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="./vendor/autoload.php"
>
<testsuites>
<testsuite name="Test Suite">
<directory>./Tests/</directory>
</testsuite>
</testsuites>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to configure the KERNEL_DIR env variable, so the kernel is autoloaded & instantiated

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only autoloaded if you use KernelTestCase with PHPUnit.
I personally prefer to explicitly load AppKernel as it will work with any test framework.

<filter>
<whitelist>
<directory>.</directory>
<exclude>
<directory>./Resources</directory>
<directory>./Tests</directory>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also exclude vendor from the code coverage collection, otherwise it will make the report totally unusable (and might create weird issues when it will load bin files in the vendors)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

</exclude>
</whitelist>
</filter>
</phpunit>

With these two simple steps it becomes possible to run the test suite with the following command:

.. code-block:: bash

$ vendor/bin/phpunit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use $ "vendor/bin/phpunit". This doesn't cause problems on Unix PCs and does also work on Windows (the current command doesn't)


Functional Web Tests
~~~~~~~~~~~~~~~~~~~~

As advised in the official best practices (smoke testing), writing tests for
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of the (smoke testing). part, I propose to link "official best practices" to the article/section

controllers can be done by simply checking the status code::

// Tests/Controller/DemoControllerTest.php
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to put this in the Tests/Functional directory as well (As done in Symfony bundles)

namespace Acme\MyBundle\Tests\Controller;

use Acme\MyBundle\Tests\AppKernel;
use Symfony\Component\HttpFoundation\Request;

class DemoControllerTest extends \PHPUnit_Framework_TestCase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use WebTestCase here:

class DemoControllerTest extends WebTestCase { /**  * @dataProvider providePages  */ public function testPagesRunSuccessfully($url, $content = null) { $client = static::createClient(); $crawler = $client->request($url); $response = $crawler->getResponse(); $this->assertTrue($response->isSuccessfull()); if (null !== $content) { $this->assertContains($content, $response->getContent()); } } public function providePages() { return array( array('/demo', 'Hello World!'), // verifies success and some special response text array('/demo/something'), // only verifies that the response is a success ); } }
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no point in using WebTestCase here.
Also testing the content defeats the point of the article (which is promoting smoke testing as suggested in the official best practices).
This approach is driven by the fact that the content will change often making them fragile and the value provided by those checks don't make it worth it.

{
private $app;

protected function setUp()
{
$this->app = new AppKernel('test', false);
$this->app->boot();
}

public function testItRunsSuccessfully()
{
$headers = array('CONTENT_TYPE' => 'application/json');
$content = array('parameter' => 'value');
$request = Request::create('/demo', 'POST', array(), array(), array(), $headers, $content);

$response = $this->app->handle($request);

$this->assertSame(200, $response->getStatusCode(), $response->getContent());
}
}

Functional CLI Tests
~~~~~~~~~~~~~~~~~~~~

As advised in the official best practices (smoke testing), writing tests for
commands can be done by simply checking the exit code::

// Tests/Command/DemoCommandTest.php
namespace Acme\MyBundle\Tests\Command;

use Acme\MyBundle\Tests\AppKernel;
use Symfony\Bundle\FrameworkBundle\Console\Application;
use Symfony\Component\Console\Tester\ApplicationTester;

class DemoCommandTest extends \PHPUnit_Framework_TestCase
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using a KernelTestCase instead?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the point of using KernelTestCase here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a bootstrap to resolve the kernel name through the PHPUnit config and eventually some other config parameters, nothing more

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, but in this precise situation it's not really necessary since we're already loading the Kernel.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should, as it makes things easier (for instance, only having to change phpunit.xml.dist when moving the kernel directory) and less to set-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't there be a problem with composer autoloading?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, when configuring the kernel dir as a server parameter (using pHPunit config), the kernel is automatically autoloaded by the KernelTestCase.

{
private $application;

protected function setUp()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be refactored in:

protected function setUp() { self::bootKernel(); $this->application = new Application(self::$kernel); $this->application->setAutoExit(false); }

if use KernelTestCase

{
$kernel = new AppKernel('test', false);
$application = new Application($kernel);
$application->setAutoExit(false);
$this->application = new ApplicationTester($application);
}

public function testItRunsSuccessfully()
{
$exitCode = $this->application->run(array(
'command_name' => 'acme:demo',
'argument' => 'value',
'--option' => 'value',
));

$this->assertSame(0, $exitCode, $this->application->getDisplay());
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would add a note to @matthiasnoback service validator perhaps

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean add a section about testing service definitions? I'm not sure what's the policy about pointing to external resources.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm 👍 on referring to external resources if they provide real value (which they do in this case).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I think we should just reference it and not show a code example of it. Mathias actually has 2 great PHPunit libraries which are very usefull for bundle maintainers (extension test and configuration test).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about showing an example as follow: https://github.com/gnugat/redaktilo-bundle/blob/master/Tests/ServiceTest.php

And then suggest to do more thorough tests using Matthias's library?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnugat That's a good idea.


Service Definition Tests
~~~~~~~~~~~~~~~~~~~~~~~~

It is possible to check that a service is correctly defined using a test::

// Tests/ServiceDefinition/DemoServiceDefinitionTest.php
namespace Acme\MyBundle\Tests\ServiceDefinition;

use Acme\MyBundle\Tests\AppKernel;

class DemoServiceDefinitionTest extends \PHPUnit_Framework_TestCase
{
private $container;

protected function setUp()
{
$app = new AppKernel('test', false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here (about using KernelTestCase)

$app->boot();
$this->container = $app->getContainer();
}

public function testItIsDefinedCorrectly()
{
$demo = $this->container->get('app.demo');

$this->assertInstanceOf('Acme\Service\Demo', $demo);
}
}

.. tip::

Matthias Noback's library, `SymfonyServiceDefinitioValidator`_, is also
a good way to check service definitions.

Conclusion
----------

By creating a minimal ``AppKernel`` in a third party bundle it becomes possible
to run it on its own which can be useful for showcases, but most importantly: It
makes it possible to write automated tests.

.. _SymfonyServiceDefinitioValidator: https://github.com/matthiasnoback/symfony-service-definition-validator
1 change: 1 addition & 0 deletions cookbook/map.rst.inc
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
* :doc:`/cookbook/bundles/extension`
* :doc:`/cookbook/bundles/configuration`
* :doc:`/cookbook/bundles/prepend_extension`
* :doc:`/cookbook/bundles/tests`

* :doc:`/cookbook/cache/index`

Expand Down