I have a pretty big Laravel 5.6 application I'm currently developing. By default, Larvel comes with a tests directory along with a tests/Unit and tests/Feature sub directories.
When I first started developing my app, any test I wrote that didn't require a full HTTP life cycle to take place, I would put in the tests/Unit directory.
Example: Testing a custom local query scope on a model.
Similarly, any test that did require a full HTTP life cycle I would put in the tests/Feature directory.
Example: Testing controllers using a GET, POST, etc request and making sure I got back a 200 response or equivalent.
Now that my application has grown to be quite massive, I'm starting to think that there should be a better way to organize my tests directory. I'm also noticing that parts of my code base do not have test coverage based on reports from codecov.io.
In my mind, it would make sense to create a tests/App directory and have that mirror all my files and classes in the app directory of the project root. Each class and/or file would have a corresponding test. This way, I'll have a better idea of which parts of my app have test coverage and which ones do not. I haven't seen any other projects that do this so I'm hesitant to apply this approach.
So, to summarize my question:
Are there any pitfalls with structuring the tests directory exactly like the app directory?
Is there an alternative way to structure your test directory so that you can tell with ease which classes/files have corresponding tests?
Side Note: I'm well aware that my unit tests don't adhere to the "normal" convention of mocking out your dependencies and testing in isolation. This is a separate issue but in short I feel that mocking doesn't allow you to refactor you code with ease. More info on what I'm talking about in my defense: https://www.youtube.com/watch?v=LdUKfbG713M
I'm also using the structure that Laravel provided. but I have made little changes. for example for functional testing, I have added an admin folder so I can write tests for admin APIs or for unit testing I have added (unit/cron, unit/middleware, unit/jobs, ...) this way I can control the directories.
tests/functional/admin
tests/unit/cron
tests/unit/middleware
tests/unit/jobs
tests/unit/policy
tests/unit/models
...
And also to see which API or function has a test or not it's a little hard to have it. If you are following the TDD principle then no need to worry about this part because you know that everyone who writes a piece of code he/she will write the test for it. but if you are not following the TDD and you have just started with writing tests for your project then you need to run PHPUnit coverage with --HTML tag and it'll return for you the HTML graph which can tell you how much code coverage your project has.
All Laravel projects start from an initial template,
if you check the phpunit.xml which the template normally includes, you should find:
...
<testsuites>
<testsuite name="Unit">
<directory suffix="Test.php">./tests/Unit</directory>
</testsuite>
<testsuite name="Feature">
<directory suffix="Test.php">./tests/Feature</directory>
</testsuite>
</testsuites>
All this does is tell PHPUnit to execute/run Unit folder's tests before Feature folder's tests.
The reason for that being;
If a normally small Unit is failing any test,
Then it makes no sense to search for the bug in normally large Feature(s),
And PHPUnit's report shows Unit folder's test-results at top,
which suggests that those should be fixed first, because debugging an entire Feature is normally harder than debugging a small Unit.
Example
Knowing above should help you decide for yourself;
But if you ask me, I would simply create an App folder in Unit folder, or add an additional testsuite to phpunit.xml file, right after Unit entry, like:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
>
<testsuites>
<testsuite name="Unit">
<directory suffix="Test.php">./tests/Unit</directory>
</testsuite>
<testsuite name="Feature">
<directory suffix="Test.php">./tests/Feature</directory>
</testsuite>
<testsuite name="App">
<directory suffix="Test.php">./tests/App</directory>
</testsuite>
</testsuites>
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
</filter>
<php>
<server name="APP_ENV" value="testing"/>
<server name="BCRYPT_ROUNDS" value="4"/>
<server name="CACHE_DRIVER" value="array"/>
<!-- <server name="DB_CONNECTION" value="sqlite"/> -->
<!-- <server name="DB_DATABASE" value=":memory:"/> -->
<server name="MAIL_MAILER" value="array"/>
<server name="QUEUE_CONNECTION" value="sync"/>
<server name="SESSION_DRIVER" value="array"/>
<server name="TELESCOPE_ENABLED" value="false"/>
</php>
</phpunit>
Because having one test-file beside each php-file is a Unit-testing style, not Feature-testing, even if the test requires "full HTTP life cycle".
Related
I have an API method that is called by clicking a buttom in a form and inside that API method there is a request to another API of another project using guzzle. That works fine.
My problem is I'm doing behat tests and I want to test my API method, but I need to mock the request to the external API (because I don't need to test it). Is there any way Laravel detects if I am in a test environment and mock the request and if I'm in a normal environment leave it without mock?
There should be a file in your project called phpunit.xml and you can set the APP_ENV there.
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
...
processIsolation="false"
stopOnFailure="false">
<testsuites>
...
<testsuite name="Unit">
<directory suffix="Test.php">./tests/Unit</directory>
</testsuite>
</testsuites>
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
</filter>
<php>
<!-- HERE -->
<env name="APP_ENV" value="testing"/>
<env name="CACHE_DRIVER" value="array"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="QUEUE_DRIVER" value="sync"/>
</php>
Then you can use whatever logic you like with if(env('APP_ENV') == 'testing')
I am trying to use the feature in PhpStorm for code coverage but it shows me this error saying
"No whitelist is configured, no code coverage will be generated."
I am running PHP 7.2 on my local Mac machine.
EDIT: I have already added phpunit settings for whitelisting and logggin
This is what my phpunit.xml file looks like:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit printerClass="Codedungeon\PHPUnitPrettyResultPrinter\Printer"
backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false">
<testsuites>
<testsuite name="Feature">
<directory suffix="Test.php">./tests/Feature</directory>
</testsuite>
<testsuite name="Unit">
<directory suffix="Test.php">./tests/Unit</directory>
</testsuite>
</testsuites>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
</filter>
<logging>
<log type="coverage-html" target="./report" charset="UTF-8"
yui="true" highlight="true"
lowUpperBound="50" highLowerBound="80" />
</logging>
<php>
</php>
</phpunit>
I have also tried setting
processUncoveredFilesFromWhitelist
instead of
addUncoveredFilesFromWhitelist
But still the same error
I ran into the same issue. This might help.
First, make sure you can run code coverage on the command line.
Then, look at what command Phpstorm is trying to run. It's given in the test output, second line:
Testing started at 16:44 ...
C:\...\php.exe -dxdebug.coverage_enable=1 C:/.../vendor/phpunit/phpunit/phpunit --coverage-clover C:\.\XXX.xml --no-configuration --filter "/(::XXX)( .*)?$/" App\XxxTest C:\XXX.php --teamcity
PHPUnit 7.3.3 by Sebastian Bergmann and contributors.
Error: No whitelist is configured, no code coverage will be generated.
I tried to run this on the command line myself, changing the paths.
What I noticed was "--no-configuration". This means it's not even reading my phpunit.xml file. To fix this, set
Settings -> Languages & Frameworkds -> Php -> Test Frameworks -> "Default configuration file" to your phpunit.xml file.
You might have a different problem, but this might point you in the right direction.
You need to configure whitelisting, that's a phpunit setting, see https://phpunit.de/manual/6.5/en/code-coverage-analysis.html#code-coverage-analysis.whitelisting-files
Run below command. It worked for me. I am using PHP 7.1, Symfony(1.x) and PHPUnit(5.7) legacy version. So directory structure could be different according to the version.
symfony/lib> vendor/bin/phpunit --whitelist ../[valid dir name]/
--coverage-clover test.xml UnitTest ../[dir]/[file_name]
symfony/lib> vendor/bin/phpunit --whitelist ../plugins/
--coverage-clover test.xml UnitTest ../test/PluginAllTests.php
OR
symfony/lib> vendor/bin/phpunit --whitelist ../tests/
--coverage-clover test.xml UnitTest ../test/PluginAllTests.php
I am using Laravel 5.5, phpunit 6.5.5 on Homestead-7 (I think).
I am trying this tutorial: https://laravel-news.com/your-first-laravel-application (And this should tell you a lot about my experience with the framework)
Testing fails (due to TokenMismatchException) and I have managed to track down the root cause to the APP_ENV variable being set to local, although I've tried many ways of setting it to testing.
At the end, what allowed me to overcome the problem was to set the variable like this:
APP_ENV=testing vendor/bin/phpunit
Then, the tests completed successfully.
My question is, what am I doing wrong? The above hack is obviously not the best way to do it. There must be a way to properly do this.
Update
Contents of phpunit.xml:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false">
<testsuites>
<testsuite name="Feature">
<directory suffix="Test.php">./tests/Feature</directory>
</testsuite>
<testsuite name="Unit">
<directory suffix="Test.php">./tests/Unit</directory>
</testsuite>
</testsuites>
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
</filter>
<php>
<env name="APP_ENV" value="testing"/>
<env name="CACHE_DRIVER" value="array"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="QUEUE_DRIVER" value="sync"/>
<env name="DB_CONNECTION" value="sqlite"/>
<env name="DB_DATABASE" value=":memory:"/>
</php>
</phpunit>
Thank you in advance for your time.
From the Laravel docs:
When running tests via phpunit, Laravel will automatically set the configuration environment to testing because of the environment variables defined in the phpunit.xml file.
...so I would bet that your phpunit.xml file is missing or misconfigured. It should be in your project's root directory and should contain the following:
<php>
<env name="APP_ENV" value="testing"/>
...
</php>
If you do have a phpunit.xml file in the right place and it does contain the part above, and it's still not working, then try clearing your config cache:
php artisan config:clear
If that still doesn't fix it, then I'd check for something odd with the phpunit.xml file, such as being misnamed or containing a syntax error.
And here's a link to the original phpunit.xml file, to help you restore it if needed.
I managed to find the problem with my system. In my homestead box, I had included in Homestead.yaml setting of global environment variable APP_ENV:
variables:
- key: APP_ENV
- value: local
This was provided as an example in the instructions for Homestead set up. I followed that without realising what I was doing.
Setting again the variable through .env or phpunit.xml just did not work. I removed the definition from Homestead.yaml and it works as I would have expected.
I have what I think is an uncommon issue but I'll try to explain as clear as I can. I have created a simple artisan command that does this
/**
* Execute the console command.
*/
public function handle()
{
$this->info("==> Cleaning up reports and docs...");
$command = new Process("rm -f tests/docs/* && rm -rf test/reports/*");
$command->run();
$this->warn("==> Reports and docs are clean");
$this->info("==> Executing Tests Suite...");
$command = new Process("vendor/bin/phpunit --coverage-html tests/reports --testdox-html tests/docs/reports.html -v --debug");
$command->run();
$this->info($command->getIncrementalOutput());
$this->warn("==> report generated >> test/reports. Documentation generated >> test/docs/reports.html");
}
This could seems kinda odd but it is actually pretty useful, it launch the PHPUnit with the coverage support and other stuff. The problem is that if I run this command like php artisan ludo237:full-test it will completely ignore the phpunit.xml in fact it will display and error saying that the MySQL database does not exists, even though I've set the sqlite connection inside my phpunit.xml speaking of which you can clearly see that is correct:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
backupStaticAttributes="false"
bootstrap="bootstrap/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false">
<testsuites>
<testsuite name="Application Test Suite">
<directory suffix="Test.php">./tests</directory>
</testsuite>
</testsuites>
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
</filter>
<php>
<env name="APP_ENV" value="testing"/>
<env name="APP_DEBUG" value="true"/>
<env name="APP_URL" value="http://localhost:8000"/>
<env name="DB_CONNECTION" value="sqlite"/>
<env name="DB_DATABASE" value=":memory:" />
<env name="CACHE_DRIVER" value="array"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="QUEUE_DRIVER" value="sync"/>
<env name="MAIL_DRIVER" value="log"/>
<env name="MAIL_PRETEND" value="true"/>
</php>
</phpunit>
I know that I can simply create a bash alias for that command line, in fact this is my current hot fix, but at this point I'm just curious to understand why when I launch the phpunit command from the artisan command it ignores the XML file.
Does anyone have a clue about this? Thank you!
It took me really a lot to debug this, but in the end I found why it is not working.
First of all, let me clarify that is nothing wrong with your code, even there is nothing wrong with the Process class, you will get the same behavior even by using native exec function.
Second, the phpunit.xml file is totally being read, no issues at all.
Said that, the real issue resides, on PHPUnit that doesn't allow you to re-define ( or override ) any env variable, because there is a check that prevents that.
This is because Laravel is not parsing the phpunit.xml file itself, and in the moment you're calling it, it's already too late and the environment variables are defined, either via putenv, $_SERVER and/or $_ENV.
Because of this, I created an Issue on PHPUnit, which even in latest versions ( as of today ), this issue persist, even if this was already asked in the past.
Unfortunately I have no solution yet for you, but let's cross our fingers and let's wait for PHPUnit to add the proposed force attribute :)
Best regards,
Julian
I am wondering if it is possible to configure PHPStorm to exclude the Test folder from the percentages, since my module has 100% coverage but the Test folder is bringing the overall value down to 50%.
This could be unrelated, but I also have an issue where the coverage window never displays the actual coverage data, neither in the separate tab nor viewing the file itself.
Chances are I have mis-configured something, or it is something to do with my project setup (composer based Magento project with symlinking) but I'm banging my head against the wall a bit. Any suggestions welcome.
Update to include phpunit.xml:
<?xml version="1.0"?>
<!-- initial phpunit configuration file, that you can modify for your project needs -->
<phpunit cacheTokens="true"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
strict="false"
stderr="true"
verbose="false"
bootstrap="app/code/community/EcomDev/PHPUnit/bootstrap.php">
<listeners>
<listener file="app/code/community/EcomDev/PHPUnit/Test/Listener.php" class="EcomDev_PHPUnit_Test_Listener"/>
</listeners>
<testsuite name="Magento Test Suite">
<file>app/code/community/EcomDev/PHPUnit/Test/Suite.php</file>
</testsuite>
<filter>
<blacklist>
<!-- Exclude Magento Core files from code coverage -->
<directory suffix=".php">app/code/core</directory>
<!-- Exclude EcomDev_PHPUnit classes from code coverage -->
<directory suffix=".php">app/code/community/EcomDev/PHPUnit</directory>
<directory suffix=".php">lib/EcomDev/Utils</directory>
<directory suffix=".php">lib/EcomDev/PHPUnit</directory>
<directory suffix=".php">lib/Spyc</directory>
<directory suffix=".php">lib/vfsStream</directory>
<!-- Exclude Mage.php file from code coverage -->
<file>app/Mage.php</file>
<!-- Exclude template files -->
<directory suffix=".phtml">app/design</directory>
<!-- Exclude Varien & Zend libraries -->
<directory suffix=".php">lib/Varien</directory>
<directory suffix=".php">lib/Zend</directory>
<directory suffix=".php">lib/Magento</directory>
</blacklist>
</filter>
<logging>
<log type="coverage-html" target="var/phpunit/coverage" charset="UTF-8" yui="true" highlight="false" lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="var/phpunit/coverage.xml"/>
<log type="junit" target="var/phpunit/junit.xml" logIncompleteSkipped="false"/>
</logging>
</phpunit>
The reason I thought it might be PHPStorm is the fact that the HTML version of the coverage works perfectly, correctly excluding the Test folder.
This has nothing to do with PhpStorm. You simply need to configure a whitelist for your project.