[Bug]: After upgrading to 3.0 taking almost an hour to run and complete test cases

[vasanth-kumar-m-y]

Preconditions

Using this bundle version 3.0.1 with Symfony 6.4, PHP 8.1.28 Mysql Distrib 5.6.51 (mysqldump Ver 10.13), for Linux (x86_64).

Previously with version 2.9.0 the entire test case was taking around 8 minutes to complete. No with new version its taking around 50 minutes to completely run all the test cases.

Is it due to cache_db option is not present in version 3.0.0, a a result the fixtures are loaded on every test case sowing down.

Even tried switching back to previous version but test cases fails with mysqldump: unknown variable 'column-statistics=0'

@alexislefebvre Please guide me here

Steps to reproduce

version 3.0.0 configuration

liip_test_fixtures:
    keep_database_and_schema: true
    cache_metadata: true

Fixture class

use Doctrine\Common\DataFixtures\ReferenceRepository;
use Liip\FunctionalTestBundle\Test\WebTestCase;
use Liip\TestFixturesBundle\Services\DatabaseToolCollection;
use Liip\TestFixturesBundle\Services\DatabaseTools\AbstractDatabaseTool;

class FixturedTestCase extends WebTestCase
{
    /**
     * @var ReferenceRepository
     */
    protected $referenceRepository;

    /**
     * @var AbstractDatabaseTool
     */
    protected $databaseTool;

    /**
     * Setup test class.
     */
    protected function setUp(): void
    {
        $this->databaseTool = static::getContainer()->get(DatabaseToolCollection::class)->get();
        $this->referenceRepository = $this->databaseTool->loadFixtures(Fixtures::FIXTURES)->getReferenceRepository();
    }

    /**
     * Tear down test class.
     */
    protected function tearDown(): void
    {
        unset($this->databaseTool);
        unset($this->referenceRepository);
        parent::tearDown();
    }

An example of test case

class UserRepositoryTest extends FixturedTestCase
{
    /**
     * Setup test class.
     */
    protected function setUp(): void
    {
        parent::setUp();
    }
    
    public function testSomeUser()
    {
        // loading fixture reference
        $user = $this->referenceRepository->getReference(
            'user_some-user'
        );
    }

Expected result

Taking around 50 minutes to completely run all the test cases

Actual result

Was completed within 10 minutes previously with version 2.9.0

[alexislefebvre]

Someone else reported the same issue:

• [3.x] chore: remove backup #289 (comment)

So we consider restoring the backup functionality.

In the meantime, stay on the 2.x version of LiipTestFixturesBundle.

[vasanth-kumar-m-y]

Okay, i tried reverting back to 2.9.0 but the few test cases are failing with Entity of type 'App\Entity\User' for IDs id(6) was not found

Running Symfony 6.4, PHPUnit 9.6.5, PHP 8.1.28 on CENTOS 7.

Where it works with fixture bundle version 3.0.1 but takes too long to execute.

Could you please guide me on this @alexislefebvre

and the fixtures data of this table is emty/deleted

[alexislefebvre]

@vasanth-kumar-m-y You should be able to revert the commit where you upgraded LiipTestFixturesBundle, and your project will be back to the previous state when it worked. (I hope that your composer.json and composer.lock files are versioned)

[vasanth-kumar-m-y]

Yes, will revert it. Thank you @alexislefebvre

[alexislefebvre]

Backups have been reverted on an alpha release, please see here to test it and give feedback:

• [3.x] Revert "chore: remove backup" #316 (comment)

[vasanth-kumar-m-y]

sure, will test this

[vasanth-kumar-m-y]

have tested with the above version, still taking an hour to complete all the test cases @alexislefebvre

[alexislefebvre]

have tested with the above version, still taking an hour to complete all the test cases @alexislefebvre

This is strange because 2.9.0 and 3.1.0-alpha2 have marginal differences: 2.9.0...3.1.0-alpha2

Another user had the same performance with these 2 versions: #316 (comment)

[vasanth-kumar-m-y]

1. With v2.9

• Time: 09:45.757, Memory: 695.00 MB

2. With v3.1.0-alpha1

• Time: 01:31:09.110, Memory: 705.00 MB

3. With v3.1.0-alpha2

• Time: 01:19:03.753, Memory: 729.00 MB

[alexislefebvre]

Did you configure cache_db: … when using 2.9.0 and 3.1.0-alpha2?

[vasanth-kumar-m-y]

yes, no changes in config

liip_test_fixtures:
    cache_db:
        mysql: 'Liip\TestFixturesBundle\Services\DatabaseBackup\MysqlDatabaseBackup'

[alexislefebvre]

Thanks for the feedback, unfortunately I don't know why you test suite is still slow.

A new stable release, that support backups, has been released: https://github.com/liip/LiipTestFixturesBundle/releases/tag/3.1.0

[vasanth-kumar-m-y]

@alexislefebvre Sorry, the issue was with the machine that was used to run the tests. Yes, it takes the same amount of time as version 2.9.
This issue can be closed. Thank you!