Use compiled config for generated data instead of general config

[ilnytskyi]

Description (*)

When we used multi-container ASG we noticed that magento loads a lot of data from shared cache instance (Redis).
That during deploys of new containers leads to downtime or unstable behavior during containers swapping.
We decided to move instance-sensetive cache types such as compiled DI, events, to containers itself.
That prevents cache overriding in shared instance while old container is alive a few minutes after deploy.

... 'compiled_config' => [ 'frontend' => 'file_cache' ], ... 

So we checked what caches lead to such result, and moved some core classes from using shared config type to compiled_config because this data is instance-sensitive and not necessary should be shared.
This also reduses excessive network transfer and overhead of data that depended on certain version of code.

Magento\Framework\Interception\PluginList\PluginList - if new plugin cached in the shared config, the instance with older version can be down if did not find a new class. New instance may not execude the new code if this cache key was overriden by older container.
Magento\Framework\Event\Config\Data - as above, related to adding new events/observers
Magento\Framework\App\Router\ActionList - as above but related to adding new routes and controllers
Magento\Ui\Config\Reader\Definition\Data - related to new Ui components

Once those caches are generated, they are the same until you deploy new version of code.

Related Pull Requests

Fixed Issues (if relevant)

1. Fixes magento/magento2#<issue_number>

Manual testing scenarios (*)

1. Cache keys event_config_cache, ui_component_configuration_definition_data, app_action_list,plugins are stored in compiled config instance.
2. Running two servers with different code version and isolated compiled config does not lead to cache overriding or application downtime or unexpected behavior caused by using the cache from different app version.

Questions or comments

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] Use compiled config for generated data instead of general config #38785: Use compiled config for generated data instead of general config

[m2-assistant]

Hi @ilnytskyi. Thank you for your contribution
Here is some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[ilnytskyi]

@magento run all tests

[mrtuvn]

@magento run Functional Tests CE, Functional Tests B2B, Functional Tests EE

[mrtuvn]

@magento run Functional Tests B2B, Functional Tests EE

[mrtuvn]

CC @sivaschenko look great with final results

[ilnytskyi]

up

[ilnytskyi]

up

[ilnytskyi]

up

[ilnytskyi]

up, it's going to be 3 years for this small change
@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Hotel]

@magento create issue

[engcom-Hotel]

Hello @ilnytskyi,

Thanks for the contribution!

The changes seem good to me, hence approving the PR.

Thanks

[engcom-Charlie]

@magento run all tests

[engcom-Charlie]

Hi @ilnytskyi,

Thank you for your valuable contribution!

Can you please elaborate more on the Steps to reproduce. Kindly mention detailed steps in order to reproduce this issue as well as test the given solution in PR.

Thank you!

[ilnytskyi]

@engcom-Charlie
As mentioned in the description.
The following cache keys event_config_cache, ui_component_configuration_definition_data, app_action_list,plugins will be stored in compiled config instance.

In case of changing the storage for that case from redis to files in env.php

'compiled_config' => [ 'frontend' => 'file_cache' ], ``` Related keys will be stored on disk instead of redis 

[engcom-November]

@ilnytskyi,

As you have added the compiledConfig classes in the di.xml for the instance-sensetive cache. This PR can be verified by looking at the cache generated in the var/cache directory.
As it will have the cache instance of the class for which you have injected the dependency.

Please let us know if we can proceed with this approach.

[ilnytskyi]

@engcom-November yes I think your suggested approach is also viable

[engcom-November]

Hello @ilnytskyi,

Thank you for the collaboration and contribution.

✔️ QA Passed

As the instance-sensetive cache is moved to compiled config, this can be verified by looking at the cache generated at var/cache directory. As it will have the cache instance of the class.

Before: ✖️
There is no cache instance for the respective class.

After: ✔️
It can be seen that CompiledCache instance is present for the respective class.

This can be seen with all the four classes that has been moved to compiled config.

Thank you.

[engcom-November]

@magento run all tests

[engcom-November]

@magento run Functional Tests B2B, Functional Tests CE, Functional Tests EE

[engcom-November]

The repeating failures in Functional tests B2B, CE and EE are flaky and are known issues.

Functional Tests B2B
Run 1:

Run 2:

Functional Tests CE
Run 1:

Run 2:

Functional Tests EE
Run 1:

Run 2:

Known issues:
ACQE-6856 - SlideItemValidateVideoBackgroundVideoURLs
ACQE-6774 - StorefrontCashOnDeliveryPaymentForSpecificCountryTest
ACQE-6695 - TierPricingWhenPriceScopeIsWebsiteWorkingProperlyWithMultipleCurrenciesConfiguredTest
ACQE-6799 - StorefrontLoginFormCheckDuplicateValidateMessageTest
ACQE-6786 - StorefrontAddBundleDynamicProductToShoppingCartWithDisableMiniCartSidebarTest

Hence moving it to Merge In Progress