Skip to content

Conversation

igorwulff
Copy link
Contributor

@igorwulff igorwulff commented Oct 21, 2022

The magic calls from the DataObject are being called over a 100k when there is no block_cache available and depending on the page often thousands of times when all block_cache is available.

The small change for addData() is just included because its a tiny optimization and prevents a bug from happening. When a $value is an array the original call to setData would override the internal $_data array, this is not expected behavior and just setting it directly to $_data is faster anyway.

The other change is in the __call magic method and the _underscore function and the change is quite radical. Since this method can be called so often, we want to prevent further function calls if possible. Functional calls are normally not that expensive, but will become more expensive if you use this extensively (thousands of even hundred of thousands extra calls):

These are my own benchmarks based on a local environment with opcache and with PROD mode.

HOMEPAGE transaction

  • All cache enabled except for FPC (__call is called 975 times): 4.318307ms -> 1.469747ms
  • All cache enabled except for FPC + Block_html (__call is called 100379 times): 385.36412ms -> 88.412914ms

PLP transaction:

  • All cache enabled except for FPC (__call is called 4628 times): 19.625463ms -> 5.754514ms
  • All cache enabled except for FPC + Block_html (__call is called 112591 times): 404.84066ms -> 99.390374ms

Now this PR also skips some checks. It assumes if a $method call start with 'g' it will be for 'get' for example, this is done in my PR to get the most optimal performance result but might not be preferred. I also removed an index check with $args[1] for the setter since it isn't used anywhere, but might also be something we would want to change.

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

  1. Fixes magento/magento2#<issue_number>

Manual testing scenarios (*)

  1. ...
  2. ...

Questions or comments

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

Resolved issues:

  1. resolves [Issue] Improve performance of DataObject magic calls and addData() #37637: Improve performance of DataObject magic calls and addData()
The magic calls from the DataObject are being called over a 100k when there is no block_cache available and depending on the page often thousands of times when all block_cache is available. The small change for addData() is just included because its a tiny optimization and prevents a bug from happening. When a $value is an array the original call to setData would override the internal $_data array, this is not expected behavior and just setting it directly to $_data is faster anyway. The other change is in the __call magic method and the _underscore function and the change is quite radical. Since this method can be called so often, we want to prevent further function calls if possible. Functional calls are normally not that expensive, but will become more expensive if you use this extensively (thousands of even hundred of thousands extra calls): These are my own benchmarks based on a local environment with opcache and with PROD mode. HOMEPAGE transaction All cache enabled except for FPC (__call is called 975 times): 4.318307ms -> 1.469747ms All cache enabled except for FPC + Block_html (__call is called 100379 times): 385.36412ms -> 88.412914ms PLP transaction: All cache enabled except for FPC (__call is called 4628 times): 19.625463ms -> 5.754514ms All cache enabled except for FPC + Block_html (__call is called 112591 times): 404.84066ms -> 99.390374ms Now this PR also skips some checks. It assumes if a $method call start with 'g' it will be for 'get' for example, this is done in my PR to get the most optimal performance result but might not be preferred. I also removed an index check with $args[1] for the setter since it isn't used anywhere, but might also be something we would want to change.
@m2-assistant
Copy link

m2-assistant bot commented Oct 21, 2022

Hi @igorwulff. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE,
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

For more details, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

@ihor-sviziev ihor-sviziev self-assigned this Oct 21, 2022
@ihor-sviziev ihor-sviziev added Award: category of expertise Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Area: Performance and removed Progress: review labels Oct 21, 2022
@ihor-sviziev
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

11 similar comments
@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

@igorwulff
Copy link
Contributor Author

Hi @ihor-sviziev We've been analyzing and testing this specific patch and think a change is required for the magic 'get' call, since we are seeing some issues in various areas, what do you think?

case 'get': if (isset($args[0]) && $args[0] !== null) { return $this->getData( self::$_underscoreCache[$method] ?? $this->_underscore($method), $args[0] ); } return $this->_data[ self::$_underscoreCache[$method] ?? $this->_underscore($method) ] ?? null; 
@hostep
Copy link
Contributor

hostep commented Aug 24, 2023

So you are closing a proven solution just because it doesn't match exactly with the numbers in the description?

@ihor-sviziev what do you make of this? If it improves performance, even if not 100% exact the same as written in the description and causes no regressions, why don't we go through with it?

@ihor-sviziev ihor-sviziev reopened this Aug 24, 2023
@m2-github-services m2-github-services added Partner: Youwe partners-contribution Pull Request is created by Magento Partner labels Aug 24, 2023
@m2-assistant
Copy link

m2-assistant bot commented Aug 24, 2023

Hi @igorwulff. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.

Add the comment under your pull request to deploy test or vanilla Magento instance:
  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:
  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.


For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

@engcom-Hotel
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

@magento run all tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

@magento run Functional Tests B2B, Functional Tests CE, Functional Tests EE, Integration Tests, Unit Tests, WebAPI Tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

engcom-Hotel commented Aug 25, 2023

✔️ QA Passed

While testing this PR we have below observations.

In order to test the performance improvement we have used the below profiler:

https://github.com/NoiseByNorthwest/php-spx

HomePage: With FPC Disable

2.4-develop
2 4-develop_homepage

After PR Changes
PR_homepage

HomePage: With FPC and block_html cache Disable

2.4-develop
2 4-develop_homepage_fpc_blockhtml

After PR Changes
PR_homepage_fpc_blockhtml

PLP: With FPC Disable

2.4-develop
2 4-develop_PLP

After PR Changes
PR_PLP

PLP: With FPC and block_html cache Disable

2.4-develop
2 4-develop_PLP_fpc_blockhtml

After PR Changes
PR_PLP_fpc_blockhtml

As per the above screenshot, we can see the major improvements in the HomePage rendering after PR changes but it looks similar in the case of PLP after and before PR changes.

Hence we can move forward with the PR, but due to some tests are still failing, moving this PR into Extended Testing.

Thanks

@engcom-Hotel
Copy link
Contributor

@magento run Functional Tests CE, Functional Tests EE, Unit Tests, WebAPI Tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

@magento run Functional Tests EE, WebAPI Tests

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

@magento run Functional Tests EE

@magento-automated-testing
Copy link

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please message the #magento-devops slack channel if they don't show in a reasonable amount of time and a representative will look into any issues.

@engcom-Hotel
Copy link
Contributor

All tests are green, hence moving this PR into Merge in Progress

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Performance Award: category of expertise Partner: Youwe partners-contribution Pull Request is created by Magento Partner Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: accept Project: Community Picked PRs upvoted by the community Triage: Dev.Experience Issue related to Developer Experience and needs help with Triage to Confirm or Reject it