Skip to content

Set all instances of cache context to be transient #29699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

mlambley
Copy link

@mlambley mlambley commented Aug 20, 2020

Description (*)

Cache context has no way to know which items have been cleaned. When an item is added to the context and then cleaned, this is not an issue. But as more items are added and cleaned, those earlier items are cleaned again and again. This results in an exponential performance drain. This PR seeks to remedy this by making all instances of cache context transient.

Fixed Issues (if relevant)

  1. Fixes Cache cleaner performance issue when saving multiple products #29275

Manual testing scenarios (*)

$skus = [1000 product skus];
$products = [];
foreach ($skus as $sku) {
	$products[] = $this->productRepository->get($sku, true, 0, true);
}

foreach ($products as $product) {
	$product->setPrice(12);
	$this->productRepository->save($product);
}

Before applying these changes, put a debug in \Magento\Indexer\Model\Indexer\CacheCleaner looking at the value of $identities in cleanCache(). Watch the array grow with each product save, with the same identities being cleaned multiple times. Now apply the changes and see the $identities not grow.

Questions or comments

Please give feedback. Thank you.

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • All automated tests passed successfully (all builds are green)

@m2-assistant
Copy link

m2-assistant bot commented Aug 20, 2020

Hi @mlambley. Thank you for your contribution
Here is some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE,
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

@ihor-sviziev
Copy link
Contributor

@magento run all tests

Copy link
Contributor

@ihor-sviziev ihor-sviziev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mlambley,
I general your changes looks good to me, but I see that webapi tests are failing now, so seems like your change did some regression.
Could you review failing tests and adjust your code changes accordingly?

Also would be good to cover your changes with some kind of tests, I believe integration tests will fit there. Could you do that?

@ihor-sviziev ihor-sviziev added Auto-Tests: Not Covered Changes in Pull Request requires coverage by auto-tests Severity: S3 Affects non-critical data or functionality and does not force users to employ a workaround. Progress: needs update and removed Progress: needs update labels Aug 21, 2020
@sidolov sidolov added the Priority: P3 May be fixed according to the position in the backlog. label Aug 26, 2020
@sidolov
Copy link
Contributor

sidolov commented Sep 4, 2020

@mlambley I am closing this PR now due to inactivity.
Please reopen and update if you wish to continue.
Thank you for the collaboration!

@sidolov sidolov closed this Sep 4, 2020
@m2-assistant
Copy link

m2-assistant bot commented Sep 4, 2020

Hi @mlambley, thank you for your contribution!
Please, complete Contribution Survey, it will take less than a minute.
Your feedback will help us to improve contribution process.

@duckchip
Copy link
Contributor

I think this PR should be openend again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Tests: Not Covered Changes in Pull Request requires coverage by auto-tests Component: Catalog Component: CatalogInventory Component: CatalogRule Priority: P3 May be fixed according to the position in the backlog. Progress: needs update Release Line: 2.4 Severity: S3 Affects non-critical data or functionality and does not force users to employ a workaround.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache cleaner performance issue when saving multiple products
5 participants