[8.x] Pluginmanager clean after mutate (backport #17203)

[mergify]

Release notes

• FIX: The plugin manager now correctly cleans up plugins and gem dependencies that are no longer needed after invoking install, remove, or update.
• FIX: The plugin manager's update command no longer fails to upgrade plugins that were initially installed with --version flag

What does this PR do?

• Takes the previously-reviewed guts of plugins: add clean command to remove inactive plugins/dependencies #16998 and attaches the clean operation to the successful completions of the update, remove, and install commands instead of as its own stand-alone command.
• Fixes [Bug]🐛 a plugin installed with --version is skipped when updating #17196, in which the update command fails to update plugins that were initially installed with a --version flag

Why is it important/What is the impact to the user?

• For users who update, install, or remove plugins, cleaning out deactivated dependencies can reduce the disk footprint of the logstash installation
• For building Logstash artifacts, this allows us to build a true subset of the dependency graph after removing plugins that are not needed in a minimalized artifact

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Likely the easiest is to install, remove, or update plugins, observe the output, and check how the vendor/bundle was changed:

1. Installing a plugin should remove overwritten plugins and orphaned dependencies

   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt bin/logstash-plugin install --version 4.1.2 logstash-output-email find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt diff -u vendor_snapshot.txt vendor_compare.txt 

2. Updating a plugin should remove the previous version (and its shared dependencies)

   bin/logstash-plugin install --version 4.1.2 logstash-output-email find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt bin/logstash-plugin update logstash-output-email find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt diff -u vendor_snapshot.txt vendor_compare.txt 

3. Removing a plugin should remove the plugin gem and its unshared dependencies:

   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt bin/logstash-plugin remove logstash-integration-aws find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt diff -u vendor_snapshot.txt vendor_compare.txt 

Related issues

• Resolves: [Bug]🐛 a plugin installed with --version is skipped when updating #17196
• Supersedes: plugins: add clean command to remove inactive plugins/dependencies #16998

---

This is an automatic backport of pull request #17203 done by [Mergify](https://mergify.com).

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 4872d06

Failed CI Steps

• 🥼 Integration Tests / part 1
• 🥼 Integration Tests / part 1
• 🥼 Integration Tests / part 1
• 🥼 Integration Tests / part 1
• 🥼 IT Persistent Queues / part 1
• 🥼 IT Persistent Queues / part 1
• 🥼 IT Persistent Queues / part 1
• 🥼 IT Persistent Queues / part 1

History

cc @yaauie

[donoghuc]

Closing in favor of #17270