Skip to content

Conversation

andsel
Copy link
Contributor

@andsel andsel commented Apr 7, 2022

Release notes

Enable the usage of not shadowed Java plugins

What does this PR do?

Update the Java plugins loading from gems to use multijar plugins.
Before this PR the example Java plugins come with a single shadowed jar with all the dependent jars squashed into.
The shadowed jar also bundle Logstash classes and logging libraries, driving to an increase of file size, but also polluting the classpath.
With PR logstash-plugins/logstash-input-java_input_example#15 it fixes the problem on plugin side.
Logstash needs to recognize and properly load multijar plugins.
With this PR if the Ruby gem contains a single jar, then it behaves like before and uses that jar as the container of the plugin's root class. If the gem file contains multiple jar that is return the jar file that contains the Logstash-plugin property is its MANIFEST.mf.

Why is it important/What is the impact to the user?

Let the developer to implement pure Java Logstash plugins that could be used into the project.

The native Java plugins generally bundles all the the transitive dependencies inside an uber jar. However, due to the Gradle's shadow plugin, this generates a problem because the uber jar contains also Logstash's core classes, and this could generate a problem with class loading. On the other side, mixed Ruby/Java plugins can ship multiple jars used by the plugins itself.
This PR aims to bundle multiple jars inside a native Java plugins, permitting the correct loading of the plugin's main class through the usage of manifest file.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • [ ]

How to test this PR locally

Test as described in PR logstash-plugins/logstash-input-java_input_example#15

Related issues

Use cases

As a developer I want to bundle only the required libraries used by a native plugin, bundling them and permitting the correct loading of the plugin's main class.

Screenshots

Logs

Error happening when loading a Java plugin with multuple jars contained:

[FATAL] 2022-08-29 09:40:26.675 [main] Logstash - Logstash stopped processing because of an error: (LoadError) Java plugin 'logstash-input-java_input_example' does not contain a single jar file with the plugin's name and version org.jruby.exceptions.LoadError: (LoadError) Java plugin 'logstash-input-java_input_example' does not contain a single jar file with the plugin's name and version	at home.andrea.workspace.logstash_andsel.logstash_minus_core.lib.logstash.plugins.registry.load_available_plugins(/home/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/plugins/registry.rb:159) ~[?:?]	at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1865) ~[jruby.jar:?]	at home.andrea.workspace.logstash_andsel.logstash_minus_core.lib.logstash.plugins.registry.load_available_plugins(/home/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/plugins/registry.rb:153) ~[?:?]	at home.andrea.workspace.logstash_andsel.logstash_minus_core.lib.logstash.plugins.registry.setup!(/home/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/plugins/registry.rb:131) ~[?:?]	at RUBY.<class:Runner>(/home/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/runner.rb:262) ~[?:?]	at RUBY.<main>(/home/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/runner.rb:55) ~[?:?]	at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1017) ~[jruby.jar:?]	at home.andrea.workspace.logstash_andsel.lib.bootstrap.environment.<main>(/home/andrea/workspace/logstash_andsel/lib/bootstrap/environment.rb:89) ~[?:?] 
@andsel andsel self-assigned this Apr 7, 2022
@andsel andsel changed the title Update the java plugin loading logic to process the manifest files in… Update the java plugin loading logic to process the manifest file. Apr 7, 2022
@andsel
Copy link
Contributor Author

andsel commented Apr 7, 2022

Jenkins test this please

@andsel
Copy link
Contributor Author

andsel commented Dec 1, 2022

The CI fails is on integration test:

./gradlew clean runIntegrationTests -PrubyIntegrationSpecs="specs/install_java_plugin_spec.rb" 

with the error:

 Starting Logstash: /home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/bin/logstash ["-f", "/tmp/studtmp-cdbbf93353cd54233dc1f5b5673a7c672d9b3325dca6fbc098596664f4a3", "--path.settings", "/tmp/studtmp-0fcba89b07f26b3d5f27f2ab6f8f4d490f8609192c84228c1a6dfa19532b"] (pwd: /home/andrea/workspace/logstash_andsel/qa/integration) Logstash started with PID 190329, using java: bundled java Using bundled JDK: /home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/jdk [FATAL] 2022-12-01 18:59:43.603 [main] Logstash - Logstash stopped processing because of an error: (NoMethodError) undefined method `relative_path' for nil:NilClass org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `relative_path' for nil:NilClass at RUBY.find_plugin_jar(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/plugins/registry.rb:176) ~[?:?] at home.andrea.workspace.logstash_andsel.build.logstash_minus_8_dot_7_dot_0_minus_SNAPSHOT.logstash_minus_core.lib.logstash.plugins.registry.load_available_plugins(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/plugins/registry.rb:184) ~[?:?] at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1865) ~[jruby.jar:?] at RUBY.load_available_plugins(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/plugins/registry.rb:182) ~[?:?] at RUBY.setup!(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/plugins/registry.rb:131) ~[?:?] at RUBY.<class:Runner>(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/runner.rb:262) ~[?:?] at RUBY.<main>(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/logstash-core/lib/logstash/runner.rb:55) ~[?:?] at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1017) ~[jruby.jar:?] at home.andrea.workspace.logstash_andsel.build.logstash_minus_8_dot_7_dot_0_minus_SNAPSHOT.lib.bootstrap.environment.<main>(/home/andrea/workspace/logstash_andsel/build/logstash-8.7.0-SNAPSHOT/lib/bootstrap/environment.rb:89) ~[?:?] 
@andsel andsel force-pushed the fix/java_plugins_utils_improvements branch 2 times, most recently from 3a4fbf2 to cccdb95 Compare May 25, 2023 08:49
@andsel andsel marked this pull request as ready for review May 25, 2023 15:10
@andsel andsel force-pushed the fix/java_plugins_utils_improvements branch from b34d353 to 267d203 Compare July 12, 2023 15:03
Copy link
Contributor

mergify bot commented Mar 5, 2025

This pull request does not have a backport label. Could you fix it @andsel? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
  • backport-8.x is the label to automatically backport to the 8.x branch.
Copy link
Contributor

mergify bot commented Mar 5, 2025

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Mar 5, 2025
Copy link
Contributor

mergify bot commented Mar 5, 2025

This pull request is now in conflicts. Could you fix it @andsel? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream git checkout -b fix/java_plugins_utils_improvements upstream/fix/java_plugins_utils_improvements git merge upstream/main git push upstream fix/java_plugins_utils_improvements 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-8.x Automated backport to the 8.x branch with mergify enhancement spacetime status:needs-review

2 participants