Mark deprecated SSL settings as obsolete

This commit updates SSL settings to be marked as obsolete: - Replace `ssl` with `ssl_enabled` - Replace `ca_file` with `ssl_certificate_authorities` - Replace `ssl_certificate_verification` with `ssl_verification_mode` `setup_ssl_params!` has been updated to only handle SSL inference when not explicitly configured. All changes have been updated in tests and in docs. The preparation for releasing a new major version is also included.

Author: donoghuc

CHANGELOG.md

@@ -1,3 +1,11 @@
+## 5.0.0
+ - SSL settings that were marked deprecated in version `4.17.0` are now marked obsolete, and will prevent the plugin from starting.
+ - These settings are:
+ - `ssl`, which should bre replaced by `ssl_enabled`
+ - `ca_file`, which should bre replaced by `ssl_certificate_authorities`
+ - `ssl_certificate_verification`, which should bre replaced by `ssl_verification_mode`
+ - [#213](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/213)
+
 ## 4.20.5
  - Add `x-elastic-product-origin` header to Elasticsearch requests [#211](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/211)
 

docs/index.asciidoc

@@ -478,6 +478,8 @@ Enable SSL/TLS secured communication to Elasticsearch cluster.
 Leaving this unspecified will use whatever scheme is specified in the URLs listed in <<plugins-{type}s-{plugin}-hosts>> or extracted from the <<plugins-{type}s-{plugin}-cloud_id>>.
 If no explicit protocol is specified plain HTTP will be used.
 
+When not explicitly set, SSL will be automatically enabled if any of the specified hosts use HTTPS.
+
 [id="plugins-{type}s-{plugin}-ssl_key"]
 ===== `ssl_key`
  * Value type is <<path,path>>
@@ -608,12 +610,12 @@ option when authenticating to the Elasticsearch server. If set to an
 empty string authentication will be disabled.
 
 
-[id="plugins-{type}s-{plugin}-deprecated-options"]
-==== Elasticsearch Input deprecated configuration options
+[id="plugins-{type}s-{plugin}-removed-options"]
+==== Elasticsearch Input Removed configuration options
 
-This plugin supports the following deprecated configurations.
+The following configurations will be removed in version 5.0.0.
 
-WARNING: Deprecated options are subject to removal in future releases.
+WARNING: These options have been deprecated and will be removed in version 5.0.0. Please use their replacements instead.
 
 [cols="<,<,<",options="header",]
 |=======================================================================
@@ -625,7 +627,9 @@ WARNING: Deprecated options are subject to removal in future releases.
 
 [id="plugins-{type}s-{plugin}-ca_file"]
 ===== `ca_file`
-deprecated[4.17.0, Replaced by <<plugins-{type}s-{plugin}-ssl_certificate_authorities>>]
+deprecated[4.17.0, Will be removed in 5.0.0, replaced by <<plugins-{type}s-{plugin}-ssl_certificate_authorities>>]
+
+This setting will be removed in 5.0.0. Please use <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> instead.
 
 * Value type is <<path,path>>
 * There is no default value for this setting.
@@ -634,7 +638,9 @@ SSL Certificate Authority file in PEM encoded format, must also include any chai
 
 [id="plugins-{type}s-{plugin}-ssl"]
 ===== `ssl`
-deprecated[4.17.0, Replaced by <<plugins-{type}s-{plugin}-ssl_enabled>>]
+deprecated[4.17.0, Will be removed in 5.0.0, replaced by <<plugins-{type}s-{plugin}-ssl_enabled>>]
+
+This setting will be removed in 5.0.0. Please use <<plugins-{type}s-{plugin}-ssl_enabled>> instead.
 
 * Value type is <<boolean,boolean>>
 * Default value is `false`
@@ -645,7 +651,10 @@ server (i.e. HTTPS will be used instead of plain HTTP).
 
 [id="plugins-{type}s-{plugin}-ssl_certificate_verification"]
 ===== `ssl_certificate_verification`
-deprecated[4.17.0, Replaced by <<plugins-{type}s-{plugin}-ssl_verification_mode>>]
+deprecated[4.17.0, Will be removed in 5.0.0, replaced by <<plugins-{type}s-{plugin}-ssl_verification_mode>>]
+
+This setting will be removed in 5.0.0. Please use <<plugins-{type}s-{plugin}-ssl_verification_mode>> instead.
+
 
 * Value type is <<boolean,boolean>>
 * Default value is `true`

lib/logstash/inputs/elasticsearch.rb

@@ -198,23 +198,12 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base
  # Set the address of a forward HTTP proxy.
  config :proxy, :validate => :uri_or_empty
 
- # SSL
- config :ssl, :validate => :boolean, :default => false, :deprecated => "Set 'ssl_enabled' instead."
-
- # SSL Certificate Authority file in PEM encoded format, must also include any chain certificates as necessary
- config :ca_file, :validate => :path, :deprecated => "Set 'ssl_certificate_authorities' instead."
-
  # OpenSSL-style X.509 certificate certificate to authenticate the client
  config :ssl_certificate, :validate => :path
 
  # SSL Certificate Authority files in PEM encoded format, must also include any chain certificates as necessary
  config :ssl_certificate_authorities, :validate => :path, :list => true
 
- # Option to validate the server's certificate. Disabling this severely compromises security.
- # For more information on the importance of certificate verification please read
- # https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
- config :ssl_certificate_verification, :validate => :boolean, :default => true, :deprecated => "Set 'ssl_verification_mode' instead."
-
  # The list of cipher suites to use, listed by priorities.
  # Supported cipher suites vary depending on which version of Java is used.
  config :ssl_cipher_suites, :validate => :string, :list => true
@@ -242,7 +231,7 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base
  config :ssl_truststore_password, :validate => :password
 
  # The JKS truststore to validate the server's certificate.
- # Use either `:ssl_truststore_path` or `:ssl_certificate_authorities`
+ # Use either `:ssl_truststore_path`
  config :ssl_truststore_path, :validate => :path
 
  # The format of the truststore file. It must be either jks or pkcs12
@@ -264,6 +253,11 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base
  # If set, the _source of each hit will be added nested under the target instead of at the top-level
  config :target, :validate => :field_reference
 
+ # Obsolete Settings
+ config :ssl, :obsolete => "Set 'ssl_enabled' instead."
+ config :ca_file, :obsolete => "Set 'ssl_certificate_authorities' instead."
+ config :ssl_certificate_verification, :obsolete => "Set 'ssl_verification_mode' instead."
+
  # config :ca_trusted_fingerprint, :validate => :sha_256_hex
  include LogStash::PluginMixins::CATrustedFingerprintSupport
 
@@ -408,8 +402,6 @@ def setup_client_ssl
  ssl_options[:ssl] = true if @ssl_enabled
 
  unless @ssl_enabled
- # Keep it backward compatible with the deprecated `ssl` option
- ssl_options[:trust_strategy] = trust_strategy_for_ca_trusted_fingerprint if original_params.include?('ssl')
  return ssl_options
  end
 
@@ -473,38 +465,11 @@ def setup_client_ssl
  end
 
  def setup_ssl_params!
- @ssl_enabled = normalize_config(:ssl_enabled) do |normalize|
- normalize.with_deprecated_alias(:ssl)
- end
-
- # Infer the value if neither the deprecate `ssl` and `ssl_enabled` were set
- infer_ssl_enabled_from_hosts
-
- @ssl_certificate_authorities = normalize_config(:ssl_certificate_authorities) do |normalize|
- normalize.with_deprecated_mapping(:ca_file) do |ca_file|
- [ca_file]
- end
+ # Only infer ssl_enabled if it wasn't explicitly set
+ unless original_params.include?('ssl_enabled')
+ @ssl_enabled = effectively_ssl?
+ params['ssl_enabled'] = @ssl_enabled
  end
-
- @ssl_verification_mode = normalize_config(:ssl_verification_mode) do |normalize|
- normalize.with_deprecated_mapping(:ssl_certificate_verification) do |ssl_certificate_verification|
- if ssl_certificate_verification == true
- "full"
- else
- "none"
- end
- end
- end
-
- params['ssl_enabled'] = @ssl_enabled
- params['ssl_certificate_authorities'] = @ssl_certificate_authorities unless @ssl_certificate_authorities.nil?
- params['ssl_verification_mode'] = @ssl_verification_mode unless @ssl_verification_mode.nil?
- end
-
- def infer_ssl_enabled_from_hosts
- return if original_params.include?('ssl') || original_params.include?('ssl_enabled')
-
- @ssl_enabled = params['ssl_enabled'] = effectively_ssl?
  end
 
  def setup_hosts

logstash-input-elasticsearch.gemspec

@@ -1,7 +1,7 @@
 Gem::Specification.new do |s|
 
  s.name = 'logstash-input-elasticsearch'
- s.version = '4.20.5'
+ s.version = '5.0.0'
  s.licenses = ['Apache License (2.0)']
  s.summary = "Reads query results from an Elasticsearch cluster"
  s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"

spec/inputs/elasticsearch_spec.rb

@@ -58,6 +58,19 @@
  end
  end
 
+ describe 'handling obsolete settings' do
+ [{:name => 'ssl', :replacement => 'ssl_enabled', :sample_value => true},
+ {:name => 'ca_file', :replacement => 'ssl_certificate_authorities', :sample_value => 'spec/fixtures/test_certs/ca.crt'},
+ {:name => 'ssl_certificate_verification', :replacement => 'ssl_verification_mode', :sample_value => false }].each do | obsolete_setting|
+ context "with obsolete #{obsolete_setting[:name]}" do
+ let (:config) { {obsolete_setting[:name] => obsolete_setting[:sample_value]} }
+ it "should raise a config error with the appropriate message" do
+ expect { plugin.register }.to raise_error LogStash::ConfigurationError, /The setting `#{obsolete_setting[:name]}` in plugin `elasticsearch` is obsolete and is no longer available. Set '#{obsolete_setting[:replacement]}' instead/i
+ end
+ end
+ end
+ end
+
  context "against not authentic Elasticsearch" do
  before(:each) do
  Elasticsearch::Client.send(:define_method, :ping) { raise Elasticsearch::UnsupportedProductError.new("Fake error") } # define error ping method