Skip to content

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 27, 2025

The queue factory setting "queueMaxBytes" used to be a long link:

long queueMaxBytes = RubyFixnum.num2long(args[6]);

But with the refactoring of #18180, it became an Integer by mistake:

.queueMaxBytes(getSetting(context, settings, QUEUE_MAX_BYTES).toJava(Integer.class))

Which prevents queues being bigger than 2^31-1, causing crashes to anyone setting their PQ to 2GB or more:

queue.max_bytes: 2147483647 # 2GB-1 OK queue.max_bytes: 2147483648 # 2GB FAIL

This change makes queueMaxBytes a Long again, with a test confirming that queues larger than 2ˆ31-1 (max int) can be created.

Without this change the new test fails:

❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb ..F. Failures: 1) LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error Failure/Error: expect { queue = subject.create(settings) }.to_not raise_error expected no Exception, got #<RangeError: too big for int: 2147483648> with backtrace: # org/logstash/ackedqueue/QueueFactoryExt.java:97:in `create' # org/logstash/ackedqueue/QueueFactoryExt.java:88:in `create' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./lib/bootstrap/rspec.rb:36:in `<main>' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./lib/bootstrap/rspec.rb:36:in `<main>' Finished in 0.03879 seconds (files took 0.03787 seconds to load) 4 examples, 1 failure Failed examples: rspec ./logstash-core/spec/logstash/queue_factory_spec.rb:84 # LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error 

But with the change it passes:

❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb -fd LogStash::QueueFactory when `queue.type` is `persisted` returns a `WrappedAckedQueue` per pipeline id subdirectory creation creates a queue directory based on the pipeline id when queue.max_bytes is larger than Java int does not raise error when `queue.type` is `memory` returns a `WrappedSynchronousQueue` Finished in 0.03292 seconds (files took 0.04052 seconds to load) 4 examples, 0 failures ```<hr>This is an automatic backport of pull request #18366 done by [Mergify](https://mergify.com). 
…8366) The queue factory setting "queueMaxBytes" used to be a long [link](https://github.com/elastic/logstash/pull/18180/files#diff-a377851670e06cd113751aa73b726b446dfb721d03d6979b9195afff7facd9f5L60): ```ruby long queueMaxBytes = RubyFixnum.num2long(args[6]); ``` But with the refactoring of #18180, it [became an Integer by mistake](https://github.com/elastic/logstash/pull/18180/files#diff-01eec670d2beabcd43041ff7fca8bc907eb227471dce886bb757b6414a655429R125): ```java .queueMaxBytes(getSetting(context, settings, QUEUE_MAX_BYTES).toJava(Integer.class)) ``` Which prevents queues being bigger than 2^31-1, causing crashes to anyone setting their PQ to 2GB or more: ```yaml queue.max_bytes: 2147483647 # 2GB-1 OK queue.max_bytes: 2147483648 # 2GB FAIL ``` This change makes queueMaxBytes a Long again, with a test confirming that queues larger than 2ˆ31-1 (max int) can be created. Without this change the new test fails: ``` ❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb ..F. Failures: 1) LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error Failure/Error: expect { queue = subject.create(settings) }.to_not raise_error expected no Exception, got #<RangeError: too big for int: 2147483648> with backtrace: # org/logstash/ackedqueue/QueueFactoryExt.java:97:in `create' # org/logstash/ackedqueue/QueueFactoryExt.java:88:in `create' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./lib/bootstrap/rspec.rb:36:in `<main>' # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>' # ./lib/bootstrap/rspec.rb:36:in `<main>' Finished in 0.03879 seconds (files took 0.03787 seconds to load) 4 examples, 1 failure Failed examples: rspec ./logstash-core/spec/logstash/queue_factory_spec.rb:84 # LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error ``` But with the change it passes: ``` ❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb -fd LogStash::QueueFactory when `queue.type` is `persisted` returns a `WrappedAckedQueue` per pipeline id subdirectory creation creates a queue directory based on the pipeline id when queue.max_bytes is larger than Java int does not raise error when `queue.type` is `memory` returns a `WrappedSynchronousQueue` Finished in 0.03292 seconds (files took 0.04052 seconds to load) 4 examples, 0 failures ``` (cherry picked from commit 6c48e51)
@github-actions
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@jsvd jsvd merged commit a0e5260 into 9.2 Oct 27, 2025
11 of 12 checks passed
@jsvd jsvd deleted the mergify/bp/9.2/pr-18366 branch October 27, 2025 11:07
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

cc @jsvd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 participants