RabbitMQ 3.13.0-beta.5

[michaelklishin]

RabbitMQ 3.13.0-beta.5

RabbitMQ 3.13.0-beta.5 is a preview of a new feature release.

Highlights

This release includes several new features, optimizations, and graduates (makes mandatory) a number of feature flags.

The user-facing areas that have seen the biggest improvements in this release are

• Support for consumer-side stream filtering
• MQTTv5 support
• A new common message container format used internally, based on the AMQP 1.0 message container format
• Improved classic non-mirrored queue performance with message sizes larger than
  4 KiB (or a different customized CQ index embedding threshold)
• Classic queues use version 2 of the storage implementation (CQv2).
  This should significantly improve performance of non-mirrored classic queues

This release also features many internal API improvements in preparation to 4.0
with Khepri.

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Release Artifacts

RabbitMQ preview releases are distributed via GitHub.

Community Docker image is another installation option
for previews. It is updated with a delay (usually a few days).

Erlang/OTP Compatibility Notes

This release requires Erlang 26.0 or later.

Provisioning Latest Erlang Releases explains
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x.

Upgrading to 3.13

Documentation guides on upgrades

See the Upgrading guide for documentation on upgrades and RabbitMQ change log
for release notes of other releases.

Required Feature Flags

TBD

Mixed version cluster compatibility

RabbitMQ 3.13.0 nodes can run alongside 3.12.x nodes. 3.12.x-specific features can only be made available when all nodes in the cluster
upgrade to 3.13.0 or any other patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes is covered below.
Once all nodes are upgraded to 3.13.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).

Compatibility Notes

TBD

Minimum Supported Erlang Version

Starting with this release, RabbitMQ requires Erlang 26.0 or later versions. Nodes will fail to start
on older Erlang releases.

Client Library Compatibility

Client libraries that were compatible with RabbitMQ 3.12.x will be compatible with 3.13.0.

Getting Help

Any questions about this release, upgrades or RabbitMQ in general are welcome on the RabbitMQ mailing list.

Changes Worth Mentioning

Release notes are kept under rabbitmq-server/release-notes.

Core Server

Enhancements

• Messages are now internally stored using a new common heavily AMQP 1.0-influenced container format. This is a major step towards a protocol-agnostic core:
  a common format that encapsulates a sum of data types used by the protocols RabbitMQ supports, plus annotations for routng, dead-lettering state,
  and other purposes.

  AMQP 1.0, AMQP 0-9-1, MQTT and STOMP have or will adopt this internal representation in upcoming releases. RabbitMQ Stream protocol already uses the AMQP 1.0 message container
  structure internally.

  This common internal format will allow for more correct and potentially efficient multi-protocol support in RabbitMQ,
  and that most cross-protocol translation rough edges can be smoothened.

  GitHub issue: #5077

• Target quorum queue replica state is now continuously reconciled.

  When the number of online replicas of a quorum queue goes below (or above) its target,
  new replicas will be automatically placed if enough cluster nodes are available.
  This is a more automatic version of how quorum queue replicas have originally been grown.

  For automatic shrinking of queue replicas, the user must opt in.

  Contributed by @SimonUnge (AWS).

  GitHub issue: #8218

• Reduced memory footprint, improved memory use predictability and throughput of classic queues (version 2, or CQv2).
  This particularly benefits classic queues with longer backlogs.

  Classic queue v2 (CQv2) storage implementation is now the default. It is possible to switch
  the default back to CQv1 using rabbitmq.conf:

  # uses CQv1 by default classic_queue.default_version = 1

  Individual queues can be declared by passing x-queue-version argument and/or through a queue-version policy.

  GitHub issue: #8308

• Non-mirrored classic queues: optimizations of storage for larger (greater than 4 kiB) messages.

  GitHub issue: #6090, #8507

• A subsystem for marking features as deprecated.

  GitHub issue: #7390

• Plugins now can register custom queue types. This means that a plugin now can provide
  a custom queue type.

  Contributed by @luos (Erlang Solutions).

  GitHub issues: #8834, #8927

Bug Fixes

This release includes all bug fixes shipped in the 3.12.x series.

• Feature flag discovery on a newly added node could discover an incomplete inventory of feature flags.

  GitHub issue: #8477

• Feature flag discovery operations will now be retried multiple times in case of network failures.

  GitHub issue: #8491

• The state of node maintenance status across the cluster is now replicated. It previously was accessible
  to all nodes but not replicated.

  GitHub issue: #9005

Stream Plugin

Enhancements

• Support for (consumer) stream filtering.

  This allows consumers that are only interested in a subset of data in a stream to receive
  less data. Note that false positives are possible, so this feature should be accompanied by
  client library or application-level filtering.

  GitHub issue: #8207

MQTT Plugin

Enhancements

• Support for MQTTv5 (with limitations).

  GitHub issues: #7263, #8681

• Negative message acknowledgements are now propagated to MQTTv5 clients.

  GitHub issue: #9034

• Potential incompatibility: mqtt.subscription_ttl configuration setting is now deprecated in favor of
  mqtt.max_session_expiry_interval_seconds that targets MQTTv5.

  GitHub issue: #8846

AMQP 1.0 Plugin

Bug Fixes

• During AMQP 1.0 to AMQP 0-9-1 conversion, the Correlation ID message property is now stored as x-correlation-id (instead of x-correlation) for values longer than 255 bytes.

  This is a potentially breaking change.

  GitHub issue: #8680

Dependency Changes

• ra was upgraded to 2.6.3
• osiris was upgraded to 1.6.2

Source Code Archives

To obtain source code of the entire distribution, please download the archive named rabbitmq-server-3.13.0.tar.xz
instead of the source tarball produced by GitHub.

---

This discussion was created from the release RabbitMQ 3.13.0-beta.5.