Feature: Add command to detect new queues

[furan917]

Description

The setup:db:status command does not pick up newly defined or missing queues, this means anyone trying to get close to true green/blue deployments has no quick way to check if a setup:upgrade is required if a new queue (mysql or amqp) has been added and no other changes are made.

This PR adds the console command queue:config:status, which checks for new queues. In line with other status commands; 0 = no new queues, exit code 2 = new unconfigured queues.

Important architecture info: The command resides in MessageQueue, and it uses DI to hydrate any ChangeDetectors - which there are 2 of, one in MysqlMq and one on Amqp, this avoids any additional module dependency. MysqlMq checks if the queue exists in the DB; Amqp, if you have rabbitmq/amazonmq/etc, checks if the queue exists in the broker.

Related Pull Requests

Follows on from the similar PR #186 as a concentrated effort to improve Blue/Green deployment efforts.

Manual testing scenarios

cat > app/code/Magento/ImportExport/etc/queue_topology.xml <<'EOF' <?xml version="1.0"?> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd"> <exchange name="magento"> <binding id="exportBinding" topic="import_export.export" destination="export"/> </exchange> <exchange name="magento" connection="amqp" type="topic"> <binding id="exportAmqpBinding" topic="import_export.export" destination="test.amqp.queue" destinationType="queue"/> </exchange> </config> EOF 

2. bin/magento queue:config:status ## exit code 2
3. bin/magento setup:upgrade
4. bin/magento queue:config:status ## exit code 0
5. git checkout -- app/code/Magento/ImportExport/etc/queue_topology.xml

Questions or comments

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

Prexisting unit & integration test failures stop me from checking all tests passing box - though the unit tests related to new code do all pass.

[rhoerr]

Code looks good to me. Can you confirm that this detects amqp discrepancies too? Or can it be expanded for that purpose?

I had a case just last week where a RabbitMQ container rebuilt at the host, and lost all the queues from memory. Had to run setup to fix it. (I don't think Magento self-heals in that situation--may be wrong.) It would help if the command could detect that.

[furan917]

@rhoerr It doesnt cover that yet. My original commit had a secondary command that checks against amqp directly but was affected by cache giving false results and I ran out of time on my machine so stripped it out for now rather than commiting junk. But I may extend it later in the night. (Command worked in that if functioned, but new queues only got seen on a cache clean cause I tried to use a different process to the original command)

Can check here for an idea of what it looked like: 8fcc1cc before being stipped (The justification was that infra failure should result in an external process anyways, but would be nice to have - I have faced that issue in the past myself hence its original inclusion)

[furan917]

@rhoerr

Actually, let me change this up a little. I'm going to put the command in MessageQueue, and have MysqlMQ and Amqp modules hydrate a changeDetectors like attribute via DI so I dont have to worry about creating weird dependencies.

[furan917]

@rhoerr There we go, updated it while kids where sitting down for dinner.

Refactored + added amqp checking so if your RMQ gets nuked it should inform you the queues are out of date on the command run

[rhoerr]

I need to test yet, but visually this code looks good to me. Thank you Francis.