Skip to content

[ MFTF ] Mess in ActionGroups: invalid names, multiple nodes. #26331

New issue
New issue
Closed
Closed
[ MFTF ] Mess in ActionGroups: invalid names, multiple nodes.#26331
Assignees
lbajsarowiczVladimirZaets
Labels
Component: TestFixed in 2.4.xThe issue has been fixed in 2.4-develop branchIssue: Clear DescriptionGate 2 Passed. Manual verification of the issue description passedIssue: ConfirmedGate 3 Passed. Manual verification of the issue completed. Issue is confirmedIssue: Format is validGate 1 Passed. Automatic verification of issue format passedIssue: Ready for WorkGate 4. Acknowledged. Issue is added to backlog and ready for developmentReproduced on 2.4.xThe issue has been reproduced on latest 2.4-develop branchimprovement
@lbajsarowicz

Description

@lbajsarowicz
lbajsarowicz
opened on Jan 9, 2020

Summary (*)

Goal of this ticket is to explain Why I put so much effort to cleanup the Magento's Funcionatl Testing Framework:

  1. MFTF Best Practices states that Action Group name should be consistent with the filename (We're moving towards having it as a requirement)
  2. Practical side: When looking for ActionGroup - it's enough if you search for a file with its "what we expect" name (eg. PHPStorm: CTRL + SHIFT + N and type AdminLoginActionGroup)
  3. Maintainability: Only one ActionGroup with the same name is created in the module (when trying to create another one with the same name - you'll hit existing file).
  4. Extensibility: I want to introduce ActionGroup lazy loading. When you're running even single Test, MFTF traverses all the files of ActionGroups to parse them and build the dependency of necessary files. I want to change that and use the only files that we need.

Examples (*)

Report from my tool - for example for Customer module.

Multiple ActionGroups in single file

 /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontOpenCustomerAccountCreatePageActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerLogoutActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontAddCustomerAddressActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/SignUpNewUserFromStorefrontActionGroup.xml --- 13 /app/code/Magento/Customer/Test/Mftf/ActionGroup/OpenEditCustomerFromAdminActionGroup.xml --- 4 /app/code/Magento/Customer/Test/Mftf/ActionGroup/DeleteCustomerActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminEditCustomerAddressesFromActionGroup.xml --- 4 /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCustomerSubscribeNewsletterActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCreateCustomerWithWebsiteAndStoreViewActionGroup.xml --- 2 /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerSubscribeNewsletterActionGroup.xml --- 2

Inconsistent ActionGroup names

/app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminSelectAllCustomersActionGroup.xml - AdminSelectAllCustomers /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerDefaultBillingAddressActionGroup.xml - AdminAssertCustomerDefaultBillingAddress /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminEditCustomerAddressNoZipNoStateActionGroup.xml - AdminEditCustomerAddressNoZipNoState /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminFilterCustomerGridByEmailActionGroup.xml - AdminFilterCustomerGridByEmail /app/code/Magento/Customer/Test/Mftf/ActionGroup/AssertCustomerLoggedInActionGroup.xml - AssertCustomerWelcomeMessageActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerDefaultShippingAddressActionGroup.xml - AdminAssertCustomerDefaultShippingAddress /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontAssertSuccessLoginToStorefrontActionGroup.xml - StorefrontAssertSuccessLoginToStorefront /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerNoDefaultBillingAddressActionGroup.xml - AdminAssertCustomerNoDefaultBillingAddress /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminSelectCustomerByEmailActionGroup.xml - AdminSelectCustomerByEmail /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerActionGroup.xml - CustomerLogoutStorefrontByMenuItemsActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AssertStorefrontPasswordAutocompleteOffActionGroup.xml - AssertStorefrontPasswordAutoCompleteOffActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerElementNotVisibleActionGroup.xml - StorefrontCustomerReorderButtonNotVisibleActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertNumberOfRecordsInCustomersAddressGridActionGroup.xml - AdminAssertNumberOfRecordsInCustomersAddressGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/LoginToStorefrontWithEmailAndPasswordActionGroup.xml - LoginToStorefrontWithEmailAndPassword /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCustomerGridActionGroup.xml - AdminFilterCustomerByEmail /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerAccountInformationActionGroup.xml - AdminAssertCustomerAccountInformation /app/code/Magento/Customer/Test/Mftf/ActionGroup/NavigateCustomerActionGroup.xml - NavigateToAllCustomerPage /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminFilterCustomerByNameActionGroup.xml - AdminFilterCustomerByName /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminResetFilterInCustomerGridActionGroup.xml - AdminResetFilterInCustomerGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerAddressBookNotContainsActionGroup.xml - StorefrontCustomerAddressBookNotContains /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminResetFilterInCustomerAddressGridActionGroup.xml - AdminResetFilterInCustomerAddressGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerGroupOnCustomerFormActionGroup.xml - AdminAssertCustomerGroupOnCustomerForm /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminSaveCustomerAndAssertSuccessMessageActionGroup.xml - AdminSaveCustomerAndAssertSuccessMessage /app/code/Magento/Customer/Test/Mftf/ActionGroup/OpenStorefrontStoredPaymentMethodsPageActionGroup.xml - OpenStorefrontCustomerStoredPaymentMethodsPageActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontRegisterCustomerFromOrderSuccessPageActionGroup.xml - StorefrontRegisterCustomerFromOrderSuccessPage /app/code/Magento/Customer/Test/Mftf/ActionGroup/SetGroupCustomerActionGroup.xml - SetCustomerGroupForSelectedCustomersViaGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminUpdateCustomerGroupActionGroup.xml - AdminUpdateCustomerGroupByEmailActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerSubscribeNewsletterActionGroup.xml - AdminAssertCustomerIsSubscribedToNewsletters - AdminAssertCustomerIsSubscribedToNewslettersAndSelectedStoreView /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCustomerSubscribeNewsletterActionGroup.xml - AdminSubscribeCustomerToNewsletters - AdminSubscribeCustomerToNewslettersAndSelectStoreView /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminEditCustomerAddressSetDefaultShippingAndBillingActionGroup.xml - AdminEditCustomerAddressSetDefaultShippingAndBilling /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerGroupPresentInGridActionGroup.xml - AdminAssertCustomerGroupPresentInGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminConfigCustomerActionGroup.xml - SetCustomerDataLifetimeActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontAssertRegistrationPageFieldsActionGroup.xml - StorefrontAssertRegistrationPageFields /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerAddressBookNumberOfAddressesActionGroup.xml - StorefrontCustomerAddressBookNumberOfAddresses /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCustomerSaveAndContinueActionGroup.xml - AdminCustomerSaveAndContinue /app/code/Magento/Customer/Test/Mftf/ActionGroup/NavigateCustomerGroupActionGroup.xml - NavigateToCustomerGroupPage /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminDeleteAddressInCustomersAddressGridActionGroup.xml - AdminDeleteAddressInCustomersAddressGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminEditCustomerAddressesFromActionGroup.xml - AdminEditCustomerAddressesFrom - AdminEditCustomerAddressSetDefaultShippingAndBilling - AdminEditCustomerAddressNoZipNoState - SelectDropdownCustomerAddressAttributeValueActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminEditCustomerInformationFromActionGroup.xml - AdminEditCustomerAccountInformationActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertAddressInCustomersAddressGridActionGroup.xml - AdminAssertAddressInCustomersAddressGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/VerifyGroupCustomerActionGroup.xml - VerifyCustomerGroupForCustomer /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerGoToSidebarMenuActionGroup.xml - StorefrontCustomerGoToSidebarMenu /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontCustomerAddressBookContainsActionGroup.xml - StorefrontCustomerAddressBookContains /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerNoDefaultShippingAddressActionGroup.xml - AdminAssertCustomerNoDefaultShippingAddress /app/code/Magento/Customer/Test/Mftf/ActionGroup/SwitchAccountActionGroup.xml - SignOut /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerGroupOnProductFormActionGroup.xml - AdminAssertCustomerGroupOnProductForm /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertCustomerInCustomersGridActionGroup.xml - AdminAssertCustomerInCustomersGrid /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminAssertErrorMessageCustomerGroupAlreadyExistsActionGroup.xml - AdminAssertErrorMessageCustomerGroupAlreadyExists /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminCustomerShopingCartActionGroup.xml - AdminAddProductToShoppingCartActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/StorefrontAddCustomerAddressActionGroup.xml - StorefrontAddNewCustomerAddressActionGroup - StorefrontAddCustomerDefaultAddressActionGroup /app/code/Magento/Customer/Test/Mftf/ActionGroup/AdminFilterCustomerAddressGridByPhoneNumberActionGroup.xml - AdminFilterCustomerAddressGridByPhoneNumber

Proposed solution

Currently I work with all the Open Source and Commerce modules to address the issue, as well as to report other issues - like workarounds applied to the tests, that create overhead when it comes to time of execution.

CC: @okolesnyk @dmytro-ch @ihor-sviziev @rogyar @soumyau @nuzil

Metadata

Metadata

Assignees

Labels

Component: TestFixed in 2.4.xThe issue has been fixed in 2.4-develop branchIssue: Clear DescriptionGate 2 Passed. Manual verification of the issue description passedIssue: ConfirmedGate 3 Passed. Manual verification of the issue completed. Issue is confirmedIssue: Format is validGate 1 Passed. Automatic verification of issue format passedIssue: Ready for WorkGate 4. Acknowledged. Issue is added to backlog and ready for developmentReproduced on 2.4.xThe issue has been reproduced on latest 2.4-develop branchimprovement

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions