Automation of IPP Flex applications Gokuldas K Pillai @gokool
Objective • Identify a tool to automate the functional testing of a Flex application. In my application, we have the following use-cases: – Create/Edit Invoice – Search for a customer
Tools Evaluated • FlexMonkey 1.0 by Gorilla Logic • SilkTest 2009 R2 by Borland (Plugin for Flex) • Ranorex Studio • Other tools (not evaluated/less popular) – QTP – Selenium4Flex – RiaTest
Automation support in Flex • Adobe provides automation delegate implementations for the framework components – Automation.swc (delegation implementations) – Automation_agent.swc (Facilitate communication with a tool) – Automation_dmv.swc (delegates for charts and advanced datagrid) • Third party tool developers leverage these and build on top of it. – The tool developers are limited by what Adobe exposes via the automation hooks. • Custom components • Components that extend framework components – Automation delegates supplied should work • Components that do NOT extend framework components (extend UIComponent instead) – Developer of component needs to provide automation delegates and override event handlers.
Agent mechanism
Tool #1 : Flex Monkey • Quick Facts – Open Source – Ver 1.0 GA released – Adobe AIR application – Scripting language – Actionscript – Support for: • Record/Playback • Validation • Timing delays • Exporting script – integration with ANT script • Pluses – Simple to setup – Works on both OS (Windows/Mac) – Works with both IE and Mozilla • Negatives – Does not scale beyond simple usecases – Sparse documentation for advanced usage – Community is in infancy (Opensource like support ) – Recently moved out of Google code and the transition was not smooth for forum users – Core developers offer paid consulting gigs for training and help with tool.
Tool #2 : SilkTest Flex plugin/Silk4J • Quick facts – 2009 R2 – Flex support since 2008 R2 – Silk4J – Eclipse plugin – Scripting language – Java or Silk script – Object identification using Xpath • Pluses – JUnit like test generation (Silk4J) – Familiar tool(Silk not Silk-Flex) for QA teams in Intuit – Enterprise support plan • Negatives – Performance hog (Seems like it is broken) – Tool is not very developer friendly. – Does not work with IPP native flex apps. – Support from Ireland (timezone) – primarily e-mail based
Tool #3: Ranorex Studio • Quick facts – Ver 2.2 (2.3 coming out next week, 3.0 targeted for Fall ’10) – Built on .NET framework – Scripting support – C#, Python and VB.NET • Pluses – Mature tool – very intuitive interface – Object recognition using custom XPath – System integration testing with Quickbooks possible (not tested out) – Support for AJAX/Javascript testing – In-tool support for the whole object tree once the application is instrumented. – Has support for object preview. – Once the test is fully baked, you can export it as an executable (.exe). This can then be run simply by double clicking the exe. – Executable can be passed in command line params • Negatives – ?
Criteria for selection • Excellent support for object identification – Custom components like custom combo-box, item renderers • Easy to setup and ramp up • Mature support for scripting using standard languages • Ability to integrate with CI build • Performance • Good record and replay capability • Support for IE and Firefox
Quick Summary • FlexMonkey dropped from the list since it could not recognize some of the non-trivial objects like Custom ComboBox and DataGridColumn with item Renderer. • SilkTest 2009 is a good attempt but has a long way to go to be more developer/QA friendly in terms of the tool. Also, it has a higher cost for the license. • Ranorex satisfies our current requirements for automated testing, goes beyond what Adobe provides for automation support and delivers a great experience for the developer.

Automating functional testing of Flex applications.

  • 1.
    Automation of IPPFlex applications Gokuldas K Pillai @gokool
  • 2.
    Objective • Identify atool to automate the functional testing of a Flex application. In my application, we have the following use-cases: – Create/Edit Invoice – Search for a customer
  • 3.
    Tools Evaluated • FlexMonkey1.0 by Gorilla Logic • SilkTest 2009 R2 by Borland (Plugin for Flex) • Ranorex Studio • Other tools (not evaluated/less popular) – QTP – Selenium4Flex – RiaTest
  • 4.
    Automation support inFlex • Adobe provides automation delegate implementations for the framework components – Automation.swc (delegation implementations) – Automation_agent.swc (Facilitate communication with a tool) – Automation_dmv.swc (delegates for charts and advanced datagrid) • Third party tool developers leverage these and build on top of it. – The tool developers are limited by what Adobe exposes via the automation hooks. • Custom components • Components that extend framework components – Automation delegates supplied should work • Components that do NOT extend framework components (extend UIComponent instead) – Developer of component needs to provide automation delegates and override event handlers.
  • 5.
  • 6.
    Tool #1 :Flex Monkey • Quick Facts – Open Source – Ver 1.0 GA released – Adobe AIR application – Scripting language – Actionscript – Support for: • Record/Playback • Validation • Timing delays • Exporting script – integration with ANT script • Pluses – Simple to setup – Works on both OS (Windows/Mac) – Works with both IE and Mozilla • Negatives – Does not scale beyond simple usecases – Sparse documentation for advanced usage – Community is in infancy (Opensource like support ) – Recently moved out of Google code and the transition was not smooth for forum users – Core developers offer paid consulting gigs for training and help with tool.
  • 7.
    Tool #2 :SilkTest Flex plugin/Silk4J • Quick facts – 2009 R2 – Flex support since 2008 R2 – Silk4J – Eclipse plugin – Scripting language – Java or Silk script – Object identification using Xpath • Pluses – JUnit like test generation (Silk4J) – Familiar tool(Silk not Silk-Flex) for QA teams in Intuit – Enterprise support plan • Negatives – Performance hog (Seems like it is broken) – Tool is not very developer friendly. – Does not work with IPP native flex apps. – Support from Ireland (timezone) – primarily e-mail based
  • 8.
    Tool #3: RanorexStudio • Quick facts – Ver 2.2 (2.3 coming out next week, 3.0 targeted for Fall ’10) – Built on .NET framework – Scripting support – C#, Python and VB.NET • Pluses – Mature tool – very intuitive interface – Object recognition using custom XPath – System integration testing with Quickbooks possible (not tested out) – Support for AJAX/Javascript testing – In-tool support for the whole object tree once the application is instrumented. – Has support for object preview. – Once the test is fully baked, you can export it as an executable (.exe). This can then be run simply by double clicking the exe. – Executable can be passed in command line params • Negatives – ?
  • 9.
    Criteria for selection •Excellent support for object identification – Custom components like custom combo-box, item renderers • Easy to setup and ramp up • Mature support for scripting using standard languages • Ability to integrate with CI build • Performance • Good record and replay capability • Support for IE and Firefox
  • 10.
    Quick Summary • FlexMonkeydropped from the list since it could not recognize some of the non-trivial objects like Custom ComboBox and DataGridColumn with item Renderer. • SilkTest 2009 is a good attempt but has a long way to go to be more developer/QA friendly in terms of the tool. Also, it has a higher cost for the license. • Ranorex satisfies our current requirements for automated testing, goes beyond what Adobe provides for automation support and delivers a great experience for the developer.