Uploaded byZoltan Micskei
Model-based Regression Testing of Autonomous Robots
The document discusses a model-based regression testing approach for autonomous robots, focusing on projects like R3-COP and R5-COP. It outlines the limitations of current testing methods and proposes a new approach that integrates regression test selection and domain-specific languages to efficiently classify existing test scenarios. The findings highlight the scalability of the technologies used and the utility of the generic approach across various testing iterations and DSLs.
Model-based Regression Testing of Autonomous Robots
- 1. Budapest University ofTechnology and Economics Department of Measurement and Information Systems Fault Tolerant Systems Research Group Model-based Regression Testing of Autonomous Robots David Honfi, Gabor Molnar, Zoltan Micskei, Istvan Majzik 1 18th International Conference on System Design Languages (SDL 2017) DOI: 10.1007/978-3-319-68015-6_8
- 2.
- 3. Context: R3-COP andR5-COP projects 3 Household service robots Automated forklifts http://www.elettric80.com/ http://www.care-o-bot.de Search and rescue robots http://www.piap.pl
- 4. Testing approaches Simulating robotand environment • Not yet widespread (but changing) Replaying captured sensor data • Based on real data, but coverage? Testing with real robot in “real” environment • Expert operators, experience-based • Resource- and time-intensive 4
- 5. Standard test method:DHS-NIST-ATSM Testing robots in physical environment Standardized apparatus and procedure E.g.: ramp, gap, sand, sign, door opening 5 Source: NIST. Guide for Evaluating, Purchasing, and Training with Response Robots Using DHS-NIST-ASTM International Standard Test Methods, 2014 Source: ASTM E2801-11, Standard Test Method for Evaluating Emergency Response Robot Capabilities: Mobility: Confined Area Obstacles: Gaps, ASTM International, West Conshohocken, PA, 2011 5
- 6. Previous work: Model-basedapproach 6 Z. Micskei, Z. Szatmári, J. Oláh, I. Majzik: A Concept for Testing Robustness and Safety of the Context- Aware Behaviour of Autonomous Systems, TruMAS 2012. DOI Context model Configuration model Safety properties Test data generation Few, but diverse scenario Test execution
- 7. New challenges: autonomy,modularity… Frequent reconfiguration New tasks (+ new SW) Swapping HW modules New, unknown environment 7
- 8.
- 9. Reconfiguration Regressiontesting Regression test selection (RTS) o Rich related work for code o Categorization: Re-usable, Re-testable, Obsolete, New Model-based development o Domain-specific languages (DSL) 9 How to perform regression test selection on DSLs?
- 10. Regression test selectionmetamodel 10 Represent changes in different DSLs in one model E.g.: (Config) Robot has a motor. (Context) Test scenario 1 has sand terrain. (Mapping) Motor is tested by sand in test scenario 1.
- 11. Architecture of theprototype tool 11 Input: Models describing the application + tests Output: classification of existing test cases Adapters to various models and changes RTS algorithm is implemented only once
- 12. Proposed Workflow Context and Component Models Model Transfor- mation Changein the Input Models Reduced Set of Necessary Tests 12 MDD/DSL expert Test engineer
- 13.
- 14. Experiences Used technologiesscale well (EMF, VIATRA…) (see paper for evaluations) Generic approach proved useful o Several DSLs, several iterations Not easy to get abstraction level right 14
- 15. Model-based regression testingin action 15 Input Output
- 16.