Download as PDF, PPTX
Uploaded byAlan Richardson
Black Ops Testing Workshop from Agile Testing Days 2014
The document summarizes notes from a testing workshop on the Redmine project management application. Participants tested various aspects of Redmine and identified 15 bugs, including issues with PDF exports not containing all data, character encoding problems in CSVs, ability to delete the only admin account, and other data validation and synchronization bugs. Workshop leaders provided hints and interrupts to prompt deeper investigation and testing of areas like HTTP traffic, DOM manipulation, and breaking file exports.
Black Ops Testing Workshop from Agile Testing Days 2014
- 1. Black Ops Testing Workshop Agile Testing Days Tony Bruce Steve Green Alan Richardson
- 2. January 2015 Workshop,London www.BlackOpsTesting.com
- 3. Introduction ● 3* 5 minute lightning talks ● We expect you to test stuff ● We will coach & ask questions as you test ● We may periodically debrief
- 4. What you areabout to test ● Redmine ○ www.redmine.org ● Project Planning App ○ GUI, Rest API, Feeds, DB, Web Server
- 5. Alan’s Bit atthe start ● Model, Observe, Interrogate, Reflect, Manipulate ● Tools help me observe and manipulate ● Note Taking
- 6. A model ofhow Alan tests ● Model ○ What I think I understand. Different viewpoints. ● Observe ○ at different points to corroborate/invalidate model ● Reflect ○ find gaps, lack of depth, derive intent ● Interrogate ○ Focussed, deep dive observation with intent ● Manipulate ○ Hypothesis exploration and “how we do stuff”
- 7. Tools help me... … Observe and Manipulate
- 8.
- 9.
- 10.
- 11.
- 12. Note Taking ●Why: ○ Questions, Ideas, Risks, etc. ● What: ○ ToDos, Issues, Observations, Notes, etc. ● When: ○ Timestamps, sequential order ● Where: ○ urls, environment, users, etc. ● How: ○ commands, methods, tools used, etc. Evidence: ● logs, screenshots, output, files, etc.
- 13. Tony’s bit ●Test ideas ● Tools ● Information & Intelligence
- 14. Explore for testideas Prep - Notes - Summary - Important bits - Ideas - Comments - Questions - Thoughts - Six Honest Men "I Keep Six Honest Serving Men ..." I KEEP six honest serving-men (They taught me all I knew); Their names are What and Why and When And How and Where and Who. …….. Rudyard Kipling
- 15. Tools Browser -Add-ons - Extensions - Developer tools - Others - Accessibility - Security - Debugging -
- 16. Bob Stewart LeadershipUnder Pressure: Tactics from the Front Line
- 17.
- 18. Steve’s bit Whatare we going to test? ● What’s new? ● What’s changed? ● What’s important? ● What are known buggy areas? ● What has not been tested previously?
- 19. What are wegoing to test?
- 20. What might bedifficult?
- 21. What might bedifficult? ● How can we create enough data? ● How can we test time-related features? ● How do we know if the right thing happened?
- 22. Lightning Talk Debrief Extras? Questions? Comments?
- 23. Collaboration Rules ●Don’t load test the app, we are all using it ○ If you accidentally bring it down through a clever then that’s fine, ● Don’t change data you didn’t create
- 24. Where is theapp? ● You can install it locally if you want ○ http://redmine.org ● You can get an install or VM from bitnami ○ https://bitnami.com/stack/redmine ● You can use the redmine demo ○ http://demo.redmine.org/ ● You can use our server ○ ….
- 25. Where is theapp? ● Links removed as only valid at the time we conducted the workshop
- 26. Testing Phase 1 ● Consider what we said ● Test the app in new ways, take notes, try new tools ● Black Ops Team will mingle - do ask for help ● We will debrief prior to the break
- 27. “...no plan ofoperations extends with any certainty beyond the first contact with the main hostile force.” Field Marshall Helmuth Carl Bernard Graf von Moltke, 1871
- 28. Test the ****out of Redmine
- 29. Hints... Are youmonitoring the HTTP traffic? Have you changed the DOM? Cookies? Internationalisation? Logged issues? ...
- 30.
- 31. Break (30 mins) Feel free to carry on testing if you want
- 32. Testing Phase 2 ● You tell us
- 33. Interrupt 1 Structureyour data to make testing easier. ● Unique values (where possible) ● Sequenced
- 34. Interrupt 2 Testingmaxlength and truncation.
- 35. Interrupt 2 Testingmaxlength and truncation. 0005x0010x0015x0020x0025x0030x0035x004 0x0045x0050x0055x0060x0065x0070x0075x0 080x0085x0090x0095x0100x0105x0110x0115 x0120x0125x0130x0135x0140x0145x0150x01 55x0160x0165x0170x0175x0180x0185x0190x 0195x0200x0205x0210x0215x0220x0225x023
- 36. Interrupt 2 Testingmaxlength and truncation. http://bit.ly/1B7gQlx
- 37. Interrupt 2a ●We have a broken app - can you get in? ● Links removed as only valid at the time of the workshop
- 38. Interrupt 3 Recordingdata flow.
- 39. Interrupt 4 Canwe break the CSV, Atom or PDF exports? What might do that?
- 40.
- 41. Bugs we found1 ● The PDF does not contain the Start Date, Estimated Time, % Done or File Description for the attachments that are included in the New Issue form. ● The PDF does not contain the Target Version and Spent Time values that are included in the View Issue form.
- 42. Bugs we found2 The Atom feed from the View Issue page has no content when it is viewed immediately after creating an issue. It did have content after adding a quote to the issue.
- 43.
- 44. Bugs we found3 The File Description for an image is not saved if too many characters are entered in the New Issue form. We did not investigate where the boundary is.
- 45. Bugs we found4 The PDF that is generated from the Gantt page always shows the default zoom level regardless of the zoom level that has been selected. The URL of the PDF link contains a ‘zoom’ parameter (which does nothing). Changing the ‘months’ parameter has the desired effect.
- 46. Bugs we found5 Some non-Roman characters are displayed correctly on all HTML pages but they are not displayed in PDFs.
- 47. Bugs we found6 Some non-Roman characters are not displayed correctly in CSVs.
- 48. Bugs we found7 You’re able to delete all user accounts, including admin. There is only 1 admin (as far as I could see) Tell us how….
- 49. Bugs we found7 cont. Record browser traffic while deleting a account. Find the delete POST
- 50. Bugs we found8 "Your account has been activated. You can now log in. " System says I am already "logged in as eris" and I am on my account page Minor issue about wording
- 51. Bugs we found9 Error message about emails already in use when registering - privacy concern “Email has already been taken”
- 52. Bugs we found10 Maximum length of email is 60 chars but needs to accept 254
- 53. Bugs we found11 Can use an invalid language when registering a user.
- 54. Bugs we found12 Truncation on project identifier with no error or warning message i.e. create project with 255 char identifier - truncated to 100
- 55. Bugs we found13 When creating a project, the ID and name are populated via javascript but if I change the name then the identifier is not kept in sync.
- 56. Bugs we found14 Can create an invalid enabled_modules entry by submitting a module name which does not exist when creating a project
- 57. Bugs we found15 Change url to have csv or pdf views System should respond differently to csv and pdf on projects when GUI request rather than an API request 406 is better for API, 404 with html or 406 with html payload might be better
- 58. Rathole 1 -Password Alan thought there was a bug with password lengths, and storing in a varchar 40, since password can be very long. But, a ‘hash’ is stored, not the password, this took time to discover.