Tuesday, October 30, 2012

Acceptance testing through the UI layer

This is a subject I have been thinking about for a few weeks now. On a recent project I was working on  we ran an automated test suite that handled all acceptance tests through specflow + webdriver. I'm not sure I would attempt this again but find myself inheriting a system that sounds like it is based on a lot of the same principles. Here are a few challenges I faced that I hope to avoid:

  • Asynchronous integration with external systems
    • In my previous case, we had an external system that outputted emails that needed to be checked for content. This could take between 1 second and 3 mins to complete and was of extremely high business value. 
    • Solved by having an AssertWithRetry that would check every every 500 ms and sleep until it timed out
    • Problem: Delayed feedback cycle. Many test runs had multiple waits and could delay the test suite
  • Flakiness of Selenium due to Ajax + javascript errors
    • Lead to perception that tests could not be trusted by dev team
    • "Flaky test" vs. "Legitimate bug" fights with devs. 
    • Frequent quarantining of tests without enough investigation. Client + Devs unwilling to accept time it took for debugging.
    • Solved with re-run
  • Edge Case testing through UI
    • Can be solved with high level of parallelization. Not always an option
  • Zero Failed test before check-in
  • Redundancy in Specflow Steps
  • Test IE first vs. Firefox?

1 comment:

  1. Hello William,
    The Article on Acceptance testing through the UI layer, gives amazing information about it. Thanks for Sharing the information about the features of Acceptance testing through the UI layer, For More information check the detail on the User Acceptance Testing here
    Software Testing Services

    ReplyDelete