ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel

Agile Testing Explained

Updated on June 21, 2011


The role of testers and testing in Agile development is largely misunderstood. Although there are many ways that testers can help in this maturing area (several books have been written on the subject) the main role of testers is to take user stories and turn them into business-facing tests by way of examples. The majority of these are then automated to provide ongoing regression testing.

Figure 1 illustrates the Agile Testing approach.

Unit Testing and Business-Facing Testing take place in parallel, both creating automated tests that form part of the Continuous Build. There is no waiting for code drops and a close working relationship between developers, business analysts and testers.

Figure 1 - Agile Testing
Figure 1 - Agile Testing

Figure 1 Notes

  1. Testers work closely with Business Analysts (or people with relevant domain knowledge) to take user stories and construct table-based examples that can be shared with developers. For the most part these are then automated and included in the Continuous Build.

    The mindset of testers is different to that of business analysts and developers. They're adept at defining both positive and negative test cases. Over time the goal should be to build up extensive test suites that verify not just the happy path that's of most interest to business users, but also negative tests to confirm that the application is sufficiently robust.
  2. The use of concrete examples acts as a vehicle for confirmation of detailed business needs and provides developers with a clearer basis on which to construct the solution. Business Analysts, Developers and Testers meet regularly to clarify requirements.

    One deficiency of Agile development is a lack of detailed functional requirements. Developers traditionally draw knowledge of what's needed from these and other detailed design documents. Also, because of the short development cycles (e.g. a typical "Scrum" Sprint is 4 weeks), there's a tendency to focus on developing and unit testing individual pieces of functionality with less attention paid to the overall business workflow and rules, Sometimes important elements of end-to-end functionality are missed. Use of detailed end-to-end examples helps to fill these gaps. They also add a layer of traceability that gives stakeholders and the business greater confidence in the development process.
  3. Unit Testing is conducted by developers, typically as a by-product of Test Driven Development (TDD). Tools such as JUnit are used to automate these tests and they are included in the Continuous Build Process for ongoing Regression Testing.

    Traditional testers don't normally get involved with Unit Testing. They should however have visibility of the Unit Testing process, confirming that developer testing is in line with an agreed and acceptable approach. Try to put yourself in the shoes of the stakeholder. What assurances will stakeholders be looking for as the project moves forward? Confirmation from Quality Assurance that developers are properly testing what they build is one of these.
  4. Calculation examples are (often) the simplest and most common form of tests with each one represented by a single table row that includes both input and expected output. The same style can also be used for decision-type tests (e.g. with Yes / No output).
  5. List examples produce sets of results i.e. multiple rows. They're more complicated than calculation tests with expected outputs varying depending on filtering, duplication etc.
  6. Process examples allow the execution of steps and an assessment of success/failure along the way. They are typically used to test workflow.
  7. Mixed examples are the reality of Agile Testing. A way to build up more complex test suites that accurately reflect the execution of complete user stories and epics.
  8. Fixtures allow communication between test examples and the application under test. How fixtures are developed and used varies depending on the Testing Framework used. In its simplest form a developer extends a base class (these exist for each of the test types: Calculation, List and Process) and codes in the relevant attributes and method names to facilitate linkage.

    The development of Fixtures is a potentially onerous task and requires programming skills. There’s a real need out there for testers who have a coding bent!
  9. The Testing Framework provides the infrastructure for execution of business-facing tests. It scans the tables of test examples, instantiates the relevant fixtures and requests execution against the Application Under Test (AUT). On return it completes a comparison of actual versus expected values and then logs the result.

    There are several offerings in the market including:

    FIT (Framework for Integrated Testing ... the first of its kind 2002)
    Fitnesse (using FIT)
    Fitnesse (using SLIM)
    Robot Framework
  10. One of the key advantages of combining examples with a Testing Framework is the ability to automate business-facing tests and include them in the Continuous Build. These act as Regression Tests ensuring that desired business functionality continues to operate correctly as changes are made by developers.
  11. Inclusion of unit and business-facing tests is usually a simple matter e.g. adding a reference to (or the example tables themselves) a directory structure that’s used as input for the build.

Agile Development Testing Context

The context within which Agile Testing occurs varies from project to project depending upon the Agile methodology used and the delivery and quality assurance requirements of individual stakeholders.

Figure 2 illustrates a common Agile Testing situation in the context of Scrum-based Agile Development using Sprints. User Acceptance is conducted following release of significant areas of functionality.

Figure 2 - Agile Development - Testing Context
Figure 2 - Agile Development - Testing Context

Figure 2 Notes

  1. Sprint Zero is a preparation sprint, a time to sort out setup of the Testing Framework, test environment, how each sprints will work etc.
  2. The makeup of a sprint consists of initial planning and then the day to day development and creation of business-facing tests. It culminates with manual regression testing, presentation to the business and refactoring of code and tests.
  3. Releases typically consist of end to end functionality that marks a significant milestone for the business. Functionality is deemed to be complete in terms of business rules provided.
  4. User Acceptance Testing (UAT) is checking by the business to verify that they’re getting what they want and have paid for. The value of business-facing tests comes into its own here. As input to UAT the business may even decide that these tests are sufficient on their own to cover off the verification. The bottom line is that the extent of UAT conducted is up to the business.
  5. The product backlog is a list of user stories requested by the business. It typically gets larger, not smaller, with the business adding more requests as the project progresses. At the end of the project the product backlog often contains undelivered stories.
  6. A planning session is conducted at the beginning of each sprint to decide which user stories will be developed and the tasks, including resources and priorities, involved. It is normal for user stories to be grouped into epics for this purpose.
  7. The Sprint Backlog typically consists of the developer tasks to be completed during the Sprint e.g. test preparation, coding, testing.
  8. A selection of user stories, usually combined into Epics, is assigned for the creation of business-facing tests.
  9. There will be a significant number of business-facing tests that require too much effort or cost to automate. These should be prepared to assist with development and executed either towards the end of the Sprint or at some time prior to release for UAT.
  10. A retrospective is an opportunity for the business to see the delivered functionality for the Sprint. It also acts as a UAT primer, giving the business a chance to vet what’s been done and request changes.
  11. Time is usually allowed towards the end of the Sprint for rationalisation of both programming code and tests.

Frequently Asked Questions

  1. Who creates the fixtures?

    Either a developer or, and ideally, a tester who has programming skills. There's a real niche out there for traditional QA-style testers who can take the load off developers and develop their own fixtures.

  2. What are the biggest traps?

    The biggest trap is trying to do traditional system testing with Agile development i.e. waiting for a code-drop and then testing against it. It doesn't work because traditional system testing relies on the existence of functional specifications ... that typically don't exist in Agile development.

    Another trap is trying to automate "all" of the business-facing tests. Some of them will be just too difficult to build fixtures for in which case you should simply allocate time towards the end of each sprint for manual testing.

  3. What's the reality? Is Agile Testing truly viable?

    It's more than viable. Creating examples of business rules and using them in place of functional specifications to aid development, as well as being able to test what's built ongoingly, gets results faster and to higher quality.

    There's much for QA-style testers to do both in creating (and agreeing with the business) the business-facing tests and also helping to integrate / automate the tests with the application under test.

    Fixture creation is often not straight forward and "simple" tests are rarely the norm. It's common to have a combination of calculations, set verifications and process checking all in a single test. So get ready for some complexity!
  4. What else should QA-Testers be doing?

    There are books written that cover "the whole" of an Agile Tester's lot including areas such as Exploratory Testing and non-functional test areas e.g. performance testing, that Agile development tends to overlook. Avoid getting caught up in these areas if you're just starting out. Putting the business-facing tests together and automating and including them in the continuous build process has to be front and centre.
  5. What framework should I use?

    For me GreenPepper is the way of the future. It's well documented and well connected in terms of associations with recognised vendors (i.e. Atlassian). It's a good fit with tools like Jira. I also like the fact that you have to pay for it (albeit nominally for small businesses) which brings with it support and a better future (possibly).

    There are functional reasons for my preference as well e.g. its editing facilities are better and the fixtures are completely hidden from the test-writer which is a good thing if developers are going to create the fixtures.

    By all means check out some of the other offerings that are out there including Fitnesse, FIT, SLIM and Concordian.
  6. How does this measure up to standard System Testing?

    I point to things like traceability and practical input and integration with development as key ingredients to both creating quality applications and satisfying stakeholder concerns.

    Creating examples for the business rules as part of the development effort creates not only good input material for the developers and ongoing regression testing during (and after) development, these also trace back directly to user stories and, in turn, business requirements. So business-facing tests stack up well in terms of traceability and they also lead in favourably to the UAT process (see below).

  7. Are business-facing tests the same as UAT?

    UAT means different things to different organisations depending on their circumstances. The business-facing tests created by QA-style testers provide a great starting point for the business to do their Acceptance Testing. And if the business are able to use the same table-based standard for the definition of further "UAT" tests, there's no reason why they can't also be automated and included in the build process.

    So, to answer the question, the business-facing tests created by QA-style testers "could" indeed be deemed the same as or at least part of UAT if the business is satisfied that they provide sufficient evidence of what they expect the application to do.

    Alternatively the business can prepare and execute completely different tests to satisfy themselves that all's well.


    0 of 8192 characters used
    Post Comment
    • Russell O'Brien profile imageAUTHOR

      Russell O'Brien 

      8 years ago from Sydney

      Fair comment. Agile Testing is largely about putting together examples. I'll see what I can do.

    • profile image


      8 years ago

      Interesting description as far as it goes. Could do with a real example, especially as this seems to be about creating examples.


    This website uses cookies

    As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

    For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

    Show Details
    HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
    LoginThis is necessary to sign in to the HubPages Service.
    Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
    AkismetThis is used to detect comment spam. (Privacy Policy)
    HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
    HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
    Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
    CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
    Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
    Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
    Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
    Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
    Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
    Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
    VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
    PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
    Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
    MavenThis supports the Maven widget and search functionality. (Privacy Policy)
    Google AdSenseThis is an ad network. (Privacy Policy)
    Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
    Index ExchangeThis is an ad network. (Privacy Policy)
    SovrnThis is an ad network. (Privacy Policy)
    Facebook AdsThis is an ad network. (Privacy Policy)
    Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
    AppNexusThis is an ad network. (Privacy Policy)
    OpenxThis is an ad network. (Privacy Policy)
    Rubicon ProjectThis is an ad network. (Privacy Policy)
    TripleLiftThis is an ad network. (Privacy Policy)
    Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
    Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
    Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
    Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
    ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
    Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)
    ClickscoThis is a data management platform studying reader behavior (Privacy Policy)