Acceptance Criteria in Agile Testing 

In the Agile tapestry, acceptance criteria are like golden threads that connect user stories to their final form. These criteria help testers shape testing strategies and act as a critical threshold for verifying functionality and quality assurance.

Acceptance criteria promise that the delivered product will align with customer expectations.  With clearly defined customer needs, they mitigate possible misunderstandings and promote transparency. This alignment with customer expectations directly contributes to customer satisfaction. 

How To Develop and Document Acceptance Criteria 

Developing and documenting acceptance criteria is crucial to the Agile framework. It requires a collaborative effort. It also sets a well-defined outline for what’s required for a user story to be considered complete. 

User Stories and Acceptance Criteria

User stories and acceptance criteria are closely related elements within Agile development, where user stories act as a narrative placeholder for a requirement or feature, and acceptance criteria define the conditions that must be met for the story to be considered complete.

User Stories

These are short, simple descriptions of a feature or functionality from an end-user perspective. They typically follow a format like “As a [type of user], I want [some goal] so that [some reason].” User stories capture what needs to be accomplished without going into exhaustive detail.

Acceptance Criteria

These conditions or requirements must be met for the user story to be considered finished. They outline the boundaries and details of the functionality described in the user story. Acceptance criteria define the “definition of done” for that particular story.

The relationship between them is that acceptance criteria are derived from and closely tied to user stories:

1. Derived from User Stories: Acceptance criteria are extracted from and directly linked to user stories. They elaborate on the specifics of what needs to be delivered to fulfill the story’s objectives.
2. Detail and Validate User Stories: Acceptance criteria provide the necessary details to ensure that the user story is well-defined and testable. They help in validating that the implemented functionality meets the user’s expectations.
3. Mutual Understanding: Together, user stories and acceptance criteria provide a shared understanding among the development team, stakeholders, and customers about what needs to be delivered and how its completion will be measured.
4. Testable Conditions: Acceptance criteria serve as a basis for creating tests to validate that the user story has been implemented correctly. They outline specific scenarios that can be tested to ensure the functionality works as expected.

In essence, user stories set the stage and context for what needs to be accomplished, while acceptance criteria provide the detailed conditions and requirements necessary to meet the objectives outlined in the user story. 

Example of User Stories and Acceptance Criteria

Let’s consider a real-life scenario in an e-commerce application:

User Story: As a customer, I want to be able to track my orders to see the status of my purchases.

Acceptance Criteria: 

1. When logged in: When a user is logged in, they should have a “My Orders” section visible in the navigation bar.
2. Order History: In the “My Orders” section, the user should see a list of all their past orders, displaying order number, date, and status (e.g., processing, shipped, delivered).
3. Order Details: Clicking on an order should display detailed information, including items purchased, quantity, price, and shipping details.
4. Order Status Updates: The order status should update in real-time. For example, if an order changes from processing to shipped, the status should reflect this change immediately.
5. Email Notifications: Users should receive email notifications for order status changes. Emails should include a summary of the status change and a link to view details on the website.

In this scenario, the user story outlines the high-level need from the customer’s perspective: the ability to track orders. 

The acceptance criteria detail the specific functionalities and behaviors required to fulfill that user story. They define what constitutes a completed and functional feature, providing guidance for development and testing.

These acceptance criteria ensure that the tracking orders feature meets the customer’s expectations by providing a clear structure for what the feature should include and how it should behave in the e-commerce application.

Who Defines Acceptance Criteria? 

Acceptance criteria are typically defined collaboratively by a combination of roles within a software development project. Here are the key stakeholders involved in defining acceptance criteria:

• Product Manager: They often take the lead in defining acceptance criteria since they’re responsible for understanding user experience and needs, market requirements, and overall product vision. They work closely with stakeholders to gather requirements and ensure the acceptance criteria align with the user’s perspective.
• Business Analysts: These professionals play a crucial role in understanding business needs, analyzing requirements, and translating them into actionable acceptance criteria. They bridge the gap between business stakeholders and the development team.
• Development Team (Developers and Testers): Developers and testers provide insights into the technical aspects and feasibility of meeting certain criteria. Testers, especially, contribute by ensuring the criteria are testable and verifiable.
• Stakeholders: Other stakeholders, including end-users or customer representatives, might contribute to defining acceptance criteria by providing insights into user expectations and needs.

In an Agile environment, collaboration among these roles is key. The Product Owner often takes the lead in prioritizing and finalizing acceptance criteria, but it’s a team effort to ensure that the criteria are comprehensive, clear, and aligned with the project’s overall goals.

Best Practices for Developing Acceptance Criteria 

Developing effective acceptance criteria is crucial for successful software development. The goal is to create a set of criteria that are testable and directly related to the user story they support. Here are some best practices to consider:

• Collaborate: Involve all stakeholders. Collaboration ensures that diverse perspectives are considered, leading to a comprehensive understanding of requirements. It fosters a shared ownership of the acceptance criteria and minimizes misunderstandings.
• Use clear, concise, measurable language: Clear language helps in avoiding confusion or misinterpretation. Measurable language sets specific expectations that can be objectively validated, ensuring a shared understanding of success.
• Make criteria specific and testable: Specific criteria avoid ambiguity, providing clear guidance on what needs to be delivered. Testable criteria enable validation, ensuring that the requirement can be objectively verified.
• Align/link criteria with user stories or features: Connecting acceptance criteria to user stories or features provides context and relevance, ensuring that each criterion directly relates to a specific functionality or user need.
• Prioritize: Prioritizing acceptance criteria helps in managing time and resources effectively. It ensures that essential functionalities or requirements are addressed first.
• Be explicit and use examples: Avoiding ambiguity in acceptance criteria is crucial. Using examples or scenarios clarifies expectations, leaving little room for misinterpretation.
• Regularly update criteria as needed: Acceptance criteria may evolve as the project progresses or as new information becomes available. Regular reviews and refinements ensure that criteria remain aligned with project goals and stakeholder needs.
• Ensure stakeholders agree on the criteria: Consensus among stakeholders about acceptance criteria is crucial. It ensures that everyone has a shared understanding and commitment to the criteria, reducing the likelihood of conflicts or misunderstandings later in the project.

Good Acceptance Criteria Examples

Here are some examples of acceptance criteria that adhere to best practices:

User Story: As a registered user, I want to reset my password to regain access to my account if forgotten.

Acceptance Criteria:

1. When accessing the ‘Forgot Password’ link:
   1. Given I’m on the login page, when I click on the ‘Forgot Password’ link, then I should be directed to a password reset page.
2. Entering email for password reset:
   1. Given I’m on the password reset page, when I enter my registered email address and submit, then a confirmation message should appear indicating that an email has been sent for resetting the password.
3. Receiving password reset email:
   1. Given I’ve requested a password reset, when I check my email, then I should receive an email with a password reset link.
4. Clicking the password reset link:
   1. Given I’ve received the password reset email, when I click the reset link in the email, then I should be directed to a page where I can enter a new password.
5. Setting a new password:
   1. Given I’m on the password reset page, when I enter a new password and confirm it, then a success message should confirm that my password has been updated.
6. Logging in with the new password:
   1. Given I’ve reset my password, when I try to log in using the new password, then I should be able to access my account successfully.

These acceptance criteria exemplify best practices by being specific, testable, measurable, aligned with user needs, and comprehensive. These criteria help ensure that the password reset functionality meets user expectations and is functional across the entire process, from initiating a reset request to successfully accessing the account with a new password.

By following best practices, teams can create acceptance criteria that are comprehensive, clear, and aligned with stakeholder expectations, leading to successful project outcomes and improved communication within the team.

Gherkin Syntax

Gherkin is a human-readable language used primarily for behavior-driven development (BDD). It provides a structured way to define and document software behaviors in a format that is easy to understand for both technical and non-technical stakeholders.

Gherkin uses syntax with keywords like Given, When, Then, And, and But to describe the behavior of a system in a way that can be translated into automated tests using tools like Cucumber or SpecFlow.

While Gherkin is commonly used due to its clarity and its alignment with BDD principles, Agile teams might use other formats or tools to define acceptance criteria. Some teams might use simple user stories, detailed checklists, plain text descriptions, or specific templates tailored to their project needs.

Ultimately, the choice of how to document and define acceptance criteria in Agile depends on the team’s preferences, the nature of the project, the tools available, and the level of technical expertise within the team.

Writing Acceptance Criteria in Gherkin

Here’s an example of how acceptance criteria might be written in Gherkin format for a simple feature like user authentication:

In this example:

• Feature defines the overall functionality being described.
• Scenarios describe specific instances or behaviors related to user login.
• Given, When, Then, And are keywords used to outline the steps of the scenario. They represent the initial context, actions, and expected outcomes of the described feature.

These scenarios can serve as both documentation for what the feature should do and as a basis for automated tests. Automated testing tools that understand Gherkin syntax can translate these scenarios into executable tests, verifying that the application behaves as specified.

This is just one way to write acceptance criteria in Gherkin format. Teams might adapt or extend this structure based on the feature’s complexity or the project’s specific needs.

Tools for Documenting Acceptance Criteria 

Several tools can aid in documenting and managing acceptance criteria effectively:

• Jira: Jira is a widely used project management tool that helps teams manage their work through customizable workflows, issue tracking, and real-time collaboration. Jira’s collaborative strengths lie in its ability to centralize project information, facilitate communication among team members, and enable the creation of user stories and acceptance criteria within its platform, ensuring clear visibility into the progress of work items among team members
• Confluence: Often used alongside Jira, Confluence is a collaborative workspace tool used for creating, organizing, and discussing project documentation. Confluence serves as a centralized platform where teams can document and share acceptance criteria, user stories, and other project-related information.
• TestRail: TestRail is a test management tool designed to help teams organize, manage, and track their software testing efforts. While it’s not directly involved in creating acceptance criteria, TestRail allows users to create and manage test cases linked directly to acceptance criteria in an issue-tracking tool, making tracking and ensuring test coverage easy.
• Trello: Another popular project management tool, Trello allows the creation of cards containing user stories and their associated acceptance criteria on a collaborative Kanban-style board. 

Regardless of what tool you use, its documentation should be accessible to all team members and updated as new insights are gained and processes evolve. 

Acceptance Criteria and Testing Phases 

Acceptance criteria guide the software testing process at every stage. Here’s a table format summarizing the relationship between acceptance criteria and different testing phases in software development:

Testing Phase	Role of Acceptance Criteria
Test Planning	-Form the basis for developing test plans and strategies -Help in identifying what needs to be tested
Test Design	-Act as a reference for designing test cases and scenarios -Each criterion becomes a test condition
Test Execution	-Provide the basis for validating implemented functionality -Testers ensure that criteria are met during execution
Defect Reporting	-Criteria deviations are documented as defects or issues -Helps in capturing discrepancies between expected and actual behavior
Validation	-Serve as a benchmark for validating software against expectations -Determine if the software meets customer requirements
Regression Testing	-Guide regression tests to ensure new changes don’t affect fulfilled criteria -Criteria help in identifying areas needing revalidation

Test Case Creation and Acceptance Criteria

The development of test cases heavily relies on acceptance criteria as they provide the necessary guidance, structure, and specifics that testers use to design, execute, and validate test cases, ensuring that the software meets the expected standards and requirements.

Let’s consider an example:

Acceptance Criteria:

For an e-commerce platform, the acceptance criteria for the checkout functionality might include:

1. Users should be able to add items to their cart.
2. Users should be able to proceed to checkout from the cart.
3. During checkout, users should enter shipping and billing information.
4. Users should be able to select a payment method.
5. After successful payment, users should receive an order confirmation.

Development of Test Cases:

Using these acceptance criteria, test cases are designed to validate the functionality of the checkout process:

1. Test Case 1: Adding Items to Cart
   1. Steps: Navigate to product page > Add item to cart
   2. Expected Outcome: The item is added to the cart
2. Test Case 2: Proceeding to Checkout from Cart
   1. Steps: Click on ‘Proceed to Checkout’ from the cart
   2. Expected Outcome: The user is directed to the checkout page
3. Test Case 3: Entering Shipping and Billing Information
   1. Steps: Fill in shipping and billing details
   2. Expected Outcome: Information is successfully entered and saved
4. Test Case 4: Selecting Payment Method
   1. Steps: Choose a payment method (e.g., credit card, PayPal)
   2. Expected Outcome: Payment method is selected without errors
5. Test Case 5: Order Confirmation
   1. Steps: Complete payment
   2. Expected Outcome: The user receives an order confirmation message or email

In this scenario, acceptance criteria define the high-level requirements and functionalities expected from the checkout process and test cases are derived from these criteria and outline specific steps and expected outcomes to validate each aspect of the criteria. 

Each test case aligns with a particular criterion and ensures that the system behaves as specified in the criteria. These test cases are executed to verify that the implemented checkout functionality meets the user expectations as outlined in the acceptance criteria. Any discrepancies found during testing are recorded as defects, initiating a feedback loop to address and align the software with the defined criteria.

How to Validate and Verify Acceptance Criteria 

Acceptance criteria ensure that the development process meets user requirements, but what if the criteria are ill-defined? 

Navigating Ambiguity in Acceptance Criteria 

If the acceptance criteria are unclear, the developed features may not meet the user’s needs, leading to longer development time, late delivery, and a software product that doesn’t satisfy customers. 

For example, poorly defined acceptance criteria for an online payment system might read: “Customers should be able to pay online.”  

This sentence must include more detailed information about the steps, outcome, and delivery time. 

Conversely, a well-defined acceptance criterion might read: “Given that customers select items to buy and enter valid payment, they should get a confirmation message and email receipt immediately when confirming the purchase.” 

Ensuring Clarity and Completeness 

The INVEST criteria are guidelines for assessing and creating well-formed user stories in agile software development. While primarily used for user stories, these criteria can also be applied to validating and verifying acceptance criteria to ensure they are effective. Here’s how the INVEST method applies:

• Independent: Criteria should stand alone
• Negotiable: Criteria should be open to discussion and adjustment
• Valuable: Criteria should deliver meaningful value
• Estimable: The estimated effort to complete the criteria should be clear and realistic
• Small: Criteria should be specific and manageable
• Testable: Criteria should be verifiable through testing

Suppose you’re developing a feature for an e-commerce website. This feature involves calculating and displaying the total price of items in a shopping cart. 

• Independent: The acceptance criterion for the total price should not rely on other modules. For example, this functionality should not depend on the shipping module. 
• Negotiable: The criterion should always be open to discussion and adjustment. If pricing rules are changed, the criterion may need to be adjusted. 
• Valuable: This example passes the “Valuable” check because an accurate total price is very valuable to the end user customer.
• Estimable:  The team should be able to calculate how much effort it will take to calculate the total price accurately and realistically.
• Small: The criterion should be as specific as possible and focus on a particular behavior, such as calculating the total price while considering taxes and discounts.
• Testable: The criterion should be possible to test, ensuring that the total price is always calculated correctly for the end user. 

Applying the INVEST method helps teams create criteria that are clear, manageable, and aligned with the Agile principles of delivering value to the customer.

Bottom line

Acceptance criteria are essential to agile teams as they offer precise, shared definitions of what constitutes a completed task, ensuring clarity among team members and stakeholders. They enable focused efforts by outlining priorities and guiding development towards valuable features. 

Crucially, these criteria serve as the basis for testing and validation, ensuring that the product’s deliverables meet the specified requirements. 

By fostering a shared understanding and clearly defining “done,” acceptance criteria facilitate incremental development, continuous improvement, and effective collaboration within agile teams.