End-to-End Testing is a crucial aspect of software development as it ensures that all components of a system are functioning correctly together. CodeceptJS is an efficient and robust End-to-End automation framework with which you can avoid vendor lock-in and benefit from rich locators, interactive debugging, and much more. When combined with Playwright, it becomes a powerful tool for automating web, mobile, and even desktop (Electron.js) applications. In this article we will explore how to build a End-to-End testing pipeline with CodeceptJS, Playwright and GitHub Actions.
GitHub Actions
GitHub Actions is a powerful and flexible CI/CD platform that enables you to automate your software development workflows. With GitHub Actions, you can quickly automate your development, testing, or operational processes directly within your GitHub repository. GitHub Actions seamlessly integrates with CodeceptJS and Playwright, providing a reliable solution for your project’s test automation needs.
Prepare the environment
We will start with configuring the Environment Variables that will define how the pipeline will be running as well as control its behaviour. Typically projects run multiple environments, therefore we will define BASE_URL variable as a minimum to alternate between running environments.
Navigate to your GitHub repo settings and under Secrets and variables section select Actions. Clicking on Variables tab will show the list of all variables for this repo.
GitHub repository environment variables settings
Click on New repository variable to create BASE_URL that will point test automation to your web application url.
Other useful settings that can be implemented using environment variables:
HEADLESS=true - to run headless mode inside the pipeline or headful on your local machine to debug test scenarios.
DEVICE_SCALE_FACTOR=1 - avoid screen flickering while developing test scenarios locally (use 2 for MacBook screens).
ACCESSIBILITY=true - include accessibility reports in your End-to-End Testing.
RECORD_HAR=true - record network traffic in a HTTP archive to use for debugging purposes.
Configure the pipeline
Create a new file in YAML format to configure the pipeline and put it in .github/workflows folder.
CodeceptJS will show detailed steps only for failed scenarios and this is what is needed in most cases.
Use consistent helper dependencies
We are using Playwright as a CodeceptJS helper in this case, so make sure Playwright npm package version matches the Docker image version in the pipeline.
Running Playwright automation in Github Actions fails for Firefox with the error message: Running Nightly as root in a regular user’s session is not supported.
GitHub Actions pipeline log with Running Nightly as root in a regular user’s session is not supported error.
To fix this issue we need to workaround the problem by adding a HOME environment variable into the pipeline.
Building an end-to-end testing pipeline with CodeceptJS, Playwright, and GitHub Actions provides a powerful solution for automating your test scenarios. By leveraging the capabilities of CodeceptJS and Playwright, you can easily automate testing across different browsers and environments. GitHub Actions allows you to seamlessly integrate and automate your testing process within your GitHub repository. With the configuration and setup outlined in this article, you can ensure that your software development workflows are efficient and reliable, enabling you to deliver high-quality software with confidence.
