Cypress vs Selenium, which tool will you choose?

Intro

Selenium has grown over time to become the world’s de facto web testing tool. Supported by the Browser vendors such as Google and Microsoft, It has even received an endorsement from W3C (World Wide Web Consortium), the community which works on web standards development. Over the years, it has become the biggest open source automation tool for the web. It has many benefits such as multi-platform, multi-language, multi-browser, Selenium Grid for parallel test execution and integrating into every framework. But it has drawbacks in that it is just an automation tool, NOT a testing tool, for testing we have to add other dependencies such as test frameworks, assertion libraries, reporting tools, browser drivers etc. and this can be unwieldly for those new to coding and automation.

A new(ish) tool – Cypress is currently gaining in popularity. Like Selenium it is a web testing tool, but also incorporates API testing and also a mock/stub server. Unlike Selenium, it works directly within the browser, whereas Selenium uses browser servers to connect to the browser. It is multi-platform, multi-browser, but only one language – JavaScript. It runs on NodeJS. The main advantage it has for the newcomer to test automation is that it is a complete package. It comes with a BDD style framework (Mocha), and Assertion Library (Chai) a Test Runner, and reporter. It even captures videos & screenshots by default. So install is very simple with no searching for dependencies.

Architecture

Selenium uses browser drivers to connect to the web browser:

selenium architecture

These browser drivers are separate to Selenium WebDriver and need to be updated as new versions of web browsers are released.

Cypress uses a different architecture. The Cypress engine directly operates inside the browser. In other words, it is the browser that is executing your test code. It also means it has native access to your DOM Document and all the web elements on your page, giving you absolute control.

cypress architecture

This also means that Cypress executes a lot quicker than Selenium does.

Flexibility

Cypress is a complete testing tool (as of version 6.3.0 it also comes with a recording tool). It provides a testing framework (Mocha), an Assertion Library (Chai) as well as other tools (JQuery, Test Runner, Recorder, Locator Playground), so all you need to get going is an IDE (Such as Microsoft Visual Studio Code) and NodeJS installed. As it is built on NodeJS, it only offers JavaScript as a programming language, but is very quick to deploy and get started.

Selenium, on the other hand, offers many language bindings (Java, JS, Ruby, Python, C#), but after that it is up to you which framework you wish to use (NUnit, XUnit, JUnit, BDD frameworks such as Cucumber, SpecFlow, Mocha, TDD frameworks etc.), which Assertion libraries (NUnit, Chai, Hamcrest etc.), which browser drivers you need, and an IDE (IntelliJ, Eclipse, VS Code, Visual Dev Studio etc.). Also Selenium was designed as an API that could be expanded and incorporated into other testing tools, so we have other options like ProtractorJS, mobile testing with Appium, as well as others. So Selenium is a well-supported and flexible API.

Both tools integrate very easily into any of the main Continuous Integration tools (TeamCity, Bamboo, Azure DevOps, Jenkins etc.).

Reporting

A key to good test automation is providing good reporting. Both tools have support for standard testing report formats that can be interpreted by other tools (such as your CI tools), so things like JUnit reports are easy to implement in both tools. If you run Cypress Tests through the Cypress Test Runner, it provides fantastic reporting by default, you can move through screenshots of every test step, read the console output, and it even videos the test run for you automatically (all built-in). Much of this is possible in Selenium WebDriver, but requires extra dependencies & coding.

Cypress.io also provide a free (limited, but they also provide a paid for version) of something they call Dashboard. This is hosted by Cypress.io in the cloud, and when we run a Test Suite we can record the results to the dashboard. The dashboard is then available to the whole team, and provides summary as well as full results from the test executions, these can also be tied to Git Commits. Very nice!

cypress dashboard

Skill Support

Part of choosing a web testing tool is what support you may need access to when developing and maintaining your Test Suite. Part of this maybe determined by the programming language, and the browsers supported. You may also need to consider other types of testing (RESTful APIs, Mobile, Performance).

As Selenium has been around a long time now, there are many developers & testers on the market with those skills (but they may be language dependent). Cypress is quite new, but as it is JavaScript (one of the top 5 most popular languages in the World), anyone with JavaScript skills should be able to pick it up very quickly.

So it should be easy to get good support for either tool.

Integration

Both tools can be used via the command line, so both integrate into CI tools very easily. In Selenium, the way this is approached is language/framework dependent, so for Java you may have your Selenium Tests within a Maven or a Gradle project, which can then fetch dependencies to run on agent machines. In Cypress, we use npm (as it is nodeJS) to manage our dependencies.

So both tools can manage integrations easily.

One thing to note is that the Cypress team have developed a very nice GitHub Workflow Action to execute tests in the cloud – worth having a look at: https://www.edgewordstraining.co.uk/2021/02/02/cypress-github-action-to-execute-tests/

Comparison

Here is a simple feature comparison table

  Cypress Selenium
Languages JavaScript JavaScript, Java, Ruby, C#, Python
Platforms Windows, Mac, Linux Windows, Mac, Linux
Browser Support Chrome, Firefox, Chromium, Electron, Edge, Canary Chrome, Chromium, Firefox, IE11, Safari, Opera, Edge
Recording Yes, from 6.3.0 via Test Runner Yes, via Selenium IDE
Parallel Test Execution Yes, but only through multiple machines (Can only run one instance on one machine), and using a CI tool. Yes, multiple browsers & test on a single machine with Selenium Grid
Command Line execution Yes Yes
Assertion Library Chai (Built-in) Third party dependency, many options
Test Framework Mocha (Built-in) Third party dependency, many options
Setup Very simple npm install cypress –save-dev more complex, as need to include dependencies
Ease of use Very simple, you can be up and running with Cypress in minutes and writing tests is easy More complex
Feature rich? Has built-in API testing & also built-in mock server, but no mobile. It also comes with the Test Runner which makes developing tests very quick and easy Very fully featured, including Appium (mobile), and gestures etc.

API testing & mock servers for stubbing would require other dependencies

Website https://docs.cypress.io/guides/overview/why-cypress.html#In-a-nutshell https://www.selenium.dev/
Edgewords is a training company, providing training courses in all the most popular testing tools: Selenium, Cypress, Ranorex, UFT, Octane, Cucumber, SpecFlow, Appium, API Testing.

For more information, please visit: https://www.edgewordstraining.co.uk/

Or call +44 (0)333 939 8884,   info@edgewordstraining.co.uk