What is Automation Testing?
Overview
Automation testing is a process of writing scripts to execute test cases. It's an entire automation framework where test data, logs, reports, and screenshots are all generated via script. Writing automation test cases is best suited when test cases are redundant, tedious, and time-consuming. Writing automation test cases is an investment that gives more returns in a few weeks than manual test cases.
What is Automation Testing?
Automation testing uses automation tools to write test cases, execute them, and generate test reports which in manual testing is done by a tester manually. In manual testing, the tester writes test cases in an excel file, passes test data, notes the expected vs. actual result, and hence passes or fails the test case.
In automation testing, a tester creates a testing framework that is the structure for the code. Uses testing tools like Selenium, TestNG, Log4j, Allure reports, Jenkins, etc., to run the test case and pass the test data through either database or excel. The test case executes, automatically generates the report, and attaches a screenshot.
The most crucial element is to create the suited framework based on the testing case scenario, which will talk in detail below.
Why Test Automation?
There are several strong reasons why organizations choose to implement test automation. Here are some key points on why test automation is beneficial:
- Test automation allows for faster and repeated execution of tests, providing quicker feedback on software quality.
- Automated tests cover a wide range of scenarios, including negative and edge cases, ensuring comprehensive testing coverage.
- Test automation saves time and reduces reliance on manual resources, resulting in long-term cost savings.
- Automated tests eliminate human errors, providing consistent and reliable results.
- Test automation is scalable and allows for the reuse of test scripts or components, making it efficient for large and complex systems.
- Early bug detection is possible through automated tests, reducing the cost and effort required to fix issues later in the development cycle.
- Test automation promotes collaboration by enabling the sharing of test scripts, data, and results among team members.
- Automated tests generate detailed reports and logs, providing valuable insights for analysis and continuous improvement.
::
What kinds of Tests Should be Automated?
Now we know what is automation testing, let's discuss the test cases that should be automated for better efficiency of the testing process.
1. Most commonly used:
Some test cases are always found in many applications. Hence, it makes sense to take time to create such test cases that can be used for different applications or under similar case scenarios. For instance, to check if a link is working or not. A tester can create a script that could be used in various applications.
2. Performance driven: Load testing, volume testing, and stress testing cannot be achieved manually. We can't make lakhs of people sit and send requests to the server. Here writing automation scripts become mandatory.
3. Time-consuming and repetitive:
Any` data-driven test case must be automated because manually validating an excel sheet or a database is not feasible. Automation scripts and libraries are used to write test cases.
4. Complex events:
All the test cases where we cannot physically replicate the test case environment should also be automated. How the code would work in different hardware configurations, devices, etc., must be automated.
What Not to Automate?
1. User Experience Test cases:
Test cases that involve user experience, look, and feel of the application can not be automated. Only a manual tester can do the usability testing of the application.
2. Test case with low ROI:
Some rare test cases that are limited to one application and are complex to write should not be automated because these scripts would not be reusable.
3. Test cases involving the integration of many APIs:
Automating test cases involving the integration of many applications should be manually tested as manual testing here would be faster. Also, more test cases could be covered during integration testing.
4. Customer support Test cases:
Customer support phone calls are meant to provide the human touch. It's not always about providing the correct solution but ensuring customer trust remains intact. And they feel they are emotionally understood, which cannot be automated`.
Is 100% Automation Testing Possible?
We do automation testing for better ROI(Return On Investment). Manual testing is a creative process where we find multiple ways to test and validate a feature. In automation testing that creativity is not possible. Even artificial intelligence requires learning models and massive data sets to train.
We cannot 100% automate the entireSTLCprocess as what's the priority, which one to test, and writing test scenarios are all manually driven. Even if we try to automate everything, the budget, time, and increasing Requirement Traceability Matrix could not be achieved.
Automated Testing Process
1. Understanding the requirement:
Requirement analysis and generating a requirement traceability matrix (RTM) is the most crucial step. This step would help determine the scope of automation and find the correct tool.
2. Scope of Automation:
Not all test cases need to be automated. This step defines what needs to be automated in the defined period. How much UI automation, `API automation, or performance testing is needed?
3. Choosing the appropriate automation tool:
Once you know what to automate now need to decide the tools to choose. Tool selection is based on budget constraints and ensuring the employees must be trained on the same: for example, Katalon, RPA, orseleniumfor `UI automation.
4. Designing the test framework:
Designing the correct framework is very important. That would help to make the code maintainable. There are many frameworks linear framework, modular framework, and data-driven based on the case scenario. The framework also generates reports and attach logs and screenshot to test cases.
5. Test Execution:
Writing multiple test cases in the framework to maximize the test coverage, getting test data from a data source, executing the script, and generating reports. The failed test cases are then assigned to the developer with the bug id.
6. Automation Maintenance:
Maintenance signifies that when new functionalities are added to the application, the framework and scripts can be reused to add more test cases. No drastic change should happen.
Test Automation Best Practices
1. Automate reusable test cases:
Test cases that can be used in different case scenarios, require enormous test data, or can be part of multiple projects and are time-consuming must be automated.
2. Maintainable automation framework:
A framework that is maintainable and appropriate based on the scenarios to automate. For example, suppose a lot of test data is involved in the data-driven framework used. In that case, there is a lot of keyword action-driven tasks, and then a Keyword drove framework should be used.
3. Choosing a reliable automation tool:
Many new automation tools are coming up with good codeless automation, which has a lot of issues. So, the one sustained in the market for an extended time should be used like Selenium, SOAPUI, Jmeter, etc.
4. Writing cost-efficient test cases:
Writing complex and time-consuming test cases which return low ROI should be avoided.
Test Automation Framework
The framework is a code structure that needs to follow for any code that gets added. Hence, every test case must confine to some norms. Features of the test automation framework are:
- Connection to the data source
- Using design approaches like page object models
- Using OOPs concepts like Inheritance, Abstract Class, Interface, etc.
- Using proper naming conventions like Class starts with uppercase, the package starts with lowercase, etc.
- Modular approach so that all components are separate
- Non-technical testers can be involved using a Behaviour driven approach
- A separate file for username, URL, and password called data.properties
Below are a few different types of automation frameworks we have:
- Linear Scripting Framework
- Modular Testing Framework
- Data-driven testing framework
- Keyword-driven testing framework
- Behavior-driven testing framework
- Hybrid testing framework
Popular Test Automation Testing Tools
1. Selenium:
Selenium is the most trending tool for browser automation. Selenium has three tools- grid for cross-browser testing, web driver, and ide for code-less automation
2. Appium:
Appium is an automation tool built on top of selenium for mobile automation. It automates the emulator to test user experience on a mobile application.
3. SoapUI:
SoapUI is used for web services and API testing. It is more for backend testing. If APIs work correctly, response status, data, headers, and all are tested using SoapUI. It is hands down one of the most trending tools for backend automation.
4. Postman:
Postman is another tool for API testing which is equally popular with developers as with testers for API and web services testing. It is GUI based automation tool for code-less automation.
5. Jmeter:
Apache Jmeter is a widely used tool for performance testing. It is used for load and stress testing on servers. It is built on the multithreading concept of JAVA to send concurrent requests at a given time to a server.
6. Katalon Studio:
Katalon studio, in the last few years, has gained recognition for UI testing for codeless automation. It is also built on the Selenium tool only.
7. LambdaTest:
LambdaTest is a cloud testing platform that provides a test environment for test cases. They provide virtual machines for different systems and application configurations like running Selenium grid on Mac system chrome driver 104 etc.
8. Cypress Testing:
Cypress is automation built on javascript for an end to end automation. The beauty of it is that both websites and testing use Javascript, so developers can also use this tool.
How to Choose a Test Automation Tool
Many factors help to decide which automation tool to use some of them are below:
- Test scenarios
- Subscription price of the tool
- Language the tool support
- Test the environment the project is using
- Report generation feature of the tool
- How long has the tool been in the market
- Cost of training the employees on the same
Benefits of Automation Testing
1. Extensive report generation:
Most automation teams in association with the reports tool give extensive detailed reports `within seconds. Allure reports and extended reports are examples of the same.
2. Saves time:
Automation testing executes hundreds of test cases within minutes. When attached to CI/CD, tools like Jenkins can be scheduled to generate the report at any time of the day
3. Saves cost in the longer run:
Automation testing gives higher ROI. Frameworks are maintainable and can be reused in multiple types.
4. Reduce human dependency:
There is a massive dependency on the tester and his experience in manual testing. Product and company face enormous challenges when a manual tester leaves as there is an investment in him to learn the product details.
5. Detailed testing:
Accurate, time-based, event-based data is not possible by manual testing, whereas in automation, testing can be quickly done
6. Code reusability:
Code could be reused and would remain. In the future products or features same with slight tweaks would produce accurate results.
Types of Automated Testing
There are multiple types of automation testing. Below are the most in-demand skill set
1. UI testing:
UI testing is half of the automation testing industry. Selenium, Appium, Katalon Studio, and Cypress are all commonly used tools for the same.
2. API testing:
API testing is equally in demand with UI testing. SoapUI, Postman, and Rest Assured are the most used tools
3. Performance testing:
Apache Jmeter is used for load, and stress testing
4. Data testing:
Data-driven framework and testing involves cleaning and validating data in excel or a database.
5. Security testing:
Security testing differs from the rest as it focuses on cyber security, penetration testing, etc., concepts that are entirely different domains.
Difference between Manual and Automation Testing
Conclusion
- Here we discussed what is automation testing. It involves writing scripts to execute test cases.
- Most commonly used, time-consuming, and redundant test cases must be automated.
- Usability test cases and time-consuming scripts involving multiple APIs integrated` must not be automated as they would have low ROI.
- 100% automation is not possible as it won't be feasible and would not fit in time and budget constraints as well.
- Automation testing process includes Understanding the requirement, Scope of Automation, Choosing the appropriate automation tool, Designing the test framework, Test Execution, Automation Maintenance
- There are many types of automation frameworks based on test scenarios automated Linear Scripting Frameworks, Modular Testing Frameworks, Data driven testing frameworks, Keyword driven testing frameworks, Behavior-driven testing frameworks, Hybrid testing framework
- Manual testing is a creative process where random testing is possible, which cannot be done in `automation testing