According to an IBM study conducted, searching, managing, maintaining, and generating test data consumes 30-60% of the tester’s time. It is undeniable that data preparation is the most time-consuming phase of software testing. Nonetheless, it is a fact across many disciplines that most data scientists spend 50% to 80% of their model development time organizing data. And now, taking into account the legislation as well as the Personally Identifiable Information (PII), the testers’ engagement in the testing process is overwhelmingly positive.
Today, the credibility and dependability of test data are regarded as critical components for business owners. The product owners see ghost copies of test data as the huge challenge, reducing the reliability of any app at this unique time of client demand/requirements for quality assurance. Given the importance of test data, the vast majority of software owners will not accept tested apps that contain fake data or have inadequate security measures.
Why don’t we go over what Test Data is right now? When we begin writing test cases to verify and validate the given features and developed scenarios of the application under test, we require information that will be used as input to perform the tests for identifying and locating defects. And we know that this information must be precise and complete in order for the bugs in production to be removed. It’s referred to as test data. To be precise, names, countries, and so on are not sensitive, whereas personal data pertaining to contact information, medical history, SSN, and credit card information are.
The information could be in any format, such as:
- System tests data
- SQL test data
- Performance test data
- XML test data
In this article:
- Strategies for Test Data Preparation
- How to Prepare Data that will assure Maximum Test Coverage
- Data for Black Box Testing
- Establishment of manual data for testing Open EMR application
- Properties of a Good Test Data
- Test Data and its Significance
- What are the types of test data?
- What are the different ways of preparing test data?
- Test Data Sourcing Challenges
- FAQs
Strategies for Test Data Preparation
We know from daily practice that the players in the testing industry are constantly experimenting with new ways and means to improve testing efforts and, most importantly, its cost efficiency. We have seen in the short history of information and technology evolution that when tools are integrated into production database or testing environments, the level of output significantly increases.
The completeness and full coverage of testing are primarily determined by the data quality. Because testing is the foundation for achieving quality software, test data is a critical component in the testing process. Flat file generation based on mapping rules. It is practical to create a subset of the data you require from the production environment where the application was designed and coded. Indeed, this approach reduces the testers’ data preparation efforts and maximizes the use of existing resources to avoid additional expenditures.
Typically, we must create or at least identify the good data based on the type of requirements that each project has from the start.
You can apply the following strategies handling the process of TDM:
- Data from the manufacturing environment
- SQL queries that extract data from the client’s existing databases are being retrieved.
- Tools for Automated Data Generation
The testers must back up their testing activities with complete data by taking into account the elements depicted in Figure 3 here. In agile development teams, the testers generate the data required to execute their test cases. When we speak of test cases, we are referring to cases for various types of testing such as white box, security, black box, and performance. At this moment, we know that data for performance testing should be able to determine how quickly a system responds under a given workload in order to be very close to a real or live large volume of data with significant coverage.
Developers prepare their required data for white box testing to cover all paths in the program source code, as many branches as possible, and the negative Application Program Interface (API). Finally, we can say that everyone involved in the software development life cycle (SDLC), such as BAs, Developers, and product owners, should be actively involved in the Test Data preparation process. It could be a collaborative effort. Let us now focus our attention to the problem of corrupted test data.
Corrupted Test Data
Before running any test cases on existing data, check sure the data isn’t faulty or obsolete, and that the application under test can access the data source. When more than one tester is working on separate modules of an AUT in the same testing environment, the risks of data corruption are very high.
The testers adjust the existing data in the same environment to meet their needs/requirements for the test cases. When the testers are through with the data, they usually leave it alone. There’s a chance that the next tester will pick up the updated data and run the test again, which will result in a test failure that isn’t due to a coding fault or flaw.
What is the ideal test data?
Data is said to be ideal if, for the smallest amount of data set, all application errors are identified. Try to prepare data that incorporates all application functionality while not exceeding the cost and time constraints for data preparation and testing.
How To Prepare Data That Will Assure Maximum Test Coverage
Create test data with the following categories in mind:
1) No data: Run your test cases with the default or blank data. Check to see if appropriate error messages are generated.
2) Valid data set: Create it to ensure that the application is functioning properly and that valid input data is saved in the database or files.
3) Invalid data set: Create an invalid data set to test the application’s behavior with negative values and alphanumeric string inputs.
4) Illegal data format: Create a single data set in an illegal data format. Data in an invalid or illegal format should not be accepted by the system.
5) Boundary Condition dataset: Dataset containing data that is out of range. Determine application boundary cases and prepare a data set that includes both lower and upper boundary conditions.
6) The dataset for performance, stress testing and load: This data set should be quite large in size.
Creating separate datasets for each test condition in this manner ensures complete test coverage.
Test data preparation techniques
There are two ways to prepare test data:
Method 1: Insert New Data
Create a new database and insert all of the data specified in your test cases. Once you’ve entered all of your required and desired data, begin executing your test cases and filling the ‘Pass/Fail’ columns by comparing the ‘Actual Output’ with the ‘Expected Output.’ Sounds easy, doesn’t it? But, wait a minute, it’s not that simple.
Few essential & critical concerns are as follows:
- It is possible that an empty instance of the database is not available.
- In some cases, such as performance and load testing, the test data inserted may be insufficient.
- Due to database table dependencies, inserting the required test data into a blank DB is a difficult task. Data insertion can become difficult for the tester as a result of this unavoidable constraint.
- Inserting limited test data (only what is required by the test case) may obscure some issues that would be visible only with a large data set.
- Complex queries or procedures may be required for data insertion, and sufficient assistance or help from the DB developer(s) would be required.
Method 2: Determine sample data subset from actual DB data
This is a more feasible and practical method for preparing test data. It does, however, necessitate strong technical skills as well as detailed knowledge of DB Schema and SQL. You must copy and use production data in this method, replacing some field values with dummy values. This is the best data subset for testing because it is representative of the production data. However, due to data security and privacy concerns, this may not always be possible.
Data for Black Box Testing
System testing, Integration testing, and acceptance testing, also known as black box testing, are all performed by Quality Assurance Testers. The testers in this methodology of testing do not work on the internal structure, design, or code of the application under test. The primary goal of testers is to identify and locate errors. In doing so, we employ either functional or non-functional testing techniques, as well as various black box testing techniques.
Establishment of manual data for testing Open EMR application
Let’s move on to creating manual data for testing the Open EMR application against the given data set categories.
1) No Data: The tester validates the Open EMR application URL as well as the “Search or Add Patient” functions without providing any data.
2) Valid Data: With Valid data, the tester validates the Open EMR application URL and the “Search or Add Patient” function.
3) Invalid Data: With invalid data, the tester validates the Open EMR application URL and the “Search or Add Patient” function.
4) Illegal Data Format: With invalid data, the tester validates the Open EMR application URL and the “Search or Add Patient” function.
5) Boundary Condition Data Set: Its testing purpose is to find input values for boundaries that are inside or outside of the given data values.
6) Equivalence Partition Data Set: It is a testing technique that divides your input data into valid and invalid input values.
7) Decision Table Data Set: It is a method of qualifying your data by using a combination of inputs to produce a variety of results. This black box testing method assists you in reducing your testing efforts in verifying each and every combination of test data. Furthermore, this technique can ensure that you have complete test coverage.
8) State Transition Test Data Set: It’s a testing technique that allows you to give validation to the state transition of the Application Under Test (AUT) by providing input conditions to the system.
9) Use Case Test Date: It’s the testing method that identifies our test cases, which capture the end-to-end testing of a specific feature.
Properties of a Good Test Data
Test Data Properties
The test data should be carefully chosen and have the four characteristics listed below:
Realistic: If testing is done with realistic test data, the app will be more robust because most of the possible bugs can be captured with realistic data. Another benefit of realistic data is its reusability, which saves us time and effort in creating new data over and over.
When we’re talking about realistic data, I’d like to introduce you to the golden data set concept. A golden data set is one that covers almost all of the possible scenarios that may occur in a real-world project. We can provide maximum test coverage by utilizing the GDS. In my organization, I use the GDS for regression testing, which allows me to test all possible scenarios that could occur if the code is put into production.
Practically valid: This is similar to but not the same as realistic. This property is more related to AUT’s business logic, for example, a value of 60 is realistic in age field but practically invalid for a candidate of a Graduate or Masters Program. In this case, a reasonable age range would be 18 to 25 years (this might be defined in requirements).
Versatile to cover scenarios: There may be many subsequent conditions in a single scenario, so choose the data wisely to cover the maximum aspects of a single scenario with the smallest set of test data, for example, when creating test data for the result module, do not only consider the case of regular students who are successfully completing their program. Pay special attention to students who are repeating the same course but are from different semesters or even different programs.
Exceptional data (if applicable/required): There may be some exceptional scenarios that occur less frequently but require special attention when they do, such as issues involving disabled students.
Test Data And Its Significance
Test data is data that will be used to test a specific piece of software program. While some data is used to obtain confirmatory results, other data may be used to test the software’s capability. There are numerous methods of testing for obtaining appropriate test data for testing a system. A tester or a program can generate test data for a specific system.
For example, the testing team may want to see if the software produces the desired result. It would enter the data into the system and run it. It would then analyze the results to determine whether or not the expected results were obtained. The software should at the very least produce the desired results without any glitches. After all, this was the primary reason for its creation, and it must fulfill it.
In contrast, if it is given non-standard input, it should not produce unexpected, unusual, or extreme values. There must be enough test data to test both the positive and negative scenarios. This is to ensure that the software continues to run smoothly even if the end user enters incorrect information while using it or chooses to do so on purpose in order to play with the system.
Experts disagree on whether real production data or synthetic data should be used for testing. There are specific scenarios in which each of them is appropriate. Synthetic data, for example, performs better in narrowly focused tests. However, if a close simulation of the real system is desired during testing, production test data is preferable. Many times, production data is masked before being used for testing.
When performing negative testing, QA might practice submitting incorrect data.
What Are The Types Of Test Data?
| Boundary Test Data: | This type of data aids in the removal of defects that are associated with the processing of boundary values. This data type contains a collection of boundary values that are sufficient to handle the application. And, if the tester goes beyond this, the application may be broken. |
| Valid Test Data: | These data types are valid, and the application supports them. These aid in the verification of system functions, and when an input feed is provided, it aids in the receipt of the expected output. |
| Invalid Test Data: | Unsupported data formats are among these data types. The data is used by the teams to determine whether or not the application is functioning properly. By entering invalid values, the app should display a relevant error message and notify the user that the data is insufficient to function. |
| Absent Data: | The term “no data” or “blank files” refers to files that do not contain any data. The use of blank data aids in testing how the app reacts when no or no data is entered into the software. |
What Are The Different Ways Of Preparing Test Data?
Manual Test Data Creation:
This test data generation method is accurate method to data generation. This method includes test data of the following types: valid, invalid, null, standard production data, and data set for performance. The benefit of using this test data type is that it does not require any additional resources; it is created using the testing team’s skills and ideas. However, it takes longer and produces less productivity. If the tester lacks the necessary domain knowledge, this method may be hampered, resulting in flawed data.
Back-end Data Injection:
Back-end servers with a huge database are used in this method. This data generation method eliminates the need for front-end data entry and allows for faster data injection. Furthermore, this method does not necessitate the assistance of experts in order to create backdated entries. However, there are some drawbacks that can endanger the database testing and application if the technique is not used correctly.
Automated Test Data Generation:
Data generation tools are used in this method to process and achieve better results with large amounts of data. Web Services API and Selenium are popular tools for this automated test data generation method. The benefit of this type of data generation is that the data generated by automation tools will be of high quality and accurate. There is no need for human invention because the output is delivered at a faster rate. However, there are some drawbacks, such as cost factors and a lack of talented resources.
Third-party Tools:
Using third-party tools makes it easier to create and inject data into the system. Because these tools have a thorough understanding of the back-end applications, they will be able to produce data that is very close to real-time. The benefit of these tools is the accuracy of the data and the scope provided to users for performing the required tests on historical data. Furthermore, the disadvantage of this method is that it is too expensive and has a work limitation.
Test Data Sourcing Challenges
Data sourcing requirement for sub-set is one of the areas in test data generation that testers consider. For example, suppose you have over a million customers and need a thousand of them for testing. And the sample data should be consistent and statistically representative of the targeted group’s distribution. In other words, we’re supposed to find the right person to test, which is one of the most effective ways to test use cases.
There are also some environmental constraints in the process. One of them is PII policy mapping. Because privacy is a significant barrier, the testers must classify PII data. The Test Data Management Tools are intended to address the aforementioned problem. These tools make policy recommendations based on the standards/catalog they have. However, it is not a very safe exercise. It still provides the option of auditing one’s actions.
To keep up with present and future challenges, we should always ask questions like, “When/where should we begin the conduct of TDM?” What should be done automatically? How much should businesses invest in testing in areas such as human resource ongoing skill development and the use of newer TDM tools? Should we begin with functional and non-functional testing? And they are far more likely to be questions.
Below are some of the most-common challenges:
- The teams may lack complete knowledge and skills regarding test data generator tools.
- Test data coverage is frequently insufficient.
- During the data collection phase, there was less clarity in the data requirements covering volume specifications.
- The data sources are not accessible to the testing teams.
- Developers’ failure to provide testers with production data access.
- Based on the developed business scenarios, production environment data may not be fully usable for testing.
- Large amounts of data may be required in a short period of time.
- Some of the business scenarios will be tested using data dependencies/combinations.
- The testers spend more time than necessary communicating with architects, database administrators, and business analysts in order to gather data.
- The majority of the data is generated or prepared during the course of the test.
- Versions of applications and data
- Several applications are subject to continuous release cycles.
- Personal Identification Information Protection Legislation (PII)
- Testing may be delayed if data from the development teams is not received. Data is frequently requested from them, and this can sometimes be delayed as if they are approaching for other reasons.
- In most cases, testing teams are not given the necessary permission to access the tools for obtaining data sources.
- There may be scenarios where a larger amount of data is required in a shorter period of time, which can be difficult to achieve if the necessary tools are not available to assist the testing teams.
- There may be a significant risk of affecting the software if data defects are not identified as soon as possible.
- Because the majority of data creation occurs during execution, the time required to collect the data is longer, and it even extends the testing time.
- Test data management process necessitates that the testing team have in-depth knowledge of alternative data creation solutions, which may not be available to all testers.
This is where QAs must collaborate with developers to expand testing coverage of AUT. One of the biggest challenges is incorporating all possible scenarios (100% test case) as well as every single possible negative case. Other information could be utilized to test the program’s capacity to respond to uncommon, severe, exceptional, or unexpected input.
FAQS
What is DevOps?
DevOps is the combination of people, process, and technology to continuously give value to consumers. It is a compound of development (Dev) and operations (Ops).
What does DevOps imply for organizations?
DevOps allows previously compartmentalized disciplines like development, IT operations, quality engineering, and security to interact and coordinate to create better, more reliable products. Teams may better respond to customer requests, boost confidence in the apps they produce, and achieve business goals faster by adopting a DevOps culture and using DevOps principles and tools. Other data may be used in order to challenge the ability of the program to respond to unusual, extreme, exceptional, or unexpected input.
