Designed for Agile
The High Cost of Poor Software Quality
Organizations are investing heavily in digital transformation, Big Data analytics, artificial intelligence and machine learning. They are developing these advanced technologies as they engineer digital information systems to become more competitive, customer centric and operationally efficient.
However, many of these systems will experience costly and preventable problems with software quality. In a recent study, The Cost of Poor Software Quality in the US: A 2020 Report, CISQ estimates a staggering $2.08 trillion was lost last year alone because of software defects, fragile legacy systems, cybersecurity failures and failed IT projects.
This has led to increased focus on quality through automated testing, as well as the test data used to validate software. GenRocket has developed a self service platform to automate the design and deployment of the test data needed to conduct comprehensive testing that will maximize test coverage, ensure quality and reduce cost.
Test Data is Critical to Software Quality
Why is test data critical to software quality? Because the data used for testing determines the execution path of the code. Without the right test data variations, large blocks of code can go untested, reducing coverage and allowing defects to go undetected.
The volume of test data is also critical to software quality. In a production environment, transaction processing systems are subject to varying load conditions. Applications must be stressed with high volume test data to validate performance under peak and sustained load conditions.
Test data should be carefully designed to meet test case objectives. This implies control over the patterns, permutations and boundary values needed to fully test the code. After its requirements are determined, the time to provision test data must be accelerated to keep pace with a continuous integration and delivery process. The importance of test data to ensuring quality can be summarized as three key provisioning goals:
The only way to achieve all of these critical test data requirements is with the use of a self service platform that provides Synthetic Test Data Automation.
The Challenge of Traditional Test Data Provisioning
According to the World Quality Report, using spreadsheets to manually generate test data is the most popular method of provisioning test data, used by 69% of the testers surveyed. The second most common test data provisioning method is copying production data and anonymizing it before using it for testing. This method is used by 65% of testers.
Based on these methods, it’s no surprise that over 50% of QA professionals report challenges with test data, and they are spending as much as 60% of their time provisioning it. The lack of automation during the test data provision process is the main reason why most organizations have automated less than 10% of their test cases. Test data provisioning is a major challenge preventing organizations from fully testing their code and contributes to the high cost of poor software quality.
Current Test Data Challenges
Using spreadsheet data for testing is popular because it allows testers to have the exact data they need for any given test case. The challenge of spreadsheet data is the labor intensive nature of creating each test dataset. Time and resources limit the volume and variety of data that can be effectively provisioned to keep pace with automated testing.
Additionally, spreadsheet data is 2-dimandsional by nature, making it impossible to accurately reproduce the complex data structures associated with modern enterprise databases.
As a result, many testers find it easier to simply request a test data subset to be copied from a production database. Before it can be used for testing it must be anonymized to remove sensitive customer or patient data. The challenge of production data is it can take days to provision thereby introducing significant delays in the testing process.
After test data is delivered to the QA team, it must then be queried for the needed data values. Most of the time, production data is missing many of the required data values to ensure full coverage and must be augmented with manually created data.
Traditional test data provisioning methods impact both test coverage and speed. The lack of data variations – combinations, permutations, edge case data, negative data, dynamic data – reduces coverage and compromises the accuracy of the testing process.
The limitations on test data volume can impact the robustness of load and performance testing. And the time lag to provision test data can delay the testing process and/or reduce the number of tests that can be run during a sprint.
Test Data for Automating the Full CI/CD Pipeline
The limitations imposed by traditional test data provisioning has led to the growing use of Synthetic Test Data Generation. That’s because synthetic test data can be designed for any use case and generated on-demand for any test case.
ANY TEST DATA
- Positive
- Negative
- Unique
- Permutations
- Workflows
- Transactions
- Data Feeds
- X12 EDI
- Messages
- IoT Sensor
- Dynamic
- Salesforce
ANY TYPE OF TEST
- Unit
- API
- Integration
- Functional
- System
- Security
- Smoke
- Load
- Performance
- Compatibility
- Acceptance
- Regression
With GenRocket’s self service platform for Synthetic Test Data Automation platform, the design and deployment of test data is placed directly in the hands of the tester. GenRocket allows QA professionals to fully automate the test data provisioning process and seamlessly integrate it with the test automation process.
Test Data for Automating the Full CI/CD Pipeline
With GenRocket’s Self Service Test Data Automation platform, developers and testers have access to real-time test data on-demand. This allows more testing to be performed and conducted earlier in the SDLC.
Developers can easily generate the precises data needed for unit testing whenever they commit a new code build to the CI/CD pipeline. This allows the early detection of software defects prior to its integration with the main code branch.
Once code enters the staging environment, the QA team can use GenRocket to generate test data for a full range of testing during the continuous integration process. This allows for more automated testing at an accelerated pace prior to releasing code to production.
Designed for Agile and DevOps
GenRocket enables a faster and easier transformation to a scalable Agile framework. As Agile teams design and develop new product features, they create Epics and Stories to define their customer value, functional requirements and acceptance criteria.
As each user story is translated to an executable test script, testers can follow an easy and consistent process for defining the test data that will maximize coverage. The GenRocket Methodology is a 4-step process for modeling the data structure, designing the required data values, deploying the data into an automated test environment, and managing test data projects for reusability.
With GenRocket, Test Data Projects are organized as Test Data Epics, Test Data Stories and Test Data Cases that correspond with an Agile-defined test plan.