Blog

Test Data Map Design Pattern by Hycel Taylor on May 05, 2014

Prologue

If you have access to or the ability to automate and generate large amounts of simple to complex test data, it will inevitably force you to rethink how you go about testing your software.

We’ve spent a lot of time learning, crafting, and perfecting the best practices for testing with good test data.  In that time, we’ve come up with great design patterns that have enabled us to test more thoroughly, promote projects to production much quicker and with way fewer bugs.  One such design pattern is the Test Data Map (TDM) Design Pattern.

Introduction

The Test Data Map (TDM) design pattern provides a way to map synthetically generated test data relationships to true primary identifiers generated by a target platform.

Intent

The Test Data Map design pattern helps solve the challenge of building complex test data for data models that may be heavily related, requiring complex referential integrity without having to define complex test data scenarios to build all of the related data at once. Instead, one or more test data scenarios may be used to generate less demanding relationships and allow the referencing to happen when the test data is being loaded into entities where referential integrity is required.

Solution

  • Define and generate test data with attributes containing synthetic referential integrity values used to reference other generated test data.
  • Populate entities using the generated test data.
  • Using a namespace, store a true id generated from the target platform with its synthetically generated equivalent.
  • Retrieve a true id with a given namespace and synthetic id to attain a platform generated entity.

 

Test Data Map

The TDM design pattern must use a database table or an in memory structure to map synthetically generated values to the true values generated by the target platform. Each synthetic_id/true_id pair must be associated to a namespace in order to be uniquely identified. This table or in memory structure is key to the TDM design pattern.

TestDataMapTable

  • name_space – allows the synthetic_id/true_id pair to be uniquely identified.
  • synthetic_id – stores the synthetically generated id.
  • true_id – stores the value generated by the target platform.

 

Application-specific example using the TDM design pattern

Let’s discuss the design pattern with an example that stores test data into specific tables in a relational database.  We’ll use two data models, one containing three related tables and the other containing four related tables to illustrate the TDM design pattern.

Data Model One

Data Model One contains three tables: organization, department and user.  The relationships infer that a user must belong to one and only one department and a department must belong to one and only one organization.

TDMExample1

Data Model Two

Data model two contains four tables: organization, department, user and department_user.  The relationships infer that a user may belong to one or more departments and a department must belong to one and only one organization.

TDMExample2

Generate Test Data

The goal is to populate either of the two data models via business methods using the TDM design pattern; the first step in this process is to generate test data.

The test data to be generated should not necessarily be created to directly model entities in a database, but to model method parameters that contain business logic to populate such entities.  From the two data models defined above, only three test data solutions are needed as long as there are services that contain business logic to populate entities defined in either data model.

Organization Test Data

The example XML solution contains synthetic test data that can be used to populate the organization table.

Department Test Data

The example XML solution contains synthetic test data that can be used to populate the department table.  The organization attribute defines values that will reference values generated in the organization XML solution for the id attribute.

User Test Data

The example XML solution contains synthetic test data that can be used to populate the department table.  The department attribute defines values that will reference values generated in the department XML solution for the id attribute.

Populating The Organization Table

The service method below (written in Groovy code) uses the TestDataLoader (TDL) design pattern to load the data from the XML solution into memory.

For each organization:

  • The synthetic id is stored in a local instance variable.
  • The organization is saved and its true id is created by the target platform.
  • The synthetic id and true id are saved to the test_data_map with the namespace, ‘com.acme.organization’.

 

Table organization

When completed, the data in the organization table will contain three rows of data.

OrganizationTable

test_data_map

The test_data_map will contain the following three rows of data mapping the originally generated synthetic id to the platform generated true id.

TestDataMap1

Populating The Department Table

The service method below (written in Groovy code) uses the TestDataLoader (TDL) design pattern to load the data from the XML solution into memory.

For each department:

  • The synthetic id is stored in a local instance variable.
  • The synthetic organization id is stored in a local instance variable.
  • The organization’s true id is retrieved from the test_data_map.
  • The organization is retrieved via its true id.
  • The department is saved and its true id is created by the target platform.
  • The synthetic id and true id are saved to the test_data_map with the namespace, com.acme.department.

 

Table Department

When completed, the data in the department table will contain the six rows of data.

DepartmentTable

test_data_map

The test_data_map table will contain six new rows of data mapping the originally generated synthetic id to the platform generated true id.

TestDataMap2

Populating The User Table

The service method below (written in Groovy code) uses the TestDataLoader (TDL) design pattern to load the data from the XML solution into memory.  The source code below populates user data in the second model where users may belong to more than one department; thus the user and department_user tables will be populated.

For each user:

  • The synthetic id is stored in a local instance variable.
  • The synthetic department id is stored in a local instance variable.
  • The departments’ true id is retrieved from the test_data_map.
  • The department is retrieved via its true id.
  • The user is saved and its true id is defined.
  • The department_user is saved.

 

Discussion

In the application-specific example:

  • We used two data models to illustrate the use of the TDM design pattern using three XML solutions containing synthetically generated test data that have attributes relating to each other.
  • We used the organization’s test data to populate the organization table and mapped the organization’s synthetic id to its true id generated by the database.
  • We used the department’s test data to populate the department table, used a namespace & organization synthetic id to retrieve its true id from the test_data_map to get an actual organization and mapped the departments synthetic id to its true id generated by the database.
  • Lastly we used the user’s test data to populate the user table, used a namespace & department synthetic id to retrieve its true id from the test_data_map to get an actual department and populated the department_user table.

 

The application-specific example for populating three tables that are simply related to each other via grandparent, parent and grandchild could be implemented in many other ways without using the TDM design pattern.  However, the application-specific example does a fair job of illustrating the usefulness of the TDM design pattern.

Summary

The TDM design pattern may be used with great effect when generating test data for highly complex data models containing a large number of heavily related entities. The solution to building complex test data scenarios is defining many smaller test data scenarios, each concerned with generating data for only a few related entities. Executing a set of scenarios in a specific sequence using the TDM design pattern can greatly simplify populating even the most complex data models with useful test data.

Related Design Patterns

The Test Data Loader (TDL) design pattern provides a model for loading test data using a common set of components that interact with each other via a common interface.