Transformation – Introduction

The user can apply transformation to one or multiple columns in a dataset.  

Step 1: Configure the data source to which you want to apply your transformations. [Refer to this Link: Creation of Data Source. Create a new Workflow. Once data is imported, click on transformation.  

Step 2: Drag and drop the transformation node onto the main screen. Connect the two nodes. (Refer to the image below).  

Step 3: Once a successful connection is made between ‘data import and transformation’, click on the select button. A pop-up appears with the columns present in the data.   

Step 4: Columns can be filtered on various options available. Individual columns can also be selected for filtering. (Refer to the image below).

Step 5: You can add multiple custom fields and the same is reflected in the table. You can also delete the custom field(s) that has been added.  

Step 6: Click on ‘Apply’.

Step 7: Click on configure to make changes to the data. A data transformation pop-up appears. Choose the columns that need change by clicking on the dropdown. (Refer to the image below). 

Step 8: Once the data configuration has been done, click on the run button to run the workflow.   

Step 9: Click on the “Show Results” on the bottom line to view the result.  

Step 10: In the example below, “Units_Sold” has been renamed as “Unit Sales”. The data is reflected based on the operations selected. (Refer to the images below).

Actions

Replace

This action replaces a particular entity with a new replace value. 

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Replace’ option from the dropdown.  

Step 4: Enter the old replace value and the new replace value. Click on the ‘Apply’ button. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the images below). 

Prepend

The user can add a prefix to the data in each column.  

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Prepend’ option from the dropdown.  

Step 4: Enter the prepend value and click on the ‘Apply’ button. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.

Step 6: Click on the “Show Results” on the bottom line to view the result. 

Step 7: The data is reflected based on the operations selected. (Refer to the images below). 

Pattern

Pattern is similar to replace. It matches a specific pattern within column values and replaces it with the given value. 

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Pattern’ option from the dropdown.  

Step 4: Enter the old pattern value and the new pattern value. Click on the ‘apply’ button. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the image below). 

Deidentification

The user can make use of this function within a dataset to mask the original data with duplicate or false data. There are various functions available to achieve this.  

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Deidentification’ option from the dropdown. 

Step 4: From the dropdown, select the type of generator as per your requirement. Now, select either ‘Random or Consistent’. Click on ‘Apply’. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the image below). 

Deidentification Sub-types:

State Generator

The user is able to deidentify the state from address. (Refer to the image below). 

First Name Generator

The user is able to deidentify the first name. (Refer to the image below).

Last Name Generator

The user is able to deidentify the last name. (Refer to the image below).

Zip Generator

The user is able to deidentify Zip from address. (Refer to the image below). 

Date Generator Past

The user is able to deidentify the date column with value earlier than the source value. (Refer to the image below).

DOB Generator

The user is able to deidentify the date of birth. (Refer to the image below).

Alpha Numeric Generator

The user is able to deidentify the text column with alpha-numeric values. (Refer to the image below). 

Email Generator

The user is able to deidentify E-mail. (Refer to the image below). 

Alpha Generator

The user is able to deidentify the text column with alpha value. (Refer to the image below). 

City Generator

The user is able to deidentify the city from the address. (Refer to the image below). 

Decimal Generator

The user is able to deidentify numeric column with that of decimal value. (Refer to the image below).

Numeric Generator

The user is able to deidentify the numeric column with a numeric value. (Refer to the image below).

Phone Generator

The user can deidentify the phone. (Refer to the image below).

Random Existing Data Generator

The user can deidentify using random values from an existing list of values. (Refer to the image below). 

Address 1 Generator

The user is able to deidentify address 1 from address. (Refer to the image below).

Address 2 Generator

The user is able to deidentify address 2 from address. (Refer to the image below). 

Full Address Generator

The user can deidentify the full address. (Refer to the image below).

Full Name Generator

The user is able to deidentify the full name. (Refer to the image below). 

Date Generator Future

The user is able to deidentify the date column with a value that is later than the source value. (Refer to the image below). 

Static Generator

The user can enter any value as desired and the same will be reflected in the column. (Refer to the image below).  

Deidentification Methods

Random: The deidentified value is randomly generated. 

Consistent: The deidentified value is consistently generated. 

Pass Through: The deidentified value is the same as the source value.

Rename

The user is able to rename the header of the selected column.  

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Rename’ option from the dropdown. 

Step 4: Enter the rename value and click on the ‘Apply’ button. (Refer to the image below).

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the images below). 

Tag

Assign a tag name to specify related columns. This is used specifically to deidentify the ‘Address Fields’. Assign a unique tag name for multiple related address columns such as Address1, Address2, City, State and Zip.   

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Tag’ option from the dropdown. 

Step 4: The user is able to assign a tag name to specify related columns. After selecting the desired column, click on ‘Apply’. (Refer to the image below). 

Create New

A new column is created. The new column is a copy of the selected column.  

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Create New’ option from the dropdown. 

Step 4: The user can add another action to a column that has already been de-identified by clicking on the + option. Enter the new column name and click on ‘Apply’. (Refer to the image below).

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result. 

Step 7: The data is reflected based on the operations selected. (Refer to the images below).

Data Type Conversion

The user can change the schema of the columns of a dataset. 

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Data Type Conversion’ option from the dropdown.  

Step 4: Select the conversion type and click on the ‘Apply’ button. (Refer to the image below).

Step 5: Once the data configuration has been done, click on the run button to run the workflow.

Step 6: Click on the “Show Results” on the bottom line to view the result. 

Step 7: The data is reflected based on the operations selected. (Refer to the images below).

Note: In the example above, decimals have been converted into integers.

Expression

Expressions are used to perform mathematical operations on the columns.  

The syntax is as follows in the case of multiplication: 

@Column_name*2 

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Expression’ option from the dropdown.  

Step 4: Enter the expression value. Click on the ‘Apply’ button. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the images below).

Append

The user can add a suffix to the data in a given column. 

Step 1: Once data is imported, click on transformation.  

Step 2: Drag and drop the data transformation node onto the main screen. Connect the two nodes. 

Step 3: Click on configure to make changes to the data. A data transformation pop-up appears. Select the columns that need change and choose the ‘Append’ option from the dropdown.  

Step 4: Enter the append value and click on the ‘Apply’ button. (Refer to the image below). 

Step 5: Once the data configuration has been done, click on the run button to run the workflow.   

Step 6: Click on the “Show Results” on the bottom line to view the result.  

Step 7: The data is reflected based on the operations selected. (Refer to the images below). 

Note: After configuring a node, ensure you click “Save” to retain the changes. If you need to undo the configuration, click “Discard.” Failing to choose either “Save” or “Discard” will trigger a warning pop-up. (Refer to the image below).