Migrate from Data-Add to Data-Correction Schema

This guide explains how to migrate an existing schema built with the Data-Add module to one using the Data-Correction module in WriteBackExtreme. The goal is to shift from a many-to-many mapping model to a precise one-to-one model for better control, traceability, and data integrity.


Background: Module Differences

Data-Add Module:

  • Uses a many-to-many relationship.

  • Stores mapping records in the _mapping table in the repository.

  • Every new datapoint written creates a new row.

Data-Correction Module:

  • Uses a one-to-one relationship.

  • Data is written directly into the schema table.

  • Enables inline editing and record replacement.


Why Migrate?

Benefit

Description

Data Integrity

Prevents duplicates by updating existing records.

Traceability

Maps each entry directly to a Tableau datapoint using a unique key.

Simplified Workflow

Removes complexity by eliminating mapping logic for updates.


Migration Overview

  1. Create a new schema using the Data-Correction module.

  2. Include all original fields from the Data-Add schema.

  3. Add one new column: mapping_key — a unique identifier for Tableau datapoint.

  4. Export existing data from the Data-Add schema.

  5. Modify the data: Add a mapping_key column in Excel or CSV.

  6. Import modified data into the new Data-Correction schema.

  7. Update Tableau to use the mapping_key for correction mapping.

Step-by-Step Instructions

1. Create a New Schema

  • In the Management Console, create a new schema.

  • Select the Data-Correction module.

  • Do not base it on the existing Data-Add schema.

2. Add Fields

  • Recreate all fields from the original Data-Add schema.

  • Add a new column:

    • mapping_key (type: String)

    • This will serve as the primary link to Tableau datapoints.

3. Export Existing Data

  • Use the extension to export existing data from the Data-Add schema.

  • Alternatively, request a database export from your DBA.

4. Add mapping_key to Your Data

  • In Excel/CSV, create a mapping_key column.

  • Populate it using a logic that combines one or more fields that identify the Tableau record.

Example:

If you have:

product_id = "ABC123"
country = "US"

Create:

mapping_key = "ABC123_US"

Use a Tableau calculated field that generates the same key.

5. Import into Data-Correction Schema

  • Upload the modified data with the mapping_key column.

  • Verify all required fields match.

  • In Tableau, ensure your dataset includes the same calculated field mapping_key.

  • Use it as the identifier field when configuring the extension.


Field Mapping Example:

Original Data-Add Schema (dataadd_product_pricing_for_countries):

Field

Type

id

UUID

price

Decimal

New Data-Correction Schema (datacorrect_product_pricing_for_countries):

Field

Type

id

UUID

mapping_key

String

price

Decimal


Important Notes

  • mapping_key must be uniquely reproducible in Tableau and your schema.

  • Use calculated fields to generate the same value on both sides.

  • Be sure the new schema is tested in a staging dashboard before replacing production dashboards.


For questions or help, contact [email protected].

Last updated

Was this helpful?