Academy · Modelling & Integration

Modelling & Integration

Designing the canonical model and building the pipelines that feed it — domains, dictionary, sources, mappings, ingestion and reference data. About 230 minutes of reading, seven checkpoint quizzes, and the certification exam at the end.

MDM Studio Design Map Govern Trust
Sign in to track your progress

Ask the study tutor

Stuck on a concept? Ask in your own words — answers come from the course material with links to the source lessons. Signed-in learners only; daily limit applies.

Before you start

Prerequisite course
This course assumes you know
  • Model, domain, entity, attribute, and the Data Dictionary
  • What a golden record is and where it comes from
  • Source systems versus connections
  • That a model is versioned and checked out like code
You will also need
  • A developer, modeller or admin role, and a working model you may check out.
  • Enough SQL to read a SELECT. You will not be asked to write one to pass.
  • Read access to at least one source system you can point the hub at.

Recommended, not required — the exam is open to anyone. If you have not taken Foundations, module 0 covers what this course assumes in about ten minutes.

Haven't taken it? Module 0 — Before you start — the assumed ground covers what this course assumes, in about ten minutes, with a self-check at the end.

Before you start — the assumed ground

The vocabulary and the shape of a model, condensed. Skip it if the Associate material is already familiar.

A model is a repository, not a database

This module is a recap, not a substitute for the Associate track. If the words below are already comfortable, go straight to module 1.

A model is the versioned container for an entire design: domains, entities, the Data Dictionary, relationships, mappings, match and survivorship rules, standardization and jobs. The mental model that fits best is a repository — you check it out to change its definition, you check it back in, and what you changed is reviewable on the way out.

Inside it: a domain is a top-level subject area (Customer, Product, Supplier); an entity is a master object within one; its fields are attributes, defined once in the Data Dictionary and referenced everywhere else. A relationship connects two entities and carries its own cardinality and key columns.

Outside it: a source system is where raw records come from, and a connection holds the credentials to reach it. Connections live in a global registry shared across every hub engine — deliberately outside any one model, because the same source usually feeds several.

What you are modelling towards

Everything a modeller builds points at one output: a golden record — the surviving best version of one real-world thing, with cross-references back to every source record that contributed to it.

The path there has a fixed shape, and each stage is something you will design in this course:

  • Connect a source and discover its schema.
  • Map its fields, stage by stage, into your entity.
  • Standardize on the way in, so comparison happens on cleansed values.
  • Ingest on a schedule, through jobs and job groups.
  • Match to decide which records are the same thing, then survive to decide which value wins per attribute.
  • Publish, and serve.

Two boundaries are worth fixing now because the rest of the course leans on them. Matching and survivorship are separate decisions with separate rules. And an approved mapping specification freezes an immutable version — the pipeline runs what was approved, not what the designer happens to look like today.

Designing the model

Models as versioned containers; domains, entities and a dictionary that drives everything downstream.

Models, lifecycle and checkout

A model is the versioned container of an entire design — domains, entities, dictionary, mappings, rules. Models are created, cloned, imported/exported as XML and activated under Administration → Deployment Models; creating one also auto-provisions its target database and pipeline schemas (SQL Server, PostgreSQL, MySQL). You design inside the active working model.

Structural change happens under checkout: an exclusive edit lock that makes the model read-only for everyone else until you check in (reviewing your changes) or grant control to a requester. Treat models like code: clone to experiment, keep the production model governed, and let checkout serialise design work.

Domains, entities and relationships

A domain frames a subject area (Customer, Product); entities are its master objects (Customer, Address, Contact). Model only what you intend to master — an entity earns its place by having sources, rules and consumers, not by existing in some upstream schema.

Relationships connect entities (a Customer has many Addresses) so the ERD, lineage and reporting understand the joins. The Data Modelling (ERD) view renders the design graphically — the fastest way to spot a shape that grew wrong.

The Data Dictionary: the contract

The Data Dictionary defines each entity's attributes: name and business name, data type and length, key/identifier flags, required flags — and the sensitivity level. It is the contract everything else reads: mapping targets, match rules, survivorship, quality and masking all refer to these attributes.

Two decisions deserve the most care: the business key (what makes two records "the same" — matching leans on it) and classification (label confidential fields at design time, when it costs nothing). A sloppy dictionary taxes every later stage; a precise one makes them almost mechanical.

The Business Glossary and its spine

The Business Glossary holds plain-language definitions — what "active customer" means, how "primary email" is chosen — so stewards and analysts read intent, not just column names. Terms move DRAFT → PUBLISHED through approval. Keep it next to the dictionary and write it as you design, not after.

Underneath sits the glossary spine, which turns a word list into a structure: typed relations between terms (SYNONYM, RELATED, BROADER, NARROWER, SEE_ALSO), many-to-many bindings from terms to the assets they govern, and term-level sensitivity classes (none < personal < financial < special) where the most restrictive binding wins. A term bound to an asset is how one classification decision reaches every column that means the same thing.

The Schema Importer plus AI naming can bootstrap entities and business names from an existing database — imported drafts you then refine, never blindly accept.

Hierarchies and tiers: different things with one word

The word "hierarchy" means three distinct features in MDM Studio, and confusing them wastes days. Two are covered here; the third — value hierarchies, declared containment between governed code values — has module D5 to itself.

Managed hierarchies (Modelling Studio → Hierarchies) are trees over your master data: named, ragged and alternate hierarchies, node CRUD, levels, top nodes, roll-ups, integrity and placement checks — plus immutable version snapshots you can diff and export to CSV. Think organisational structures, product taxonomies, territory trees: things that change and whose past shape you must be able to prove.

Hierarchy tiers are something else entirely: four tier names configured on the model, bound per entity to attributes, and materialised as tier columns stamped at every pipeline stage and carried onto the golden record. They are the reporting dimension — region → country → branch → team — not a tree you edit. Tier presets and a tier advisor sit on the model-create form; the advisor may consult a language model, and falls back to a curated industry library when no key is configured. The names are set once on the model and can be overridden per domain — a Sales domain may call tier 2 "Territory" where Finance calls it "Cost centre" — with the model's names as the fallback wherever a domain says nothing, and every surface that prints a tier name asks the domain first. On the entity form, all four tier bindings are edited inline together, so the reporting dimension of an entity is one decision made in one place.

Rule of thumb: if you would draw it as an org chart with editable nodes, it is a managed hierarchy. If you would put it on the rows and columns of a report, it is tiers. A model that needs both should have both.

Both meanings are managed in one place: the Hierarchies & Tiers workbench (Modelling Studio → Hierarchies tab). Per-entity tier bindings show coverage bars, and a null bar means not yet stamped — which is not the same as zero; a Backfill action stamps already-published golden records, and a tier-level advisor helps choose levels. For hierarchies generated from your data there are What changed?, Re-sync and Refresh measures actions — Re-sync always previews its changes before applying them, and refresh-measures reports set, unplaced and non-numeric counts separately rather than as one blended number. Right-click a hierarchy for Open, Copy and Delete — delete states the node count and asks for confirmation first.

Trees are only navigable if you can read them, so every hierarchy resolves a label attribute: nodes show the category name with the code muted after it. You choose it at from-parent creation (the "Node labels from" picker) and can change it later from any surface that renders the tree — the change is saved on the hierarchy and every surface follows.

As-Is Assessment: knowing where you are

Modelling Studio → As-Is Assessment is the planning surface: a ten-dimension maturity assessment per domain scored L1–L5 with weights, plus organisational readiness, criticality, dependencies, a generated roadmap, a portfolio view across domains, and stakeholder governance — a RACI letter per stakeholder, power/interest positioning, an engagement ladder and a per-activity RACI matrix. A sign-off locks a round so the baseline cannot drift after the fact.

Be clear about its status: the assessment is advisory. Nothing in it gates a pipeline, blocks a publish or enforces a roadmap. Its value is the conversation it structures and the baseline it freezes — it is how you argue for sequencing Customer before Product, with evidence, and how you show progress a year later.

The Catalog: a read model, not a second repository

Modelling Studio → Catalog projects a searchable read model over metadata that other modules already own — glossary terms, domains, entities, attributes, connections, mappings. It does not become a second place to define things; the modules stay authoritative and the catalog is rebuilt from them. Assets are joined by a URN, and an asset that vanishes is tombstoned, never hard-deleted.

What it gives you: search and coverage across the estate, facets (owner, steward and the Consulted/Informed sides of RACI), steward proposals — term bindings, classifications, new terms, asset descriptions — that apply only when someone explicitly accepts them, and an impact view answering "what breaks if I change this?". Editing a facet is the one metadata write in the product that routes through Change Governance.

Three honest limits to carry. Search is lexical, not semantic by default: it compares shared words and word shapes and reports semantic:false; only an externally configured embedding model makes it semantic. The index is a rebuild, not a live feed — by default it refreshes when somebody presses Reindex, and a scheduled drain only caps staleness. And some changes emit no marker at all — connections, mapping sets, relationships, attribute sources and trust scores change the model silently, which the status panel reports as sources not emitting. Trust the catalog for orientation; confirm in the owning module before you act.

Statistics only: the catalog indexes profile statistics, never raw minimum, maximum or frequent values — which is what makes it safe to show broadly.

Connecting & mapping

From source systems to model attributes: connections, the Mapping Designer and standardization.

Sources and connections

A source system is the business identity of an origin (CRM, Billing); a connection is the technical access to it. Fifteen connection types are supported: databases and warehouses (SQL Server, Azure SQL, Synapse, PostgreSQL, MySQL, Oracle, Teradata, Snowflake, BigQuery, Databricks), Salesforce, and files/endpoints (CSV, Excel, FTP, REST API) — with live previews, drafted source profiles with defensible trust scores, and a Test button that proves reachability before anything depends on it. Credentials are envelope-encrypted in the secrets vault.

One caveat worth knowing before you promise a connector: the drivers for Snowflake, BigQuery, Databricks, Salesforce and Teradata are optional dependencies loaded on demand. A stock install without them cannot reach those five, and their live connectivity is verified against the customer's own account rather than proven in-house. Plan a connectivity spike for those, not for PostgreSQL.

Connections form one global registry across engines: defined once, usable everywhere, self-healing across hubs. Name sources for the business, not the hostname — "Billing (SAP)" outlives "PRDSQL07".

Sources & Connections has a third tab: Source Objects — the registered inventory of the tables, files and endpoints your pipelines actually read. Registering an object snapshots its field inventory; you mark fields in or out of scope (coverage KPIs always show their denominators), and re-snapshotting performs drift detection: added, changed and removed fields are listed explicitly, removed fields are kept struck-through rather than silently dropped, and a removed field that a live mapping still reads is flagged BREAKING by name. This is where "the source changed under us" stops being a surprise.

The Mapping Designer

A mapping says "this source column becomes that model attribute". In the Mapping Designer you pick the source and target entity, load the source's columns, and map them onto dictionary attributes — with per-field transforms where shapes differ.

Be precise about transforms, because the platform now is. UPPER, LOWER, TRIM and EXPRESSION transforms execute — they are applied by the pipeline's generated extract SQL, so what you configure is what lands. Two transforms — CONCAT and LOOKUP — do not execute at run time, and the designer says so: their picker entries are labelled, and saving a mapping that carries a non-executing or unknown transform is refused with a suggested alternative. If your hub predates transform execution, know that transforms used to be silently ignored — the transform-impact report names every mapping whose output changes now that they run.

Approval freezes what runs. Approving a mapping set freezes an immutable, numbered specification version, and the pipeline executes the approved version, not the editable draft. A set with no approved version keeps running its draft, exactly as before. The Versions panel states which of three states the set is in — no approved version (the draft runs), approved and matching, or diverged (the approved version still runs until you approve again) — with a field-level diff showing both values for every change, and a spec-impact report names the diverged sets. The Scripts tab previews; it does not promise: every SQL box is read-only, the Extract tab shows the same builder the pipeline runs, DDL and MERGE are labelled Preview only, and there is no Save — saved custom SQL was never executed by the pipeline, and the tab no longer implies otherwise.

Mappings are stage-aware and stage-isolated: an Extract set reads the source; every later stage (Land → Stage → Standardize → Match → Survive → Publish) reads only the previous stage's output, and the form only offers valid choices. Once Extract sets exist, Build pipeline sets generates the whole stage-to-stage lineage per entity in one click. Several sources map onto the same entity — CRM and Billing both feeding Customer is exactly the point of MDM — and each keeps its own mapping, so a source change touches one mapping, not the model.

Standardization: clean on the way in

Standardization rules normalise values during ingestion: casing, trimming, phone and date formats, code translations (with reference-domain suggestions). They run before matching — which is why they matter: comparators score consistent input far better than raw variants.

Put normalisation here rather than in per-source SQL: rules are declared once, visible, testable (preview the cascade before committing) and applied uniformly to every source that feeds the attribute.

Schema import and AI naming

When the structure already exists in a database, the Schema Importer reads tables and columns and drafts entities and attributes in one pass. AI naming proposes business names, descriptions, categories, PII flags — and a sensitivity (data-access) level plus a golden-record eligibility score — from physical names ("CUST_NM" → "Customer Name"), with a deterministic heuristic fallback, so the feature always works.

The importer is stage-aware: pick a pipeline stage and it browses that stage's schemas, tables and previews — post-Extract stages show the mapping feed from the previous stage rather than pretending everything is an Extract. Add all links columns to existing attributes case-insensitively (no duplicate attributes minted by casing) and registers the mappings against the selected stage; columns that match no attribute at that stage are reported in an "unmatched at this stage" banner rather than silently skipped. Every registered mapping can be removed individually, with a confirmation dialog — here and on every other surface that shows mappings — and the import queue starts collapsed so the page leads with the schema, not the backlog.

Everything imported is a draft for a designer's review: confirm keys, fix names the inference got wrong, classify sensitive fields, and delete what you won't master. Bootstrap fast; curate deliberately.

Pipelines & reference data

Ingestion jobs and orchestration; governed reference data and controlled write-back.

Ingestion jobs

An ingestion job executes a mapping: reads the source, applies transforms and standardization, and stages rows in the platform. Create jobs by dragging mapping sets onto the jobs panel (or let Auto-allocate propose the missing ones); jobs run on demand or on a schedule, record per-run history (rows read, loaded, warned, rejected), and the All Runs view unifies job, rule, match and profiling history. Stage rows carry a lifecycle — STAGED, WARN, RETIRED — and retired rows never flow forward.

Watch the reject counts, not just the green ticks: a job that "succeeds" while quietly rejecting 5% of rows is a data loss you'll discover at reporting time. Job history is the first place to look when golden data looks stale.

Orchestration: groups and packages

Job groups run related jobs together; execution packages chain the full pipeline with dependencies — load CRM, then Billing, then match, survive and publish — as one repeatable, schedulable unit.

Packages are how a domain stays fresh without a human conducting the orchestra. Design them so a failure stops the chain at the failing step (half-processed pipelines are worse than late ones), then re-run the package once the cause is fixed.

Reference data (RDM)

RDM governs shared code sets — country codes, statuses, industry classifications — as versioned domains with immutable releases: publish a release and it never changes; corrections become the next release. Crosswalks map between code systems.

A read-only serving API with service tokens exposes released reference data to other applications, making MDM Studio the organisation's source of truth for codes — not just for master records. Model attributes bind to reference domains so validation and standardization can enforce them. Each code carries a display value, and that display value is the standardization: wherever a governed value is shown, the display value is what appears, and wherever it is compared or matched, the code is what counts — so a change to the display value changes what people read without changing what the platform trusts.

Write-back and lineage

Write-back closes the loop from the golden record to the source, through a governed sequence: eligibility opt-in per attribute or entity with an authority role, then build → preview → impact → per-item approve/reject → apply → verify → rollback.

Be precise about what "apply" means, because the name over-promises. There are three apply modes and the default is SCRIPT: the platform generates an idempotent, drift-guarded UPDATE script for a human to run against the source. Nothing is written by the platform in that mode. PROC and DIRECT do mutate the source — DIRECT executes guarded, chunked updates with before-images for rollback — but both must be chosen deliberately. Tell a source owner "we will hand you a script" unless someone has explicitly opted into DIRECT.

And one hard limit: write-back is not available on an Oracle hub. The runner returns a clear not-implemented error there; ports exist for SQL Server, PostgreSQL and MySQL only. If the hub engine is Oracle, write-back is not part of the design, whatever the licence says.

Data lineage ties the whole build together: from a golden field, back through survivorship and mapping, to the exact source column and run that delivered it. When someone asks "where did this value come from?", lineage is the designed-in answer.

The Real-time API: serving and decisioning

Mastering → Real-time API exposes two genuinely synchronous operations, in-app and to machine consumers over a token-authenticated API: match an inbound record against the published golden master (returning ranked candidates and an AUTO_MERGE / REVIEW / NO_MATCH decision), and get golden by entity and golden key. Both run in-request, and match reuses the batch engine's scoring so an online decision and a batch decision agree.

What it is not: the real-time lane does not master. Nothing in it creates, merges or updates a golden record. It reads the last published master — produced by the batch MATCH → SURVIVE → PUBLISH pipeline on a schedule, a manual run or a job package — and answers questions about it. If a caller needs their record mastered, it goes through the pipeline like everything else.

Two operational facts to design around. A synchronous match loads a bounded candidate set (a hard cap in the thousands), so it is a decisioning call, not a bulk reconciliation tool. And event intake is queued, not streaming: posting an event enqueues it durably and processing is an explicit call — replayable and auditable, but not a stream processor.

Sell it accurately: "real-time serving and decisioning over the last published master" is true and valuable. "Real-time mastering" is not something this product does.

Beyond the platform edge

Registering the assets you do not own, and governing the specifications that point at them — without pretending either one moves data.

The external-asset register

Your estate does not stop at the platform's edge. A finance warehouse consumes your Party golden records; a downstream report is built on a table nobody here owns. Modelling Studio → External Assets is the curated register of those things.

Nothing crawls. This is the first thing to internalise, because it inverts the instinct: the platform never reaches into your network to discover what exists. A steward records an asset because it matters, and that record is an assertion by a person. Assets are addressed ac:x/{system} or ac:x/{system}/{name}, they carry no model (a warehouse is not a fact about one version of your design), and they link to your own assets three ways whose direction is the meaning: CONSUMES (it reads from you), FEEDS (it supplies you), DESCRIBES (it documents you). A glossary term can govern one exactly as it governs your own columns.

Two rules that look inconsistent until you see why. Registering is audited but not review-gated — recording that a warehouse exists is documentation, and a review queue in front of documentation is how registers go stale; whereas editing a catalog facet stays governed, because a statement about ownership and meaning should be reviewable. And retiring never deletes: retire states the link count first, then the next reindex tombstones the asset in the catalog, keeping its history and its human-set facets.

The honest limit, which belongs in every steward's head: external assets are the only assets in the catalog whose truth the platform cannot check. Everything else is projected from a module that owns it, so a wrong entry there is a bug. An external asset is a person's statement about somewhere else — it stays true only while somebody maintains it. Give the register an owner.

Specifications that target outside

A mapping set names a target kind: ENTITY (one of your model's entities — the normal case) or EXTERNAL (a registered external asset). Exactly one, never both. An EXTERNAL-target specification is how you write down and govern Hub Party → FinanceDW DIM_CUSTOMER, field by field, with the same immutable approved versions and the same field-level diff as any other set.

Now the part that is examined, because getting it wrong is expensive: no pipeline will ever run one. Ingestion refuses to create a job for it, the pipeline builder refuses to generate stages for it, and the field guard refuses it in both places — each by name, saying which set and why. This is not an unfinished feature. Documenting a target is not moving data to it. Moving data outward is Write-Back, which is a separate mechanism you choose deliberately, with its own approvals, its own script-by-default apply mode and its own rollback.

What they do give you is lineage: they emit as MAPPING_SET catalog assets and contribute external FEEDS edges, so an impact analysis can follow your model outward into the systems that read it — which is exactly the question "what breaks downstream if I change this attribute?"

Reading the External specifications section honestly

Pipeline Health's source-to-target view carries external specifications as their own External specifications section, with its own list and its own counts, deliberately kept out of the attribute-coverage numbers beside them. The reason is worth stating: an external specification does not cover one of your attributes, so folding it into a coverage percentage would make one number mean two things — and a number that means two things is a number nobody can act on.

One reading rule. On a hub whose specification version tables predate the release, a set's version state reads NOT MEASURED, not zero. Nothing recorded yet and nothing there are different facts, and the section keeps them apart. If you see NOT MEASURED, the answer is "this hub has not been through that migration", never "this specification has no versions".

Containment, and rules that explain themselves

Value hierarchies declared and governed, containment drawn on the ERD, code sets built from columns, and a standardization builder whose every candidate names where it came from.

Value hierarchies: containment you can govern

The third meaning of "hierarchy" (D1L5) is containment between values: South Africa contains Gauteng, Gauteng contains Johannesburg. Tiers cannot say this — a tier declares that PROVINCE is level 1 and CITY is level 2, but nothing in a tier records which province contains which city. A value hierarchy does, and that one fact is what makes a roll-up trustworthy.

The shape. A value hierarchy is a named chain — GEO_ZA: Country ▸ Province ▸ City — in which every level is a governed reference domain and every containment link joins a value in one code set to its parent in the code set exactly one level above. Because the levels are governed, the links are too: a link to a value not in the parent's code set is refused ("the link would dangle"), a cross-level link is refused with the levels named, a code set can never contain itself, and depth carries a stated cap that refuses rather than truncating.

Declaring and binding are two separate acts. The Value Hierarchies page manages the chain; binding it to an entity — saying Customer's PROVINCE and CITY carry these levels — is per entity, and is deliberately a different binding from tier bindings. An entity can carry both, and a surface filtering by both AND-combines two named filters, never one blended answer.

The advisor measures; it never guesses. Point it at two bound levels and it reads your published golden records and proposes links with the evidence — agree / disagree / blank counts. A unanimous value is proposed; a contested value (Gqeberha claimed by two provinces) is proposed as nothing — it sits in the tree as a finding until a human resolves it. The contested values are precisely the ones a guess would get wrong.

Two switches, both off until you choose them. Derive is display only — a record whose province is blank but whose city resolves under Gauteng can be shown there, labelled as derived; nothing is written. The IN_HIERARCHY rule operator lets a DQ rule flag records whose two bound values contradict the containment — contradiction only; a blank parent is a completeness question and an unknown value is a mapping question, each with its own finding. No rule ships using the operator, and a rule whose hierarchy is missing or capped fails its run rather than passing every record.

Containment on the canvas — and code sets from columns

Value-hierarchy containment is drawn on the ERD alongside the foreign-key relationships, styled so the two can never be confused: a declared containment is a violet dashed edge between the participating columns, a proposed one a fainter amber dot-dash. Participating columns carry small ⊃ / ⊂ chips (amber while only proposed), and the toolbar's Contains button shows a live count of declared and proposed edges with an eye toggle to hide them on a crowded canvas — your choice remembered.

Every containment edge answers a click. A proposed edge is labelled proposed — click to review: the review names the entity, the chain, and which ends rest on inference, and accepting writes only the ends still inferred — a declared end is never re-stamped. A declared edge opens a read-only statement of the recorded fact, with one button to the Value Hierarchies governance page — read-only on purpose, because un-declaring a containment changes what every consumer of the chain means. And a level the product cannot name gets no line at all: no edge is drawn to a column it would have to guess.

A code set can be created from the column itself. Declaring containment on a column that names no governed code set offers the virtual ref list: preview-then-commit, where the first click writes nothing and states exactly what would happen — a new DRAFT reference domain, seeded once with N distinct values, with a sample, the blank count and a declared cap. The commit re-runs the preview (it cannot state one thing and do another); values are trimmed and de-duplicated case-insensitively, blanks are counted but never seeded, and a re-run is additive-idempotent. A masked column refuses before a single value is read — a code list built from a masked column would be the extraction the mask exists to prevent — and an all-blank column refuses rather than creating an empty domain. Publishing the new domain stays with the normal reference-data release flow.

The rule builder: candidates that name their provenance

Creating or editing a standardization rule opens a builder that grounds every suggestion in something you can check. Candidates arrive in layers, each naming its provenance: the governed layer (the attribute's logical type and reference domain, read live from the catalog — never a copy), the evidence layer (every DQ rule on the column, with its fix pairing), and the pattern layer (the curated transform library). A suggestion that cannot be attributed cannot be judged.

Describe what the rule should do. The builder starts with an intent box: type "remove dialling code e.g. +27 / 0027" and the platform reads the verbs and literals out of the sentence and composes a portable expression that runs identically on all four hub engines. This layer is deterministic and needs no AI key; what it cannot do portably it says rather than drops. Its candidates come first, badged DERIVED — this came from your sentence, not from anything anybody approved — and the box states what it read back to you. Where a sentence has two honest readings — delete +27, or substitute the national trunk prefix — both are offered and the choice is named, because a confident single answer would simply be one of them, half the time wrongly.

Try one value. Type a single value and every candidate on screen shows your value → result, computed through the same compiler and executor the pipeline runs. Three outcomes, all named: transformed, unchanged (a real and common answer, said in a word), and failed with the database's own words. Nothing is written; no population is read.

Preview before you commit. A transform can fix every failing value and still be wrong — a left-pad that turns '99' into '0099' also turns '12345' into '2345', which passes the length check and is a different number. The preview runs the whole cascade over the column's real population and buckets every value: fixed, still failing, broken (was passing, now fails) and rewritten (still passes, but the stored value changed — the quiet one). It answers with a verdict sentence"Fixes 805 failing record(s) and CHANGES 421 record(s) that were already passing" — never a coverage percentage, and it refuses on a masked column. A transform the product proposed cannot be saved until it has been previewed against the real column, and editing the expression afterwards invalidates the preview; an expression you typed yourself saves as it always has.

Formats, calculations and what never changes the data

Where a number gets its appearance, where a derived field gets its SQL, and why neither touches the value.

Display formats belong on the attribute

An attribute in the Data Dictionary can carry a display format — a named preset such as a thousands-separated integer, a currency, a percentage, a date style or a masked identifier. Set it once here and every surface honours it: grids, report tables, chart data labels, axis ticks, KPI cards and record screens. When you import a schema, the importer proposes a format from the source column's type and scale; you accept or change it.

A display format is display only. It changes what a value looks like and never what it is. Filtering, copying, CSV and Excel export, generated SQL, data-quality rules, mapping specifications and every API payload carry the raw value.

The reason is worth holding as a modelling principle rather than a footnote: if a format changed what a filter matched, the screen and the data would disagree about the same record, and every downstream count would depend on who was looking. Appearance is a property of the view; the value is a property of the model.

Calculated fields compile; they are not pasted

A calculated field is an expression saved on a report and used like any other field. What matters for a modeller is what happens to it on the way to the database: expressions are compiled to SQL for whichever hub engine the model runs on. Nothing an author typed is passed to the database as text.

That is the same compiler the advanced data-quality rule authoring surface uses, where @Column references compile to portable per-engine access. One consequence you can rely on: an expression that is legal in a DQ rule is legal in a report, on every engine, because there is one compiler rather than one per surface.

It is also why the Check SQL door exists on rule authoring — it asks the database whether an expression is legal before the rule is saved, rather than discovering it at 3am in a pipeline run.

Ready to prove it?

MDM Studio Certified Modeler & Integrator — Exam — 54 questions, 65 minutes, 70% to pass.

Go to the exam