Data Mesh: Domain-Driven Data Products

Date: November 29, 2022

Author: Howard Diesel

Credit to Yolanda Smit (BSG)

As with all technologies on the hype curve, we need to understand what obstacles it's tackling and what challenges we will face due to its current level of maturity.

Yolanda Smit (she/her) did an outstanding job of helping us understand, using a Data-to-Wine metaphor for clarification.

As Data Management programmes and professionals, we must find the value that can be delivered using this technology.

The value list that I derived is as follows:

1. Data Products by Business Value Chain Stage

2. Consumer Data Product focus

3. Consumer-aligned Data Ownership

4. Appropriate levels of governance

Data Products by Business Value Chain Stage

I like this definition of a Value chain from Wikipedia:

"A value chain is a progression of activities performed .... to deliver a valuable product to the end customer."

It's a great confirmation to see the Data Value chain mapping to the Business Value Chain. It was a revelation to see that each stage of the Business Value Chain can deliver a Data Product.

Several comments in the chat box lamented that we have largely ignored the Business Value Chain as Data Programmes and Professionals.

Consumer Data Product Focus

Credit to Yolanda Smit (BSG)

Consumer describes both internal and external parties that require a data product.

The Data Product feature list represents what consumers (people & programs) need from a Data Catalogue:

1. Discoverable: can be found online

2. Accessible: consumers may gain access

3. Understandable: all the required context information is available

4. Trustworthy: meets the consumers' fit-for-purpose expectations

5. Interoperable/Composable: Data Products can be part of other products further down the chain

6. Valuable on its own: it provides economic value without any other dependency

7. Secure: adheres to business security requirements

This list reads like a set of Data Quality dimensions for Data Products.

Consumer-Aligned Data Ownership

Credit to Yolanda Smit (BSG)

Data Ownership is undoubtedly a thorny topic within the Data Mesh environment. We had questions and comments on the approach to governing and defining Ownership.

The Data Mesh technology creates isolated platforms to deliver data products by federated domain.

Isolated Domains certainly help with reducing bottlenecks but increase interoperability requirements.

Choosing the Domain isolation level for Ownership is tricky, and we face similar problems with Stewardship and Master Data Domains.

The three Ownership Archetypes are as follows:

1. Source-aligned Domain Ownership

2. Aggregated Data Domain Ownership

3. Consumer-aligned Domain Ownership

The consumer-aligned approach benefits from a single view of fit-for-purpose (jobs-to-be-done) data products.

An appropriate level of governance

Credit to Yolanda Smit (BSG)

To avoid bottlenecks, we typically distribute services to improve throughput and reduce queuing. The downside of the federated approach is an increase in standardisation and governance to support interoperability. Examples of required standardisation are:

1. Federated Data Management Operating Models

2. Canonical Data Models for Domain-Driven Development

The suggested approach to reducing the need for standardisation is to allow economic forces to determine the survival of data products. If your data products do not adhere to the Consumer Data Product dimensions, the demand will drop, and it will not be economically viable.

This approach works for Business products.

Click here to see the original article on LinkedIn

If you want to receive the recording, kindly contact Debbie (social@modelwaresystems.com)

Don’t forget to join our exciting LinkedIn and Meetup data communities not to miss out!