The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “#datamesh”

19 Apr 2023

Unlocking the Vault Podcast – What’s Coming in 2023

A few weeks back I was privileged to one of the guests on Dan Linstedt’s podcast, Unlocking the Vault, along with a bevy of other well know data visionaries, thought leaders, and practitioners.

Our topic was –  it’s 2023 and we’re swimming in buzzwords like data mesh, decentralization, data lake house, etc. What’s actually the next big thing that we should take seriously? Are we forgetting something obvious or leaving something behind?

We had quite the discussion on what we see happening in 2023, where the industry is going, agile, DataOps, and loads more.

Listen in!

Enjoy!

Kent

The Data Warrior

P.S. If you enjoyed that discussion, lots more like that to come for an entire week at WWDVC 2023. It is not too late to sign up and join us in Stowe, Vermont! Register here today!

Posted in Data Vault and tagged , , , , , , | Leave a comment
12 Apr 2023

Federated Governance: Why it Matters and Why it’s Essential for Data Mesh

How do you balance agility and governance, particularly when you now have the capability to ingest, deploy and transform massive amounts of data at a scale that no-one had previously dreamed of? And often this is sensitive data. Perhaps you’re working a highly regulated area like healthcare or financial services.  

How can you ensure domain teams adhere to the rules and standards while also giving them freedom and autonomy – the responsibility – to manage and develop data products appropriate to their domains, without it turning into a free-for-all? 

The concept of federated governance isn’t new but the popularity of data mesh means it’s become a much hotter topic, and a challenge for teams to solve. One of the four pillars of data mesh, a federated data governance framework provides for some centralized management, for enterprise level rules and standards,  but the ultimate responsibility to apply and execute the rules is decentralized to the domains.  

In short, it’s a way to ensure good governance with the right rules being applied in the right ways to the right data but through a more flexible approach, one that empowers domain team developers and supports collaborative working.   

Read the rest of the post here: Federated Governance: Why it Matters and Why it’s Essential for Data Mesh

Ciao for now!

Kent

The Data Warrior

Posted in Big Data, thought leadership and tagged , , , | Leave a comment
10 Jan 2023

Panel Discussion: Data Modeling in a Data Mesh World

Happy New Year y’all!

Last month I had the pleasure of chatting with two of my data modeling pals about how to think about data modeling in a #DataMesh architecture. The recording was hosted by the Data Mesh Learning Community in partnership with Data Mesh Radio.

Juha Korpela (Chief Product Officer at Ellie Technologies) facilitated this panel with Veronika Durgin (Head of Data at Saks) and myself (The Data Warrior). Watch (or listen) to learn why Veronika and I consider Data Vault to be the best choice available right now, why only focusing on the technical aspects of data modeling instead of the business will lead you astray, how to build in iterations, change management for data modeling, and much more!

Watch the video replay here: Panel: Data Modeling in Data Mesh

If you want to just listen, check out the podcast audio, which has some nice notes from Scott Hirleman from the Data Mesh Learning Community.

Enjoy!

Kent

The Data Warrior

Posted in Data Modeling, thought leadership and tagged , , , , | Leave a comment
08 Dec 2022

The Role of DataOps in a Data Mesh Architecture

Hot off the press – a new post discussing two of today’s hottest topics – #DataMesh and #DataOps.

Data Mesh is a decentralized architecture that organizes data by specific business domains and teams, leveraging a self-service approach. It’s designed to provide greater ownership and responsibility to Data Product teams and, it is hoped, more effective outputs in a timely manner. 

Data mesh is founded on four pillars: 

  • Empowered domain ownership – letting the experts who know the data best do their jobs. 
  • Cross organizational transport – enabling consumption through data products. 
  • Self-service analytical platform – reducing costs, improving agility. 
  • Federated governance – the ‘glue’ that holds the process together, enabling interoperability and compliance. 

Is DataOps simply an enabler for more sophisticated design patterns like Data Mesh?  If we carry on with DevOps thinking – considering how the involvement of end users in a product lifecycle went from passive to truly active, one starts to see that it’s more than that. 

Read the rest of the post here – The Role of DataOps in a Data Mesh Architecture

Happy reading.

Kent

The Data Warrior

Posted in Big Data, thought leadership and tagged , , , , | Leave a comment
06 Oct 2022

The Hype:  Data Mesh Will Become Obsolete – Really?

Data Mesh is indeed a hot topic these days. Is it the answer to all our big data problems? Probably not, but it will help some organizations. Is it obsolete? See what I think in this recent blog post…

In the summer of 2022, Gartner® published its Hype Cycle on Data Management, and this was not without some controversy.

In particular, it asserted that Data Mesh will be obsolete before it reaches what Gartner calls the Plateau of Productivity. It is hard to fathom how they came to that conclusion when there is such a vibrant Data Mesh Community complete with a very active Slack channel, and when DataOps.live customers like Roche and OneWeb have successfully built a Data Mesh using DataOps.live and Snowflake. But we’ll come back to that in a moment.

Data Mesh is in essence a conceptual approach and architectural framework, not a specific off-the-shelf tool or technology. One of the main precepts for its creation is angled towards improving productivity and time to value by eliminating the perceived bottlenecks that have been experienced by organizations building out large-scale enterprise data warehouses and data lakes.  Zhamak Deghani, the creator of the Data Mesh concept, has referred to it as a decentralized sociotechnical approach – meaning it involves people, processes and technology. So, it seems likely that no two implementations will be the same by their nature. How one organization defines productivity based on its specific needs and outputs on its own application of Data Mesh will differ from another. Yes, Data Mesh is (kind of) new and yes, it is maturing, but that does not mean we should ignore or discount it this early in the game.

It is hard these days to separate the hype from the reality so check out the entire post over at the DataOps.live blog.

Cheers.

Kent

The Data Warrior

Posted in Big Data, Data Warehouse, thought leadership and tagged , , , , , | Leave a comment

Post Navigation

← Older Entries