The Data Warrior

Changing the world, one data model at a time. How can I help you?

Mass Data Fragmentation: Reducing ‘Data Puddles’

Shortly before leaving Snowflake last year, I was interviewed for this post about one of the worst case examples of data siloes I had seen – we called them data puddles!

A few years ago, Kent Graziano joined a big organization to work on its data. The first problem was that nobody really knew what and where all the data was. Graziano took his first three months on the job investigating data sources and targets, ultimately creating an enterprise data map to illustrate all the flows. It wasn’t pretty.

“In the end, I discovered that the same data was being sent to three or four places,” he said. In one case raw data was transformed and stored in a data warehouse, then moved from there into another warehouse—which was also pulling in the original raw data.

Graziano, who recently retired from his post as Chief Technical Evangelist at Snowflake, said this scenario is entirely common. Data scattered and copied in lakes, warehouses, data marts, SaaS platforms, spreadsheets, test systems, and more. That’s mass data fragmentation, or, more colloquially, data sprawl or data puddles. 

Indeed, 75% of organizations do not have a complete architecture in place to manage an end-to-end set of data activities including integration, access, governance, and protection, according to IDC’s State of the CDO research, December 2021. This lack of governance combines with legacy systems, shadow IT, and good intentions to pave the road to a lot of fragmentation.

Check out the rest of the post to learn how data sprawl hurts businesses and what to do about it. Read it all here!

Try not to step into any of those puddles!

Kent

The Data Warrior

Thank You Snowflake!

(Artwork by Francis Mao)

After six years and one month as the global evangelist for Snowflake (and almost 40 years in IT), I’ve decided to slow down and begin easing into retirement. As such, today is my last day at Snowflake.

I loved the Snowflake product so much that I gave up independent consulting to sign on (as employee 105) with this scrappy little startup in Silicon Valley (my first) and take a chance. Now at the end of 2021, the product is even more amazing than when I started and with #TheDataCloud it is changing the world of data.

The people and culture established by the founders, Benoit and Thierry, along with the leadership of Bob Muglia, Denise Persson, Chris Degnan, and Frank Slootman, have made this the best job (and longest) of my career. I cannot imagine a better way to bring this chapter to an end.

Thank you, Benoit and Thierry, for your vision and for inventing a new architecture for databases.

Thank you, Kyle Rourke and Todd Beauchene for introducing me to this amazing technology at that tiny meetup in Denver. Know you changed my life!

And finally, thank you Francis Mao for your incredible artwork (above) and especially for my Data Superhero uniform and avatar. The #DataWarrior never looked better!

It has been an honor and a pleasure to work with Snowflake from startup thru hypergrowth to the largest software IPO in history! What a wild and adventurous ride it has been. And I know the team will continue to go far and continue to disrupt the way we manage and get value from data long into the future (I am counting on it…after all I am a shareholder too!).

#LetItSnow

#OneTeam #OneDream

Kent Graziano

Chief Technical Evangelist (Emeritus), Snowflake

3 Key Resources for Data Vault on Snowflake

By now you surely know that you can build a Data Vault on Snowflake. In fact we have many customers doing so today. So much so that we formed a Snowflake Data Vault User Group.

Over the years I have had hundreds of calls and meetings with organizations around the world discussing this topic from just basic Data Vault 101 type questions to best practices to who is doing Data Vault on Snowflake. Because of that we developed a Data Vault Resource Kit that points you to all the key blog posts, videos, and customer stories on the topic (scroll down to see everything!). Be sure to bookmark that page. Most of your questions on this topic can be answered there.

To take it a step further and to a deeper level, I partnered up with Snowflake Field CTO Dmytro Yarashneko (CDVP2) and wrote a post with reference architectures and discussions related to doing real time feeds into a Data Vault 2.0 on Snowflake. Check that out here. This article even has code!

And, at long last, for those that want to jump in feet first and try it for yourself, the team built a Data Vault Quickstart , based on the above article and a hands on lab from WWDVC 2021, that gives you a step-by-step guide and all the code to build and load a Data Vault 2.0 system, including an information mart on top of the Data Vault, all in your very own Snowflake account.

So, what is your excuse now? You have all the resources you need to give it a go!

And please, bookmark this post and/or the links above so you don’t lose them!

Model on!

Kent

The Data Warrior

Data Mesh Learning – Interview with The Data Warrior

Last week I had the privilege of being interviewed by Nick Heudecker (former Gartner analyst and current Senior Director at Cribl) for the Data Mesh Learning Community. In our interview, we covered the idea of empowering business domains to really own and manage their data via things like templates and a center of excellence, not to just give them the responsibility of owning their data and leaving them to figure the rest out on their own. We also discussed the need for organizations to focus on investing in growing a data culture, not just investing in the newest cloud based tooling. Really, how do we lower the barriers to accessing, sharing, and leveraging data and get people to really think about data-as-a-product.

Like Agile before it, Data Mesh is as much about changing the way an organization thinks and works as it is about technology. I argue that the people and organization aspects of adopting a data mesh approach are more important than the technology aspects. Without the right approach, the best technology (like Snowflake), is not going to solve your organization’s data problems.

See the full interview here:

So what do you think about all this data mesh stuff?

Cheers!

Kent

The Data Warrior

P.S. For much more on the thoughts about #datamesh, check the other podcasts and videos listed on my Snowflake Resources page.

Snowday! Sunny, with a 100% Chance of Innovation

What a great event! So many announcements and great demos, plus and awesome live Q&A with our Snowflake leaders.

At Snowday 2021, Snowflake announced exciting new product capabilities that expand what is possible in the Data Cloud. In addition to announcing Python support in Snowpark (currently in private preview), these latest innovations make it easier for organizations to maintain business continuity across clouds and regions; help data engineers and data scientists build pipelines, ML workflows, and data applications faster; and remove the complexity of getting the right data into the hands of customers.

The Snowflake Data Cloud is a global network connecting organizations through data, creating new opportunities for collaboration to improve business outcomes, and fundamentally changing what is possible across industries. For Kraft Heinz, its data science teams are able to build and test models dramatically faster in Snowflake compared with its prior data lake. For NBCUniversal, it’s building brand-new advertising targeting and measurement products, in a secure and privacy-compliant way using Snowflake’s governance and data sharing capabilities. And for 84.51°, it’s built a Collaborative Cloud that takes complexity off the table and unlocks new possibilities for grocers and CPGs sharing and collaborating on data.

Snowflake continues to expand the scope and possibilities of the Data Cloud, delivering unique innovations that enable customers to:

  • Operate globally
  • Eliminate silos
  • Build faster
  • Create new businesses

Catch up on all the details on the  Snowflake blog.

Check it out!

Kent

The Data Warrior

P.S. Registration for Snowflake Summit 2022 is now open!

 

Post Navigation

%d bloggers like this: