The Data Warrior

Changing the world, one data model at a time. How can I help you?

3 Key Resources for Data Vault on Snowflake

By now you surely know that you can build a Data Vault on Snowflake. In fact we have many customers doing so today. So much so that we formed a Snowflake Data Vault User Group.

Over the years I have had hundreds of calls and meetings with organizations around the world discussing this topic from just basic Data Vault 101 type questions to best practices to who is doing Data Vault on Snowflake. Because of that we developed a Data Vault Resource Kit that points you to all the key blog posts, videos, and customer stories on the topic (scroll down to see everything!). Be sure to bookmark that page. Most of your questions on this topic can be answered there.

To take it a step further and to a deeper level, I partnered up with Snowflake Field CTO Dmytro Yarashneko (CDVP2) and wrote a post with reference architectures and discussions related to doing real time feeds into a Data Vault 2.0 on Snowflake. Check that out here. This article even has code!

And, at long last, for those that want to jump in feet first and try it for yourself, the team built a Data Vault Quickstart , based on the above article and a hands on lab from WWDVC 2021, that gives you a step-by-step guide and all the code to build and load a Data Vault 2.0 system, including an information mart on top of the Data Vault, all in your very own Snowflake account.

So, what is your excuse now? You have all the resources you need to give it a go!

And please, bookmark this post and/or the links above so you don’t lose them!

Model on!

Kent

The Data Warrior

Single Post Navigation

10 thoughts on “3 Key Resources for Data Vault on Snowflake

  1. Regarding pk being an MD5 hash doesn’t that mean given the super high cardinality on the pk, there wouldn’t exist an optimized way to choose how to physically cluster data (either automatically by snowflake or manually with us telling what column(s) made up cluster key on the table ?

  2. In many cases on Snowflake, unless you have massive tables, you should not be manually clustering anyway. And in DV 2.0 folks use MD5, SHA1, SHA2 and sometimes natural keys. Lots of discussion on the approach out on the data vault alliance site too. Bottom line though – customers are not having performance issues on queries that can’t be resolved with PIT and BRIDGE tables or in many cases upsizing their virtual warehouse a little. Check this post too for more details and discussion https://www.linkedin.com/pulse/data-vault-20-snowflake-hash-question-patrick-cuba/

    Best advice is to try it for your data, then evaluate all the pros and cons.

    The fact that I used a diagram showing MD5 for the PK should not be taken as that being the only way, or best way, to do it in all cases. 🙂

    • thanks so much for the link, i was aware of the basic of data vault but interesting the PiT and Bridges snapshots to boost performance in some scenarios..

      Im follower of yours, so big embrace from Europe 🌍

      • Emanueoi, you could watch the Snowflake webinar delivered in Oct about test and query patterns as it pertains to data vault. It expands on the article Kent referenced above. Click here: https://bit.ly/3qdpIDv
        passcode is: S5sa4&ZY

      • Thanks Patrick for the access to your webinar above. I could neither find your presentation slides nor the code you demo’ed from your GitHub site. Could you please share the link to these resources as well? Many thanks!

      • You should take a look at my Agile Data Engineering book on Amazon and Dan’s Data Vault 2.0 book to learn more about PITs and Bridges. There is a lot more to DV 2.0 tha hubs, links and sats.

      • Thanks @patrickcuba8 for the access.

        @Kent Graziano could you share exact book names or urls to be sure ? Thanks so much.

      • Click the images on the right side of the blog. That will get you there.

  3. Thanks Kent for the awesome resources you shared above. I didn’t know there was a user group so I just joined the Snowflake DV User Group at https://usergroups.snowflake.com/data-vault/. Looks like I missed a few past events, however, when I click on these past events, it doesn’t provide a way for me to play the recording. Were these recorded by any chance?

Leave a Reply to Sam_P Cancel reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: