The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “Data Modeling”

It Depends (or Does It?) – Episode 13

A few weeks back, right after attending Snowflake Summit, I had the pleasure of chatting with Sanjeev Mohan on his podcast – It Depends. We spent an hour or so chatting about trends in modern data management, data mesh, data vault, my now semi-retired life as an advisor to several Snowflake partners, and my love of martial arts.

So sit back, relax (I did), and enjoy!

Aloha.

Kent

The Data Warrior

Life and Times of The Data Warrior

Please join me on March 30th at 11AM CDT for my first official post-retirement talk!

I will be chatting with Mike Lampa from Great Data Minds about my career and plans post-Snowflake. I expect we will cover a wide range of topics including what community and technical evangelism is all about, as well as what made me jump from nearly 30 years in the Oracle world to a small startup with less than 100 people that claimed to have re-invented data warehousing in the cloud.

You can register here for this free event.

See you then!

Kent

The Data Warrior

3 Key Resources for Data Vault on Snowflake

By now you surely know that you can build a Data Vault on Snowflake. In fact we have many customers doing so today. So much so that we formed a Snowflake Data Vault User Group.

Over the years I have had hundreds of calls and meetings with organizations around the world discussing this topic from just basic Data Vault 101 type questions to best practices to who is doing Data Vault on Snowflake. Because of that we developed a Data Vault Resource Kit that points you to all the key blog posts, videos, and customer stories on the topic (scroll down to see everything!). Be sure to bookmark that page. Most of your questions on this topic can be answered there.

To take it a step further and to a deeper level, I partnered up with Snowflake Field CTO Dmytro Yarashneko (CDVP2) and wrote a post with reference architectures and discussions related to doing real time feeds into a Data Vault 2.0 on Snowflake. Check that out here. This article even has code!

And, at long last, for those that want to jump in feet first and try it for yourself, the team built a Data Vault Quickstart , based on the above article and a hands on lab from WWDVC 2021, that gives you a step-by-step guide and all the code to build and load a Data Vault 2.0 system, including an information mart on top of the Data Vault, all in your very own Snowflake account.

So, what is your excuse now? You have all the resources you need to give it a go!

And please, bookmark this post and/or the links above so you don’t lose them!

Model on!

Kent

The Data Warrior

Building a Real-time Data Vault in Snowflake?

Yes you can! The #DataCloud loves #DataVault!

In this day and age, with the ever-increasing availability and volume of data from many types of sources such as IoT, mobile devices, and weblogs, there is a growing need, and yes, demand, to go from batch load processes to streaming or “real-time” (RT) loading of data. Businesses are changing at an alarming rate and are becoming more competitive all the time. Those that can harness the value of their data faster to drive better business outcomes will be the ones to prevail.

One of the benefits of using the Data Vault 2.0 architecture is that it was designed from inception not only to accept data loaded using traditional batch mode (which was the prevailing mode in the early 2000s when Dan introduced Data Vault) but also to easily accept data loading in real or near-realtime (NRT). In the early 2000s, that was a nice-to-have aspect of the approach and meant the methodology was effectively future-proofed from that perspective. Still, few database systems had the capacity to support that kind of requirement. Today, RT or at least NRT loading is almost becoming a mandatory requirement for modern data platforms. Granted, not all loads or use cases need to be NRT, but most forward-thinking organizations need to onboard data for analytics in an NRT manner.

See all the details (and some code) in the full post over on Data Vault Alliance.

Happy Vaulting!

Kent

The Data Warrior

Tips for Optimizing the #DataVault Architecture on Snowflake (Part 3) 

For this last post in my current Data Vault (DV) series, I will discuss two more cool features of Snowflake Cloud Data Platform (@Snowflakedb) that you can take advantage of when building a DV on our platform. If you are not familiar with the DV method, please read my introductory blog post and part 1 of this series before reading this post.

Get the details here: Tips for Optimizing the Data Vault Architecture on Snowflake (Part 3)

Happy Data Vaulting!

Kent

The Data Warrior

Post Navigation

%d bloggers like this: