The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the category “Data Warehouse”

Combine COVID and POS Data to Ensure In-Stocks with #Snowflake and #Alteryx

Reposting this blog as it is an example of how data related to #COVID19 is being used outside of the obvious #healthcare use cases. In this case, something near and dear to us all – grocery store inventories!

We all know this is a truly unprecedented time in the world, and although sometimes we feel like we’ve been through something similar before, the reality is that we haven’t. Sure – we’ve been through natural disasters. We’ve watched smaller outbreaks from afar. We’ve all made a stock-up trip to the grocery store “just in case” our power went out, or “just in case” we were stuck in our house during a cold-stretch for a couple days, but none of those compare to what we’re seeing in our world right now. COVID-19 has drastically changed the way that the world works – both for now and for the future – and although we’ve never experienced anything like this before, we’ve also never been equipped to respond like we are today.

Get the full story here: Combine COVID & POS Data to Ensure In-Stocks with Snowflake & Alteryx

Stay safe out there!

Kent

The Data Warrior and Chief Technical Evangelist at Snowflake

Tips for Optimizing the #DataVault Architecture on #Snowflake (Part 2)

SETTING UP FOR MAXIMAL PARALLEL LOADING!

In this post, I discuss how to engineer your Data Vault load in Snowflake Cloud Data Platform for maximum speed.

Because Snowflake separates compute from storage and allows the definition of multiple independent compute clusters, it provides some truly unique opportunities to configure virtual warehouses to support optimal throughput of DV loads.

Along with using larger “T-shirt size” warehouses to increase throughput, using multi-cluster warehouses during data loading increases concurrency for even faster loads at scale.

Get the details –  Tips for Optimizing the Data Vault Architecture on Snowflake (Part 2)

Enjoy!

Kent

The Data Warrior & Chief Technical Evangelist for Snowflake

Tips for Optimizing the #DataVault Architecture on #Snowflake

Data Vault is an architectural approach that includes a specific data model design pattern and methodology developed specifically to support a modern, agile approach to building an enterprise data warehouse and analytics repository.

Typical Data Vault Design with Hubs, Sats, and a Link

Snowflake Cloud Data Platform was built to be design pattern agnostic. That means you can use it with equal efficiency 3NF models, dimensional (star) schemas, DV, or any hybrid you might have.Snowflake supports DV designs and handles several DV design variations very well with excellent performance.

This series of blog posts will present some tips and recommendations that have evolved over the last few years for implementing a DV-style warehouse in Snowflake.

Here is the first set of tips:Tips for Optimizing the Data Vault Architecture on Snowflake (part 1)

I hope you find this helpful!

Kent

The Data Warrior and Chief Technical Evangelist for Snowflake

Data Engineering Podcast – What is Snowflake?

A few months back I had the privilege of being interviewed by Tobias Macey on his Data Engineering Podcast show. This came about because Tobias actually Tweeted at me about wanting to do the interview! In this episode we spent an hour discussing the ins and outs of the Snowflake Cloud Data Platform. You can find it here. Hope you enjoy it!

Interview Outline

  • How did you get involved in the area of data management?
  • Can you start by explaining what Snowflake is for anyone who isn’t familiar with it?
    • How does it compare to the other available platforms for data warehousing?
    • How does it differ from traditional data warehouses?
      • How does the performance and flexibility affect the data modeling requirements?
  • Snowflake is one of the data stores that is enabling the shift from an ETL to an ELT workflow. What are the features that allow for that approach and what are some of the challenges that it introduces?
  • Can you describe how the platform is architected and some of the ways that it has evolved as it has grown in popularity?
    • What are some of the current limitations that you are struggling with?
  • For someone getting started with Snowflake what is involved with loading data into the platform?
    • What is their workflow for allocating and scaling compute capacity and running analyses?
  • One of the interesting features enabled by your architecture is data sharing. What are some of the most interesting or unexpected uses of that capability that you have seen?
  • What are some other features or use cases for Snowflake that are not as well known or publicized which you think users should know about?
  • When is Snowflake the wrong choice?
  • What are some of the plans for the future of Snowflake?

This is a great podcast series, so you might want to add it to your regular list!

Cheers.

Kent

The Data Warrior & Chief Technical Evangelist at Snowflake

New Snowflake Community Advocate Program

I am happy to announce that The Snowflake Community—a hub for Snowflake users to connect, share, and learn from each other both offline and online—is launching a new program called the Select Star Program.

What is a Select Star?

The Select Star program recognizes and rewards our most engaged community members who go above and beyond to help other users around the globe. The program offers many opportunities, including speaking at meetups or during webinars, publishing technical tutorials and blog posts, answering questions on Stack Overflow, and sharing your own Snowflake story with the community. The more you engage and offer support, the more award points you’ll earn. Once you’ve collected 1,000 points, we’ll officially accept you into the Select Star program.

Get the rest of the details and how to sign up here: 4 Ways to Become a Snowflake Select Star

Cio for now.

Kent

The Data Warrior

Post Navigation

%d bloggers like this: