The Data Warrior

Changing the world, one data model at a time. How can I help you?

Archive for the tag “@SnowflakeDB”

19 Apr 2021

Building a Real-time Data Vault in Snowflake?

Yes you can! The #DataCloud loves #DataVault!

In this day and age, with the ever-increasing availability and volume of data from many types of sources such as IoT, mobile devices, and weblogs, there is a growing need, and yes, demand, to go from batch load processes to streaming or “real-time” (RT) loading of data. Businesses are changing at an alarming rate and are becoming more competitive all the time. Those that can harness the value of their data faster to drive better business outcomes will be the ones to prevail.

One of the benefits of using the Data Vault 2.0 architecture is that it was designed from inception not only to accept data loaded using traditional batch mode (which was the prevailing mode in the early 2000s when Dan introduced Data Vault) but also to easily accept data loading in real or near-realtime (NRT). In the early 2000s, that was a nice-to-have aspect of the approach and meant the methodology was effectively future-proofed from that perspective. Still, few database systems had the capacity to support that kind of requirement. Today, RT or at least NRT loading is almost becoming a mandatory requirement for modern data platforms. Granted, not all loads or use cases need to be NRT, but most forward-thinking organizations need to onboard data for analytics in an NRT manner.

See all the details (and some code) in the full post over on Data Vault Alliance.

Happy Vaulting!

Kent

The Data Warrior

Posted in Data Vault, SnowflakeDB and tagged , , , , , , , , , , | Leave a comment
12 Feb 2021

The Data Cloud Tour: Mobilize a World of Data

6 INDUSTRY EVENTS

4 REGIONS!

Most years I have been at Snowflake, we have done Data for Breakfast events all over the world to kick off a new year. But this year is different, so instead we are launching a virtual #DataCloud Tour! Join us to learn how the Data Cloud can power your data strategies and help you deliver innovative and industry-leading products and services.

In this tour we will actually run seven different events in each region – one for the generalist and then separate sessions for these industries:

  1. Healthcare and Life Sciences
  2. Financial Services
  3. Public Sector
  4. Retail & CPG
  5. Media, Entertainment, & Advertising
  6. Manufacturing

Each event will bring you industry SMEs talking about how the Data Cloud has helped them be data leaders and succeed in getting the more value from their data in the most cost effective manner. In the talks you will learn how to break down data silos with The Data Cloud and how you can put the Snowflake platform to work for your company.

The Data Cloud is a global network where thousands of organizations mobilize data with near-unlimited scale, concurrency, and performance. Inside the Data Cloud, you can unite your siloed data, easily discover and securely share governed data, and execute diverse analytic workloads.

See the full agenda and reserve your spot (#Free) here: The Data Cloud Tour

#MobilizeYourData

Kent

The Data Warrior & Chief Technical Evangelist, Snowflake

Posted in SnowflakeDB and tagged , , , , , , | Leave a comment
18 Dec 2020

Snowflake and #NetZero: The Case for Data Decarbonisation (Part One)

I love the fact that Snowflake is getting involved in the betterment of our planet!

This is the first in a series of three posts from one of my colleagues examining the concept of net zero data and how advances in technology can help the world’s largest world’s largest organisations—especially those which are particularly emissions-intensive like oil and gas—reduce the carbon emissions footprint of their data. 

Read the whole post here –  Snowflake and Net Zero: The Case for Data Decarbonisation (Part One)

Enjoy!

Kent

The Data Warrior & Chief Technical Evangelist at Snowflake

#LetItSnow

 

 

Posted in SnowflakeDB and tagged , , , ,
11 Dec 2020

Data Vault 2.0 Automation with erwin and Snowflake

I am seeing a HUGE uptick in interest in Data Vault around the globe. Part of the interest is the need for agility in building a modern data platform. One of the benefits of the Data Vault 2.0 method is the repeatable patterns which lend themselves to automation.  I am please to pass on this great new post with details on how to automate building your Data Vault 2.0 architecture on Snowflake using erwin! Thanks to my buddy John Carter at erwin for taking this project on.

The Data Vault methodology can be applied to almost any data store and populated by almost any ETL or ELT data integration tool. As Snowflake Chief Technical Evangelist Kent Graziano mentions in one of his many blog posts, “DV (Data Vault) was developed specifically to address agility, flexibility, and scalability issues found in the other mainstream data modeling approaches used in the data warehousing space.” In other words, it enables you to build a scalable data warehouse that can incorporate disparate data sources over time. Traditional data warehousing typically requires refactoring to integrate new sources, but when implemented correctly, Data Vault 2.0 requires no refactoring.

Successfully implementing a Data Vault solution requires skilled resources and traditionally entails a lot of manual effort to define the Data Vault pipeline and create ETL (or ELT) code from scratch. The entire process can take months or even years, and it is often riddled with errors, slowing down the data pipeline. Automating design changes and the code to process data movement ensures organizations can accelerate development and deployment in a timely and cost-effective manner, speeding the time to value of the data.

Snowflake’s Data Cloud contains all the necessary components for building, populating, and managing Data Vault 2.0 solutions. erwin’s toolset models, maps, and automates the creation, population, and maintenance of Data Vault solutions on Snowflake. The combination of Snowflake and erwin provides an end-to-end solution for a governed Data Vault with powerful performance.

Get the rest of the details here: Data Vault Automation with erwin and Snowflake

Vault away my friends!

Kent

The Data Warrior

Posted in Data Vault, SnowflakeDB and tagged , , , , , , , , , , , , , | Leave a comment
23 Nov 2020

How Snowflake’s Founders Architected the Rise of the #DataCloud

On the heels of a very successful #DataCloud Summit, I am pleased to let you all know that Snowflake CEO Frank Slootman is publishing a book that really illuminates the Data Cloud and how we got here.

According to Gartner, the public cloud services market continues to grow, largely due to the data demands of modern applications and workloads. And data is one of the leading factors in this transition. In recent years, organizations have struggled with processing big data, sets of data large enough to overwhelm commercially available computing systems.

For a long time, the only real solution was data warehousing services. These services relied on specialized computer hardware to increase the scale of data processing. But these systems had major drawbacks in terms of their extremely high cost and performance constraints. Increasing scale this way wasn’t feasible for many or even most companies. With demand continuing to explode, the world desperately needed a more democratic solution for big data delivery.

A new book, The Rise of the Data Cloud, looks at how that problem can be solved over a few short years. As the founders of Snowflake came together to design a better big data solution, they built an entirely new class of cloud computing in the process.

Get all the details here –  The Rise of the Data Cloud

Enjoy

Kent G

The Data Warrior

P.S. If for some reason you missed the Snowflake Data Cloud Summit, you can still view on the content by signing into the Summit site here.

Posted in SnowflakeDB and tagged , , , , , , | Leave a comment

Post Navigation

← Older Entries
%d bloggers like this: